It is becoming increasingly common to move the learning process from the cloud to the device itself, driven by the needs of new ML applications in edge intelligence circumstances. One of the most important concerns in the cutting-edge study of on-device learning is the implementation of a high-efficiency learning framework and the facilitation of system-level acceleration. Taking this into consideration, this chapter takes an in-depth look at the most recent developments in the field and identify prospective avenues for further study and development. Model-level computing acceleration, algorithm-level training coordination, and data-level feature perception are all discussed in this chapter to show where research is headed. Edge intelligence applications may benefit from a high-performance TinyML system that can be used to integrate on-device learning methods into the realistic downstream tasks.