Unleash All Cores: Asymmetry-aware Scalable DNN Inference on Mobile CPUs

Abstract

Asymmetric multiprocessing CPUs are central to mobile devices, but naive DNN scheduling across heterogeneous cores can degrade throughput because of workload imbalance. SANI combines an affinity-aware kernel issuer, an adaptive-granularity scheduler, and an on-demand kernel switcher to preserve core-kernel affinity while dynamically balancing work. Across five mobile SoCs, SANI reduces inference latency by 17.6%–23.7% on average, reaches up to 29.5% on individual models, and lowers energy consumption by up to 39% compared with state-of-the-art baselines.

Qianlong Sang

Fifth-Year Computer Science
Ph.D. Student

My research focuses on operating systems, mobile and edge systems, and efficient on-device AI inference.