Closed-loop Adaptive System for Efficient AI-centric Perception

Note: Please view the desktop site in order to see images.

Advances in sensing technologies (cameras, LiDAR, radar) and large-scale machine learning models have enabled AI systems to capture, interpret, and act on high-dimensional data — powering applications from smartphones to autonomous vehicles and AR/VR.

However, these capabilities come with high costs: training and inference are energy-intensive, and transmitting massive sensor data to cloud-based models demands exponential growth in compute and communication infrastructure. Data generation is expected to soon outpace global transmission capacity.

To sustain progress, computation must move closer to the sensor. This shift requires new strategies for deploying large models on resource-constrained edge devices. Yet, existing approaches typically remain limited in 2 ways:(1) sensing and inference systems are often statically optimized, assuming that all input samples can be uniformly resolved with a single optimized solution; and (2) dynamic optimization methods typically treat the sensor and the model as separate entities, missing out on opportunities from joint optimization.

To address this, we propose closed-loop adaptive perception systems that integrate feedback from inference into sensing, dynamically coordinating both to balance performance and energy. 

Our current work focuses on adaptive inference, which reduces cost by scaling computation to input complexity. Techniques like Mixture-of-Experts, early exiting, and layer skipping embody this approach, but introduce new design challenges such as routing overhead and dynamic resource allocation. We aim to develop new methods that address these challenges and make adaptive inference practical for deployment on resource-constrained edge devices.

We also demonstrated joint sensing–computation adaptation with mmWave radar for pedestrian gait analysis. By dynamically adjusting radar resolution and waveform based on scene complexity — inspired by cognitive radar — and using reinforcement learning to balance accuracy and cost, we significantly improve efficiency and performance.

Publications

[1] Soheil Hor and Amin Arbabian, “Sense, Predict, Adapt, Repeat: A Blueprint for Design of New Adaptive AI-Centric Sensing Systems,” arXiv preprint arXiv:2312.07602, 2023, doi:10.48550/arXiv.2312.07602.

[2] S. Hor, M. Pilanci and A. Arbabian, "A Data-Driven Waveform Adaptation Method for Mm-Wave Gait Classification at the Edge," in IEEE Signal Processing Letters, vol. 29, pp. 26-30, 2022, doi: 10.1109/LSP.2021.3122355.