AdaptAV: Continuous Adaption of Vision Models for Autonomous Vehicles Using Cloud-based Oracle
Published in VTC 2024, 2024
Deploying vision perception models in autonomous vehicles requires that we prioritize inference speeds, resulting in a model with shallower architectures and lesser model parameters (i.e., more pruned). Such small models do not generalize well, which could result in poor performance when encountered with novel scenarios. We propose a system that overcomes this by continuously retraining the vision models on the cloud with data uploaded by vehicles. We leverage the abundant compute resources, including machine learning accelerators, of the cloud to run a state-of-the-art, highly accurate oracle model that will guide the retraining process of the on-vehicle model. This newly trained model is transmitted to the vehicle over the network and is utilized by the vehicle for perceptions, leading to improved inference accuracy over time.
Recommended citation: Yuheng Z., Dhruva U., Boluo G. & Man-Ki Y. (2024). AdaptAV: Continuous Adaption of Vision Models for Autonomous Vehicles Using Cloud-based Oracle. Proceedings of the 100th IEEE Vehicular Technology Conference, Oct. 2024