DyRoNet:
Dynamic Routing and Low-Rank Adapters for Autonomous Driving Streaming Perception
The advancement of autonomous driving systems hinges on the ability to achieve low-latency and high-accuracy perception. To address this critical need, this paper introduces Dynamic Routering Network (DyRoNet), a low-rank enhanced dynamic routing framework designed for streaming perception in autonomous driving systems. DyRoNet integrates a suite of pre-trained branch networks, each meticulously fine-tuned to function under distinct environmental conditions. At its core, the framework offers a speed router module, developed to assess and route input data to the most suitable branch for processing. This approach not only addresses the inherent limitations of conventional models in adapting to diverse driving conditions but also ensures the balance between performance and efficiency. Extensive experimental evaluations demonstrating the adaptability of DyRoNet to diverse branch selection strategies, resulting in significant performance enhancements across different scenarios. This work not only establishes a new benchmark for streaming perception but also provides valuable engineering insights for future work.
The videos presented here demonstrate the advantages of our proposed DyRoNet. From the videos, it is evident that DyRoNet can achieve performance equal to or even surpassing all sub-models in the Model Bank (in these examples, DyRoNet is consisting of StreamYOLO s and StreamYOLO m) in complex and dynamic autonomous driving scenarios.
In static scenarios, DyRoNet's performance is equivalent to that of any individual sub-model in the Model Bank.
The following table illustrates the specific performance of DyRoNet under different Model Bank configurations:
| Model Bank | sAP (0.50:0.95) | sAP (0.50) | sAP (0.75) | sAP s | sAP m | sAP l |
|---|---|---|---|---|---|---|
| StreamYOLO S + M | 33.7 | 53.9 | 34.1 | 13.0 | 35.1 | 59.3 |
| StreamYOLO S + L | 36.9 | 58.2 | 37.5 | 14.8 | 37.4 | 64.2 |
| StreamYOLO M + L | 35.0 | 55.7 | 35.5 | 13.7 | 36.2 | 61.1 |
| LongShortNet S + M | 30.5 | 51.2 | 30.2 | 11.3 | 31.1 | 56.1 |
| LongShortNet S + L | 37.1 | 58.3 | 37.6 | 15.1 | 37.6 | 63.7 |
| LongShortNet M + L | 36.9 | 58.2 | 37.4 | 14.9 | 37.5 | 63.3 |
| DAMO-StreamNet S + M | 35.5 | 56.9 | 36.2 | 14.4 | 36.8 | 63.2 |
| DAMO-StreamNet S + L | 37.8 | 59.1 | 38.7 | 16.1 | 39.0 | 64.2 |
| DAMO-StreamNet M + L | 37.8 | 58.8 | 38.8 | 16.1 | 39.0 | 64.0 |
Citation
x@misc{huang2024dyronet, title={DyRoNet: Dynamic Routing and Low-Rank Adapters for Autonomous Driving Streaming Perception}, author={Xiang Huang and Zhi-Qi Cheng and Jun-Yan He and Chenyang Li and Wangmeng Xiang and Baigui Sun and Xiao Wu}, year={2024}, eprint={2403.05050}, archivePrefix={arXiv}, primaryClass={cs.CV}}