Why is edge analytics important for software-defined vehicles?
In a world where vehicle features and functionality are mainly software-driven, groups of applications perform specific tasks. Due to increasing expectations for vehicle functionality, multiple groups of applications will fulfill different use cases, such as AI assistants, commercial fleet management apps, and gaming apps.
These applications are continuously updated, and new functionality is constantly rolled out via software upgrades. However, when cars are on the road, frequent changes to the vehicle setup can introduce failures. Edge analytics plays a key role in staying reactive and getting immediate notifications about failures in application chains and their root causes.
Edge analytics adds value not only during the production phase, but also during integration, when you can monitor all your deployments and identify failures before cars are on the road.
Why would an SDV use AI at the edge instead of sending all data to the cloud?
There are several reasons for that.
First, sending a huge amount of data from a large number of vehicles to the cloud is not scalable in terms of resource usage. Tokens that are processed with LLMs are expensive.
Additionally, network capacity is limited, and the network should not be overwhelmed by sending large amounts of data from each vehicle to the cloud.
Most importantly, edge analytics keeps most of the data private on the vehicle where it is generated. These factors are also important for design decisions when doing edge analytics for software-defined vehicles.
“Identifying SDV deployment and application issues on the fly accelerates fixes, keeping end users happy and ensuring a positive experience.”
Can you give an example of how AI-driven edge analytics could improve the driving or user experience?
Identifying SDV deployment and application issues on the fly accelerates fixes, keeping end users happy and ensuring a positive experience. It contributes to a high quality deployment process and always pays off in terms of customer trust and repeat business.
What are some basic challenges of running AI models directly on vehicle hardware?
Vehicles are embedded platforms that are inherently limited in terms of resources and optimised for cost and usage. Although GPUs are available for efficient AI inference, they are not as easily accessible or scalable as they are in the cloud.
Additionally, ADAS applications prioritise access to GPUs for safety and driving experience reasons. Therefore, running Gen AI models for other tasks alongside ADAS applications remains challenging. However, small language models that perform specific tasks are the preferred choice. This is because running big LLM models with their billions of parameters is quite resource-intensive. These models require a lot of GPU storage (VRAM), which is not as readily available in vehicles as it is in the cloud.
Nevertheless, smaller models and software frameworks with optimisations are constantly improving, as are in-vehicle AI platforms that include optimised runtimes and inference frameworks that are optimised for the underlying hardware. The hardware itself is also becoming increasingly powerful.
Learn more about this topic at the SDV Community Days, 24-25 February, in Bonn.
Oliver Kral is a Senior Software Engineer at Elektrobit and works in the automotive industry after he has subsequently gained experience in IT security, cloud engineering and containerisation. After starting his automotive journey in ADASIS projects, he is a principal contributor to the open source embedded software orchestrator Eclipse Ankaios. He has several years of experience in programming Rust and is constantly broadening his AI knowledge through various projects. Oliver has presented at the Eclipse Community Day, Eclipse SDV Hackathons (as hack coach), and Eclipse Software Orchestration workshops, where he has served as one of the workshop leaders.
