As of April 2023, one-third of American adults have used generative AI. OpenAI launched ChatGPT in November 2022, triggering a global AI craze.

Currently, most generative AI applications run in the cloud, but their workloads also impose additional equipment and operational cost burdens on the cloud. Therefore, as applications such as ChatGPT and Midjournal become increasingly widely used, these additional AI workloads are prompting people to reassess the optimal deployment of AI models.

One of the most promising strategies is to migrate some or all of the AI workload to edge terminals, such as smartphones, laptops, and XR headsets, which have powerful terminal side AI processing capabilities. To implement this terminal side AI deployment strategy, it is necessary to optimize the AI model for edge terminals to fully utilize the AI accelerators supported by the terminals.

Generative AI requirements that can be deployed locally include text generation, image and video generation, enhancement or modification, audio creation or enhancement, and even code generation.

At the World Mobile Conference in early 2023, Qualcomm demonstrated the text generated image generation AI model, Stable Diffusion, on smartphones equipped with the second-generation Snapdragon 8.

Recently, Qualcomm announced plans to support the deployment of the Meta Llama 2 based Large Language Model (LLM) on the Snapdragon platform in 2024. Optimizing these neural networks can reduce memory and processing requirements, making them suitable for the processing capabilities of mainstream edge terminals.

Although the performance improvement speed of mobile system level chips (SoCs) is not as fast as the parameter growth rate of generative AI applications such as ChatGPT, there are currently many generative AI models with parameters below 10 billion that are suitable for terminal side processing, and the number of these AI models will continue to increase.