Artificial Intelligence Race seems to tire many companies. Artificial intelligence solutions, which we have been talking about on the corporate side for a long time, can now be easily used on our mobile phones. One of the main reasons for this is the capabilities of the chips used. Artificial intelligence is now in our pockets and in our hands.

Smartphone manufacturers such as Vivo have also entered the artificial intelligence race. kThe dates were November 1, and Chinese smartphone manufacturer Vivo’s Artificial Intelligence Solution Director Xie Weiqin searched the following sentence on his phone at the Vivo Developer Conference. “Find a photo of a girl smiling while wearing a trash bag in the rain” then the photo appeared on the Teflon in almost seconds.

Meanwhile, Xie was showing off the smartphone assistant powered by Vivo’s latest artificial intelligence model. In addition to searching for photos and files, this tool also allows users to organize photos and extract highlights from articles. And it can also help create social media topics based on keywords or images. These new smartphone functions are enabled by so-called edge AI technology.

Edge AI allows calculations to be run on local devices rather than a centralized cloud computing facility. Unlike cloud-based AI, which relies on data centers to process resource-intensive tasks, edge AI is capable of making real-time decisions without the internet and can be applied to smart devices in daily use.

The release of OpenAI’s ChatGPT chatbot in late 2022 has launched a global Artificial Intelligence Race around large language models (LLMs), a type of AI that can mimic human intelligence, and sparked competition between device makers and chipmakers.

Following Vivo, South Korean smartphone giant Samsung Electronics showcased its Gaussian generative AI model on November 8, and Oppo launched its AndesGPT solution later in the same month. Xiaomi and Honor Technology announced progress in the development of LLMs in late October. Huawei also released Wide Language Model integration for its smartphones in August.

In late October, US chip giant Qualcomm introduced the Snapdragon 8 Gen 3 processor, the world’s first processor to support generative artificial intelligence models with up to 10 billion parameters. Taiwanese chip manufacturer MediaTek previously announced its collaboration with Oppo and Vivo in the field of Large Language Model (LLM). JC Hsu, MediaTek’s senior vice president, said that “a fierce Artificial Intelligence Race has begun” in the field of generative AI.

Companies and investors predict that the adoption of new artificial intelligence technology will revive the flagging consumer electronics market. Data from market research firm Counterpoint shows that global smartphone shipments in the third quarter of 2023 decreased for the ninth consecutive quarter, shrinking by 8% on an annual basis and reaching the lowest third quarter levels in the last decade. Global PC shipments also fell 9% in the third quarter.

Counterpoint said AI-powered computers are very likely to drive a recovery in shipments in 2024 and will dominate the PC market with a penetration rate of over 50% after 2026.

However, challenges remain with edge AI models due to high processing power requirements and intensive memory and storage usage, which can significantly increase the cost. Analysts said Wide Language Models for mobile devices face challenges in energy consumption, as battery energy density is a hardware bottleneck that is difficult to overcome in the short term.

A Broad Language Model engineer said it could take six months to a year for consumers to experience the qualitative change brought by edge AI.

Artificial Intelligence Race Will Positively Affect End Users’ Lives

The most common application of the Broad Language Model on smartphones is smart assistants. These assistants have evolved from voice interactions to supporting multiple inputs such as voice, text, images, and documents. They progressed from passively following instructions to participating in natural conversations and conducting summarization, debriefing, and multilingual translations.

“Talking to AI in the past required careful consideration, similar to caring for a child,” said Luan Jian, head of the Xiaomi Technical Committee Artificial Intelligence Laboratory Large Model Team. But now, Broad Language Models can help users communicate with AI more naturally and conveniently.

AI can also assist users with music and image production on their local devices. During the Snapdragon Summit, Qualcomm demonstrated its “photo augmentation” feature, where a photo can be enhanced with AI-generated landscapes. MediaTek demonstrated the rapid generation of emojis. Intel CEO Pat Gelsinger demonstrated an artificial intelligence-supported computer that could produce songs in the style of Taylor Swift at Intel Innovation 2023 in September.

A year ago there were only one or two use cases for generative AI, now there are hundreds, Qualcomm CEO Cristiano Amon said in a speech. This number will reach thousands by 2024. “Running AI pervasively and continuously on the device will transform our user experience,” says Amon. Of course, another company in the artificial intelligence race is Oppo.

The Biggest Problem is Costs

“We can spend $1 billion to build a trillion-parameter model in the cloud, but how do we get hundreds of millions of people to use it?” Gelsinger asked this question in a September interview with Caixin.

This needs to be achieved by not just allowing users to access the cloud but by bringing the technology towards the customer, Gelsinger said, adding that running cloud AI models on PCs can ensure privacy and data security.

Smartphone and chip developers in the Artificial Intelligence Race have repeatedly talked about the advantages of edge AI models in protecting privacy and security in the last six months. “User data, including call logs, photos, fingerprints and faces, will all be consumed by large models for analysis and inference. Can you accept that if everything is sent to the cloud, especially to third parties?” said Luo Xuan, co-founder of the RWKV Language Model.

Honor CEO Zhao Ming said that cloud AI models address how to better integrate human knowledge, but edge AI models analyze personal data, behaviors and habits to provide services. “If cloud AI models know everything about you, such as your ID card, phone number, address and weight, it is very scary,” Zhao said. Edge AI models can avoid these issues because both data and model outputs remain on the local device.

During the Vivo Developer Conference, Vivo vice president Zhou Wei said that the minimum cost of using a cloud AI model once is 0.012 yuan (0.17 cents USD), and the current cost is around 0.015 yuan. “With 300 million users using it 10 times a day, the bill amounts to about 10 billion yuan a year,” Zhou said.

Rising costs are also pushing aside the Broad Language Model. Cloud AI models often have tens of billions or hundreds of billions of parameters, making inference computation expensive.

“Today, no cloud AI model is profitable due to massive computing power consumption,” says Zhao. Most computations do not require cloud-side solutions. In the future, the industry plans to use edge and cloud AI models collaboratively.

The Search for Balance Continues

Companies in the Artificial Intelligence Race have a lot of investments to make. Edge AI models are trained with billions of parameters, limited by the processing power, memory, storage capacity and battery life of smart devices.

Luo said that in the next six months to a year, smartphones will be able to run LLMs with up to 14 billion parameters, while PCs are expected to host models with 60 billion parameters.

“It is likely that in the future there will be a large model with 14 billion parameters, which acts as the ‘engine’ of the operating system in smartphones. On the cloud side, it will be a model even larger than GPT-4, Luo said. “They act as the basis of the next generation internet. “They will complement each other, similar to the existing relationship between native software and the internet.”

A large model capable of understanding context requires at least 13 billion parameters, which takes up significant smartphone memory and impacts performance, Luo says. Considering the difficulty of increasing battery capacity, energy consumption is a more significant bottleneck than memory usage.

Edge AI Will Inevitably Require Hardware Upgrades

“The biggest challenge is user privacy, computing power, and low power consumption. If any AI application cannot balance these three factors, it cannot provide a better experience to consumers,” says Zhao.

Many in the AI industry say edge AI models are still in their infancy and the industry is still exploring future applications.

Luan says that the first step to improve the ability of AI-powered devices to handle Large Language Models is to increase memory capacity and bandwidth. The second step is to increase or optimize the computing power to efficiently support the networking of Large Language Models.

Additionally, improvements in inference algorithms as well as continued investigation of model compression and quantization are required to reduce computational power demands.

“Yet edge AI models will definitely change the way smartphones are used,” says Luan.

Read this news  iPhone 15 Problems and Solutions