Edge Computing Promoting the Development of Large Models

With the development of generative AI, the performance of large models in multi-model fields such as images, audio, and video showcases significant potential for industrial applications, particularly in enhancing efficiency and predictability. The development of computing power, information processing, storage requirements, edge computing, and edge-end models has become a focal point for the industry, addressing challenges related to precision, local reasoning, and deployment.

On the occasion of the launch of the Chongqing Edge Computing Laboratory, jointly established by Terminus Group and the Chinese Academy of Sciences, we interviewed Dr. Yang Yang, Chief Scientist of Terminus Group. During the interview, we discussed the development and application of edge models in the realm of large-scale models.

The Chongqing Edge Computing Laboratory aims to significantly advance scientific research and development in the areas of edge intelligence and computing services, low-carbon technology, and urban IoT operations. The laboratory seeks to facilitate the progress of major national projects and the formulation of technical standards, enhance the cultivation of industry-relevant talent, promote the development of innovative ecological industries and the digital economy, and explore the implementation of edge computing driven by LLMs technology in Terminus Chongqing AI CITY.

Q1：What’s your understanding of edge computing?

The concept of edge computing was proposed around 2017 and has significantly influenced both daily life and industry since 2020, primarily due to the rapid proliferation of Internet of Things (IoT) devices. Previously, users transmitted information online through numerous IoT devices, making it challenging to evaluate the significance of the digital information and determine whether it had changed compared to previous information. This situation necessitated cloud processing, which was often difficult to implement effectively. Therefore, adding a layer of edge computing became essential to facilitate digital information processing. If edge computing is unable to process the information, it will be sent to the cloud; however, not all digital information will be transmitted to the cloud for processing. For instance, if there is no change in temperature information, there is no need to upload it online or resend it to users.

Q2: How do you perceive the edge computing market?

At present, there is considerable demand for edge computing in the market. The number of users is limited, while the number of sensors is difficult to quantify. In telecommunications networks, the number of sensors far exceeds the number of users. Edge computing facilitates the development of IoT scenarios and increasingly impacts a growing number of users, primarily in the following aspects:

Firstly, there is an increasing amount of digital information that cannot be uploaded to communication networks, and much important information is being affected by inaccurate digital information.

Secondly, network congestion has resulted in increased service latency. Similar to a newly constructed highway, which initially accommodates fewer vehicles without congestion, as traffic volume rises, congestion occurs, leading to decreased efficiency. Additionally, security is enhanced through localized processing, which ensures that digital information is accessible only to the users themselves. This process requires multiple uploads to succeed, thereby helping to protect privacy and security.

Taking the intelligent manufacturing scenario as an example, the manufacturing industry has high efficiency requirements. Previously, numerous workers on the production line were responsible for checking for production defects. Currently, a large number of sensors capture product images to identify any defects. If this information can only be processed in the cloud, it results in long processing times and slow response speeds, potentially leading to significant waste on the assembly line. Therefore, factories typically require that information be processed locally. They prefer not to transmit digital information to the cloud through operators and seek fast responses, hoping to immediately detect and address any problems as they arise.

Q3: How will Terminus perform in the field of edge computing?

With the advancement of IoT technology, there is an increasing need for the integration of the communication and computing industries to tackle emerging challenges. Terminus can analyze, modelize, conduct collaborative research, and make adjustments in conjunction with the industry to create value for users and address problems through AIoT-enabled technologies.

Q4: What is the significance of combining large model technology with edge computing?

The challenges presented in real-world scenarios cannot be entirely addressed by large models alone. Therefore, it is necessary to partition the large model at the edge nodes and deploy certain capabilities to smaller or industrial models. This approach allows local, data-driven models to be processed at the edge nodes. The models do not need to be excessively large or costly to train; they can effectively process information that is relatively important to users. This represents the application of large models on the edge.

Developers of cloud-based models aim to address real-world problems. However, these models are often impractical for industrial users. In contrast, edge models, which are driven by local information, can effectively meet the needs of specific vertical domains and tackle smaller tasks, thereby delivering maximum value to industrial users.

Q5: How will the collaboration between edge models and cloud models address real-world challenges in various industries?

Large models in the industrial market are increasingly focused on LLMs, but they often lack interaction with various scenario requirements. The large model that Terminus emphasizes is based on AIoT technology and edge computing, which enhances the understanding of LLMs. This approach promotes the implementation of LLMs in conjunction with business scenarios and customer needs, facilitating interaction between large models and the real physical world.

In addition, Terminus has proposed a LLMs-System-Integration technology road map, which aims to integrate large models, traditional small models, and existing information systems. This approach leverages edge computing to bring large models closer to the edge and end-user devices. Some large models are integrated with end-to-end devices, transforming them into intelligent devices. Terminus has also developed a series of edge computing products capable of deploying large models at a scale of 10 billion parameters. Furthermore, the company has created large models for end-user devices, edge environments, and cloud-based systems, which will be deployed and collaborated on according to different scenarios to meet the diverse needs of customers.

Q6: How do you perceive the future development opportunities of edge computing?

The application scenarios for edge computing are extensive. Numerous end-to-end devices, including virtual reality devices, perception devices, wearable devices, and more, need edge computing to enhance their functionality. Additionally, the market is expected to see a growing number of perception devices emerging.

At the same time, the decline in computing power costs will accelerate the emergence of large models on the edge and end devices. The cost of inference is anticipated to decrease at a rate of approximately tenfold per year. Several factors contribute to this reduction in costs, including the decrease in computing power expenses, improvements in the performance of inference frameworks, and enhancements in model training capabilities.

Nowadays, many enterprises and institutions are researching small models, and the capabilities of these models are nearing those of larger models. Some 1 billion parameter models have already demonstrated capabilities comparable to those of 10 billion parameter models. From the standpoint of cost and model performance, edge computing and end-user computing scenarios are poised for rapid development.

Q7: What are the opportunities and directions for cooperation between Terminus and the Chinese Academy of Sciences in jointly developing the Chongqing Edge Computing Laboratory?

The Chinese Academy of Sciences emphasizes fundamental research. Terminus has a diverse array of application scenarios and understands the genuine needs of various industries and markets. By collaborating with the Chinese Academy of Sciences, the company aims to facilitate the application of research findings in real-world markets. This partnership will form a robust alliance, concentrating on:

Joint Talent Development: Collaboratively establishing a postdoctoral program in Chongqing to cultivate talent in the field of technology.

Jointly addressing technical challenges, including major national and provincial projects, and collaboratively solving key technical issues in the fields of national science and technology.

Conducting collaborative research in fields such as embodied intelligence.

The industrialization of technology implementation is a key focus area. The Chinese Academy of Sciences possesses a robust technical foundation and has successfully completed numerous landmark research projects in Chongqing. Terminus, a market-oriented company, engages in collaborative product research and development to create a core competitive solution for the market.

Terminus possesses extensive practical experience in production and finance. The company will collaborate with the Chinese Academy of Sciences to develop high-quality projects, incubate promising enterprises, and foster the development of a robust supply chain ecosystem.

Dr. Yang Yang, Chief Scientist of Terminus Group