With global technology moving forward at an incredible rate, the metaverse is now poised to become the next powerhouse to promote economic and social development.
This buzzword has multiple application scenarios, from the OASIS-like physical interaction device featured in the movie "Ready Player One" to using VR medical technology to do surgery remotely. Among them, the basis of efficient interaction is low-latency data transmission.
In this article, we invited Mr. Chen Ce, specialist in streaming media from NetEase.IM, to share his ideas on transmission architecture, edge access, large-scale network transmission, and applications of transmission network architecture in the metaverse.
The Ultra-low Latency Transmission Architecture of NetEase.IM
In terms of data input, the main types of metaverse transmission can be divided into three distinct categories: control signaling, audio/video data, and VR/AR depth map data.
Control signaling: This is the command for controlling the operation of a communication system. The data volume of this type is relatively small, but it is vital and cannot be lost. In contrast to audio/video, VR/AR depth maps contain much larger data volumes, usually exceeding 15Mbps, and the low latency transmission required for such applications is a major bottleneck for the development of metaverses today.
When dealing with these three types of data, two major issues must be addressed to efficiently run these metaverse services across global users thousands of miles apart.
Edge access: The first issue with real-time communication systems is the most complicated of them all since the global edge real-time network is extremely different from the other networking systems.
Large-scale network transmission: The shortest path for cross-country transmission must be used to minimize the end-to-end delay for global users.
Edge Access
Edge deployment
A direct approach to improving user access is to bring edge servers closer to the users. Due to this, NetEase.IM has enabled the deployment of edge solutions in major countries worldwide, in addition to many provinces and cities within China. Additionally, all servers are routinely checked for network quality, and a "horse racing mechanism" is activated to ensure that high-quality servers are close to users.
Intelligent dispatching
As a follow-up to solving the edge deployment problem, it is necessary to guide end-users to the best nodes. Despite the notion that it contradicts the principle of edge dispatching, why would we need complicated scheduling when servers are already located close to users? Due to the complexity of user networks and carriers, proximity is not necessarily synonymous with optimization. For example, one may consider Southeast Asia, where there are many different network providers, and Indian users may connect to Singapore nodes faster and more reliably than local Indian nodes.
Static dispatching: Nodes closest to the user are selected based on their geographical location;
Users' historical login success rate: whitelist of access nodes;
Historical user service status: jitter rate, latency, and other indicators;
Real-time detection: RTT, packet loss rate, jitter;
Traffic aggregation: 95 percentile peaks, traffic balancing.
Weak network confrontation.
In a real-time network system, weak network confrontation is the most complex, and our strategies vary based on the data type and scenario.
Control signaling: Signaling usually involves a small amount of data with a very high priority. We utilize QUIC for transmission and incorporate high redundancy, which can withstand packet loss of 80%.
Video and audio data: encoded with a self-developed encoder and congestion control technology. Both technologies can adapt the smoothness of the picture quality based on the service type.
AR/VR depth map: Utilizing DataChannel in transmission, 15Mbps of data transmission can be implemented in real-time.
Static resources such as images and files: NetEase.IM's HTTP acceleration proxy service provides global edge access points, and the shortest RTT between China and the United States is approximately 160 microseconds.
Large-scale Network Transmission
WE-CAN (Communications Accelerating Network) is a large-scale distributed transmission network developed by NetEase.IM that is based on the public Internet. This network improves data transmission and reduces transmission costs by intelligent dispatching of various resources.
For instance, the RTT of the public network link between Beijing and Los Angeles is approximately 250ms, which is unstable. It is common to experience data delays of several seconds before the connection is established.
WE-CAN's solution:
- Determine the best route for the public network between Beijing and Los Angeles.
- Ideally, the shortest routes between nodes should not overlap excessively, and congestion should be managed in an overall planned manner.
- Have QoS mechanisms to counteract weak networks.
- Adapt quickly to network jitter and machine failures and switch routes accordingly.
WE-CAN architecture
WE-CAN consists of four modules:
Dispatching node: responsible for access node assignment;
Access node: is responsible for internal and external protocol conversions, prioritization of services, and implementing hot fixes;
Forwarding node: the core forwarding module, which detects pairwise in real-time, forms a full-mesh network, and reports rtt, loss, jitter, and other information;
Control node: collect forwarding node reports and plan optimal routes.
Route planning
Link quality score
Based on the reported data (rtt, loss, and jitter), the link quality MOS score between any two nodes is calculated, and it is displayed as a decimal between 0 and 1.
Optimal path selection
a. Using dijkstra algorithm to calculate the shortest route between any two nodes, the multi-hop route score is the product of the quality score of each link segment; the more hops, the smaller the product, which is equivalent to the number of hops penalty. For example, the combined score of the route A->B->C->D is 0.95*0.95*0.92 = 0.83.
b. Exclude all intermediate nodes in the optimal path and continue to calculate the optimal path as the suboptimal path again, and so on to determine k optimal paths.
Congestion avoidance selects multiple optimal paths to avoid traffic congestion. Having determined the best route, it aggregates all the best paths, checks that each transit node exceeds the threshold for traffic, and if so, marks the transit node as congested, switches to the suboptimal path, and so on. Following multiple rounds of calculations, obtain the final routing table, synchronize it with the forwarding nodes, and the forwarding nodes will deliver data in accordance with the routing table.
Fast Obstacle Avoidance
In addition to shortest path planning, a sound transmission network should have the capability of fast obstacle avoidance. WE-CAN primarily addresses this problem in three steps.
Control nodes detect network jitter or machine downtime, update the routing table, and distribute it immediately.
The forwarding nodes employ mechanisms such as ARQ and FEC to overcome packet loss during bursts in the network.
Forwarding nodes switch the sending route from optimal to suboptimal when they become aware that the link RTT has surpassed a threshold.
Transmission Quality Comparison
Below is the comparison of WE-CAN and public network transmission quality in China and the United States:
The percentage of arrivals is greater than 95% across all statistical windows.
Latency: RTT
Practical Applications of Metaverse
NetEase Fuxi remote sensing system supports the digital transformation of traditional industries. The system allows users to remotely control excavators for working in outdoor environments with poor network environments while also adapting to low- and mid-range cell phones and placing complex calculations and rendering on the cloud to enhance production efficiency.
NetEase Yaotai is a first-of-its-kind practical metaverse product in China. Unlike traditional video conferencing with a single presentation, Yaotai provides a more immersive experience and meets the needs of the real world.
Previously, NetEase relocated its global investor conference to Yaotai, where over 200 participants from many countries participated in a virtual exchange on NetEase's business using their virtual images. The entire scenario was built on NetEase's WE-CAN global intelligent routing network and converged communication capabilities.
NetEase.IM's virtual human solution can be applied to such scenarios as FinTech virtual customer service, Internet medical remote consultation, virtual anchors, social entertainment digital idols, intelligent virtual assistants, and real-time virtual commerce. Unlike other virtual human solutions currently available on the market, NetEase.IM provides a one-stop SDK. The users only need to connect to the SDK to have access to several capabilities, including virtual humans, RTC, and streaming simultaneously. In addition, NetEase's client/cloud dual rendering technology meets the needs of many customers and scenarios.
Conclusion
As stated by renowned scientific fantasy writer Mr. Liu Cixin, there are two approaches before humans; one leads outwards to the starry sea, whereas the other leads inwards to virtual reality. While the former aspires to explore the vast physical universe, the latter explores the metaverse world.
In Chen Ce's view, the metaverse will offer significant potential for developing key technologies and industries, resulting in a fundamental transformation of society and business models. As part of this process, NetEase.IM will also examine specific scenarios to gain further insight into enterprise clients' true needs and provide them with better and more stable products and services.
