NavTalk Achieves 200ms Response Time for Real-Time Digital Human Experience

The article discusses the performance optimization of NavTalk, a real-time digital human system, to achieve an end-to-end latency of under 200ms. It details the challenges faced with AMD-based GPUs and the solutions implemented to leverage GPU-accelerated image processing.

Details

The article focuses on the performance optimization of NavTalk, a real-time digital human system, to achieve an unprecedented response time of under 200ms. Initially, the system faced challenges in meeting real-time requirements, as the processing time exceeded the 0.5-second audio input duration when tested on an A100 GPU environment. The root cause was identified as the use of AMD EPYC processors, which underperform in image processing tasks compared to Intel chips. To address this bottleneck, the team developed a dedicated GPU image processing tool library, leveraging PyTorch's capabilities to offload operations like resizing, blurring, sharpening, and blending to the GPU. This optimization strategy allowed the system to fully utilize the parallel computing power of the GPU, leading to significant performance improvements and the target 200ms response time. The article provides technical details on the implementation of these GPU-accelerated image processing functions, demonstrating the team's innovative approach to overcoming the limitations of the underlying hardware.