Running LTX-2.3 in Real-Time on a 4090

The author has optimized the LTX-2.3 model to run in real-time on a consumer-grade NVIDIA 4090 GPU using the open-source Scope tool. This allows for real-time text-to-video, text-to-image-to-video, and video-to-video workflows with various capabilities.

Details

The author, known as Buff, has been working on running the LTX-2.3 model as efficiently as possible on consumer hardware using the Scope open-source tool. Scope is designed for running real-time AI pipelines and has traditionally focused on autoregressive/self-forcing/causal models. However, Buff believes there is great potential in fast back-to-back bi-directional workflows, such as 'inter-dimensional TV'. By optimizing FP8 optimizations, resolution, frame count, and other parameters, Buff has managed to get LTX-2.3 running in real-time on his local NVIDIA 4090 GPU. The setup supports various capabilities, including text-to-video, text-to-image-to-video, video-to-video with IC-LoRA Union (control input), audio output, LoRA support, and randomized seeds. While there is still a slight delay between clips due to the text-encoder pushing the model out of VRAM, Buff is working on improving the performance. The software playground is completely free, and the author encourages the community to check it out and explore real-time AI visual and audio pipelines.