Setting Up and Using ONNX Runtime for C++ in Linux

This article explains how to set up ONNX Runtime, a fast inference engine for machine learning models, to run in a native C++ application on Linux. It covers installing the necessary tools, downloading the ONNX Runtime library and an ONNX model, and configuring a CMake project to use the runtime.

Details

ONNX Runtime is a practical tool for running machine learning models in a native C++ application on Linux. It provides a fast inference engine for models stored in the ONNX format, allowing developers to train or export models elsewhere and then deploy them in a lightweight C++ program without needing a full Python environment. This is especially useful when the application is already written in C++, such as a backend service, robotics stack, desktop application, or embedded system, where Linux and CMake are likely already part of the workflow. The article walks through the setup process, including installing the necessary tools, downloading the ONNX Runtime library and an ONNX image classification model, and configuring a CMake project to link the C++ application against the ONNX Runtime library. The provided CMake configuration sets up the project, enables C++17, points to the ONNX Runtime library and headers, links the executable against the library, and sets an rpath so the executable can find the ONNX Runtime shared library during local development.