YOLOv8 with ONNX Runtime C++ on CPU

This demo runs one image through a YOLOv8 nano model and prints the detected bounding boxes. ONNX Runtime executes the neural network. The C++ code performs the image preprocessing and YOLO postprocessing itself.

1. Install the Dependencies

From the repository root, install the compiler, CMake, and the OpenCV parts used to load and resize images:

sudo apt update
sudo apt install build-essential cmake libopencv-dev python3-venv

Info

OpenCV DNN is not used. Inference is performed only by ONNX Runtime.

---

ONNX Runtime C++

demos/ort_cpu_demo/onnxruntime-linux-x64-1.29.0/
├── include/onnxruntime_cxx_api.h
└── lib/libonnxruntime.so

To download and extract that package again:

cd demos/ort_cpu_demo
curl -fLO https://github.com/microsoft/onnxruntime/releases/download/v1.29.0/onnxruntime-linux-x64-1.29.0.tgz
tar -xzf onnxruntime-linux-x64-1.29.0.tgz
rm onnxruntime-linux-x64-1.29.0.tgz
cd ../..

YOLOv8 Nano Model

The converted model is already present at demos/ort_cpu_demo/yolov8n.onnx. To reproduce it, install the export tools:

python3 -m venv .venv
.venv/bin/python -m pip install ultralytics onnx onnxslim

Download the original PyTorch model:

curl -fL https://github.com/ultralytics/assets/releases/download/v8.3.0/yolov8n.pt \
  -o demos/ort_cpu_demo/yolov8n.pt

Export a fixed 640 x 640, batch-one model without embedded NMS:

.venv/bin/yolo export model=demos/ort_cpu_demo/yolov8n.pt format=onnx imgsz=640 batch=1 dynamic=False nms=False

nms=False is important because this demo expects the raw YOLOv8 output and implements confidence filtering and NMS in C++.

2. Build and Run

The bundled ONNX Runtime directory is the default:

cmake -S . -B build
cmake --build build --target ort

Run inference on an image:

./build/demos/ort_cpu_demo/ort \
  demos/ort_cpu_demo/yolov8n.onnx \
  /absolute/path/to/image.jpg

Example output:

detections: 2
class=0 confidence=0.91 x=52 y=41 width=120 height=280
class=5 confidence=0.83 x=230 y=95 width=310 height=190

The coordinates use the original image's pixels. Classes are numeric COCO class IDs because this minimal demo does not load label names.

3. Basic Idea

YOLO cannot consume the bytes returned directly by cv::imread(). The image must first be resized, padded, reordered, and normalized. The model produces candidate predictions rather than final boxes, so its output must also be decoded and filtered.

flowchart TD
    A[Image file] --> B[OpenCV BGR image]
    B --> C[Letterbox resize and padding]
    C --> D[RGB FP32 NCHW tensor]
    D --> E[ONNX Runtime session.Run]
    E --> F[Raw YOLOv8 candidates]
    F --> G[Confidence filter]
    G --> H[Restore original coordinates]
    H --> I[Class-aware NMS]
    I --> J[Final bounding boxes]

The data shapes for the bundled model are:

image                 variable height x width x 3, BGR uint8
model input           [1, 3, 640, 640], RGB float32
model output          [1, 84, 8400], float32
final detection       class + confidence + x + y + width + height

For COCO YOLOv8, 84 output channels means four box values plus 80 class scores. 8400 is the number of candidate predictions.

4. Source Code Map

The implementation is intentionally kept in one file:

ort.cpp
├── Detection                   final class, score, and source-image box
├── Letterbox                   input tensor plus resize/padding information
├── preprocess()                image to model tensor
├── intersection_over_union()   overlap measurement used by NMS
├── to_source_box()             model coordinates to original coordinates
├── postprocess()               decode, filter, and suppress candidates
└── main()                      load, validate, infer, and print

5. main() from Top to Bottom

Step 1: Read the Arguments

The executable expects a model and an image:

const std::string model_path = argv[1];
const std::string image_path = argv[2];

Step 2: Load the Image

const cv::Mat image = cv::imread(image_path);

imread() returns an interleaved unsigned 8-bit BGR image shaped conceptually as [height, width, 3].

Step 3: Create the ONNX Runtime Session

Ort::Env environment(ORT_LOGGING_LEVEL_WARNING, "yolov8-demo");
Ort::SessionOptions options;
options.SetGraphOptimizationLevel(GraphOptimizationLevel::ORT_ENABLE_ALL);
Ort::Session session(environment, model_path.c_str(), options);

• Ort::Env owns the process-level ONNX Runtime environment.
• Ort::SessionOptions controls how the model runs.
• Ort::Session loads and prepares the ONNX graph.
• No execution provider is added, so ONNX Runtime uses its CPU provider.

The objects remain alive until main() exits. Their destructors release the native ONNX Runtime resources automatically.

Steps 4-5: Inspect the Model Contract

The demo accepts exactly one input and one output. It asks the loaded model for the input shape instead of assuming 640 x 640:

const std::vector<int64_t> input_shape = session.GetInputTypeInfo(0)
    .GetTensorTypeAndShapeInfo()
    .GetShape();

The required layout is [batch, channels, height, width], with batch 1, three color channels, and fixed positive dimensions.

Step 6: Preprocess the Image

Letterbox letterbox = preprocess(image, input_width, input_height);

preprocess() returns both the tensor data and the scale/padding values needed later to restore bounding boxes.

Step 7: Wrap the Input Tensor

Ort::Value input = Ort::Value::CreateTensor<float>(
    memory,
    letterbox.input.data(),
    letterbox.input.size(),
    input_shape.data(),
    input_shape.size()
);

This does not copy the float data. Ort::Value points at letterbox.input, so the Letterbox object must stay alive through session.Run().

Steps 8-9: Select Nodes and Run Inference

Input and output node names belong to the ONNX model, so the program queries them at runtime:

auto input_name = session.GetInputNameAllocated(0, allocator);
auto output_name = session.GetOutputNameAllocated(0, allocator);

The synchronous inference call is:

std::vector<Ort::Value> outputs = session.Run(
    run_options,
    input_names,
    &input,
    input_count,
    output_names,
    output_count
);

run_options   options for this single call
input_names   model node receiving the input
&input        tensor passed to that node
input_count   number of input names and tensors
output_names  model nodes whose values are requested
output_count  number of requested outputs

Run() blocks until CPU inference finishes. The returned vector owns the output tensors allocated by ONNX Runtime.

Steps 10-12: Validate, Postprocess, and Print

The program verifies that the output is three-dimensional, then passes its float memory to postprocess():

const std::vector<Detection> detections = postprocess(
    outputs[0].GetTensorData<float>(),
    output_shape[1],
    output_shape[2],
    letterbox,
    image.size(),
    confidence_threshold,
    iou_threshold
);

The final loop prints each retained detection in source-image coordinates.

6. Preprocessing Drilldown

Letterbox Resize

Stretching a rectangular image directly to a square changes object shapes. Letterboxing instead uses one scale for both axes and fills the unused area.

original image                 640 x 640 model input

+------------------+          +--------------------------+
|                  |          |       padding = 114      |
|   original       | resize   |+------------------------+|
|   aspect ratio   | -------> || resized image          ||
|                  |          |+------------------------+|
+------------------+          |       padding = 114      |
                              +--------------------------+

The scale and centered padding are:

scale = min(model_width / image_width, model_height / image_height)
pad_x = (model_width  - resized_width)  / 2
pad_y = (model_height - resized_height) / 2

The padding value 114 matches the common Ultralytics preprocessing value.

BGR/HWC to RGB/NCHW

OpenCV and the model use different memory conventions:

OpenCV pixel memory                 YOLO input memory

B G R | B G R | B G R ...          all R values
                                    all G values
                                    all B values

HWC uint8 [0, 255]       --->       NCHW float32 [0, 1]

For pixel index i, the code writes:

input[i] = bgr[2] / 255.0F;                  // red plane
input[plane_size + i] = bgr[1] / 255.0F;     // green plane
input[2 * plane_size + i] = bgr[0] / 255.0F; // blue plane

7. YOLOv8 Output Drilldown

The raw output is stored channel-first:

                    candidate 0  candidate 1  ... candidate 8399
center_x                 x0           x1                 x8399
center_y                 y0           y1                 y8399
width                    w0           w1                 w8399
height                   h0           h1                 h8399
class 0 score            s0           s1                 ...
class 1 score            s0           s1                 ...
...                      ...          ...                ...
class 79 score           s0           s1                 ...

For each candidate, postprocess() finds the largest class score. Candidates below 0.25 are discarded. Raw YOLOv8 detection output has no separate objectness row, so the selected class score is used as confidence.

Restore Original Coordinates

YOLO emits a center-based box in letterboxed model coordinates:

left   = center_x - width / 2
top    = center_y - height / 2
right  = center_x + width / 2
bottom = center_y + height / 2

Undo the letterbox transform for every coordinate:

source_x = (model_x - pad_x) / scale
source_y = (model_y - pad_y) / scale

The result is clipped to the source image. Empty boxes are discarded.

Class-Aware Non-Maximum Suppression

Many candidates describe the same object. NMS keeps the strongest candidate and rejects weaker boxes of the same class when their intersection-over-union is above 0.45.

flowchart TD
    A[Sort candidates by confidence] --> B[Take next candidate]
    B --> C{Overlaps a kept box<br/>of the same class?}
    C -- No --> D[Keep it]
    C -- Yes, IoU > 0.45 --> E[Discard it]
    D --> F{More candidates?}
    E --> F
    F -- Yes --> B
    F -- No --> G[Final detections]

IoU measures overlap relative to the combined area:

IoU = intersection area / union area

Different classes do not suppress one another.

8. Project and CMake Flow

gst_cpp_plugin_tutorial/
├── CMakeLists.txt
└── demos/
    ├── CMakeLists.txt
    └── ort_cpu_demo/
        ├── CMakeLists.txt
        ├── README.md
        ├── ort.cpp
        ├── yolov8n.pt
        ├── yolov8n.onnx
        └── onnxruntime-linux-x64-1.29.0/
            ├── include/
            └── lib/

CMake enters each directory in order:

flowchart LR
    A[Root CMakeLists.txt] -->|add_subdirectory| B[demos/CMakeLists.txt]
    B -->|add_subdirectory| C[ort_cpu_demo/CMakeLists.txt]
    C --> D[ort executable]

The demo CMake file finds OpenCV, locates the bundled ONNX Runtime header and library, and defines the executable:

add_executable(ort ort.cpp)
target_include_directories(ort PRIVATE "${ONNXRUNTIME_INCLUDE_DIR}")
target_link_libraries(ort PRIVATE "${ONNXRUNTIME_LIBRARY}" ${OpenCV_LIBS})

The build RPATH points at the located ONNX Runtime library directory, allowing the build-tree executable to find libonnxruntime.so when it starts.

To use another extracted runtime instead of the bundled one:

cmake -S . -B build -DONNXRUNTIME_ROOT=/absolute/path/to/onnxruntime
cmake --build build --target ort

Complete example

ort.cpp

/*
 * YOLOv8 + ONNX Runtime pipeline
 * =================================
 *
 *   image file
 *   BGR, original size
 *         |
 *         v
 *   +------------------------+
 *   | Letterbox preprocessing|
 *   | - keep aspect ratio    |
 *   | - resize               |
 *   | - pad with value 114   |
 *   +------------------------+
 *         |
 *         v
 *   +------------------------+
 *   | Tensor conversion      |
 *   | BGR -> RGB             |
 *   | uint8 -> float [0, 1]  |
 *   | HWC -> NCHW            |
 *   +------------------------+
 *         |
 *         v
 *   input tensor [1, 3, H, W]
 *         |
 *         v
 *   +------------------------+
 *   | ONNX Runtime           |
 *   | Ort::Session::Run()    |
 *   +------------------------+
 *         |
 *         v
 *   output tensor [1, 4 + classes, candidates]
 *         |
 *         v
 *   +------------------------+
 *   | YOLOv8 postprocessing  |
 *   | - decode cx, cy, w, h  |
 *   | - choose best class    |
 *   | - confidence filter    |
 *   | - undo padding/resize  |
 *   | - clip to image        |
 *   | - class-aware NMS      |
 *   +------------------------+
 *         |
 *         v
 *   class, confidence, x, y, width, height
 *
 * Preprocessing records scale and padding so boxes predicted in the model's
 * letterboxed coordinate system can be mapped back to the original image.
 *
 * A raw YOLOv8 detection export (nms=False) stores one candidate per output
 * column. Its first four rows are center_x, center_y, width, and height. The
 * remaining rows are class scores. The highest class score is the candidate's
 * confidence; YOLOv8 has no separate objectness row in this output format.
 *
 * NMS (non-maximum suppression) keeps the strongest box and removes weaker,
 * highly overlapping boxes of the same class. ONNX Runtime only executes the
 * model; this file performs preprocessing and postprocessing explicitly.
 */

#include <onnxruntime_cxx_api.h>

#include <opencv2/imgcodecs.hpp>
#include <opencv2/imgproc.hpp>

#include <algorithm>
#include <cmath>
#include <iostream>
#include <stdexcept>
#include <string>
#include <vector>

struct Detection {
    int class_id;
    float confidence;
    cv::Rect box;
};

struct Letterbox {
    std::vector<float> input;
    float scale;
    int pad_x;
    int pad_y;
};

static Letterbox preprocess(const cv::Mat& image, int input_width, int input_height)
{
    const float scale = std::min(
        static_cast<float>(input_width) / image.cols,
        static_cast<float>(input_height) / image.rows
    );
    const int resized_width = std::lround(image.cols * scale);
    const int resized_height = std::lround(image.rows * scale);
    const int pad_x = (input_width - resized_width) / 2;
    const int pad_y = (input_height - resized_height) / 2;

    cv::Mat resized;
    cv::resize(image, resized, cv::Size(resized_width, resized_height));

    cv::Mat padded(input_height, input_width, CV_8UC3, cv::Scalar(114, 114, 114));
    resized.copyTo(padded(cv::Rect(pad_x, pad_y, resized_width, resized_height)));

    const size_t plane_size = static_cast<size_t>(input_width) * input_height;
    std::vector<float> input(3 * plane_size);
    for (int y = 0; y < input_height; ++y) {
        for (int x = 0; x < input_width; ++x) {
            const cv::Vec3b bgr = padded.at<cv::Vec3b>(y, x);
            const size_t index = static_cast<size_t>(y) * input_width + x;
            input[index] = bgr[2] / 255.0F;
            input[plane_size + index] = bgr[1] / 255.0F;
            input[2 * plane_size + index] = bgr[0] / 255.0F;
        }
    }
    return {std::move(input), scale, pad_x, pad_y};
}

static float intersection_over_union(const cv::Rect& left, const cv::Rect& right)
{
    const int intersection = (left & right).area();
    return static_cast<float>(intersection) /
        static_cast<float>(left.area() + right.area() - intersection);
}

static cv::Rect to_source_box(
    float center_x,
    float center_y,
    float width,
    float height,
    const Letterbox& letterbox,
    const cv::Size& source_size)
{
    float left = (center_x - width / 2.0F - letterbox.pad_x) / letterbox.scale;
    float top = (center_y - height / 2.0F - letterbox.pad_y) / letterbox.scale;
    float right = (center_x + width / 2.0F - letterbox.pad_x) / letterbox.scale;
    float bottom = (center_y + height / 2.0F - letterbox.pad_y) / letterbox.scale;

    left = std::clamp(left, 0.0F, static_cast<float>(source_size.width));
    top = std::clamp(top, 0.0F, static_cast<float>(source_size.height));
    right = std::clamp(right, 0.0F, static_cast<float>(source_size.width));
    bottom = std::clamp(bottom, 0.0F, static_cast<float>(source_size.height));

    const int x = static_cast<int>(std::floor(left));
    const int y = static_cast<int>(std::floor(top));
    const int box_width = static_cast<int>(std::ceil(right)) - x;
    const int box_height = static_cast<int>(std::ceil(bottom)) - y;
    return {x, y, box_width, box_height};
}

static std::vector<Detection> postprocess(
    const float* output,
    int64_t channels,
    int64_t candidate_count,
    const Letterbox& letterbox,
    const cv::Size& source_size,
    float confidence_threshold,
    float iou_threshold)
{
    const int class_count = static_cast<int>(channels - 4);
    std::vector<Detection> candidates;

    for (int64_t candidate = 0; candidate < candidate_count; ++candidate) {
        int best_class = 0;
        float best_score = output[4 * candidate_count + candidate];
        for (int class_id = 1; class_id < class_count; ++class_id) {
            const float score = output[(4 + class_id) * candidate_count + candidate];
            if (score > best_score) {
                best_score = score;
                best_class = class_id;
            }
        }
        if (best_score < confidence_threshold) {
            continue;
        }

        const cv::Rect box = to_source_box(
            output[candidate],
            output[candidate_count + candidate],
            output[2 * candidate_count + candidate],
            output[3 * candidate_count + candidate],
            letterbox,
            source_size
        );
        if (box.area() > 0) {
            candidates.push_back({best_class, best_score, box});
        }
    }

    std::sort(
        candidates.begin(),
        candidates.end(),
        [](const Detection& left, const Detection& right) {
            return left.confidence > right.confidence;
        }
    );

    // ponytail: O(n^2) NMS is enough for YOLOv8n; use an optimized NMS only if profiling requires it.
    std::vector<Detection> detections;
    for (const Detection& candidate : candidates) {
        const bool overlaps = std::any_of(
            detections.begin(),
            detections.end(),
            [&](const Detection& kept) {
                return candidate.class_id == kept.class_id &&
                    intersection_over_union(candidate.box, kept.box) > iou_threshold;
            }
        );
        if (!overlaps) {
            detections.push_back(candidate);
        }
    }
    return detections;
}

int main(int argc, char** argv)
try {
    // 1. Read the model and image paths from the command line.
    const std::string program_name = argv[0];
    if (argc != 3) {
        std::cerr << "usage: " << program_name << " <yolov8.onnx> <image>\n";
        return 2;
    }
    const std::string model_path = argv[1];
    const std::string image_path = argv[2];

    // 2. Load the source image. OpenCV stores color images in BGR order.
    const cv::Mat image = cv::imread(image_path);
    if (image.empty()) {
        throw std::runtime_error("cannot read image: " + image_path);
    }

    // 3. Create a CPU ONNX Runtime session and enable graph optimizations.
    Ort::Env environment(ORT_LOGGING_LEVEL_WARNING, "yolov8-demo");
    Ort::SessionOptions options;
    options.SetGraphOptimizationLevel(GraphOptimizationLevel::ORT_ENABLE_ALL);
    Ort::Session session(environment, model_path.c_str(), options);

    // 4. This minimal demo supports models with one image input and one output.
    if (session.GetInputCount() != 1 || session.GetOutputCount() != 1) {
        throw std::runtime_error("expected exactly one model input and one output");
    }

    // 5. Inspect the model instead of hard-coding its input height and width.
    const std::vector<int64_t> input_shape = session.GetInputTypeInfo(0)
        .GetTensorTypeAndShapeInfo()
        .GetShape();
    constexpr size_t batch_axis = 0;
    constexpr size_t channel_axis = 1;
    constexpr size_t height_axis = 2;
    constexpr size_t width_axis = 3;
    if (input_shape.size() != 4 ||
        input_shape[batch_axis] != 1 ||
        input_shape[channel_axis] != 3 ||
        input_shape[height_axis] <= 0 ||
        input_shape[width_axis] <= 0) {
        throw std::runtime_error("expected a static float input shaped [1, 3, height, width]");
    }

    // 6. Letterbox the image and build the normalized RGB NCHW float buffer.
    const int input_height = static_cast<int>(input_shape[height_axis]);
    const int input_width = static_cast<int>(input_shape[width_axis]);
    Letterbox letterbox = preprocess(image, input_width, input_height);

    // 7. Wrap our float buffer as an ONNX Runtime tensor without copying it.
    Ort::MemoryInfo memory = Ort::MemoryInfo::CreateCpu(
        OrtArenaAllocator,
        OrtMemTypeDefault
    );
    Ort::Value input = Ort::Value::CreateTensor<float>(
        memory,
        letterbox.input.data(),
        letterbox.input.size(),
        input_shape.data(),
        input_shape.size()
    );

    // 8. Ask the model for its actual input/output node names.
    Ort::AllocatorWithDefaultOptions allocator;
    auto input_name = session.GetInputNameAllocated(0, allocator);
    auto output_name = session.GetOutputNameAllocated(0, allocator);
    const char* input_names[] = {input_name.get()};
    const char* output_names[] = {output_name.get()};
    const Ort::RunOptions run_options{nullptr};
    constexpr size_t input_count = 1;
    constexpr size_t output_count = 1;

    // 9. Run synchronous CPU inference. This call blocks until output is ready.
    // Arguments, in order:
    // - run_options: use default options for this inference call.
    // - input_names: names of the model nodes receiving input tensors.
    // - &input: address of the first input tensor.
    // - input_count: number of input names and tensors.
    // - output_names: names of the model output nodes to request.
    // - output_count: number of requested outputs.
    // The returned vector owns the output tensors allocated by ONNX Runtime.
    std::vector<Ort::Value> outputs = session.Run(
        run_options,
        input_names,
        &input,
        input_count,
        output_names,
        output_count
    );

    // 10. Validate the raw YOLOv8 output layout before reading its memory.
    const std::vector<int64_t> output_shape = outputs[0]
        .GetTensorTypeAndShapeInfo()
        .GetShape();
    if (output_shape.size() != 3 || output_shape[0] != 1 ||
        output_shape[1] < 5 || output_shape[2] <= 0) {
        throw std::runtime_error(
            "expected raw YOLOv8 output [1, 4 + classes, candidates]; export with nms=False"
        );
    }

    // 11. Decode candidates, restore source coordinates, and apply NMS.
    constexpr float confidence_threshold = 0.25F;
    constexpr float iou_threshold = 0.45F;
    const std::vector<Detection> detections = postprocess(
        outputs[0].GetTensorData<float>(),
        output_shape[1],
        output_shape[2],
        letterbox,
        image.size(),
        confidence_threshold,
        iou_threshold
    );

    // 12. Print the final detections in original-image pixel coordinates.
    std::cout << "detections: " << detections.size() << '\n';
    for (const Detection& detection : detections) {
        std::cout
            << "class=" << detection.class_id
            << " confidence=" << detection.confidence
            << " x=" << detection.box.x
            << " y=" << detection.box.y
            << " width=" << detection.box.width
            << " height=" << detection.box.height
            << '\n';
    }
    return 0;
} catch (const Ort::Exception& error) {
    std::cerr << "ONNX Runtime error: " << error.what() << '\n';
    return 1;
} catch (const std::exception& error) {
    std::cerr << "error: " << error.what() << '\n';
    return 1;
}

CMakeLists.txt

find_package(OpenCV REQUIRED COMPONENTS core imgcodecs imgproc)

set(DEFAULT_ONNXRUNTIME_ROOT "${CMAKE_CURRENT_SOURCE_DIR}/onnxruntime-linux-x64-1.29.0")
get_filename_component(
    OLD_ONNXRUNTIME_ROOT
    "${CMAKE_CURRENT_SOURCE_DIR}/../onnxruntime-linux-x64-1.29.0"
    ABSOLUTE
)
set(ONNXRUNTIME_ROOT "${DEFAULT_ONNXRUNTIME_ROOT}" CACHE PATH "ONNX Runtime directory")
if(NOT ONNXRUNTIME_ROOT OR "${ONNXRUNTIME_ROOT}" STREQUAL "${OLD_ONNXRUNTIME_ROOT}")
    set(ONNXRUNTIME_ROOT "${DEFAULT_ONNXRUNTIME_ROOT}" CACHE PATH "ONNX Runtime directory" FORCE)
endif()

unset(ONNXRUNTIME_INCLUDE_DIR CACHE)
unset(ONNXRUNTIME_LIBRARY CACHE)

find_path(
    ONNXRUNTIME_INCLUDE_DIR
    onnxruntime_cxx_api.h
    PATHS "${ONNXRUNTIME_ROOT}/include"
    NO_DEFAULT_PATH
    REQUIRED
)

find_library(
    ONNXRUNTIME_LIBRARY
    onnxruntime
    PATHS "${ONNXRUNTIME_ROOT}/lib" "${ONNXRUNTIME_ROOT}/lib64"
    NO_DEFAULT_PATH
    REQUIRED
)

add_executable(ort ort.cpp)
target_include_directories(ort PRIVATE "${ONNXRUNTIME_INCLUDE_DIR}")
target_link_libraries(ort PRIVATE "${ONNXRUNTIME_LIBRARY}" ${OpenCV_LIBS})

get_filename_component(ONNXRUNTIME_LIBRARY_DIR "${ONNXRUNTIME_LIBRARY}" DIRECTORY)
set_target_properties(ort PROPERTIES BUILD_RPATH "${ONNXRUNTIME_LIBRARY_DIR}")