Guides

Installation and Usage

Tinyopt’s C++ API is header-only. The optional C ABI is built as tinyopt_c; it installs the C headers and shared library alongside the C++ headers. Choose the API in the application source by including either <tinyopt/tinyopt.h> or <tinyopt/c/c_api.h>.

Install

Build and install the default C++ package:

cmake -S . -B build -G Ninja -DTINYOPT_BUILD_C_LIBRARY=OFF
cmake --build build
cmake --install build --prefix "$HOME/.local"

To include the C ABI and generated fixed-size C entrypoints, configure with -DTINYOPT_BUILD_C_LIBRARY=ON:

cmake -S . -B build-c -G Ninja -DTINYOPT_BUILD_C_LIBRARY=ON
cmake --build build-c --target tinyopt_c
cmake --install build-c --prefix "$HOME/.local"

The C library is shared by default. TINYOPT_C_API_FLOAT controls float entrypoints, and TINYOPT_C_FIXED_SIZES controls the generated dimensions. The generated per-dimension headers and the precision aggregators c_api_fixed_double.h and c_api_fixed_float.h are installed under include/tinyopt/c. SuiteSparse support is separate and remains disabled unless -DTINYOPT_ENABLE_SUITESPARSE=ON is requested. Package builds can be produced with pixi run build-pkg on Linux.

C++ Consumer

Use the installed CMake package and link the tinyopt interface target. It supplies the include directory, C++20 requirement, Eigen dependency, and the compile-time features selected when Tinyopt was built:

cmake_minimum_required(VERSION 3.25)
project(cpp_example LANGUAGES CXX)

find_package(Tinyopt CONFIG REQUIRED)
add_executable(cpp_example main.cpp)
target_link_libraries(cpp_example PRIVATE tinyopt)

main.cpp can include <tinyopt/tinyopt.h>. For a non-standard install prefix, set CMAKE_PREFIX_PATH to that prefix when configuring the consumer:

cmake -S . -B build -DCMAKE_PREFIX_PATH="$HOME/.local"

C Consumer

The C ABI must have been enabled when Tinyopt was built. After find_package, locate the installed tinyopt_c library and link it to the C executable:

cmake_minimum_required(VERSION 3.25)
project(c_example LANGUAGES C)

find_package(Tinyopt CONFIG REQUIRED)
find_library(TINYOPT_C_LIBRARY NAMES tinyopt_c
  HINTS "${Tinyopt_INCLUDE_DIRS}/../lib"
  REQUIRED)

add_executable(c_example main.c)
target_include_directories(c_example PRIVATE "${Tinyopt_INCLUDE_DIRS}")
target_link_libraries(c_example PRIVATE "${TINYOPT_C_LIBRARY}")

For example, main.c can minimize a one-dimensional residual:

#include <math.h>
#include <tinyopt/c/c_api.h>

static int residual(const double *x, int dims, double *out, double **jacobian,
                    int num_residuals, void *user_data) {
  (void)user_data;
  if (dims != 1 || num_residuals != 1) return 1;
  out[0] = x[0] - 2.0;
  if (*jacobian != NULL) (*jacobian)[0] = 1.0;
  return 0;
}

int main(void) {
  double x[] = {0.0};
  tinyopt_params_t params = {x, 1, NULL};
  tinyopt_problem_t problem = {0};
  problem.type = TINYOPT_EVAL_RESIDUALS;
  problem.fn.residuals = residual;
  problem.num_residuals = 1;
  if (tinyopt_optimize(&params, &problem, NULL, NULL) != TINYOPT_STATUS_OK) return 1;
  return fabs(x[0] - 2.0) > 1e-5;
}

On Linux, a standard install prefix such as /usr or /usr/local is already on the runtime library search path. For a private prefix, add its lib directory to LD_LIBRARY_PATH when running the executable.

Tinyopt Architectural Design & System Overview

Tinyopt is a header-only C++20 numerical optimization library designed from the ground up for high computational efficiency, low latency, and zero dynamic memory allocations in critical paths.

1. System Layering

Tinyopt is organized into clear, decoupled architectural layers:

┌─────────────────────────────────────────────────────────────────┐
│                    1. High-Level User API                       │
│             Optimize(x, cost_or_residuals, options)            │
│               tinyopt/optimize.h, tinyopt/tinyopt.h             │
└────────────────────────────────┬────────────────────────────────┘
                                 │
┌────────────────────────────────▼────────────────────────────────┐
│                     2. Optimizers Layer                         │
│           Iterative loops, trust region, step acceptance        │
│          tinyopt/optimizers/ (lm.h, gn.h, gd.h, optimizer.h)    │
└────────────────────────────────┬────────────────────────────────┘
                                 │
┌────────────────────────────────▼────────────────────────────────┐
│                       3. Solvers Layer                          │
│          Linear system formation & step solution (H δx = -g)    │
│            tinyopt/solvers/ (lm.h, gn.h, gd.h, base.h)          │
└────────────────────────────────┬────────────────────────────────┘
                                 │
┌────────────────────────────────▼────────────────────────────────┐
│                 4. Differentiation & Losses                     │
│    AutoDiff (Jets), NumDiff, Robust M-Estimators (Huber, Tukey) │
│               tinyopt/diff/, tinyopt/losses/                    │
└────────────────────────────────┬────────────────────────────────┘
                                 │
┌────────────────────────────────▼────────────────────────────────┐
│               5. Core Types & Metaprogramming                   │
│     Compile-time traits, Eigen wrappers, manifold interfaces    │
│           tinyopt/types.h, tinyopt/traits.h, 3rdparty/          │
└─────────────────────────────────────────────────────────────────┘

2. Core Architectural Philosophy: The Accumulation Pattern

Traditional Least Squares (Monolithic Buffering)

Traditional non-linear least squares solvers (such as Ceres Solver or g2o) evaluate residuals by allocating arrays for:

All residuals: $r = [r_1, r_2, \dots, r_M]^T \in \mathbb{R}^M$
Full Jacobian matrix: $J \in \mathbb{R}^{M \times N}$
Assembling the normal equations via $H = J^T J$ and $g = J^T r$

When $M$ (number of observations) is large (e.g. 100,000 points in vision/robotics), storing $J$ requires massive dynamic memory allocations, triggering cache misses and memory bandwidth saturation.

Tinyopt Direct Accumulation

Tinyopt solves this through its Accumulation Pattern: Instead of storing $r$ and $J$, users and cost evaluators accumulate directly into the linear system:

$$H = \sum_{i=1}^M J_i^T J_i \in \mathbb{R}^{N \times N}$$ $$g = \sum_{i=1}^M J_i^T r_i \in \mathbb{R}^N$$

Memory Bound: Memory usage is bounded by parameter size $N \times N$, completely independent of measurement count $M$.
Cache Locality: $H$ and $g$ reside continuously in CPU L1/L2 cache throughout the accumulation pass.
Zero Allocations: Fixed-size solver state uses stack-backed Eigen types during iterations.

For deployments that require Tinyopt itself to avoid dynamic allocation on fixed-size problems, configure with -DTINYOPT_ENFORCE_NO_DYNAMIC_ALLOCATIONS=ON. This uses five-entry FIFO output histories, omits the final Hessian, disables optimizer logging, and activates Eigen’s runtime no-malloc guard. It does not apply to dynamic-size parameters or prevent heap allocations made by user residuals, accumulation functions, or callbacks.

3. Differentiation Architecture

Tinyopt provides three complementary approaches to differentiation:

1. Dual Numbers / Jets (Automatic Differentiation)

Implemented in tinyopt/diff/jet.h.
Implements forward-mode automatic differentiation using dual numbers: $$f(a + b\epsilon) = f(a) + f’(a)b\epsilon, \quad \text{where } \epsilon^2 = 0$$
Allows users to write templated functions (auto loss = [](const auto &x) { ... }) and obtain exact machine-precision derivatives without manual differentiation.

2. Analytical Differentiation

The fastest method available. Users write an accumulation lambda taking (const auto &x, auto &grad, auto &H).
Derivatives are populated directly via Eigen vector/matrix operations.

3. Numerical Differentiation (Finite Differences)

Implemented in tinyopt/diff/num_diff.h.
Uses central differences: $$\frac{\partial f}{\partial x_i} \approx \frac{f(x + \epsilon e_i) - f(x - \epsilon e_i)}{2\epsilon}$$
Primarily utilized as an automated oracle in tinyopt/diff/gradient_check.h to mathematically verify analytical Jacobians in unit tests.

4. Solvers & Damping Logic

Levenberg-Marquardt (LM)

Tinyopt’s LM optimizer solves regularized normal equations:

$$(H + \lambda D) \delta x = -g$$

Where:

$H \approx J^T J$ is the Gauss-Newton Hessian approximation.
$D$ is either the identity matrix $I$ or $\text{diag}(H)$.
$\lambda$ is dynamically adapted using the gain ratio $\rho$: $$\rho = \frac{\text{Actual Cost Reduction}}{\text{Predicted Model Reduction}}$$
- If $\rho > 0$, the step is accepted and $\lambda$ is decreased.
- If $\rho \le 0$, the step is rejected, $\lambda$ is increased, and the linear system is re-solved.

Gauss-Newton (GN)

Direct solution without damping ($\lambda = 0$). Ideal for well-conditioned, near-linear problems near the basin of attraction.

Gradient Descent (GD)

First-order steepest descent step: $\delta x = -\alpha g$, with line search or adaptive step sizing.

5. Manifold & Lie Group Integration

For parameters living on non-Euclidean manifolds (e.g. 3D rotations in $SO(3)$ or rigid poses in $SE(3)$):

Parameter updates use the exponential map (retraction / plus operator): $$x \leftarrow x \boxplus \delta x = x \circ \exp(\delta x)$$
Adapters in include/tinyopt/3rdparty/ support Sophus and Lie++.

CMake Options

Tinyopt’s CMake options control optional algorithms, dependencies, build targets, and compile-time features. Set an option when configuring the project with -DNAME=VALUE. For example:

cmake -S . -B build -G Ninja \
  -DTINYOPT_BUILD_TESTS=ON \
  -DTINYOPT_ENABLE_GRADIENT_DESCENT=ON \
  -DTINYOPT_ENABLE_LINEAR_SOLVER_QR=ON

The options and defaults below are defined in cmake/Options.cmake. Reconfigure after changing options; compile-time features are exposed through the tinyopt interface target.

Compile-Time Features

Option	Default	Description
`TINYOPT_USE_FMT`	`OFF`	Use the `fmt` formatting library when available.
`TINYOPT_ENABLE_FORMATTERS`	`ON`	Enable `std::formatter` specializations for streamable types. Disable to define `TINYOPT_NO_FORMATTERS`.
`TINYOPT_DISABLE_AUTODIFF`	`OFF`	Disable automatic differentiation in optimizers. Numerical differentiation can still be used unless separately disabled.
`TINYOPT_DISABLE_NUMDIFF`	`OFF`	Disable numerical differentiation in optimizers.
`TINYOPT_ENFORCE_NO_DYNAMIC_ALLOCATIONS`	`OFF`	For fixed-size optimization problems, prevent Eigen heap allocations in optimizer paths, retain only the first and latest history samples, omit the final Hessian, and disable optimizer logging. This cannot prevent allocations in user callbacks.
`TINYOPT_ENABLE_GAUSS_NEWTON`	`ON`	Enable the Gauss-Newton optimizer.
`TINYOPT_ENABLE_GRADIENT_DESCENT`	`OFF`	Enable the Gradient Descent optimizer.
`TINYOPT_ENABLE_CONJUGATE_GRADIENT`	`OFF`	Enable nonlinear conjugate gradient in global `Optimize()` dispatch.
`TINYOPT_ENABLE_DOGLEG`	`OFF`	Enable Powell’s DogLeg method in global `Optimize()` dispatch.
`TINYOPT_ENABLE_BFGS`	`OFF`	Enable full-memory BFGS in global `Optimize()` dispatch.
`TINYOPT_ENABLE_LBFGS`	`OFF`	Enable limited-memory BFGS in global `Optimize()` dispatch.
`TINYOPT_ENABLE_OPTIMIZERS_ALL`	`OFF`	Enable Gauss-Newton, Gradient Descent, Conjugate Gradient, DogLeg, BFGS, and L-BFGS in global `Optimize()` dispatch.

The TINYOPT_ENABLE_GAUSS_NEWTON, TINYOPT_ENABLE_GRADIENT_DESCENT, TINYOPT_ENABLE_CONJUGATE_GRADIENT, TINYOPT_ENABLE_DOGLEG, TINYOPT_ENABLE_BFGS, and TINYOPT_ENABLE_LBFGS switches control availability through the global Optimize() dispatch only. They do not prevent direct use: include an optimizer’s header and instantiate its optimizer class. For example, Conjugate Gradient can be used directly even when TINYOPT_ENABLE_CONJUGATE_GRADIENT is OFF:

#include <tinyopt/optimizers/cg.h>

Eigen::VectorXd x = Eigen::VectorXd::Ones(10);
auto cost = [](const auto &value) { return value.squaredNorm(); };
tinyopt::cg::Optimizer<Eigen::VectorXd> optimizer;
auto sum = optimizer.Optimize(x, cost);

Set TINYOPT_ENABLE_CONJUGATE_GRADIENT=ON to select it through global Optimize() using Options::Solver::ConjugateGradient. The same distinction applies to the other optimizer switches. Unit tests for each implementation are built independently of these switches.

Use -DTINYOPT_ENABLE_OPTIMIZERS_ALL=ON to enable every optimizer in one step. SuiteSparse and linear solver backends are controlled by their separate options.

Linear Solvers

Optional Eigen decompositions are disabled by default to reduce compile times. Enable only the methods required by the application.

Option	Default	Description
`TINYOPT_ENABLE_LINEAR_SOLVER_LDLT`	`ON`	Enable dense and sparse LDLT.
`TINYOPT_ENABLE_LINEAR_SOLVER_LLT`	`OFF`	Enable dense and sparse LLT.
`TINYOPT_ENABLE_LINEAR_SOLVER_LU`	`OFF`	Enable dense and sparse LU.
`TINYOPT_ENABLE_LINEAR_SOLVER_QR`	`OFF`	Enable dense and sparse QR.
`TINYOPT_ENABLE_LINEAR_SOLVER_SVD`	`OFF`	Enable dense Jacobi SVD.
`TINYOPT_ENABLE_LINEAR_SOLVER_ALL`	`OFF`	Enable all built-in dense solvers: LDLT, LLT, LU, QR, and SVD. SuiteSparse remains controlled separately.
`TINYOPT_ENABLE_SUITESPARSE`	`OFF`	Enable SuiteSparse CHOLMOD. Review the licenses of the selected SuiteSparse components. With `TINYOPT_BUILD_C_LIBRARY`, also builds the sparse C API (`c_api_sparse.h`).

Build Targets

Option	Default	Description
`TINYOPT_C_FIXED_SIZES`	`1;2;3;4;5;6;10;12`	Fixed parameter dimensions to generate for the C API. Each enabled dimension generates double (`_d`) APIs, and float (`_f`) APIs when `TINYOPT_C_API_FLOAT` is enabled.
`TINYOPT_C_API_FLOAT`	`ON`	Build and expose the float C API, including its dynamic and generated fixed-size entrypoints.
`TINYOPT_BUILD_C_LIBRARY`	`OFF`	Build the optional `tinyopt_c` C ABI library and its C tests.
`TINYOPT_BUILD_SHARED_C`	`ON`	Build `tinyopt_c` as a shared library. Set `OFF` to build a static C library instead.
`TINYOPT_BUILD_WASM`	`OFF`	Build the WebAssembly module and its JavaScript example. Requires Emscripten (`emcmake`) and `TINYOPT_BUILD_C_LIBRARY=ON`; see `pixi run build-wasm`.
`TINYOPT_BUILD_EXAMPLES`	`OFF`	Build examples.
`TINYOPT_BUILD_TESTS`	`ON`	Build the test suite.
`TINYOPT_BUILD_INSTALL_TESTS`	`OFF`	Add isolated install smoke tests.
`TINYOPT_BUILD_SOPHUS_TEST`	`OFF`	Build tests that depend on Sophus.
`TINYOPT_BUILD_LIEPLUSPLUS_TEST`	`OFF`	Build tests that depend on Lie++.
`TINYOPT_BUILD_BENCHMARKS`	`OFF`	Build benchmark targets.
`TINYOPT_BUILD_CERES`	`OFF`	Build Ceres comparison tests and benchmarks.
`TINYOPT_BUILD_G2O_BENCHMARKS`	`OFF`	Build g2o comparison benchmarks.
`TINYOPT_BUILD_GTSAM_BENCHMARKS`	`OFF`	Build GTSAM comparison benchmarks.
`TINYOPT_BUILD_PACKAGES`	`OFF`	Enable package targets.
`TINYOPT_BUILD_PIP_PACKAGE`	`OFF`	Enable the `pip-install` target.
`TINYOPT_BUILD_DOCS`	`OFF`	Enable the Sphinx and Doxygen documentation targets. Requires the documentation tools.