API

API Documentation

For C callers, including dynamic and generated fixed-size float/double entrypoints, see the C API guide. For installing Tinyopt and linking either the C++ headers or C library from CMake, see Installation and Usage.

Basic Concepts

In Tinyopt, you define two things

• The parameters x

• The Cost, Residuals or Accumulation functions

Parameters

As of today, you can only pass on one parameter x to the Optimizer. There is a trick that allows you to pack all parameters together. See here. They can be float, double or a struct/class of your preference as long as a trait is defined or the struct/class has expected members, see the Supported Types.

Cost, Residuals and Accumulation functions

Depending on what you’re trying to minimize, you have to provide different types of functions.

Cost functions (for general, unconstrained optimizations)

This is the most general function type, it takes one argument, the parameters and return a Scalar value: the cost that will be minimized. Automatic differentiation will be called in order to compute the jacobians/gradients.

Example: double(const T &x); with T being one of the supported types.

Residuals functions (for NLLS optimizations)

A Residuals function only takes a single input argument (the parameters) and must returns a Vector/Matrix or single scalar: the residuals. They are used for NLLS problems only. Automatic differentiation will be called in order to compute the jacobians/gradients.

Example: VecX(const T &x);.

Accumulation Function

The above two cost and residuals functions are converted internally to an Accumulation function with the automatic differentiation. But this step takes time and memory so tinyopt gives you a way to directly accumulate the residuals and update the gradient (and Hessian), this is what the Accumulation functions are.

An Accumulation function takes 2 to 3 input arguments:

• the parameters x,

• the gradient g,

• the Hessian H, optionally if you want to use (pseudo-)second order solvers. It must returns a single scalar, the total cost, or a Cost struct in which you can store extra info such as number of residuals, inliers ration, etc.

You are expected to accumulate all your residuals and update the gradient (and Hessian) in that Accumulation function, manually, by yourself, like a grown up!

This is the FASTEST way to optimize something so if execution time matters, use this one!

Example: Cost(const T &x, Vec3 &gradient, Mat3 &hessian);. The Gradient type must be a Eigen::Vector, The Hessian Type must be either a dense or sparse Eigen Matrix.

Full Documentation

Dive into the glorious depths of our documentation on ReadTheDocs. It’s packed with all the juicy details, and maybe a few hidden jokes if you look hard enough.

For configuration flags and their defaults, see CMake Options.

Simple API

tinyopt is inspired by the simple syntax of python so it is very developer friendly*, just call Optimize and give it something to optimize, say x and something to minimize.

Optimize performs automatic differentiation so you just have to specify the residual(s), no jacodians/derivatives to calculate because you know the pain, right? No pain, vanished, thank you Julien.

* but not compiler friendly, sorry gcc/clang but you’ll have to work double because it’s all templated.

Example: What’s the square root of 2?

Beause using std::sqrt is over hyped, let’s try to recover it using tinyopt, here is how to do:

// Define 'x', the parameter to optimize, initialized to '1' (yeah, who doesn't like 1?)
double x = 1;
Optimize(x, [](const auto &x) { return x * x - 2.0; }); // Let's minimize ε = x*x - 2
// 'x' is now √2, amazing.

That’s it. Is it too verbose? Well remove the comments then. Come on, it’s just two lines, I can’t do better.

Selecting an optimizer

Optimize() defaults to Levenberg-Marquardt. Select Powell’s DogLeg or nonlinear conjugate gradient with Options::Solver::DogLeg or Options::Solver::ConjugateGradient; their global dispatch is enabled with TINYOPT_ENABLE_DOGLEG and TINYOPT_ENABLE_CONJUGATE_GRADIENT, respectively. These CMake switches do not disable direct use of the algorithm headers:

Options options;
options.dl.radius_init = 1.0f;
const auto sum = dl::Optimizer<Mat2>(options)(x, residuals);

See CMake Options for the default values and configuration details.

BFGS and L-BFGS

BFGS and L-BFGS share the bfgs.h API but use different solvers. BFGS stores a dense inverse Hessian and requires quadratic memory in the parameter count. L-BFGS stores a bounded history of step/gradient pairs and uses the two-loop recursion, so its memory is linear in the parameter count and history length. Their global Optimize() selectors are independently enabled by TINYOPT_ENABLE_BFGS and TINYOPT_ENABLE_LBFGS; direct optimizer headers remain usable either way.

Both option groups expose initial step_size, multiplicative step_reduction after rejected steps, bounded step_growth after accepted steps, and a curvature_threshold for skipping unstable updates. L-BFGS additionally accepts history_size (default 8; its compile-time capacity can be selected on lbfgs::Optimizer when needed).

Options options;
options.lbfgs.history_size = 8;
const auto sum = lbfgs::Optimizer<VecX>(options)(x, objective);

Example: Fitting a circle to a set of points

In this use case, you’re given n 2D points your job is to fit a circle to them. Today is your lucky day, tinyopt is here to help! Let’s see how.

Mat2Xf obs(2, n); // fill the observations (n 2D points)

// loss is the sum of || ||p - center||² - radius² ||
auto loss = [&obs]<typename Derived>(const MatrixBase<Derived> &x) {
  using T = typename Derived::Scalar;
  const auto &center = x.template head<2>(); // the first two elements are the cicle position
  const auto radius2 = x.z() * x.z(); // the last one is its radius, taking the square to avoid a sqrt later on
  // Here we compute the squared distances of each point to the center
  auto residuals = (obs.cast<T>().colwise() - center).colwise().squaredNorm();
  // Here we compute the difference of of squared distances are the circle's squared radius
  return (residuals.array() - radius2).eval();
  // Make sure the returned type is a scalar or Eigen::Matrix (thus the .eval())
};


// Define 'x', the parameter to optimize, using the following parametrization: x = {center (x, y), radius}
Vec3 x(0, 0, 1);
Optimize(x, loss); // Optimize!
// 'x' should have converged to a reasonable cicle
std::cout << "Residuals: " << loss(x) << "\n"; // Let's print the final residuals

Ok so this is quite more verbose than in the first example but I’m trying to help you understand the syntax, it’s easy no?

Supported Types

With tinyopt, you can directly optimize several types of parameters x, namely

• float or double scalar types

• std:array of scalars or another type

• std::vector of a scalars or another type

• Eigen::Vector of fixed or dynamic size

• Eigen::Matrix of fixed or dynamic size

• Your custom class or struct, see User defined parameters

You can also use any levels of nesting for known types, e.g. std::array<Vec2f, 2> or std::pair<std::vector<Vec3>, VecX>.

tinyopt will detect whether the size is known at compile time and use this information to make the optimization faster.

Residuals of the following types can also be returned

• float or double (or the typename T, typically a Jet<S,N>)

• Eigen::Vector or Eigen::Matrix

Advanced API

Direct Accumulation, a faster - but manual - way.

When working with more than one residual, tinyopt lets you avoid storing a full residual vector. This is the best option when you already know the Jacobian and want the most efficient accumulation path. The manual accumulation style is usually much faster than using a residual-vector callback because it avoids temporary buffers and directly updates the gradient and Hessian in place.

For example, when the objective is a nonlinear least-squares problem with residuals $r(x)$ and Jacobian $J$, then the gradient is $J^T r$ and the Hessian approximation is $J^T J$. You can accumulate those directly:

// Define 'x', the parameter to optimize, initialized to '1'
double x = 1;

// Define the residuals / loss function, here ε² = ||x*x - 2||²
auto loss = [](const auto &x, auto &grad, auto &H) {
  float res = x * x - 2; // since we want x to be sqrt(2), x*x should be 2
  float J   = 2 * x; // residual's jacobian/derivative w.r.t x
  // Manually update the Hessian H ~= Jt * J and Gradient = Jt * residuals
  if constexpr (!traits::is_nullptr_v<decltype(grad)>) { // Gradient requested?
    H(0, 0) = J * J;   // normal matrix (initialized to 0s before so only update what is needed)
    grad(0) = J * res; // gradient (half of it actually)
  }
  // Return both the norm and the number of residuals (here, we have only one)
  return Cost(std::sqrt(res*res), 1);
};

// Setup optimizer options (optional)
Options options;
options.linear_solver = LinearSolverMethod::QR;
options.opt.grad_clipping = 10.0f;
options.stop.max_iters = 100;
// Optimize!
const auto &sum = Optimize(x, loss, options);
// 'x' is now std::sqrt(2.0), you can check the convergence with sum.Converged()

Following the iterates

options.stop.step_callback is called after every update of the parameters with the step dx that was added to them and is_rollback, which is true when a rejected step is undone (dx is then the negated step). Summing the dx of all calls therefore tracks the current parameters, so the optimizer path can be recorded or plotted. Return true to stop the optimization (StopReason::kUserStopped).

Vec2 x(-1.2, 1.0), path = x;
std::vector<Vec2> trajectory{x};
Options options;
options.stop.step_callback = [&](const VecXf &dx, bool is_rollback) {
  path += dx.cast<double>();
  if (is_rollback) trajectory.pop_back();  // the rejected point is dropped
  else trajectory.push_back(path);
  return false;  // keep going
};
Optimize(x, residuals, options);

For second order solvers, H and grad are the only things you need to update for LM to solve the normal equations and optimize x. It looks a bit rustic I know but we can’t all live in a fancy city with sleek buidlings, sometimes it’s good to go back to your (square) roots, so take your boots and start coding.

NOTE that you only need to fill the upper triangle part only since H is assumed to be symmetric. The default linear solver is LinearSolverMethod::LDLT. Solver implementations are controlled at configure time: TINYOPT_ENABLE_LINEAR_SOLVER_LDLT is ON by default; TINYOPT_ENABLE_LINEAR_SOLVER_LLT, TINYOPT_ENABLE_LINEAR_SOLVER_LU, TINYOPT_ENABLE_LINEAR_SOLVER_QR, and TINYOPT_ENABLE_LINEAR_SOLVER_SVD (all OFF by default). LU and QR select Eigen’s dense or sparse decomposition based on Hessian storage. SVD is dense-only and requires TINYOPT_ENABLE_LINEAR_SOLVER_SVD=ON. With the default threshold of 0, it uses Eigen’s default cutoff; set a positive svd_relative_threshold to compute a pseudoinverse step, treating singular values at or below svd_relative_threshold * largest_singular_value as zero:

Options options;
options.linear_solver = LinearSolverMethod::SVD;
options.svd_relative_threshold = 1e-8;

Since Tinyopt solves the accumulated normal equations, the cutoff is applied to the Hessian approximation J.transpose() * J, not directly to the Jacobian.

The optional SuiteSparse CHOLMOD backend solves sparse positive-definite systems. Enable it with -DTINYOPT_ENABLE_SUITESPARSE=ON and install SuiteSparse. Compressed Eigen sparse Hessian arrays are passed directly to CHOLMOD; Tinyopt compresses the optimizer’s Hessian in place before solving. Uncompressed matrices passed directly to SolveLinearSystem are exposed as CHOLMOD unpacked CSC views, so the sparse matrix is not copied or modified. Select it with options.linear_solver = LinearSolverMethod::SuiteSparse:

Options options;
options.linear_solver = LinearSolverMethod::SuiteSparse;

Pixi provides pixi run test-suitesparse for the dedicated solver test and pixi run bench-suitesparse for the sparse benchmark.

SuiteSparse components have separate licenses. CHOLMOD’s Cholesky module is LGPL-2.1-or-later, while its optional Supernodal module is GPL-2.0-or-later. Review the upstream SuiteSparse license notices and the modules enabled in your build when assessing distribution requirements.

Checking Gradients

Manually filling the gradient and eventually the Hessian (approximation as Jt*J) can be a shaky process because you know, your math might not be as sharp as you would like! So tinyopt provides a function that checks them for you, how nice! It does not check your second derivatives, sorry, only if H is Jt*J.

The relevant functions are CheckGradient for general optimizations and CheckResidualsGradient for non-linear least squares. You can do this type of checks for NLLS:

// First, define your residuals (for NLLS)
auto residuals = [&](const auto &x, auto &grad, auto &H) {
  float res = x * x - 2;  // since we want x to be sqrt(2), x*x should be 2
  float J = 2 * x;        // residual's jacobian/derivative w.r.t x
  // Manually update the hessian and gradient
  if constexpr (!traits::is_nullptr_v<decltype(grad)>) {
    grad(0) = J * res;
    H(0, 0) = J * J;
  }
  return res; // Returns the residual
};
// Let's check the gradient and Hessian approx.
float x = 1;
REQUIRE(diff::CheckResidualsGradient(x, residuals)); // well, let's check my math!

// Ok, let's optimize then!
auto loss = [&](const auto &x, auto &grad, auto &H) {
  const auto r = residuals(x, grad, H);
  return std::abs(r); // or r²
};
Optimize(x, loss);

Note that

• we use numerical differentiation so that you don’t need all the castings, but then you may need to adjust the epsilon, e.g. CheckResidualsGradient(x, res, 1e-3).

• you cannot check the loss function directly for NLLS, you can only check the residuals’ gradients.

For general optimization it’s simpler:

auto loss = [&](const auto &x, auto &grad) {
  double y = x - 42.0;
  double cost = 3 * y * y + std::pow(y, 4.0) - 2.0;
  if constexpr (!traits::is_nullptr_v<decltype(grad)>)
    grad(0) = 2 * 3 * y + 4 * std::pow(y, 3.0);
  return cost;
};

double x = 40.1;
REQUIRE(diff::CheckGradient(x, loss, 1e-3)); // well, let's check my math!
Optimize(x, loss); // ok, let's optimize

Sparse Systems

Ok so you have quite large and sparse systems? Just tell Tinyopt about it by simply defining you loss to have a SparseMat &H type instead of a Dense Matrix or auto.

NOTE that automatic differentation is not supported for sparse Hessian matrices but is for first order solvers. In that case, simply use this loss signature auto loss = []<typename T>(const auto &x, SparseMatrix<T> &gradient).

auto loss = [](const auto &x, auto &grad, SparseMat &H) {
  // Define your residuals
  const VecX res = 10 * x.array() - 2; // the residuals
  // Update the full gradient matrix, using the Jacobian J of the residuals w.r.t 'x'
  MatX J = Matx::Zero(res.rows(), x.size());
  for (int i = 0; i < x.size(); ++i) J(i, i) = 10;
  grad = J.transpose() * res;
  // Update the (approx. or real) Hessian
  std::vector<Eigen::Triplet<double>> triplets;
  triplets.reserve(x.size());
  for (int i = 0; i < x.size(); ++i) triplets.emplace_back(i, i, 10 * 10);
  H.setFromTriplets(triplets.begin(), triplets.end());
  // Returns the norm + number of residuals
  return Cost(res.squaredNorm(), res.size()); // you can also return Cost(res), we'll take it's squared L2 norm!
};

As an alternative, you can use the gn::Optimizer<SparseMatX> class instead of the Optimize function.

There are many ways to fill H in Eigen, have a look at tests/cpp/sparse.cpp for some examples.

User defined parameters

Let’s say you have a fancy class/struct, say a Rectangle and you want to optimize its dimensions. For instance,

template <typename T> // Template is only needed if you need automatic differentiation
struct Rectangle {
  T area() const { return (p2 - p1).norm(); }
  T width() const { return p2.x() - p1.x(); }
  T height() const { return p2.y() - p1.y(); }
  Eigen::Vector<T, 2> center() const { return T(0.5) * (p1 + p2); }
  Eigen::Vector<T, 2> p1, p2; // top left and bottom right positions
};

Now if you wan to simply call Optimize(rectangle, loss) on your rectangle struct, you need to either add specific members and methods or use a trait specialization of params_trait for you object type:

Dims is the compile-time dimension of the local update vector (the tangent or manifold dimension), and dims(value) is its runtime counterpart when that size is dynamic. For ordinary Euclidean parameters this equals the number of stored scalar coordinates. For manifold-valued parameters it is the number of degrees of freedom in a local update, not the ambient storage dimension; Tinyopt does not require a separate ambient-dimension declaration because the parameter type already carries its storage shape. PlusEq receives a delta with this local dimension.

namespace tinyopt::traits { // must be defined in tinyopt::traits

template <typename T>
struct params_trait<Rectangle<T>> {
  using Scalar = T;              // The scalar type
  static constexpr Index Dims = 4; // Tangent dimension (use Eigen::Dynamic if unknown)
  // Runtime tangent dimension [OPTIONAL, if Dims is known)
  static int dims(const Rectangle<T> &) { return Dims; }

  // Convert a Rectangle to another type 'T2', e.g. T2 = Jet<T> [OPTIONAL, if no Jet]
  // Not needed if you use manual Jacobians instead of automatic differentiation
  template <typename T2> static Rectangle<T2> cast(const Rectangle<T> &rect) {
    return Rectangle<T2>(rect.p1.template cast<T2>(),
                         rect.p2.template cast<T2>());
  }

  // Define how to update the object members (parametrization and manifold)
  static void PlusEq(Rectangle<T> &rect,
                     const Eigen::Vector<Scalar, Dims> &delta) {
    // In this case delta is defined as 2 deltas for p1 and p2: [dx1, dy1, dx2, dy2]
    rect.p1 += delta.template head<2>();
    rect.p2 += delta.template tail<2>();
  }
};

} // namespace tinyopt::traits

You can also skip the trait all together if you add these members and methods directly to the class. Now you can start optimizing your custom object, e.g.

// Let's say I want the rectangle area to be 10*20, the width = 2 * height and
// the center at (1, 2).
auto loss = [&]<typename T>(const Rectangle<T> &rect) {
  using std::max;
  Eigen::Vector<T, 4> residuals;
  residuals[0] = rect.area() - 10.0f * 20.0f;
  residuals[1] = 100.0f * (rect.width() / max(rect.height(), T(1e-8f)) -
                            2.0f); // the 1e-8 is to prevent division by 0
  residuals.template tail<2>() = rect.center() - Eigen::Vector<T, 2>(1, 2);
  return residuals;
};

Rectangle rectangle;
Optimize(rectangle, loss);
// That's it, rectangle is now fitted to your loss

How to avoid data copies?

ParamsWrapper adapts existing parameter storage so Tinyopt can update it in place. Store a reference to avoid copying the original parameters. The wrapped type must provide a Scalar, a params_trait implementation, and an operator+= that applies the tangent update (including any manifold logic). Its existing params_trait is also used to report the dimensions and cast the parameters for automatic differentiation.

#include <tinyopt/params_wrapper.h>

MyParameters parameters;
ParamsWrapper<MyParameters&> x(parameters);
Optimize(x, loss);

ParamsWrapper<MyParameters> owns its parameter value, while ParamsWrapper<MyParameters&> keeps the original object by reference. Automatic differentiation still makes a casted copy of the parameters when evaluating derivatives.

Numerical Differentiation

Not all cost functions are the same. By default, tinyopt will try to use automatic differentiation when the function has only one parameter x but if your function does not allow it, you can use a numerical differentiation one. Here is an example

auto original_loss = [&](const auto &x) -> Vec3 { return 2 * (x - y_prior); };
auto new_loss = CreateNumDiffFunc1(x, original_loss);
// you can now pass this 'new_loss' to an optimizer, e.g. Optimize(x, new_loss);

NOTE CreateNumDiffFunc1 is when using first order optimizers which use the gradient only and CreateNumDiffFunc2 for second or pseudo-second order methods, which use both gradient and Hessian.

Losses, Norms and Robust Norms

You can play with different losses, activation functions, robust norms and M-estimators, have a look at the losses folder.

All losses and other norms follow the signature: Name(x, export_or_jacobian) or Name(x, threshold, export_or_jacobian) for robust norms.

export_or_jacobian is either

• nullptr or skipped and the loss/norm will be the only output of the function.

• true, the Jacobian of the loss will be returned as well as the loss.

• a matrix/vector representing a forward Jacobian, then the second returned value will be the transformed Jacobian using the chain rule.

Here is an example of how to get a Huber norm.

double robust_norm2 = Huber(y.squaredNorm(), 0.8);

Here is another example of how to call a Huber norm and recover the scale/Jacobian of it or the transformed jacobian.

const auto &[robust_norm2, J] = Huber(y.squaredNorm(), 0.8, true);

or

const auto &[robust_norm2, J] = Huber(y.squaredNorm(), 0.8, Jy);

Robust losses can also be embedded directly in residual tuples. This is useful when a problem mixes a regular residual term with a robustified term, for example:

[&](const auto &xj) {
  const auto plain = xj - target;
  return Residuals(plain, losses::Huber(plain, 10.0));
}

The robust wrapper is unwrapped during automatic differentiation and contributes a weighted $J^T r$ and $J^T J$ update using the robust scale, so the same accumulation pattern works for both plain and robust residual blocks.

There is also various other norms, activation functions and losses. Here is an example of a loss that uses a Mahalanobis distance with a covariance C.

auto loss = [&]<typename T>(const Eigen::Vector<T, 2> &x) {
  const Matrix<T, 2, 2> C_ = C.template cast<T>();
  return MahaSquaredNorm(x - y, C_);  // return res.T * C.inv() * res
};

Tips & Tricks

Fixed-size and embedded systems

For fixed-size parameter types, enable strict allocation mode when configuring Tinyopt:

cmake -S . -B build -DTINYOPT_ENFORCE_NO_DYNAMIC_ALLOCATIONS=ON

In this mode Tinyopt uses fixed-capacity five-sample output histories, does not save the final Hessian, disables optimizer logging, and enables Eigen’s runtime no-malloc guard during optimization. This is intended for fixed-size workloads with strict memory requirements. Dynamic parameter types are not covered.

Speeding up multi-file builds

If several translation units use Optimize with the same parameter and residual-function types, explicit template instantiation can avoid compiling that specialization repeatedly. Declare the function and its pointer type, declare the specialization with extern template in a shared header, and define both in one .cpp file:

// quadratic.h
#include <tinyopt/tinyopt.h>

using ResidualFunction = tinyopt::Vector<double, 1> (*)(const double &);
tinyopt::Vector<double, 1> Residuals(const double &parameter);

extern template tinyopt::Summary tinyopt::Optimize<double, ResidualFunction>(
        double &parameter, const ResidualFunction &residuals, const tinyopt::Options &);

// quadratic.cpp
#include "quadratic.h"

tinyopt::Vector<double, 1> Residuals(const double &parameter) {
    return tinyopt::Vector<double, 1>(parameter - 2.0);
}

template tinyopt::Summary tinyopt::Optimize<double, ResidualFunction>(
        double &parameter, const ResidualFunction &residuals, const tinyopt::Options &);

Other translation units include quadratic.h, bind &Residuals to a ResidualFunction, and call tinyopt::Optimize(parameter, residuals, options); link them with quadratic.cpp. This concrete function-pointer signature uses numerical differentiation because it cannot accept Tinyopt Jet types. Explicit instantiation only avoids repeated instantiation of that exact specialization; each translation unit still parses Tinyopt’s headers, and different function-pointer types need different instantiations. A multi-translation-unit test exercises this pattern in the project test suite.

C API

Tinyopt provides a C ABI in the optional tinyopt_c library. Enable it with -DTINYOPT_BUILD_C_LIBRARY=ON; it is off by default. Include <tinyopt/c/c_api.h> for the enabled APIs, or include only <tinyopt/c/c_api_float.h> / <tinyopt/c/c_api_double.h> when a consumer needs one precision. The float entrypoints are built by default; configure with -DTINYOPT_C_API_FLOAT=OFF to omit them. For install and CMake consumer examples, see Installation and Usage.

Callbacks return 0 to continue. A nonzero result stops optimization with TINYOPT_STATUS_USER_STOPPED. User data and input parameter arrays remain caller-owned and must stay valid until return. Tinyopt optimizes a private parameter copy and copies it back on success or requested stop.

Select a tinyopt_problem_t mode:

Mode	Callback	Behavior
TINYOPT_EVAL_COST_ONLY	fn.cost	Scalar objective; Tinyopt estimates its gradient numerically. With default LM options, C cost-only mode selects BFGS.
TINYOPT_EVAL_RESIDUALS	fn.residuals	Residuals with an optional row-major Jacobian. Set *jacobian = NULL in the callback to request numerical differentiation.
TINYOPT_EVAL_GRADIENT	fn.acc_grad	Manually accumulate a scalar objective and gradient; select a first-order solver.
TINYOPT_EVAL_HESSIAN	fn.acc_hessian	Manually accumulate a scalar objective, gradient, and Hessian.

Tinyopt zeros gradient and Hessian buffers before accumulation callbacks. Either output pointer can be NULL when the selected solver does not request it. acc_grad and acc_hessian are the accumulation callback types; there are no separate acc1 or acc2 modes.

Dynamic Parameters

Dynamic parameters include their dimension and may provide a plus_eq callback to apply a parameter-space step. Set plus_eq to NULL for ordinary component-wise addition; provide a custom callback for manifold or otherwise non-Euclidean updates. Residual callbacks fill every residual entry on each invocation. The jacobian argument is a pointer to Tinyopt’s row-major Jacobian buffer. Fill it when *jacobian is non-NULL; set *jacobian = NULL to ask Tinyopt to estimate the Jacobian with finite differences. The callback may also receive a null *jacobian when only residuals are being evaluated, so check it before writing.

#include <stdio.h>
#include <tinyopt/c/c_api.h>

static void plus_eq(double *x, double *dx) {
  x[0] += dx[0];
}

static int evaluate_residuals(const double *x, int dims, double *out, double **jacobian,
                              int residual_dims, void *user_data) {
  const double target = *(const double *)user_data;
  if (dims != 1 || residual_dims != 1) return 1;
  out[0] = x[0] - target;
  if (*jacobian != NULL) (*jacobian)[0] = 1.0;
  return 0;
}

int main(void) {
  double x[] = {-3.0};
  double target = 0.75;
  tinyopt_params_t params = {x, 1, plus_eq};
  tinyopt_problem_t problem = {0};
  problem.type = TINYOPT_EVAL_RESIDUALS;
  problem.fn.residuals = evaluate_residuals;
  problem.num_residuals = 1;
  problem.user_data = &target;
  tinyopt_options_t options;
  if (tinyopt_options_default(&options) != TINYOPT_STATUS_OK) return 1;
  tinyopt_summary_t summary;

  if (tinyopt_optimize(&params, &problem, &options, &summary) != TINYOPT_STATUS_OK)
    return 1;
  printf("x = %.6f, cost = %.3g\n", x[0], summary.final_cost);
  return 0;
}

For a scalar objective, set type to TINYOPT_EVAL_COST_ONLY and provide fn.cost. The callback receives a double *cost output. With default LM options the C API selects BFGS for this scalar objective mode; an explicitly selected first-order solver is also respected.

For manually supplied derivatives, use TINYOPT_EVAL_GRADIENT with fn.acc_grad or TINYOPT_EVAL_HESSIAN with fn.acc_hessian. These callbacks accumulate the scalar objective’s gradient and, for Hessian mode, its Hessian. The gradient buffer is a vector; the Hessian buffer is column-major. Tinyopt initializes requested buffers to zero before each callback, so write contributions with += or assign the complete result.

Pass NULL as the options pointer to use Tinyopt’s default C++ Options values. The options initializer returns those defaults, after which any field can be overridden:

tinyopt_options_t options;
tinyopt_options_default(&options);
options.max_iters = 100;
options.lm_damping_init = 1e-3f;
options.log_enabled = 0;

tinyopt_options_t mirrors the value fields in tinyopt::Options: solver and linear solver selection, optimization and Hessian settings, cost scaling, stop criteria, logging, and all solver-specific settings. The two stop callbacks are also available as C function pointers. The scalar callback receives the current error, squared step norm, and squared gradient norm. Each callback receives its own user-data pointer. The vector callback receives float x and dx arrays with an explicit dimension, matching Tinyopt’s existing float callback contract.

step_callback (tinyopt_step_callback_t) reports every parameter update, which makes it possible to record the optimizer path. It receives the float step dx added to the parameters, its dimension, and is_rollback, nonzero when a rejected step is undone (dx is then the negated step). Summing the dx of all calls tracks the current parameters. Return nonzero to stop with TINYOPT_STATUS_USER_STOPPED.

static int on_step(const float *dx, int dims, int is_rollback, void *user_data) {
  double *x = (double *)user_data;  /* starts at the initial parameters */
  for (int i = 0; i < dims; ++i) x[i] += dx[i];  /* a rollback already carries the negated step */
  (void)is_rollback;
  return 0;
}
/* options.step_callback = on_step; options.step_callback_user_data = x_tracked; */

Sparse Hessians (SuiteSparse)

When Tinyopt is configured with -DTINYOPT_ENABLE_SUITESPARSE=ON together with the C library, <tinyopt/c/c_api_sparse.h> (or c_api_sparse_double.h / c_api_sparse_float.h) exposes tinyopt_optimize_sparse() and tinyopt_optimize_sparsef(). Parameters are the usual dynamic tinyopt_params_t / tinyopt_paramsf_t; the accumulation callback provides the cost, the gradient and a dynamic-size sparse Hessian as a SuiteSparse cholmod_sparse matrix. Systems are solved with CHOLMOD, using Levenberg-Marquardt by default or Gauss-Newton when selected in the options.

The callback receives a cholmod_common * to allocate with. When hessian is not NULL, set *hessian to a newly allocated dims x dims, real, int-indexed cholmod_sparse of the matching precision (CHOLMOD_DOUBLE or CHOLMOD_SINGLE); Tinyopt takes ownership and frees it. stype may be 1 (upper triangle), -1 (lower triangle) or 0 (symmetric, upper triangle is read). The gradient buffer is zeroed beforehand; hessian and gradient are NULL when only the cost is requested.

static int accumulate(const double *x, int dims, double *cost, double *gradient,
                      cholmod_sparse **hessian, cholmod_common *common, void *user_data) {
  *cost = 0.5 * x[0] * x[0];
  if (gradient != NULL) gradient[0] = x[0];
  if (hessian != NULL) {
    cholmod_triplet *t = cholmod_allocate_triplet(1, 1, 1, 1, CHOLMOD_REAL + CHOLMOD_DOUBLE, common);
    ((int *)t->i)[0] = ((int *)t->j)[0] = 0;
    ((double *)t->x)[0] = 1.0;
    t->nnz = 1;
    *hessian = cholmod_triplet_to_sparse(t, 1, common);
    cholmod_free_triplet(&t, common);
  }
  return 0;
}

double x[] = {3.0};
tinyopt_params_t params = {x, 1, NULL};
tinyopt_sparse_problem_t problem = {accumulate, NULL};
tinyopt_optimize_sparse(&params, &problem, NULL, NULL);

Only the Levenberg-Marquardt and Gauss-Newton solvers are supported; other solvers return TINYOPT_STATUS_INVALID_ARGUMENT. See examples/c/sparse_hessian.c and tests/c/test_api_sparse.c.

Fixed-Size Parameters

CMake generates fixed-size declarations, one C wrapper source per dimension and precision, and tests for every configured combination. The default dimensions are 1, 2, 3, 4, 5, 6, 10, 12; configure a subset with -DTINYOPT_C_FIXED_SIZES="2;3;6". Fixed-size parameter structs omit dims, and their optional plus_eq member has the same component-wise fallback when set to NULL:

#include <tinyopt/c/c_api.h>

static void plus_eq(float *x, float *dx) {
  for (int i = 0; i < 3; ++i) x[i] += dx[i];
}

static int evaluate_residuals(const float *x, int dims, float *out, float **jacobian,
                              int residual_dims, void *user_data) {
  const float *target = (const float *)user_data;
  if (dims != 3 || residual_dims != 3) return 1;
  for (int i = 0; i < 3; ++i) {
    out[i] = x[i] - target[i];
    if (*jacobian != NULL) {
      for (int j = 0; j < 3; ++j) (*jacobian)[i * 3 + j] = i == j ? 1.0f : 0.0f;
    }
  }
  return 0;
}

int main(void) {
  float x[3] = {2.0f, 3.0f, 4.0f};
  const float target[3] = {1.0f, 1.0f, 1.0f};
  tinyopt_params3f_t params = {x, plus_eq};
  tinyopt_problemf_t problem = {0};
  problem.type = TINYOPT_EVAL_RESIDUALS;
  problem.fn.residuals = evaluate_residuals;
  problem.num_residuals = 3;
  problem.user_data = (void *)target;
  return tinyopt_optimize3f(&params, &problem, NULL, NULL) != TINYOPT_STATUS_OK;
}

All C typedef names end in _t. Double is the default and has no precision suffix, for example tinyopt_params_t and tinyopt_optimize, or tinyopt_params3_t and tinyopt_optimize3 for fixed size. Float names keep their f precision suffix, for example tinyopt_paramsf_t and tinyopt_optimizef, or tinyopt_params3f_t and tinyopt_optimize3f for fixed size. Descriptors are passed by pointer. Options and summary may be NULL; parameter and residual descriptors must not be. The corresponding generated declarations are available from the umbrella header or the precision-only header. Each generated C wrapper is a separate translation unit so parallel build tools can compile them independently.

WebAssembly

The C API can be compiled to WebAssembly with Emscripten and called from JavaScript, see examples/wasm and the step-by-step walkthrough in wasm_tutorial. In the wasm pixi environment, pixi run build-wasm cross-compiles tinyopt_c (static, double precision, 2D fixed size) in build-wasm/ with -DTINYOPT_BUILD_WASM=ON and links it into tinyopt.mjs / tinyopt.wasm, exporting tinyopt_optimize, tinyopt_options_default, malloc and free. JavaScript fills the C structs in the module memory (wasm32 layout) and passes functions as callbacks with addFunction. C++ exceptions are enabled (-fexceptions) because the C API reports callback stops with them.

const m = await createTinyopt();
// ... write tinyopt_params_t / tinyopt_problem_t / tinyopt_options_t, set
// options.step_callback = m.addFunction((dx, dims, isRollback, user) => 0, 'iiiii');
const status = m._tinyopt_optimize(params, problem, options, summary);

Current Boundaries

The C ABI does not expose C++ templated autodiff. Fixed-size entrypoints compile with TINYOPT_ENFORCE_NO_DYNAMIC_ALLOCATIONS and Eigen’s runtime no-malloc guard; C API adapter scratch is allocated before optimization begins. The guard cannot prevent allocations made by user callbacks. Dynamic-size solver/workspace storage is runtime-sized, so dynamic parameters cannot promise zero allocation. A fixed dimension must be present in TINYOPT_C_FIXED_SIZES; dynamic entrypoints accept any positive dimension. C++ exceptions must not cross the C callback boundary.

C++ API Reference

The generated reference for the public C++ headers is available in the Doxygen API reference.