Bencher

Installation

important

For all Rust integrations, you will need the cargo-codspeed command to build and run your CodSpeed benchmarks

Install the bencher compatibility layer:

cargo add --dev codspeed-bencher-compat --rename bencher

Or directly change your Cargo.toml if you already have bencher installed:

[dev-dependencies]
bencher = { version = "*" }
bencher = { package = "codspeed-bencher-compat", version = "*" }

note

If you prefer, you can also install codspeed-bencher-compat as is and change your imports to use this new crate name.

This will install the codspeed-bencher-compat crate and rename it to bencher in your Cargo.toml. This way, you can keep your existing imports and the compatibility layer will take care of the rest.

tip

Using the compatibility layer won't change the behavior of your existing benchmark suite of the CodSpeed instrumentation environment and the benches will still run it as usual.

Usage

Creating benchmarks

Let's start with the example from the bencher documentation, creating a benchmark suite for 2 simple functions:

benches/example.rs

use bencher::{benchmark_group, benchmark_main, Bencher};

fn a(bench: &mut Bencher) {
    bench.iter(|| {
        (0..1000).fold(0, |x, y| x + y)
    })
}

fn b(bench: &mut Bencher) {
    const N: usize = 1024;
    bench.iter(|| {
        vec![0u8; N]
    });

    bench.bytes = N as u64;
}

benchmark_group!(benches, a, b);
benchmark_main!(benches);

The last step in creating the Bencher benchmark is to add the new benchmark target in your Cargo.toml:

Cargo.toml

[[bench]]
name = "example"
harness = false

And that's it! You can now run your benchmark suite with CodSpeed

Testing the benchmarks locally

Congrats! 🎉 You can now run those benchmark in your CI to get the actual performance measurements 👇.

Running the benchmarks in your CI

To generate performance reports, you need to run the benchmarks in your CI. This allows CodSpeed to detect the CI environment and properly configure the instrumented environment.

tip

If you want more details on how to configure the CodSpeed action, you can check out the Continuous Reporting section.

Here is an example of a GitHub Actions workflow that runs the benchmarks and reports the results to CodSpeed on every push to the main branch and every pull request:

.github/workflows/codspeed.yml

name: CodSpeed

on:
  push:
    branches:
      - "main" # or "master"
  pull_request:
  # `workflow_dispatch` allows CodSpeed to trigger backtest
  # performance analysis in order to generate initial data.
  workflow_dispatch:

jobs:
  benchmarks:
    name: Run benchmarks
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup rust toolchain, cache and cargo-codspeed binary
        uses: moonrepo/setup-rust@v1
        with:
          channel: stable
          cache-target: release
          bins: cargo-codspeed

      - name: Build the benchmark target(s)
        run: cargo codspeed build

      - name: Run the benchmarks
        uses: CodSpeedHQ/action@v3
        with:
          run: cargo codspeed run
          token: ${{ secrets.CODSPEED_TOKEN }}