Benchmark.js
Installationβ
First install the plugin @codspeed/benchmark.js-plugin
and benchmark
(if not already installed):
- npm
- yarn
- pnpm
npm install --save-dev @codspeed/benchmark.js-plugin benchmark
yarn add --dev @codspeed/benchmark.js-plugin benchmark
pnpm add --save-dev @codspeed/benchmark.js-plugin benchmark
Usageβ
Let's create a fibonacci function and benchmark it with benchmark.js and the CodSpeed plugin:
- JavaScript
- TypeScript
import Benchmark from "benchmark";
import { withCodSpeed } from "@codspeed/benchmark.js-plugin";
function fibonacci(n) {
if (n < 2) {
return n;
}
return fibonacci(n - 1) + fibonacci(n - 2);
}
const suite = withCodSpeed(new Benchmark.Suite());
suite
.add("fibonacci10", () => {
fibonacci(10);
})
.add("fibonacci15", () => {
fibonacci(15);
})
.on("cycle", function (event: Benchmark.Event) {
console.log(String(event.target));
})
.run();
Noticed the .mjs
extension? This is because we're using the ESM module format.
Saving our file with the .js
extension would have worked as well, but we would have needed to add "type": "module"
to our package.json
file to instruct Node.js to use the ESM module format.
If you're working with CommonJS modules, you can totally use the require
syntax for importing the libraries.
import Benchmark from "benchmark";
import { withCodSpeed } from "@codspeed/benchmark.js-plugin";
function fibonacci(n: number): number {
if (n < 2) {
return n;
}
return fibonacci(n - 1) + fibonacci(n - 2);
}
const suite = withCodSpeed(new Benchmark.Suite());
suite
.add("fibonacci10", () => {
fibonacci(10);
})
.add("fibonacci15", () => {
fibonacci(15);
})
.on("cycle", function (event: Benchmark.Event) {
console.log(String(event.target));
})
.run();
Here, a few things are happening:
- We create a simple recursive fibonacci function.
- We create a new
Benchmark.Suite
instance with CodSpeed support by using thewithCodSpeed
helper. This step is critical to enable CodSpeed on your benchmarks. - We add two benchmarks to the suite and launch it, benching our
fibonacci
function with 10 and 15.
Now, we can run our benchmarks locally to make sure everything is working as expected:
- JavaScript
- TypeScript
To run the .ts
file directly, we recommend using esbuild-register
. It allows running TypeScript & ESM files directly with Node.js.
- npm
- yarn
- pnpm
npm install --save-dev esbuild-register
yarn add --dev esbuild-register
pnpm add --save-dev esbuild-register
And... Congratsπ, CodSpeed is installed in your benchmarking suite! Locally, CodSpeed will fall back to benchmark.js since the instrumentation is only available in the CI environment for now.
You can now run those benchmarks in your CI to get consistent performance measurements.
Integrating into a bigger project, multiple benchmark filesβ
Often time you will not be writing your benchmarks in a single file. Indeed, it can become quite difficult to maintain a single file with all your benchmarks as your project grows.
You can find the source code for the following example in the examples of the codspeed-node
repository.
There are multiple examples available, for CJS, ESM, JavaScript, and TypeScript.
For these kind of situations, we recommend the following approach. Let's say you have a file structure like this, in a project with TypeScript:
.
βββ bench
β βββ fibo.bench.ts
β βββ foobarbaz.bench.ts
β βββ index.bench.ts
βββ package.json
βββ src
β βββ fibonacci.ts
β βββ foobarbaz.ts
βββ tsconfig.json
- The
src
directory contains the source code of the project. Here we have two files,fibonacci.ts
andfoobarbaz.ts
. - The
bench
directory contains the benchmarks for the project. There is a file for each source file that defines benchmarks for it. - The
bench/index.bench.ts
file is the entry point for the benchmarks. It imports all the other benchmark files and runs them.
import type { WithCodSpeedSuite } from "@codspeed/benchmark.js-plugin";
import { iterativeFibonacci } from "../../src/fibonacci";
export function registerFiboBenchmarks(suite: WithCodSpeedSuite) {
suite
.add("test_iterative_fibo_10", () => {
iterativeFibonacci(10);
})
.add("test_iterative_fibo_100", () => {
iterativeFibonacci(100);
});
}
Here we define a function that takes an instance of Bench
as a parameter and then adds some benchmarks to it. This will allow us to add benchmarks to the same suite from multiple files.
import { withCodSpeed } from "@codspeed/benchmark.js-plugin";
import Benchmark from "benchmark";
import { registerFiboBenchmarks } from "./fibo.bench";
import { registerFoobarbazBenchmarks } from "./foobarbaz.bench";
export const suite = withCodSpeed(new Benchmark.Suite());
(async () => {
registerFiboBenchmarks(suite);
registerFoobarbazBenchmarks(suite);
suite.on("cycle", function (event: Benchmark.Event) {
console.log(String(event.target));
});
await suite.run({ async: true });
})();
Here all the functions registering benchmarks are executed to import all the benchmarks from the different files.
To run the benchmarks, use the following command:
node -r esbuild-register bench/index.bench.ts
Check out the full for this example: with-typescript-cjs
in the codspeed-node
repository.
Running the benchmarks in your CIβ
To generate performance reports, you need to run the benchmarks in your CI. This allows CodSpeed to detect the CI environment and properly configure the instrumented environment.
If you want more details on how to configure the CodSpeed action, you can check out the Continuous Reporting section.
Here is an example of a GitHub Actions workflow that runs the benchmarks and reports the
results to CodSpeed on every push to the main
branch and every pull request:
name: codspeed-benchmarks
on:
push:
branches:
- "main" # or "master"
pull_request:
# `workflow_dispatch` allows CodSpeed to trigger backtest
# performance analysis in order to generate initial data.
workflow_dispatch:
jobs:
benchmarks:
runs-on: ubuntu-latest
steps:
- uses: "actions/checkout@v3"
- uses: "actions/setup-node@v3"
- name: Install dependencies
run: npm install
- name: Run benchmarks
uses: CodSpeedHQ/action@v2
with:
run: node benches/bench.mjs
token: ${{ secrets.CODSPEED_TOKEN }}