Skip to main content

Getting Started

CodSpeed is a performance testing tool helping you to consistently measure the performance of your codebase locally and in your CI/CD pipeline. More details on how CodSpeed works here: What is CodSpeed?

This guide is a quick start to get a sample Python repository on GitHub up and running with CodSpeed:

  1. Create Performance Tests
  2. Connect your Repository
  3. Run the Tests in your CI
  4. Introduce a Performance Regression

Create Performance Tests​

  1. Install the CodSpeed plugin for pytest:

    pip install pytest-codspeed

  2. Write a performance test using the @pytest.mark.benchmark marker:

    tests/test_count_even.py
    import pytest

    def count_even(arr: list[int]) -> int:
        """Count the number of even numbers in an array."""
        even = 0
        for x in arr:
            if x % 2 == 0:
                even += 1
        return even

    # Your tests can also be benchmarks
    @pytest.mark.benchmark
    def test_count_even():
        assert count_even(range(1000)) == 500

  3. Run your performance tests locally:

    Great! We set up our first performance test. Now, let's track the performance in the CI with CodSpeed!

Connect your Repository​

  1. Login on CodSpeed

  2. Go to settings and install the CodSpeed GitHub App by clicking on the "Import" button.

  3. Select the organization or the user and add the repositories you want to use with CodSpeed:

  4. Back in CodSpeed settings, enable your repositories by clicking on the "Enable" button:

  5. Copy your CODSPEED_TOKEN and add it to your GitHub repository secrets

Run the Tests in your CI​

  1. Create a new GitHub Actions workflow file to run your benchmarks:

    .github/workflows/codspeed.yml
    name: CodSpeed

    on:
      push:
        branches:
          - main # Run on pushes to the main branch
      pull_request: # Run on all pull requests

    jobs:
      codspeed:
        name: Run benchmarks
        runs-on: ubuntu-latest
        steps:
          - uses: actions/checkout@v4
          - name: Setup Python
            uses: actions/setup-python@v5
            with:
              python-version: "3.13"
          - name: Install dependencies
            run: pip install pytest pytest-codspeed
          - uses: CodSpeedHQ/action@v3
            with:
              run: pytest tests/ --codspeed
              token: ${{ secrets.CODSPEED_TOKEN }} # Optional for public repositories

  2. Create a Pull Request installing the workflow to the repository and wait for the report in the comments:

  3. Merge it and congrats 🎉, CodSpeed is installed!

Introduce a Performance Regression​

  1. Let's change the implementation of the count_even with a more concise and elegant one:

    tests/test_count_even.py
    def count_even(arr: list[int]) -> int:
        """Count the number of even numbers in an array."""
    -   even = 0
    -   for x in arr:
    -       if x % 2 == 0:
    -           even += 1
    -   return even
    +   return sum(1 for x in arr if x % 2 == 0)

  2. Open a Pull Request and wait for the CodSpeed report:

    Before merging, we can see that even if it's more concise, this new implementation is slower. Merging it would have introduced a performance regression but we caught it before shipping it!

Next Steps​