Tagging API Tests So CI Can Run the Right Subset

One of the quietest sources of CI friction I see in API test suites is the all-or-nothing run. Every push triggers the full suite, developers wait four minutes for results that are mostly irrelevant to their change, and eventually someone disables the gate entirely because it's slowing them down. The fix isn't a faster machine — it's tagging your tests so the pipeline can run exactly the subset it needs at each stage.

Tagging isn't a new idea, but most teams implement it reactively: they slap a @smoke marker on a handful of tests after someone complains about pipeline speed, and then never revisit it. That leaves the taxonomy half-baked and the CI config doing guesswork. A deliberate tagging strategy — designed upfront, enforced in code review, and wired into the pipeline with explicit commands — is one of the highest-leverage structural decisions you can make for a growing API test suite.

In this article I'll walk through the tagging patterns I rely on in pytest and Behave, the CI configuration that puts them to work, and the mistakes that make tag-based selection fall apart in practice. Everything here is something you can wire into your own suite today.

Building a Tag Taxonomy That Survives a Real API Suite

Before you write a single marker, decide what questions your tags need to answer. In my experience, three axes cover almost every CI use case:

• Speed / risk tier: smoke, regression, slow
• Functional domain: auth, payments, search, notifications
• Contract / integration boundary: contract, external, internal

The tier axis answers "how much of the suite should run right now?" The domain axis answers "which tests are relevant to this service change?" The boundary axis answers "are we safe to run this against a live third-party endpoint?" Each axis is independently queryable, and the combination is where the real power lives.

In pytest, register every marker in pytest.ini (or pyproject.toml) so that an unrecognized tag is a hard error rather than a silent no-op:

[pytest]
markers =
    smoke: fast sanity checks suitable for every PR
    regression: full-depth scenarios run on merge to main
    slow: tests that take more than 10 seconds
    auth: tests covering authentication and authorization flows
    contract: schema and contract validation tests
    external: tests that call a live third-party service

Registering markers this way means a typo like @pytest.mark.smke fails loudly instead of quietly being excluded from every run. That single config line has saved me from more than a few mysterious "why isn't this test running?" investigations.

Decorating tests is then straightforward:

import pytest

@pytest.mark.smoke
@pytest.mark.auth
def test_valid_login_returns_200(api_client):
    response = api_client.post("/auth/login", json={"user": "test", "password": "secret"})
    assert response.status_code == 200

@pytest.mark.regression
@pytest.mark.auth
def test_expired_token_returns_401(api_client):
    response = api_client.get("/profile", headers={"Authorization": "Bearer expired_token"})
    assert response.status_code == 401

Notice that a test can carry multiple markers — smoke and auth are not mutually exclusive. That's intentional. When a developer touches the auth service, they can run pytest -m auth to get every auth-related test regardless of tier. When the pre-merge gate fires, it runs pytest -m smoke and gets a fast signal. Both selections are correct, and neither requires maintaining a separate file list.

In Behave, the equivalent is a tag on the feature or scenario block:

@smoke @auth
Scenario: Valid credentials return a token
  Given I have valid credentials
  When I POST to /auth/login
  Then the response status is 200
  And the response body contains an access_token

Run it with behave --tags=smoke or combine axes with behave --tags=smoke,auth. If you're deciding between Behave and pytest for your suite, the tagging mechanics are similar enough that the choice usually comes down to whether a BDD-style feature file adds value for your stakeholders rather than anything tag-specific.

Wiring Tags Into CI So Each Pipeline Stage Runs Its Own Subset

Having well-tagged tests is only half the work. The other half is a CI configuration that actually uses them. The pattern I keep coming back to is a three-stage pipeline:

1. Pre-merge (every PR): smoke only — fast, high-signal, blocks the merge if red.
2. Post-merge to main: regression — full depth, slower, runs after the merge so it doesn't block the author.
3. Nightly or release gate: regression + slow + external — everything, including tests that hit live third-party endpoints.

Understanding where each stage fits in the broader delivery flow is worth thinking through carefully — the placement of API tests in a CI/CD pipeline has real consequences for both feedback speed and environment stability.

Here's what that looks like in a GitHub Actions workflow:

jobs:
  smoke:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
          python-version: "3.11"
      - run: pip install -r requirements.txt
      - run: pytest -m smoke --tb=short -q

  regression:
    runs-on: ubuntu-latest
    needs: smoke          # only runs if smoke passes
    if: github.ref == 'refs/heads/main'
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-python@v4
        with:
          python-version: "3.11"
      - run: pip install -r requirements.txt
      - run: pytest -m "regression and not external" --tb=short

A few things worth pointing out here. The needs: smoke dependency means the regression job won't even start if smoke is red — you don't waste runner minutes on a suite that's already known to be broken. The "regression and not external" expression uses pytest's boolean marker syntax to exclude live-endpoint tests from the post-merge run, reserving them for the nightly job where you have a controlled environment and explicit approval to call third-party services.

For the nightly job, the expression flips:

- run: pytest -m "regression or slow or external" --tb=long

That single expression picks up every test that shouldn't run on every push, without requiring a separate test file or a hand-maintained list. When you add a new @pytest.mark.external test tomorrow, it automatically appears in the nightly run and nowhere else.

One mistake I see often: teams use tags to exclude tests rather than include them. They mark a handful of tests as @slow and then run pytest -m "not slow" everywhere. This works until someone forgets to tag a new slow test, and suddenly it's in the pre-merge gate silently inflating runtime. Prefer explicit inclusion (-m smoke) over exclusion (-m "not slow") for any gate that has a hard time budget.

Tag Drift, Maintenance Debt, and Keeping the Taxonomy Honest Over Time

The hardest part of a tagging strategy isn't the initial setup — it's keeping it accurate six months later when the suite has grown and the original authors have moved on. Tag drift is real: tests get promoted from smoke to regression territory as they grow more complex but nobody updates the marker, or new tests get no marker at all and silently fall out of every targeted run.

A few practices that keep this under control in my experience:

Enforce marker presence in CI. Add a step that fails if any test file contains an unmarked test. With pytest, a custom plugin or a simple script that parses pytest --collect-only -q output and checks for the absence of known markers will do it. It's a small investment that pays off every time a new engineer adds a test without reading the tagging guide.

Track smoke suite runtime as a metric. If the smoke suite creeps from 45 seconds to 3 minutes over a quarter, something got mislabeled. A simple timestamp diff in CI output, logged to a dashboard or even just a Slack message, makes the drift visible before it becomes a complaint.

Review tags in code review the same way you review assertions. A test tagged @smoke that makes five chained API calls and seeds a database is not a smoke test. Catching that in review is cheaper than debugging a slow pre-merge gate later. I've seen this pattern show up especially often with AI-generated tests, which tend to produce thorough but slow scenarios that get tagged optimistically.

Prune the external tag aggressively. Tests marked @external are the ones most likely to cause flakiness that only surfaces in CI — network timeouts, rate limits, and third-party sandbox instability all live here. Keep this set small, document why each test needs a live call rather than a mock, and revisit quarterly. If a test can be converted to use a recorded response or a contract check, do it.

Finally, document the taxonomy in the repo itself — a short TESTING.md section that lists every registered marker, its intended meaning, and the CI stage it maps to. New contributors shouldn't have to read the CI YAML to understand what @regression means. The documentation cost is maybe thirty minutes; the onboarding value compounds every time someone new joins the project.

A well-tagged suite is ultimately a suite that respects everyone's time: the developer waiting for a pre-merge signal, the reviewer who wants to know which tests cover a changed endpoint, and the on-call engineer who needs to run only the auth-related scenarios at 2 AM. Getting the taxonomy right upfront, and maintaining it deliberately, is one of those structural decisions that quietly makes everything else in the pipeline easier.