API Responses That Vary by Query Parameter Combo
Query parameters look harmless. You tack a ?status=active onto a URL and call it a day. But the moment an API accepts two, three, or four parameters — each with its own set of valid values — you're dealing with a combinatorial space that grows fast and breaks in ways that single-parameter tests will never catch. I've seen teams ship a filter endpoint with solid individual-parameter coverage and then watch production fall over when two filters were applied together for the first time.
The core problem is that most test suites are written one scenario at a time, and the author is usually thinking about the happy path for each parameter in isolation. Nobody sits down and asks: "What does the API actually do when sort=desc is combined with page=1 and status=archived?" That question feels tedious, but it's exactly where real bugs live — in the interaction between parameters, not in any single one of them.
This article walks through how I approach this class of problem: how to map the parameter space, which combinations are worth testing explicitly, and how to structure that coverage in pytest or Behave without ending up with hundreds of near-identical test cases you'll hate maintaining.
Learn Python, Behave, GitHub Copilot, APIs, and CI/CD by building a real framework you can finish in a weekend.
Mapping the Parameter Space Before Writing a Single Test
Before touching code, I spend a few minutes with the API docs (or the source, if I have it) and build a simple grid. List every query parameter the endpoint accepts, then list the meaningful values each one can take — not every possible string, but the equivalence classes that should produce distinct behavior. For a search endpoint it might look like this:
status:active,archived,all, omittedsort:asc,desc, omittedpage:1, a middle page, the last page, beyond the last pageq(search term): a matching term, a non-matching term, empty string, omitted
Full combinatorial coverage of even this modest set is already in the hundreds. That's not realistic, and it's not necessary. What I'm looking for is pairwise coverage — every pair of parameter values appears together in at least one test. Pairwise testing catches the vast majority of interaction bugs with a fraction of the test count. Tools like allpairspy can generate these combinations programmatically, or you can sketch them by hand for smaller grids.
The grid also forces you to name the expected outcome for each combination before you write assertions. "What should the response body look like when status=archived and q= (empty)?" If you can't answer that, you don't yet understand the contract — and that's a conversation to have with the team before writing tests, not after. This upfront mapping is especially valuable when the same endpoint also drives paginated results, where parameter interactions with page boundaries create their own class of edge cases.
Structuring Parametrized Tests Without Drowning in Cases
Once I have a shortlist of combinations worth testing, I reach for pytest's @pytest.mark.parametrize. The key is to give each case an explicit ID so failures are readable, and to separate the input from the expected outcome rather than hard-coding assertions inline.
import pytest
import requests
BASE_URL = "https://api.example.com/items"
cases = [
pytest.param(
{"status": "active", "sort": "asc"},
{"min_count": 1, "first_item_status": "active"},
id="active-asc",
),
pytest.param(
{"status": "archived", "sort": "desc"},
{"min_count": 0, "first_item_status": "archived"},
id="archived-desc",
),
pytest.param(
{"status": "all", "q": ""},
{"status_code": 200},
id="all-empty-query",
),
pytest.param(
{"status": "active", "page": 9999},
{"status_code": 200, "empty_results": True},
id="active-beyond-last-page",
),
]
@pytest.mark.parametrize("params,expected", cases)
def test_item_filter_combinations(params, expected):
response = requests.get(BASE_URL, params=params)
assert response.status_code == expected.get("status_code", 200)
if expected.get("empty_results"):
assert response.json()["items"] == []
if "min_count" in expected:
assert len(response.json()["items"]) >= expected["min_count"]
A few things worth noting here. First, the expected dict is intentionally sparse — each case only asserts what's meaningful for that combination. Asserting everything in every case creates brittle tests that break on unrelated changes. Second, the id strings make failure output like FAILED test_items.py::test_item_filter_combinations[archived-desc] immediately actionable. Third, when a new combination needs coverage, it's one new pytest.param entry — no new test function, no copy-paste.
In Behave, I'd reach for a Scenario Outline with an Examples table instead. The tradeoff is readability for non-engineers (Gherkin reads well in a table) versus the flexibility of Python dicts for complex expected structures. For endpoints where the response shape itself changes by combination — not just the data — I sometimes split the outline into multiple feature sections, one per response "shape family." This mirrors the approach I use when response structure varies by user role, where the same parametrize-by-shape pattern keeps things organized.
Catching the Combinations That Actually Break Things
Pairwise coverage is a floor, not a ceiling. Beyond it, there are specific combination patterns that I've learned to test explicitly because they break more often than the math would suggest.
Conflicting filters. When two parameters logically conflict — say, status=active and include_deleted=true — the API has to make a decision. Does one win? Does it return an error? Does it silently return nothing? Test it. Don't assume the behavior is obvious, because it usually isn't documented and developers often disagree about what it should do.
Default vs. explicit values. Omitting a parameter is not the same as passing its default value explicitly, even when the docs say it is. I always include at least one test that omits a parameter and one that passes its documented default, then asserts identical responses. When they differ, you've found an undocumented inconsistency that will confuse API consumers.
Boundary values in combination. A limit=0 parameter might work fine in isolation, but limit=0&sort=desc has a habit of triggering a different code path. Similarly, an empty q= combined with a restrictive status filter is a common source of unexpected 500s. These are the cases where testing beyond the happy path pays off most — the API's error handling for bad combinations is often untested by the developers themselves.
Order sensitivity. Some backends parse query strings in order, and a bug can make ?a=1&b=2 behave differently from ?b=2&a=1. It's rare, but I've hit it. If your HTTP client lets you control parameter order (Python's requests does via a list of tuples), add one reversed-order variant for your most critical combination.
Finally, keep these combination tests tagged so CI can run them selectively. Full combinatorial sweeps are worth running on every PR for a critical filter endpoint, but you may want to gate the broader pairwise matrix to nightly runs. Tagging your test suite so CI runs the right subset at the right time is the operational habit that makes this kind of coverage sustainable rather than a burden.