Generating Test Data With AI Without Leaking Real Data

One of the first things teams reach for when they start using AI tools in their test workflow is data generation. It makes sense — describing a realistic user record or a complex nested API payload in plain English and getting back a ready-to-use JSON fixture in seconds is genuinely useful. The problem is that the shortcut people reach for most often is also the most dangerous one: pasting a real production record into the chat window to use as a "template." I've seen this happen on teams that would never dream of committing a real customer email address to a public repo, yet they'll drop a full production payload into ChatGPT without a second thought. The risk is real, and it's worth building habits that eliminate it entirely.

The good news is that AI is actually better at generating test data when you give it a schema or a description rather than a real example. A well-prompted model can produce dozens of varied, edge-case-rich records that no real dataset would ever contain — negative balances, unicode names, expired tokens, boundary-length strings — all without touching anything that came from a live system. The discipline here isn't about limiting what AI can do; it's about channeling it toward work that's both safer and more useful for testing.

In this article I'll walk through the concrete patterns I use to generate test data with AI tools like GitHub Copilot and ChatGPT while keeping real data completely out of the loop. That covers how to structure your prompts, how to keep generated data consistent across a suite, and where the remaining risks hide even after you've done everything right.

Prompt From the Schema, Never From a Real Record

The single most important rule: your AI prompt should describe the shape of data, not contain an instance of it. If you need a synthetic user object, write a prompt like this:

Generate 10 JSON user objects matching this schema:
{
  "id": "UUID v4",
  "email": "valid email format",
  "full_name": "realistic but fictional name",
  "date_of_birth": "ISO 8601, age between 18 and 90",
  "account_balance": "float, can be negative, range -500 to 100000",
  "status": "one of: active | suspended | pending"
}
Include at least one record with a negative balance, one with a name
containing non-ASCII characters, and one with the minimum valid age.

That prompt produces richer, more test-useful data than copying a real record ever would, because you're explicitly requesting the edge cases that real production data rarely surfaces. The model isn't interpolating from anything sensitive — it's constructing from a contract.

The same principle applies inside VS Code with GitHub Copilot. Rather than opening a real fixture file and asking Copilot to "generate more like this," write a factory function with typed parameters and let Copilot autocomplete the body. Here's a pattern I use regularly:

import uuid
from faker import Faker

fake = Faker()

def make_user(
    status: str = "active",
    balance: float = 0.0,
    name: str | None = None,
) -> dict:
    return {
        "id": str(uuid.uuid4()),
        "email": fake.email(),
        "full_name": name or fake.name(),
        "date_of_birth": fake.date_of_birth(minimum_age=18, maximum_age=90).isoformat(),
        "account_balance": balance,
        "status": status,
    }

With this factory in place, Copilot can suggest test cases that call make_user(status="suspended", balance=-200.0) without ever needing to know what a real user looks like. The AI is working from your typed interface, not from production data.

One pattern worth calling out: keep a dedicated fixtures/schemas/ directory that contains only sanitized schema definitions — no values, just field names, types, constraints, and allowed enumerations. That directory becomes the only input you ever give to an AI tool. Anything that touches real data never goes near it.

Keeping AI-Generated Data Consistent Across the Whole Test Suite

Generating data for a single test is easy. The harder problem is keeping that data coherent when it's used across dozens of scenarios — a user created in one step needs to match the ID referenced three steps later, and a product fixture used in a cart test needs to be the same one the inventory test expects. This is where teams often slip back toward copying real data, because "at least we know it's consistent." AI-generated data can be just as consistent if you design for it.

The approach I rely on is seeded generation. Python's Faker library accepts a seed value, and so does the standard random module. If you seed both at the start of a test session with a fixed value, every call to your factory functions produces the same output in the same order, every time:

import random
from faker import Faker

SEED = 42
fake = Faker()
Faker.seed(SEED)
random.seed(SEED)

Set that seed in a conftest.py session fixture (for pytest) or in your Behave environment.py before_all hook, and your generated data becomes deterministic without being static. You can regenerate it at any time from the seed alone — no real data required, no committed fixture files that drift out of sync with your schema.

For sharing generated fixtures safely across a Behave suite, I store the seeded outputs in a context object rather than in module-level globals. That way each scenario gets clean access to the same data without the risk of one scenario mutating a shared object that another scenario depends on.

When you use ChatGPT or Copilot Chat to generate a batch of fixtures, ask for them as a Python dict or a JSON file with a comment block at the top that records the generation prompt. That comment becomes your audit trail — if a test breaks six months later, you can re-run the exact prompt against the same schema version and understand exactly where the fixture came from. This is especially important when you're thinking about data management as part of a broader enterprise test architecture, where traceability matters as much as correctness.

# Generated by ChatGPT, 2025-06-10
# Prompt: "Generate 5 suspended user records matching /fixtures/schemas/user.json,
#          include one with a unicode surname and one at the minimum age boundary."
# Schema version: user_v3.json
# Real data used: NONE

SUSPENDED_USERS = [
    {"id": "a1b2c3d4-...", "status": "suspended", "full_name": "Søren Holm", ...},
    ...
]

That header costs thirty seconds to write and saves hours of confusion later.

Where Real Data Still Leaks — and How to Plug Those Gaps

Even with good prompt hygiene, there are a handful of places where real data sneaks in through the side door. Being aware of them is half the battle.

Error messages and logs. When a test fails against a staging environment, the assertion error often contains the actual API response — which may include real data if staging is populated from a production snapshot. I make it a habit to sanitize staging environments with generated data using the same factory functions I use in tests. If that's not possible, at minimum configure your test runner to truncate or redact response bodies in failure output before they reach any log aggregator or CI artifact.

Copilot's training context. GitHub Copilot reads your open editor tabs and recent file history when generating suggestions. If you have a file containing real data open in another tab — even a CSV you were just inspecting — Copilot may incorporate patterns from it into its suggestions. The fix is simple: close files containing real data before you start a Copilot-assisted test-writing session. Keep your editor workspace clean.

Parameterized tests seeded from real exports. A pattern I've seen trip up teams is exporting a CSV from a production database to "see what real inputs look like," then feeding that CSV directly into a @pytest.mark.parametrize decorator. The intent is reasonable — you want realistic variety — but the result is real PII sitting in your test suite. Replace that workflow with a two-step process: export the CSV once, use it only to identify the interesting structural patterns (long strings, special characters, null fields), then generate a synthetic dataset that reproduces those patterns. Delete the original export. The synthetic set goes into version control; the real one never does.

AI chat history. If you use a browser-based AI tool and your organization hasn't configured data retention policies, previous conversations — including any data you pasted — may persist. Use your organization's enterprise-licensed AI tool with appropriate data handling agreements, and treat the chat interface with the same discipline you'd apply to any other external service.

Finally, remember that data safety and test quality reinforce each other. Generated data lets you construct scenarios that real production data never would — malformed inputs, extreme boundary values, deliberate constraint violations. This is especially valuable when you're testing edge cases like paginated API responses where boundary conditions are easy to miss with organic data. AI-generated data isn't a compromise forced on you by privacy requirements — it's genuinely better test data when you use it intentionally.