A bug report arrives from a customer in Germany. Checkout fails at the final step. Nobody on the team can reproduce it. It passes locally, it passes in CI, it passes in staging. Three days later somebody works out that the payment provider serves a different form to German addresses, with one extra required field, and the validation on that field was never exercised by any test that has ever run.
The suite never had a chance. Every machine that runs it sits behind the same address.
The assumption nobody writes down
Test environments inherit one property that appears in no test plan and no acceptance criterion: the IP address the requests come from. It gets treated as plumbing. For a large class of applications it is an input, and an untested one.
The uncomfortable part is that the resulting gap is invisible from inside. Your coverage report looks the same. Your suite is green. The defects are real and they are sitting in a code path your environment cannot reach.
What actually changes with the address
More than most teams expect, and rarely in one place.
CDN and edge routing. A different point of presence means a different cache state and, during a staged rollout, potentially different asset versions.
Payment providers. Available methods, required fields, currency, and whether a strong authentication step appears at all. This is the single most common source of region-only checkout defects.
Consent and privacy notices. A banner for European addresses, different wording for California, nothing at all in some markets. That banner is a DOM element, and your selectors will meet it whether the test author expected it or not.
Feature flags and gradual rollouts keyed to region, which is how a feature ships to one market without translations for it existing anywhere.
Tax and price display, catalogue availability, content licensing, and the default language when the server resolves region before it reads the language header.
Rate limiting and bot defences, which frequently treat traffic from hosting ranges differently to traffic from consumer connections. This one is worth flagging because it produces failures that look like application bugs and are not.
Why the usual workarounds do not hold
The geolocation override in browser devtools changes what the Geolocation API returns. It does not change the address your request arrives from, so anything resolved server-side is untouched. A test that passes with the override and fails in production is not flaky. It is wrong.
Setting the language header is useful and insufficient, because a great many stacks resolve region from the address first and treat language as a preference within it.
A consumer VPN on the runner works once, by hand. As permanent infrastructure it fails on every count that matters to a test: the exit address changes without notice, the provider ranges are widely flagged, and you cannot pin a specific city. Reproducibility goes with it, and a regional failure you cannot reproduce is worse than no regional testing at all.
Mocking the geolocation service is correct for unit tests and tells you nothing about integration. It tests your mock.
What a usable setup looks like
The requirement is narrower than it first appears. You do not need a large pool of addresses. You need a small number of exit points, one per market you actually support, that stay the same from run to run.
Stability is the whole point. If the address changes between runs, a failure cannot be attributed to anything, and the team learns to ignore that job. Fixed addresses also let the infrastructure side allowlist your runners against staging environments and WAF rules, which in practice is the blocker that stops this work ever reaching CI.
That points at static addresses registered to internet service providers rather than rotating pools. An affordable isp proxy gives you a fixed address per region at a flat monthly cost, which fits a suite that runs a few hundred times a month rather than a collection job billed by the gigabyte. Two or three addresses cover most products, and the line item is small enough that it never becomes a conversation.
Getting it into the pipeline
Treat region as a test parameter, not as a separate suite. The same specs, parameterised over a list of regions, each carrying its exit address in configuration. A separate suite drifts within two sprints.
Keep regional runs off the pre-merge path. They are slower and they depend on third parties you do not control. Nightly, or on the release branch, is the right cadence.
Assert on what actually differs: which payment methods render, whether the consent banner appears and what it contains, the currency symbol and tax line, the language of server-rendered strings, and the status code of the first request.
When a regional run fails, the first question is whether the application is wrong or the environment is. A fixed address per region is what makes that question answerable in five minutes instead of an afternoon.
What it catches
Consent banners that cover the submit button at mobile widths in one market. Currency formatting with the symbol on the wrong side. Payment methods that render correctly and fail on submission. Feature flags left on in a market with no translations, so the interface falls back to keys. Font fallbacks breaking in scripts nobody on the team reads.
None of these are exotic. They are ordinary defects that survive to production for one reason: the test environment has a single address and the users do not.
About the Author
Antony Brown works on infrastructure at ProxyWing, mostly on the unglamorous side of network configuration for testing and data collection teams. He has a long-standing interest in why test environments and production environments disagree, and in the assumptions that never make it into the test plan.
Leave a Reply