Header Deduplication Techniques

When multiple infrastructure layers each inject Access-Control-* headers independently, browsers receive response messages containing the same header name more than once. The WHATWG Fetch Standard’s preflight cache algorithm treats each unique (origin, url, credentials-mode, method, header-list) tuple as a distinct cache entry — but it does not tolerate duplicate header fields within a single response. A preflight response containing two Access-Control-Allow-Origin values fails cache storage entirely, forcing the browser to re-issue a preflight on every subsequent request regardless of your Access-Control-Max-Age setting.

This page is part of Preflight Request Optimization & Caching Strategies, which covers the full set of techniques for reducing preflight overhead across real-world deployment architectures.

Spec Anchor: What the Fetch Standard Actually Checks

The WHATWG Fetch Standard, §12.5.3 “HTTP-network-or-cache fetch”, specifies that a preflight response is only stored when the parsed header list yields exactly one value per relevant Access-Control-* field. RFC 7230 §3.2.2 allows header fields to be combined using a comma-separated list when the field definition permits it, but the CORS-specific fields — Access-Control-Allow-Origin, Access-Control-Allow-Credentials, Access-Control-Max-Age — are singleton fields that must not be combined. Any response with multiple instances of these singletons is rejected by the preflight cache.

This is separate from the browser’s origin-matching logic. Even a valid, correctly-reflected origin will not be cached if the header appears twice. Duplicate headers are the silent killer of preflight cache hit rates.

Header and Parameter Reference

Multi-Layer Header Collision Detection

Modern architectures inject CORS headers at multiple layers. Uncoordinated declarations create duplicate response headers that violate the Fetch Standard cache key expectations described above.

Collision mapping by infrastructure layer:

Trace Access-Control-* headers through each tier before writing any fix. Audit overlapping Access-Control-Allow-Headers and Access-Control-Allow-Methods declarations. Correlate findings with OPTIONS endpoint design to ensure consistent response schemas across tiers before applying normalization.

Step-by-Step Implementation

Step 1 — Baseline the duplication rate

Run a curl preflight against each layer (direct to app, via proxy, via CDN) and count occurrences:

curl -sI -X OPTIONS https://api.example.com/v1/resource \
  -H "Origin: https://app.example.com" \
  -H "Access-Control-Request-Method: POST" \
  -H "Access-Control-Request-Headers: Content-Type, Authorization" \
  | grep -ic "^access-control-allow-origin:"

A count greater than 1 confirms duplication at that layer. Record counts per layer to identify which tier is the source of truth.

Step 2 — Suppress upstream declarations at the Nginx reverse proxy

location /api/ {
  proxy_pass http://backend;

  # Strip CORS headers the backend emits — the proxy owns normalization
  proxy_hide_header Access-Control-Allow-Origin;
  proxy_hide_header Access-Control-Allow-Methods;
  proxy_hide_header Access-Control-Allow-Headers;
  proxy_hide_header Access-Control-Allow-Credentials;
  proxy_hide_header Access-Control-Max-Age;

  # Emit exactly one canonical set
  add_header Access-Control-Allow-Origin $http_origin always;
  add_header Access-Control-Allow-Methods 'GET, POST, PUT, DELETE, OPTIONS' always;
  add_header Access-Control-Allow-Headers 'Content-Type, Authorization' always;
  add_header Access-Control-Max-Age 86400 always;
  add_header Vary Origin always;
}

The always flag ensures headers are emitted on all status codes including 204 and error responses. Without it, proxy_hide_header can suppress upstream headers while add_header silently skips non-2xx responses, leaving no CORS headers at all on errors.

Step 3 — Deduplicate in Node.js / Express middleware

When the application layer runs behind a proxy that cannot strip headers (for example, a managed PaaS), intercept at the framework level:

// Deduplication middleware — mount before any CORS or routing middleware
app.use((req, res, next) => {
  const _setHeader = res.setHeader.bind(res);
  res.setHeader = (name, value) => {
    if (name.toLowerCase().startsWith('access-control-')) {
      res.removeHeader(name); // evict any prior value before setting
    }
    return _setHeader(name, value);
  };
  next();
});

// Standard cors package can then run without producing duplicates
app.use(cors({
  origin: (origin, cb) => {
    const allowed = ['https://app.example.com', 'https://admin.example.com'];
    cb(null, allowed.includes(origin) ? origin : false);
  },
  methods: ['GET', 'POST', 'PUT', 'DELETE', 'OPTIONS'],
  allowedHeaders: ['Content-Type', 'Authorization'],
  maxAge: 86400,
}));

This interceptor ensures each CORS header is set exactly once by removing any existing value before writing, regardless of how many middleware functions attempt to set it.

Step 4 — Envoy / service-mesh deduplication via Lua filter

For service-mesh deployments using Envoy, a header-to-metadata extension captures the upstream value and a Lua filter re-injects a single normalized copy:

http_filters:
  - name: envoy.filters.http.header_to_metadata
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.filters.http.header_to_metadata.v3.Config
      response_rules:
        - header: access-control-allow-origin
          on_header_present:
            metadata_namespace: cors_dedup
            key: origin_value
            type: STRING
          remove: true
        - header: access-control-allow-methods
          on_header_present:
            metadata_namespace: cors_dedup
            key: methods_value
            type: STRING
          remove: true
  - name: envoy.filters.http.lua
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.filters.http.lua.v3.LuaPerRoute
      source_code:
        inline_string: |
          function envoy_on_response(response_handle)
            local meta = response_handle:streamInfo():dynamicMetadata()
            local cd = meta:get("cors_dedup")
            if cd then
              if cd["origin_value"] then
                response_handle:headers():add("access-control-allow-origin", cd["origin_value"])
              end
              if cd["methods_value"] then
                response_handle:headers():add("access-control-allow-methods", cd["methods_value"])
              end
            end
          end

This filter chain strips upstream duplicates, captures the canonical value in metadata, and re-injects exactly one normalized header before client transmission.

Step 5 — Consolidate client-side request headers

Client-side header generation directly impacts preflight trigger frequency. Each distinct combination of Access-Control-Request-Headers values produces a separate preflight cache entry. Reducing preflight frequency with header caching covers this in depth; the core practices are:

• Consolidate custom headers to minimize Access-Control-Request-Headers variations across request types
• Remove X-Requested-With unless a backend middleware explicitly requires it
• Standardize Authorization token formatting — always Bearer <token>, never a variant casing
• Batch custom tracking metadata into a single X-Client-Meta payload where feasible
• Freeze header order in HTTP client configuration so the serialized value is deterministic

Edge Cases and Security Boundaries

Subdomain normalization. https://app.example.com and https://app.example.com:443 are the same origin per the Fetch Standard, but some frameworks serialize them differently when reflecting the request Origin. This creates phantom “distinct” Access-Control-Allow-Origin values across requests, fragmenting the preflight cache. Normalize to the canonical form (strip default ports) before reflection.

The null origin. Sandboxed iframes and file:// pages send Origin: null. Reflecting null in Access-Control-Allow-Origin: null is insecure — any sandboxed document can send this origin. If your deduplication layer reflects the request origin verbatim, add an explicit null check that returns an error rather than a reflected null.

Opaque responses. When a request is made with mode: "no-cors", the response is opaque and the browser does not inspect CORS headers. Duplicate headers in opaque responses do not trigger preflight failures, but they can confuse logging and monitoring tooling that counts CORS header occurrences.

Stripping Vary: Origin during deduplication. Some proxy configurations that suppress headers also strip Vary, because Vary is treated as a generic response metadata header. Stripping it causes CDN nodes to return the same cached response — including the same reflected Access-Control-Allow-Origin — regardless of the requesting origin. This poisons multi-tenant deployments. Always explicitly preserve or re-emit Vary: Origin as part of the normalized header set.

Over-consolidating Access-Control-Allow-Headers. Merging all allowed headers across all endpoints into one exhaustive global list invalidates targeted preflight caching for specific endpoints and increases response payload size. Maintain per-route allowed-headers lists when route security profiles differ, and align with cache duration tuning and Max-Age to ensure each route’s preflight response is cached independently.

Proxy and CDN Interaction

CDN nodes cache preflight responses independently per edge location. If a CDN caches a response that contains two Access-Control-Allow-Origin headers, every request served from that cache hit will carry duplicates — even after you fix the origin server. The fix sequence must be:

1. Fix the origin server / proxy to emit single headers.
2. Purge the CDN preflight cache for the affected paths (OPTIONS method, affected Origin values).
3. Verify the CDN is forwarding Vary: Origin and using it as a cache dimension, not collapsing responses across origins.

Check CDN documentation for whether OPTIONS responses are cached by default. Cloudflare, for example, does not cache OPTIONS by default; AWS CloudFront does if you configure a Cache-Control policy that covers preflight TTL.

DevTools and curl Verification Checklist

• Run curl -sI -X OPTIONS <url> -H "Origin: ..." | grep -ic "^access-control-" — each header should appear exactly once
• Open DevTools Network tab, filter by Fetch/XHR, inspect preflight response headers — no field should appear more than once
• Confirm Vary: Origin is present in the deduplicated response
• Issue a second identical cross-origin request and confirm the preflight is not re-issued (DevTools shows no second OPTIONS request)
• Test from a second allowed origin and confirm the reflected Access-Control-Allow-Origin changes correctly without a cache collision
• Run the CI/CD diff script (below) in a staging environment before deploying changes

CI/CD header count validation script:

#!/bin/bash
TARGET_URL="${1:-https://api.example.com/v1/resource}"
ORIGIN="${2:-https://app.example.com}"

RESPONSE=$(curl -sI -X OPTIONS "$TARGET_URL" \
  -H "Origin: $ORIGIN" \
  -H "Access-Control-Request-Method: POST" \
  -H "Access-Control-Request-Headers: Content-Type, Authorization")

check_singleton() {
  local name="$1"
  local count
  count=$(printf '%s' "$RESPONSE" | grep -ic "^${name}:")
  if [ "$count" -ne 1 ]; then
    echo "FAIL: $name appears $count time(s), expected exactly 1"
    exit 1
  fi
  echo "PASS: $name ($count)"
}

check_singleton "access-control-allow-origin"
check_singleton "access-control-allow-methods"
check_singleton "access-control-allow-headers"

# Vary must include Origin
if ! printf '%s' "$RESPONSE" | grep -qi "^vary:.*origin"; then
  echo "FAIL: Vary: Origin missing from response"
  exit 1
fi

echo "PASS: all header deduplication checks passed"

Deploy this in pre-merge pipelines. Fail builds when any Access-Control-* singleton header count deviates from 1.

Common Mistakes

FAQ