01 //Introduction
A parser differential is a bug in the seam between two components. Both components read the same bytes; neither is obviously wrong in isolation; but they disagree on what the bytes mean, and an attacker who knows the disagreement can route a security decision through component A while routing the actual operation through component B. Because the bug lives in the protocol interpretation rather than in any single line of code, it slips past unit tests, slips past code review that focuses on one service at a time, and slips past WAFs — WAFs are themselves a prime source of differentials.
Every famous bug class below is a parser differential in disguise: HTTP request smuggling (proxy vs origin disagree on message framing), SSRF via URL validation bypass (validator vs fetcher disagree on hostname), SAML signature wrapping (signer vs consumer disagree on which <Assertion> is "the" assertion), HTTP/2 downgrade smuggling (HTTP/2 layer vs HTTP/1 back-end disagree on length), JWT algorithm confusion in some libraries (header parser vs key resolver disagree on the algorithm). The common review discipline is: wherever two parsers sit in the same trust chain, assume they disagree somewhere, and make sure that disagreement cannot become an escalation.
POST /api/transfer HTTP/1.1 Host: bank.example Content-Length: 13 Transfer-Encoding: chunked 0 GET /admin HTTP/1.1 Host: bank.example
- Reads
Transfer-Encoding: chunked - Stops at the
0\r\n\r\nterminator - Authorizes the POST to /api/transfer
- Forwards the stream to the origin
- Reads
Content-Length: 13 - Consumes 13 bytes of body, stops there
- Treats the remaining bytes as the next request
- Sees a smuggled GET /admin with no auth layer in front of it
The diagram above is the shape of every parser-differential bug, even ones that look nothing like HTTP smuggling on the surface. Replace "front-end proxy" with "validator," "WAF," "signature verifier," "audit logger," or "rate limiter." Replace "origin app" with "file system," "database," "HTTP client," or "authorization handler." The structural invariant — two parsers, one byte stream, divergent interpretations — is what matters.
A team reports a bug: the WAF blocks requests containing '../' but the origin still serves /etc/passwd when the path contains '..%2f'. Which framing is correct?
02 //HTTP Request Smuggling
HTTP/1.1 has two ways to tell the parser where a request body ends: Content-Length (count bytes) and Transfer-Encoding: chunked (read zero-length terminator). RFC 7230 says a message containing both is malformed and must be rejected, and that when Transfer-Encoding is present it takes precedence. In practice, every proxy, load balancer, CDN, WAF, and application server implements this slightly differently. The attack surface is the combinatorial disagreement between whatever two implementations sit next to each other in your trust chain.
Content-Length; back-end uses Transfer-Encoding.Transfer-Encoding; back-end falls back to Content-Length.Transfer-Encoding — but parse an obfuscated variant differently.Transfer-Encoding : chunked, xchunked, duplicate TE, tab vs space.Content-Length.1POST / HTTP/1.1
2Host: front-end.example
3Content-Length: 6
4Transfer-Encoding: chunked
5
60
7
8GET /admin HTTP/1.1
9Host: front-end.example
10Content-Length: 10
11
12x=yThe front-end uses Content-Length: 6, reads 6 bytes (0\r\n\r\n), forwards the whole buffer, and moves on. The back-end uses Transfer-Encoding: chunked, reads the zero-length chunk, finishes the first request, and starts parsing a second request: GET /admin HTTP/1.1. That GET was never seen by the front-end's auth layer, ACL, or WAF. It inherits the TCP connection's trust context and, on a socket that's reused by another user, it can also be spliced onto the next victim's request — which is how smuggling escalates to request hijacking.
1POST / HTTP/1.1
2Host: front-end.example
3Content-Length: 4
4Transfer-Encoding: chunked
5
65e
7POST /admin HTTP/1.1
8Host: back-end.example
9Content-Length: 15
10
11x=1
120
13
14Here the front-end trusts Transfer-Encoding and reads the chunked body to completion, including the smuggled POST. The back-end trusts Content-Length: 4 and stops after 5e\r\n, treating everything after as the next request. TE.CL is usually harder to land than CL.TE because it requires the front-end to willingly accept chunked encoding, which well-hardened CDNs often normalise or reject.
1# HTTP/2 request (to the front-end)
2:method POST
3:path /api
4:authority front.example
5content-length 0 # the H2 frame says the body is 0 bytes
6
7# Attacker-injected pseudo-header
8content-length 34 # but there is a shadow Content-Length
9
10# HTTP/2 body (rewritten by the front-end into an HTTP/1.1 request)
11POST /admin HTTP/1.1
12Host: front.example
13
14# Back-end sees HTTP/1.1 with Content-Length: 34 and happily parses a
15# second request out of the smuggled bytes.content-length) survive the translation and collide with the HTTP/2 frame length that the front-end already used for routing. Fix: reject HTTP/2 requests with conflicting framing metadata, or run HTTP/2 end-to-end.Mitigations, ranked by how well they actually work
| Mitigation | What it does | Effectiveness |
|---|---|---|
| HTTP/2 end-to-end | Eliminates the downgrade seam entirely — framing is unambiguous at both hops. | Best. Removes the bug class. |
| Reject ambiguous messages | Front-end rejects any request with both CL and TE, duplicate CL, TE with obfuscation, or TE on a non-chunked body. | Very good. Removes the most exploited variants. |
| Normalise then re-emit | Front-end parses the request to an internal representation and rewrites it on the back-end side — no byte-level passthrough. | Good if implemented consistently, but easy to bypass with novel input. |
| Close connection after each request | Prevents smuggled bytes from reaching the next user's connection. | Partial — blocks request hijacking but not same-request privilege escalation. |
| WAF signature rules | Match known payload shapes (e.g. Transfer-Encoding : chunked). | Weak. Attackers iterate faster than signatures. |
A reviewer notices the back-end framework tolerates duplicate Content-Length headers by taking the last one. The front-end CDN uses the first. Nothing else is unusual. Is this exploitable?
03 //URL Parser Disagreements
A URL looks like a simple string with a predictable grammar. It is not. RFC 3986 (generic URIs), WHATWG URL Living Standard (what browsers actually implement), and the dozen library implementations that came before either of those finalised all make slightly different choices about how to handle userinfo, percent-encoding, backslashes, IDN, and fragments. Any application that validates a URL in one library and fetches it in another is a candidate for SSRF.
http://evil.example\@internal.corp/
\, treats it as part of the userinfo.\ to / (for http:).1# Pseudocode seen in real codebases. Both halves are "correct" in isolation.
2from urllib.parse import urlparse
3import requests
4
5ALLOWED = {"images.example.com", "cdn.example.com"}
6
7def fetch_user_supplied_url(url: str) -> bytes:
8 # Validator: urllib.parse
9 host = urlparse(url).hostname
10 if host not in ALLOWED:
11 raise PermissionError(f"host {host!r} not in allowlist")
12
13 # Fetcher: requests (which uses urllib3, which re-parses the URL).
14 return requests.get(url, timeout=5).content
15
16# Attacker input:
17# http://images.example.com\@internal.corp/secret
18#
19# urlparse says hostname = "internal.corp" -> BLOCKED (safe!)
20# OR urlparse says hostname = "images.example.com" -> ALLOWED
21# requests resolves hostname = "internal.corp" -> FETCHES internal host
22#
23# Different stdlib versions give different answers for the same input.
24# You cannot rely on "my validator said no" if the fetcher re-parses.The robust fix is not to pick a better URL library. It is to change the interface: validate against the parsed object, then pass the parsed object (or an IP address resolved against an allowlist) to the fetcher, so no second parse ever happens. If the fetcher insists on a string, serialize the parsed object back to a string yourself, so the bytes the fetcher sees are ones you produced from a trusted representation, not ones the attacker supplied.
http://trusted.example\@evil.example/— backslash in userinfo (parsers split differently).http://trusted.example#@evil.example/— fragment confusion (some parsers treat#as userinfo boundary).http://trusted.example:80@evil.example/— port-in-userinfo (the @ sign steals the hostname).http://evil.example/?@trusted.example/— query string masquerading as host (naive regex validators).http://[::ffff:127.0.0.1]/— IPv4-mapped IPv6 (allowlist checks on hostname string miss the loopback).http://2130706433/— decimal-encoded 127.0.0.1 (host allowlist string match misses).http://xn--exampl-gva.com/— IDN/punycode (validator sees ASCII, browser/resolver sees Unicode).http://trusted.example.EVIL.example/— suffix allowlist check usingendsWithforgets the dot.
1from urllib.parse import urlparse, urlunparse
2import ipaddress
3import socket
4
5ALLOWED_HOSTS = {"images.example.com", "cdn.example.com"}
6
7def safe_fetch(url: str) -> bytes:
8 # 1. Parse ONCE. Everything after this point operates on the parsed object.
9 parsed = urlparse(url)
10
11 # 2. Reject schemes, userinfo, and anything exotic up front.
12 if parsed.scheme not in ("http", "https"):
13 raise ValueError("scheme not allowed")
14 if parsed.username or parsed.password:
15 raise ValueError("userinfo not allowed")
16 if not parsed.hostname:
17 raise ValueError("no hostname")
18
19 # 3. Allowlist on the parsed hostname (case-fold, no suffix tricks).
20 host = parsed.hostname.lower()
21 if host not in ALLOWED_HOSTS:
22 raise PermissionError(f"host {host!r} not allowed")
23
24 # 4. Resolve to an IP here, and block RFC 1918 / loopback / link-local.
25 ip = ipaddress.ip_address(socket.gethostbyname(host))
26 if ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_reserved:
27 raise PermissionError("internal IP")
28
29 # 5. Re-serialize from the parsed object, so the fetcher cannot re-parse
30 # attacker bytes. Pin the hostname to the resolved IP via a custom
31 # transport if the library supports it.
32 safe_url = urlunparse((parsed.scheme, host, parsed.path or "/", "", parsed.query, ""))
33 return fetch_with_pinned_ip(safe_url, ip)A reviewer sees `if url.startswith('https://api.trusted.com')` gating an outbound fetch. Why is this insufficient even if the URL is later parsed by a library?
04 //Path Normalization
Every web stack normalises request paths before authorization, routing, or filesystem access. Every web stack normalises differently. When two components normalise a path using different rules, and one of them uses the result for access control while the other uses it for action, you get authorization bypass (reach an admin endpoint the WAF thought was /public), path traversal (read /etc/passwd through a filter that never saw a ..), or static-asset leakage (serve source files that the framework thought it was routing to a handler).
Path Normalization Quirks by Stack
| Input | Interpreted as | Stack |
|---|---|---|
/admin/..;/public/ | /admin/ | Tomcat — ; starts a path parameter and is stripped after normalisation. |
/public/..%2f/admin/ | /admin/ after decode | Many frameworks — percent-decode happens after the proxy's access check. |
/admin/./ | /admin/ | All stacks — but some allowlists compare the raw bytes before collapse. |
/admin// | /admin/ | Most stacks collapse double slashes; a WAF that does not sees a "different" URL. |
/AdMiN/ | /admin/ | Case-insensitive on Windows/IIS, case-sensitive on Linux/nginx. Reverse proxies can disagree with origin. |
/admin/index.php/ | /admin/index.php | PHP path-info handling — the trailing segment is consumed by the script, not the router. |
/static/..%252f/etc/passwd | /etc/passwd after double-decode | Any stack that decodes percent-escapes twice (once in the router, once in a downstream layer). |
/%2e%2e/admin/ | /../admin/ after decode | WAF blocks ../ but not %2e%2e; origin decodes before resolving. |
1# Flask app behind nginx. nginx is configured to require auth on /admin.
2# The app trusts nginx for authn and just serves handlers based on path.
3
4# nginx.conf
5# location /admin { auth_request /auth; proxy_pass http://upstream; }
6
7# /admin/..;/public/feedback -> nginx sees /public/feedback, skips auth.
8# Tomcat (or any app that strips ;params) sees /admin/.
9# Attacker reaches /admin/ with no auth_request in front.
10
11# Fix:
12# 1. Normalise once, at the edge, before ANY auth decision.
13# 2. Use exact-path allowlists, not prefix allowlists.
14# 3. Reject requests whose normalised path differs from the raw path unless
15# you have an explicit reason to allow rewriting./admin/../../public/x routes to /public/x), then hands the original string to a filesystem call that re-resolves ... The check sees a safe path, the syscall sees /etc/passwd. Always check on the same representation you operate on.1import path from 'node:path';
2import fs from 'node:fs/promises';
3
4const SAFE_ROOT = '/var/app/uploads';
5
6export async function serveFile(userPath) {
7 // 1. Resolve to an absolute path (collapses ..).
8 const absolute = path.resolve(SAFE_ROOT, userPath);
9
10 // 2. Assert the result is STILL within the safe root.
11 // path.resolve will happily return /etc/passwd if userPath is "../../etc/passwd".
12 // Trailing separator matters to avoid /var/app/uploads-evil passing the prefix check.
13 if (!absolute.startsWith(SAFE_ROOT + path.sep) && absolute !== SAFE_ROOT) {
14 throw new Error('path traversal');
15 }
16
17 // 3. Use the resolved path for BOTH the check and the operation.
18 // Do not re-open based on the original userPath.
19 return await fs.readFile(absolute);
20}A code-review diff adds a WAF rule: `location ~* \\.\\./ { return 403; }`. The team believes this blocks path traversal. What is the problem?
05 //JSON Parser Quirks
JSON is often described as "a simple data format." It is not. RFC 8259 leaves significant behaviour undefined (duplicate keys, number precision, encoding), and every major parser makes different choices. When JSON crosses a trust boundary — signed in one service, verified in another, or rewritten by a proxy and consumed by an app — those choices become parser differentials.
JSON Behaviours That Differ Across Parsers
| Behaviour | Typical outcome | Where it hurts |
|---|---|---|
| Duplicate keys | JavaScript / Python json / Go encoding/json: last wins. Jackson: configurable (default last). Some Go parsers error. JWT libraries: varies. | Signed-by-first-parser / consumed-by-last-parser — attacker hides a second value after the signed one. |
| Numeric precision | JavaScript: IEEE-754 double only — 9007199254740993 becomes 9007199254740992. Python / Java BigInteger / Go json.Number: exact. | Monetary amounts, user IDs, nonces that cross JS and non-JS services silently lose precision. |
| Trailing bytes after root value | JavaScript JSON.parse: error. Python json.loads: error. Go encoding/json default: ignores. Some XML→JSON bridges: concatenate. | Signature covers bytes 0..N; consumer also reads bytes N..end. Smuggling via trailer. |
| Comments / trailing commas | Strict parsers: error. JSON5, JSONC, some YAML parsers: allowed. "Permissive" JSON libs: allowed. | Serialize-then-parse pipelines can lose or gain fields between the "strict" verifier and the "permissive" consumer. |
| Unicode escapes | Most parsers decode \u0000 to literal NUL. Some databases truncate at NUL. Some validators check the escaped string, app reads the decoded one. | Bypass string-length checks and null-byte injection into backend stores. |
| __proto__ and constructor keys | Node's JSON.parse produces objects with a __proto__ property that, when merged with Object.assign or similar, pollutes Object.prototype. | Prototype pollution via JSON input. |
| BOM / leading whitespace | JSON.parse rejects BOM. Some Java parsers ignore. Some Go parsers error. | Signature verifier strips BOM, consumer does not (or vice versa); canonicalisation breaks. |
1// Simplified sketch of a real CVE class.
2// Service A signs a JSON blob with an HMAC over the exact bytes.
3// Service B verifies the HMAC, then json-parses and uses the object.
4
5// Attacker-controlled JSON:
6const body = '{"user":"alice","role":"user","role":"admin"}';
7
8// A signs the bytes:
9const sig = hmac(body); // valid signature over the literal bytes above.
10
11// B verifies the HMAC -> passes.
12// B then json-parses.
13const obj = JSON.parse(body); // { user: "alice", role: "admin" } (last wins)
14
15// Depending on WHICH duplicate key the signer validated against,
16// this is an admin-privilege smuggling bug.
17//
18// The fix is not "tell people not to do this."
19// The fix is: canonicalize before signing and before verifying. JCS (RFC 8785)
20// produces a deterministic byte-exact serialization where duplicate keys are
21// disallowed. Sign and verify the canonical form, not the wire form.1// A microservice receives an order ID from a payments service:
2// {"order_id": 9007199254740993, "amount": 100}
3
4// The payments service is written in Python, so the order ID is exact.
5// The JS microservice parses the body:
6const body = JSON.parse('{"order_id": 9007199254740993}');
7console.log(body.order_id); // 9007199254740992 <-- silently rounded
8
9// Now the JS service fetches order 9007199254740992, which belongs to
10// a different user. Same JSON bytes, different numeric interpretation.
11
12// Fix options:
13// 1. Send large IDs as strings. Always.
14// 2. Use a JSON parser with BigInt support (e.g. json-bigint) at the
15// boundary that receives them, and handle BigInt throughout.
16// 3. Validate that the parsed value round-trips: JSON.stringify(parsed) === originalBytes.{"__proto__":{"isAdmin":true}} passed through Object.assign({}, req.body) mutates Object.prototype. Every object in the process now has isAdmin === true. This is a JSON parser differential because JSON.parse sets __proto__ as an own-property key rather than mutating the prototype — but any subsequent merge or clone operation treats it as a magic property. Fix: refuse __proto__, constructor, and prototype keys in input, or use Object.create(null) for bags, or use a validated schema.A team signs JWTs with HMAC-SHA256 and verifies them server-side. Someone reports that sending a header like `{"alg":"HS256","alg":"none"}` bypasses signature verification in the library they use. What is the root cause?
06 //Code-Review Checklist
Parser differentials do not appear in any one file. They appear in the seam between two files, two services, or two machines. The review discipline is therefore structural: look for places where the same bytes are parsed more than once in the trust chain, and treat every such place as a bug until proven otherwise.
- The same user-supplied string is parsed by two different libraries, or by the same library at two different trust layers. (Validator+fetcher, WAF+origin, signer+verifier, router+handler.)
- An authorization check runs against a string (URL, path, identifier, header value) and the subsequent operation runs against a re-parsed version. Bugs hide in the second parse.
- Prefix or suffix checks on URLs, hostnames, or paths:
startsWith,endsWith,contains, regex-match against a raw string. Almost always a parser differential waiting to happen. - A signature verification step and a business-logic step that both query the same signed document. If the queries differ (XPath vs getElementById, top-level-key vs nested-key), the signature does not bind what the business logic reads.
- Any code that handles Content-Length, Transfer-Encoding, or HTTP/2 pseudo-headers manually instead of delegating to a vetted HTTP library.
- JSON that is signed, verified, and then re-parsed. Unless the signature is over the canonical form, duplicate keys and numeric precision are live bugs.
- Path handling that uses string concatenation (
root + user_input) and checks containment withstartsWith(root)without a trailing separator — classic sibling-directory bypass. - A WAF rule written as a regex against raw request bytes. WAF regex vs origin parser is the textbook differential.
- Any place where a single byte stream gives rise to two different names for "the same thing" — URL vs hostname, path vs canonical path, header name case, Unicode normalisation forms.
1# A URL string is checked with startsWith/contains and later passed to a fetcher.
2rg -nE 'startsWith\(\s*["\x27]https?://' --type js --type ts
3rg -nE 'url\.startswith\(\s*["\x27]https?://' --type py
4
5# Path handling without canonicalisation.
6rg -nE 'join\([^)]*user|req\.|params\.' | rg -v 'resolve|normalize'
7rg -nE 'os\.path\.join.*request\.|open\(.*request\.' --type py
8
9# Signed-then-reparsed JSON.
10rg -n 'verify.*JSON\.parse|jwt\.decode.*JSON\.parse' --type js --type ts
11
12# Manual HTTP framing (a smell on its own).
13rg -nE 'Content-Length|Transfer-Encoding' --type go --type java --type py | rg -v '_test|mock'
14
15# WAF regex that tries to catch traversal.
16rg -nE '\\\.\\\./|%2e%2e' nginx.conf modsec/07 //XML Signature Wrapping
XML Signature (XMLDSIG) is the signature format used by SAML, WS-Security, SOAP, and several XML-based federation standards. The spec allows signatures to reference signed elements by ID rather than by position, which means the signature and the consumer must both resolve those IDs the same way — and in real deployments, they often don't. Every major SAML library has had at least one signature-wrapping CVE, and most enterprise SSO implementations have been exploited in the wild through this pattern.
<saml:Response>
<ds:Signature>
<ds:Reference URI="#authn-123"/> <!-- signature references THIS id -->
...
</ds:Signature>
<saml:Assertion ID="authn-123"> <!-- signed; verifier reads this -->
<saml:Subject>attacker@evil.com</saml:Subject>
</saml:Assertion>
<saml:Assertion ID="authn-123"> <!-- shadow: same id, unsigned -->
<saml:Subject>admin@victim.com</saml:Subject>
</saml:Assertion>
</saml:Response>getElementById("authn-123") → first match → attacker@evil.com assertion → signature is VALID.//saml:Assertion → last match wins → admin@victim.com → attacker logged in as admin.The attack works because XML has multiple ways to refer to "an element with ID X": DOM getElementById, XPath //*[@ID="X"], schema-aware ID resolution, and namespace-scoped lookups. The signature verifier uses one; the code that extracts the subject/role/permission claim uses another. The attacker crafts a document where the verifier walks to a signed, innocent assertion and the consumer walks to an unsigned, attacker-controlled assertion. Both walks succeed; both are well-formed XML; the signature is cryptographically valid; the consumer logs the attacker in as someone else.
1<samlp:Response xmlns:samlp="..." xmlns:saml="..." xmlns:ds="...">
2 <!-- A copy of the ORIGINAL signed assertion, issued to the attacker -->
3 <saml:Assertion ID="signed-attacker">
4 <saml:Subject><saml:NameID>attacker@evil</saml:NameID></saml:Subject>
5 <ds:Signature>
6 <ds:SignedInfo>
7 <ds:Reference URI="#signed-attacker">
8 <ds:DigestValue>...</ds:DigestValue>
9 </ds:Reference>
10 </ds:SignedInfo>
11 <ds:SignatureValue>VALID</ds:SignatureValue>
12 </ds:Signature>
13 </saml:Assertion>
14
15 <!-- The WRAPPED assertion the attacker wants the app to read.
16 Same ID, different content. Not covered by any signature. -->
17 <saml:Assertion ID="signed-attacker">
18 <saml:Subject><saml:NameID>admin@victim</saml:NameID></saml:Subject>
19 </saml:Assertion>
20</samlp:Response>1// Sketch. Real libraries differ in API, but the shape is universal.
2
3// 1. Parse the document ONCE into an immutable tree.
4Document doc = parseStrict(bytes); // schema-validate; reject duplicate IDs.
5
6// 2. Verify the signature against THAT tree, and capture the node set
7// the signature actually covered.
8SignedInfo info = verifyAndReturnReferencedNodes(doc);
9Element signedAssertion = info.getReferencedElement();
10
11// 3. BUSINESS LOGIC reads from signedAssertion, not from doc.
12// No second XPath query, no second getElementById call.
13String subject = signedAssertion
14 .getElementsByTagNameNS(SAML_NS, "NameID")
15 .item(0)
16 .getTextContent();
17
18// Under this pattern, a wrapped duplicate simply never gets referenced.
19// The attacker can put as many extra assertions in the document as they want;
20// the business logic walks directly from the node the signature covered.A SAML integration validates signatures with library A and extracts the user's role with library B using an XPath query. Both libraries are up to date. A pentest finds a signature-wrapping bypass. Where is the root cause?
08 //Unicode & Encoding
Unicode and character encoding are parser-differential factories. Case folding rules differ by locale, normalisation forms change byte sequences, homoglyphs look identical to humans but differ under ==, and IDN punycode mapping makes xn--exampl-gva.com and exámple.com mean the same domain in a browser but different strings to a validator. Any code that does a case-insensitive compare, a substring check, or a suffix match on a user-supplied string is a potential differential.
Unicode-adjacent bugs that come from parser mismatches
| Class | Mechanism | Example |
|---|---|---|
| Turkish I | "I".toLowerCase() is "ı" in Turkish locale, "i" in invariant locale. | Allowlist on lowercased username: "ADMIN" lowercases to "admın" on one server, "admin" on another; one passes the check, the other lets the attacker log in. |
| NFKC normalisation | Unicode normalisation collapses ffi (U+FB03) into ffi. | Validator rejects admin, accepts the ligature; downstream normalisation maps it to admin. |
| Punycode IDN | exámple.com becomes xn--exmple-3yb.com after IDNA; resolver uses punycode, validator uses the string. | Phishing domains that visually match a trusted one but differ under ASCII comparison. |
| Percent-encoding depth | %2f decodes to /; %252f decodes to %2f then to /. Some stacks decode once, some twice. | WAF sees %252f and passes it; origin decodes twice and sees /. |
| UTF-8 overlong encoding | 0xC0 0xAF decodes to / on parsers that accept overlong forms (historically IIS). | Path-traversal bypass through non-minimal UTF-8. |
| NUL truncation | Some databases and filesystems truncate a string at \0; validators may not. | safe.txt\0/../../etc/passwd passes string-length check, filesystem opens safe.txt. |
| Bidi override | RLO (U+202E) reverses display; report.%U+202Egpj.exe renders as report.exe.jpg. | Filename-based allowlists that compare displayed form to stored form. |
1// Node.js (runs in invariant locale by default)
2"ADMIN".toLowerCase() === "admin" // true
3
4// Java on a server configured with tr-TR locale
5"ADMIN".toLowerCase() // "admın" (dotless i)
6"ADMIN".toLowerCase(Locale.ROOT) === "admin" // true -- only with explicit locale
7
8// C# pre-.NET 4.0 used the current culture by default:
9"INVALID".ToLower() // "ınvalıd" on Turkish servers
10
11// The bug pattern:
12// Service A (JS): lowercases username to "admin", stores it.
13// Service B (Java, Turkish locale): lowercases "ADMIN" to "admın", looks up
14// a fresh row, and happily creates an account that is NOT the same user as
15// service A's "admin".
16//
17// Fix: always specify a locale-insensitive lower/upper (Locale.ROOT in Java,
18// CultureInfo.InvariantCulture in .NET). For security-sensitive comparisons,
19// normalize to NFC and case-fold with Unicode-aware folding, or compare bytes.A codebase normalises usernames with `username.toLowerCase()` before storing and before comparison. A reviewer asks whether this is sufficient. What answer is correct?
09 //Real-World Case Studies
The clearest way to internalise parser differentials is through incidents. Every case below turned on two components reading the same input and disagreeing about its meaning.
- James Kettle, "HTTP Desync Attacks" (Black Hat USA 2019). The modern rediscovery of HTTP request smuggling. Demonstrated CL.TE, TE.CL, TE.TE across dozens of live targets (PayPal, New Relic, AWS, Akamai customers). The remediation wave changed how front-end proxies handle ambiguous framing across the industry.
- Cloudflare ↔ origin smuggling (multiple, 2019–2022). A series of incidents where Cloudflare's parser and the origin framework's parser disagreed on chunked encoding variants. Mitigated by Cloudflare tightening its normalisation and by frameworks (Node, Go, Tornado, Tomcat, Jetty) tightening theirs independently.
- Orange Tsai, "A New Era of SSRF" (Black Hat USA 2017). Showed that every major language's URL parser disagrees with every other. Specific demos: GitLab (url_blocker), Python
urllib+ Pythonrequestsdisagreement, parser behaviour around@and#. - SAML signature wrapping (various, 2012–2019). Documented in "On Breaking SAML: Be Whoever You Want To Be" (Somorovsky et al., USENIX 2012) and reproduced in every major SAML library since. The 2019 Duo "Duo Bypass" disclosure (CVE-2019-14567 and friends) is a good modern example.
- Apache Tomcat "..;/" authentication bypass (CVE-2016-8735 and relatives). Tomcat strips path parameters after normalisation; nginx in front does not. Requests to
/admin/..;/public/reached/admin/with the front-end's auth rules applied to/public/. - urllib3 CVE-2021-33503 (Python). A URL containing leading whitespace was parsed to a different host by
urllib3than by naive string inspection, leading to SSRF bypasses in tools built on top of it. - Ruby on Rails "key wrapping" CVEs (various). Strong parameters filters key by name before the controller reads nested keys. Parsers that treated
user[admin]anduser.admindifferently, or that accepted duplicate keys in different orders, have produced repeated mass-assignment bypasses. - JSON Web Token algorithm confusion (CVE class). Libraries that read
algfrom a duplicate-key-tolerant JSON parser and passed the raw header to a separate signature-type dispatcher have been exploited repeatedly. The fix is JSON canonicalisation and parse-once semantics. - Node.js
path+ Express static file traversal (periodic). The POSIX path normaliser and the Windows path normaliser disagree on backslash handling, drive letters, and UNC paths. Any cross-platform static file server that does not lock down to a single resolver has shipped at least one traversal CVE. - Signed-JWT-with-duplicate-header (various libraries). JWT headers are JSON. When the library's JSON parser accepted duplicate
algkeys silently, attackers sent{"alg":"HS256","alg":"none"}and landed onnonein the signature dispatcher while landing onHS256in the pre-check. Closed by patches that canonicalise or reject duplicate keys.
You are triaging a security finding. The team suggests fixing it by 'upgrading the parser library to the latest version.' Under what condition is this a reasonable fix, and under what condition is it not?
10 //Detection & Tooling
Static analysis catches the easy cases — a single file that parses the same string twice — but misses most real bugs because the two parses happen in different services or across a trust boundary. The stronger detection is architectural review guided by a small set of rules, supplemented with a handful of SAST patterns and differential-fuzzing tests for the input formats that cross trust boundaries.
1# semgrep-rules/url-validate-then-reparse.yaml
2rules:
3 - id: url-validate-then-reparse
4 patterns:
5 - pattern-either:
6 - pattern: |
7 $HOST = urlparse($URL).hostname
8 ...
9 requests.get($URL, ...)
10 - pattern: |
11 $PARSED = urlparse($URL)
12 ...
13 requests.$METHOD($URL, ...)
14 - metavariable-comparison:
15 metavariable: $URL
16 comparison: $URL == $URL # same variable in both parse and fetch
17 message: >-
18 URL is parsed once for validation and then re-parsed by the HTTP client.
19 This is a parser-differential pattern; the validator and the fetcher may
20 disagree on the hostname. Pass the parsed object (or the resolved IP) to
21 the fetcher instead of re-supplying the attacker-controlled string.
22 languages: [python]
23 severity: WARNING1# semgrep-rules/saml-verify-then-requery.yaml
2rules:
3 - id: saml-verify-then-requery
4 patterns:
5 - pattern: |
6 $VERIFIER.verify($DOC, ...)
7 ...
8 $DOC.getElementsByTagNameNS(..., "Assertion")
9 - pattern-not: |
10 $VERIFIED_NODES = $VERIFIER.verifyAndReturnNodes($DOC, ...)
11 ...
12 $VERIFIED_NODES.get(...)
13 message: >-
14 Document is verified, then the assertion is re-queried from the document
15 rather than read from the set of nodes the signature actually covered.
16 Classic XML signature-wrapping shape.
17 languages: [java]
18 severity: ERROR1# URL validated by string then fetched
2rg -nE 'startsWith\s*\(\s*["\x27]https?://' --type js --type ts
3rg -nE 'urlparse\(.*\)\.hostname' --type py | rg -l 'requests\.|httpx\.'
4
5# Path-building with user input but no explicit containment check
6rg -nE 'path\.join\s*\(.*(req|request|input|params|body)' --type js --type ts
7rg -nE 'os\.path\.join\s*\(.*(request|flask\.request|input|args)' --type py
8
9# Duplicate-key-tolerant JSON parsers
10rg -n 'json\.loads' --type py # then check that duplicate_keys handling is explicit
11rg -n '"encoding/json"' --type go # then check DisallowDuplicateFields or strict decoder
12
13# JWT decode + separate header parse
14rg -n 'jwt\.decode.*verify=False' --type py
15rg -n 'jwt\.verify\(|jwt\.decode\(' --type js --type ts | rg -l 'JSON\.parse'
16
17# Explicit Content-Length / Transfer-Encoding handling in application code
18rg -nE 'Content-Length|Transfer-Encoding' --type go --type java --type py --type rbSAST / DAST Tool Coverage
| Tool | Rule / Check | Finds |
|---|---|---|
| Semgrep | python.lang.security.ssrf.allowlist-bypass | URL validated by one parser, fetched by another |
| CodeQL | js/server-side-request-forgery + dataflow from URL parse to HTTP client | SSRF via URL parser differentials in JS/TS |
| Burp Suite (HTTP Request Smuggler) | CL.TE / TE.CL / TE.TE / H2.CL probes | Live smuggling against any deployed HTTP stack |
| smuggler.py / h2cSmuggler | Explicit differential scanners | Front-end/back-end framing disagreement in CI |
| WSFuzzer / wsse-fuzz | SAML/XMLDSIG wrapping fuzzers | Signature wrapping bugs in SAML/WS-Security endpoints |
| Nuclei | Large template library for URL / path / header parser differentials | Known-shape bypasses across common stacks |
| json-canonicalize / JCS CLI | Pre-commit check that signed JSON matches canonical form | Duplicate-key and numeric-precision bugs at the signing boundary |
Which of these is the strongest CI-time signal that a parser differential exists?
11 //Differential Fuzzing
Differential fuzzing is the most reliable technique for finding parser differentials. The idea is simple: you instantiate the two parsers that sit on either side of your trust boundary, generate or mutate inputs, feed both parsers, and assert that they agree on the semantically meaningful output (the parsed hostname, the parsed path, the selected algorithm, the signed subject). Any disagreement is, by definition, a parser differential, and the fuzzer gives you the minimal input that triggers it.
1# differential_url_fuzz.py
2# Run with atheris or just with hypothesis; shown here with hypothesis
3# for readability.
4#
5# The invariant under test:
6# For every input URL accepted by both parsers, the .hostname fields agree.
7# Any disagreement is a potential SSRF allowlist bypass.
8
9from hypothesis import given, strategies as st, settings, assume
10from urllib.parse import urlparse
11import httpx # uses its own parser internally
12
13@given(st.text(
14 alphabet=st.characters(min_codepoint=32, max_codepoint=127),
15 min_size=8,
16 max_size=200,
17))
18@settings(max_examples=50_000)
19def test_parsers_agree_on_hostname(url):
20 # Only consider strings both parsers accept as URLs.
21 try:
22 a = urlparse(url)
23 b = httpx.URL(url)
24 except Exception:
25 assume(False)
26
27 # Both must produce a hostname.
28 assume(a.hostname and b.host)
29
30 # Both must be http(s).
31 assume(a.scheme in ("http", "https") and b.scheme in ("http", "https"))
32
33 # THE INVARIANT. If it fires, you have a differential.
34 assert a.hostname.lower() == b.host.lower(), (
35 f"divergence: {url!r} -> urllib {a.hostname!r} vs httpx {b.host!r}"
36 )
37
38# In a real codebase, replace the two parsers with:
39# (a) whatever your validator calls
40# (b) whatever your HTTP client / SSRF-blocker calls
41# Run in CI; archive failing inputs as regression tests.1# Sketch of a CI-friendly smuggling probe. Production tools (James Kettle's
2# HTTP Request Smuggler, smuggler.py, h2cSmuggler) are more complete; this is
3# the shape for understanding what to assert.
4
5import socket, ssl
6
7def front_back_disagree(front_host, back_host, payload):
8 """Send `payload` to the front-end, observe whether the back-end saw
9 a 'smuggled' second request. Returns True if disagreement was detected."""
10 with socket.create_connection((front_host, 443)) as s:
11 s = ssl.create_default_context().wrap_socket(s, server_hostname=front_host)
12 s.sendall(payload)
13 data = s.recv(8192)
14 # Heuristic: two response-status lines for one apparent request,
15 # or a response that reflects a request we did not visibly send.
16 return data.count(b"HTTP/1.1 ") >= 2
17
18CL_TE = (
19 b"POST / HTTP/1.1\r\n"
20 b"Host: %b\r\n"
21 b"Content-Length: 6\r\n"
22 b"Transfer-Encoding: chunked\r\n\r\n"
23 b"0\r\n\r\nG"
24)
25
26if front_back_disagree(b"front.example", b"back.example", CL_TE):
27 raise SystemExit("CL.TE smuggling surface detected in CI -- fail the build.")You set up differential fuzzing against your URL validator and your HTTP client. After a week, the fuzzer has produced 40 inputs where the two parsers disagree. Your team lead wants to ship the remediation. What is the right order of operations?
12 //Conclusion
Parser differentials are a whole class of vulnerability hiding behind many specific names — HTTP request smuggling, SSRF, signature wrapping, prototype pollution via JSON, path traversal — and the review discipline for all of them is the same: whenever a single input is parsed by two components in the same trust chain, assume they disagree until you have shown they cannot. The gap between the two parses is the vulnerability, and the structural fix is always to remove the gap: parse once, pass the structured result to every step downstream, and never let attacker bytes get a second interpretation.
startsWith/endsWith/contains on attacker-controlled strings · Differential fuzzers run nightly on every trust-boundary format · When two parsers must coexist, one of them is the authoritative one and the other is forbidden from making security decisions.Pair this module with the HTTP Request Smuggling deep-dive, the SSRF guide, and the JWT Vulnerabilities guide for the adjacent bug classes that turn on this same pattern. Done well, a single afternoon's review of your trust boundaries for parser differentials will prevent the next three CVEs in your stack.