httnope #

httnope is a HTTP/1.1 conformance corpus that supplies adversarial peers for either clients or servers. Each one tests clients or servers while probing observable application effects on fresh connections. There is a corpus of expected results.

The repository has raw response cases, request-server cases, typed-field cases, Structured Field vectors, stateful user-agent cases, and stateful cache cases. They derive from RFC 9110, RFC 9111, RFC 9112, related HTTP RFCs, live IANA registries, and edge cases found in various OCaml libraries.

Prior work and imported corpora #

The corpus and harness draw on the RFCs and the HTTP Working Group Structured Field Tests that define schema-checked JSON vectors with raw inputs, generic JSON values, required or permitted rejection, and canonical serialisations. Their three-clause BSD license and original schemas are retained.

Build and run #

``sh dune build @all dune runtest dune exec httnope -- serve --port 8080

The server binds only `127.0.0.1` unless `--public` is given. Deliberate stalls
are two seconds by default; `--delay-scale 0.1` makes exploratory runs faster.

Useful endpoints:

- `GET /manifest.json` — complete corpus, raw scripts, RFC references, and
  expected outcomes.
- `GET /manifest.jsonl` — the same data, one case per line.
- `GET /typed-fields.json` — the `http1-typed-fields` manifest.
- `GET /structured-fields.json` — the HTTPWG-compatible
  `http1-structured-fields` manifest.
- `GET /user-agent.json` — the `http1-user-agent` manifest.
- `GET /cache.json` — the `http1-cache` manifest.
- `<METHOD> /case/<id>` — run one case. The manifest specifies GET, HEAD, or
  CONNECT.
- `GET /workflow/cookie/<id>/<step>` — a stateful cookie workflow endpoint.
- `GET /workflow/reset/<case-id>` — reset endpoint attempts for one workflow.
- `GET /probe` — the second request for persistence/desynchronization cases.
- `GET /healthz` — ordinary health response.

The same metadata is available without starting a server:

```sh
httnope list
httnope list --tag chunked --tag invalid
httnope list --tag positive
httnope manifest --compact
httnope manifest --jsonl
httnope profile-list http1-typed-fields
httnope profile-list http1-structured-fields --tag dictionary
httnope profile-list http1-user-agent --tag 6265bis
httnope profile-list http1-cache
httnope profile-manifest http1-user-agent
httnope server-list
httnope server-list --tag framing --tag invalid
httnope server-list --tag positive
httnope server-manifest --compact

Corpus design #

A case keeps the stimulus and expected safe behavior together:

• stable ID, title, description, tags, and normative RFC links;
• request method and single- or two-request interaction shape;
• an exact script of writes, fragmented writes, byte-at-a-time writes, pauses, and reads of the next request, including deterministic seeded fragments and delayed drip delivery;
• an expectation expressed as allowed outcomes: a precise response sequence, an error, or a timeout, including required/forbidden header and trailer constraints and exact duplicate-preserving field-value sequences;
• a requirement class: must, should, may, or limit.

The categories cover:

• status-line grammar, versions, unknown codes, controls, obs-text, CR/LF, and arbitrary read boundaries, plus the current registered status-code matrix;
• exhaustive field-name and field-value octet matrices, whitespace, duplicates, Set-Cookie, list handling, obs-fold, limits, and unterminated sections;
• valid, duplicate, conflicting, malformed, overflowing, truncated, and under/overreported Content-Length;
• Transfer-Encoding lists, ordering, duplication, non-final chunked coding, parameters, HTTP/1.0, and TE/CL ambiguity;
• chunk-size numeric grammar and overflow, extensions, chunk-data separators, last chunks, trailers, forbidden trailer fields, truncation, and unbounded framing attacks;
• HEAD, 1xx, 204, 304, CONNECT, close-delimited bodies, upgrades, and chains of informational responses;
• persistent-connection reuse and fake-response/desynchronization payloads;
• content-coding behavior, including valid, concatenated, truncated, and trailing-junk gzip, zlib-wrapped and bare deflate, Brotli, Zstandard frame sequences and window limits, the RFC 8188 aes128gcm vector, aliases, unknown codings, and bounded coding stacks;
• prohibited and permitted trailer policy; and
• preservation of dates, locations, authentication challenges, links, cookies, content disposition, structured fields, security and cross-origin policy fields, digests, Cache-Status, and Proxy-Status.

http1-typed-fields supplies valid and invalid field-line sequences and checks the decoder's canonical representation. It covers representation metadata, conditional requests and ranges, negotiation, caching, authentication and digests, Via and Max-Forwards, RFC 9875 Cache Groups, RFC 9440 Client-Cert fields, dates, redirects, links and filenames, alternative services, structured operational fields, upgrades, and trailers.

http1-structured-fields embeds the complete HTTPWG Structured Field corpus. A parsing case supplies the original field-line sequence and expects either rejection or the exact generic JSON value and canonical serialisation. A serialisation case supplies the generic JSON value and expects the canonical field-line sequence. The manifest remains at httnope-structured-fields-v1; each case links to RFC 9651 and its pinned upstream JSON file.

Httnope.Httpwg.vector_jsont and Httnope.Httpwg.file_jsont read and write the upstream vector shape directly. Httnope.Httpwg.vendored exposes the decoded files with their parse or serialise operation. This permits an existing HTTPWG runner or OCaml Structured Fields implementation to reuse the data model while the httnope observation wrapper supplies case IDs, timings, provenance, and benchmark reports.

http1-user-agent runs an ordered request sequence through one persistent client. A request specifies its method, headers, binary content, redirect limit, and optionally a logical origin whose connections are routed to the loopback fixture. Endpoints can vary their response by attempt, send informational responses, echo a binary-safe request trace, inspect an earlier request, check a minimum retry interval, and validate Basic or Digest credentials. The profile covers cookie policy and limits, relative redirect resolution, redirect method and body handling, public-suffix and IDNA cookie policy, Basic and Digest challenge flows (including auth-int and session algorithms), configured Bearer credentials, Early Hints isolation, retries, and request replay. Concealed authentication is capability-gated because a real proof requires TLS exporter and key material; the plaintext fixture only defines the mandatory refusal case.

http1-cache uses the same stateful machinery to distinguish cache hits from network requests and to inspect emitted validators. It covers explicit and heuristic freshness, Age and Expires precedence, stale extensions, request and response directives, Vary matching, ETag and Last-Modified revalidation, authenticated responses, private/shared distinctions, HEAD interaction, unsafe-method invalidation, and Cache Group safety, case-sensitivity, and same-origin boundaries. Its score is independent of user-agent policy.

A workflow adapter sends GET /workflow/reset/<case-id> before starting each case. The reset response is no-store. This makes attempt-dependent cases repeatable on a long-running server without clearing client cookies or cache state belonging to the adapter.

Stateful workflow protocol v3 #

The user-agent and cache manifests use httnope-user-agent-v3 and httnope-cache-v3. Their additions are optional on ordinary cases, and the OCaml decoders continue to accept v1 and v2 manifests and observations.

A v2 or v3 case can set case.expected.kind to responses. Its responses member is an array of acceptable response sequences. Each response expectation has an exact status and binary-safe body, selected final headers, and an optional effective_uri. For every named expected header, its complete ordered field-line value sequence must match; other headers are unconstrained. An absolute-path effective URI expectation matches the path, query, and fragment of the adapter's absolute URI, so the dynamic fixture port does not leak into the corpus.

Adapters answer such cases with outcome: "responses" and one structured record per top-level request:

{"case_id":"cache/304-updates-headers","outcome":"responses","responses":[{"status":200,"headers":[["X-Ver","1"]],"body":"stored","effective_uri":"http://127.0.0.1:8080/workflow/cache/304-updates-headers/resource"},{"status":200,"headers":[["X-Ver","2"]],"body":"stored","effective_uri":"http://127.0.0.1:8080/workflow/cache/304-updates-headers/resource"}]}

A v3 workflow may also include a program that interleaves request actions with advance_clock controls measured in integer milliseconds. Such cases declare the virtual-clock capability; adapters without a controllable clock report them as unsupported. The fixture resets its per-case clock at /workflow/reset/<case-id> and advances it through /workflow/advance/<case-id>/<milliseconds>; a capable adapter advances its own clock and the fixture clock for each control before continuing the program.

requires declares optional capabilities such as response-metadata, challenge-authentication, bearer-authentication, logical-origins, public-suffix-list, idna, and concealed-authentication. An adapter lacking one emits unsupported; it must not approximate the case with body-only data. credentials can supply a username/password pair, Bearer token, or Concealed identity marker to the adapter's normal authentication implementation. A request origin is a logical hostname: the URI and Host value retain that name while the adapter connects to the fixture's loopback address and inherited port. This makes public-suffix, IDN, and multi-origin policy testable without public DNS.

Adapter protocol #

An adapter uses the manifest's request_paths in order and emits one JSON object per case. All paths in a sequence should use the same client/session object so connection pooling is exercised; reconnecting is allowed. For example:

{"case_id":"status/basic-11","outcome":"response","elapsed_ms":1.2,"responses":[{"status":200,"body":"hello","headers":[["content-length","5"]],"trailers":[]}]}

Binary-safe adapters use body_hex instead of body. Other outcomes are:

{"case_id":"chunk/non-hex","outcome":"error","message":"bad chunk size"}
{"case_id":"chunk/stall-size","outcome":"timeout","message":"250ms read deadline"}
{"case_id":"status/basic-11","outcome":"crash","message":"uncaught exception"}

crash is never an accepted outcome. UTF-8 headers and trailers are lists of pairs so duplicates are preserved. A pair containing invalid UTF-8 uses the binary-safe object form {"name_hex":"582d54657874","value_hex":"636166e9"}. An adapter should report the API-visible body after the client library's normal decoding policy, and document that policy.

The httnope-manifest-v1 schema applies the same rule to expected field pairs. A binary forbidden field name uses {"hex":"..."} in place of a JSON string. The header_values and trailer_values members constrain the complete ordered value sequence for one field name. Scripts and body expectations are always byte-safe.

Each references member is structured provenance. An RFC entry records its number and section and includes derived title and url members:

{"kind":"rfc","number":9110,"section":"14.1.1","title":"RFC 9110 §14.1.1","url":"https://www.rfc-editor.org/rfc/rfc9110.html#section-14.1.1"}

Check a run with:

httnope check observations.jsonl
httnope check --require-all --json observations.jsonl

Compare multiple complete or partial adapter runs, including requirement-class scores and latency percentiles:

httnope benchmark fetch-httpz=fetch.jsonl cohttp-eio=cohttp.jsonl ocurl=curl.jsonl
httnope benchmark --json fetch-httpz=fetch.jsonl cohttp-eio=cohttp.jsonl

Run the server and an adapter together with the executor harness:

httnope exec cohttp-eio httnope-adapter-cohttp-eio > cohttp-report.json
httnope exec --exit-zero ocurl httnope-adapter-ocurl > ocurl-report.json

The executor binds an ephemeral loopback server, appends its base URL to the adapter command, captures the adapter's output and standard error, and enforces a 120-second process deadline. Its httnope-execution-v1 Jsont report contains aggregate counts plus a corpus-linked record for every failure and missing case. It also records malformed, unknown, and duplicate observation rows. The command exits nonzero unless the process succeeds and every case passes exactly once; --exit-zero disables this behavior for exploratory runs.

An execution can write all three report formats directly, avoiding a second decode/render pass:

httnope exec --tag transport-variant --exit-zero \
  --output cohttp-transport.json \
  --html-output cohttp-transport.html \
  --markdown-output cohttp-transport.md \
  cohttp-eio httnope-adapter-cohttp-eio

Tagged reports record the conjunctive selected_tags scope in their JSON and display it in HTML and Markdown. A tag combination matching no cases is an error, rather than a vacuous passing report. The same direct output options are available on profile-exec and server-exec.

The shared positive tag selects cases whose judgments permit only successful outcomes. It is derived for response-client, typed-field, Structured Field, user-agent, cache, and request-server cases. Positive cases can never permit rejection, error, or timeout; httnope lint checks that invariant and also checks 53 RFC-requirement witnesses across all six profiles. For example:

httnope exec --tag positive --exit-zero \
  --output client-positive.json \
  --html-output client-positive.html \
  --markdown-output client-positive.md \
  cohttp-eio httnope-adapter-cohttp-eio

Run the independent conformance profiles with the same lifecycle:

httnope profile-exec http1-typed-fields fetch-httpz \
  httnope-adapter-fetch-httpz-typed > fetch-fields.json
httnope profile-manifest http1-structured-fields > structured-fields.json
httnope profile-exec http1-user-agent fetch-httpz \
  httnope-adapter-fetch-httpz-user-agent > fetch-httpz-cookies.json
httnope profile-exec http1-user-agent fetch-curl \
  httnope-adapter-fetch-curl-user-agent > fetch-curl-cookies.json
httnope profile-exec --exit-zero http1-user-agent ocurl-auth \
  httnope-adapter-ocurl-user-agent > ocurl-auth.json
httnope profile-manifest http1-cache > cache-manifest.json

The ocurl-auth adapter intentionally supports only authentication-tagged workflows. It uses libcurl's native Basic and Digest challenge implementation, configured Bearer headers, and reports all other workflows as unsupported. This keeps authentication failures attributable to libcurl rather than a reimplementation in the adapter.

The httnope-conformance-execution-v1 report records supported, unsupported, passed, failed, missing, invalid, and crashed cases separately. An adapter uses unsupported when its public API has no typed codec or user-agent policy for a case. Unsupported cases do not pass and do not count as conformance failures. Structured Field parsing adapters emit outcome: "parsed" with the upstream generic JSON value and a duplicate-preserving canonical string array. Serialisation adapters emit the existing outcome: "values" form.

HTML and agent reports #

Render any current execution report as a compact, self-contained HTML page:

httnope html cohttp-report.json -o cohttp-report.html
cat proffer-server.json | httnope html > proffer-server.html

Render the same JSON as Markdown for agent analysis:

httnope markdown cohttp-report.json -o cohttp-report.md

The Markdown format uses HTTNOPE_REPORT_BEGIN and HTTNOPE_REPORT_END comments around the document. Each failure, invalid observation, omitted-case batch, and process log is enclosed by matching HTTNOPE_CHUNK_BEGIN and HTTNOPE_CHUNK_END comments. A failure chunk repeats its case ID, requirement, and references, then separates the failure reason, stimulus, expected behavior, actual behavior, and rationale. Chunks remain intelligible when processed independently or split on the delimiter comments.

Build fresh reports for all reference clients and servers with:

opam exec --switch=5.5.0 -- dune build @reports

Each reference adapter is executed once to produce its JSON, HTML, and Markdown bundle. An individual output is also a Dune target, for example dune build reports/fetch-httpz-client.md. The report rules retain ordinary conformance failures but reject adapter crashes, spawn failures, and fixture readiness errors, preventing an infrastructure failure from becoming a cached report bundle.

Request-server protocol #

httnope server-exec uses a server fixture process and drives it with exact HTTP/1 request octets. The current cases cover request-line and target forms, versions, Host validation, the complete forbidden request-field control octet matrix, obs-text, Content-Length and Transfer-Encoding ambiguity, chunk grammar and trailers, Expect, size limits, fragmentation, pipelining, post-error dispatch, and response-writer containment.

A fixture binds an ephemeral loopback TCP port and writes exactly HTTNOPE_READY <port> as its first standard-output line. It exposes these reserved routes:

• GET or POST /__httnope__/effect/<token> performs the named effect after the request has been validated and its complete content consumed;
• POST /__httnope__/reset/<token> resets its counter;
• GET /__httnope__/count/<token> returns the decimal counter; and
• /__httnope__/response/... asks the server writer to exercise a named response invariant.

Each hostile exchange runs on a fresh connection. Counter resets and probes also use fresh connections, so a parser cannot pass a smuggling case by poisoning the observation channel. A result combines the raw response sequence, close/reset/timeout state, exact counter values, elapsed time, case metadata, rationale, and precise RFC links.

Reference fixtures for Proffer/httpz and Cohttp-eio are installed by the httnope-adapters package:

httnope server-exec --exit-zero proffer httnope-server-proffer \
  > proffer-server.json
httnope server-exec --exit-zero cohttp-eio httnope-server-cohttp-eio \
  > cohttp-server.json

The report uses httnope-server-execution-v1 and is covered by schema/server-execution.schema.json. The command exits nonzero unless every case passes and the fixture stops normally; --exit-zero retains a failing benchmark report for comparison. server-manifest emits the portable corpus, including binary request scripts and every permitted outcome.

OCaml adapters can depend on the httnope library and use Httnope.Adapter.observe plus Httnope.Adapter.output_jsonl. The request closure remains the adapter's responsibility. See examples/adapter_skeleton.ml.

Reference client adapters and server fixtures live in the optional httnope-adapters opam package. Start the response server in one terminal, then run:

httnope-adapter-cohttp-eio http://127.0.0.1:8080 > cohttp.jsonl
httnope-adapter-ocurl http://127.0.0.1:8080 > ocurl.jsonl
httnope benchmark cohttp-eio=cohttp.jsonl ocurl=ocurl.jsonl

Fetch adapters are provided for the raw, typed-field, and user-agent profiles through both HTTPZ and curl. The typed-field adapters exercise the same Fetch codec layer; the user-agent adapters exercise each backend's complete standard stack, including its in-memory cookie jar and redirect policy, plus an opt-in retry policy. A dedicated HTTPZ-core typed adapter exercises its ETag, Range, and Date parsers directly. See LIBRARY-NOTES.md for current results and known adapter gaps.

They deliberately omit the synthetic CONNECT cases, since neither high-level URI API can select a corpus path while emitting an authority-form CONNECT target. The benchmark reports that as missing capability coverage.

The core normative sources are RFC 9110, RFC 9111, and RFC 9112. Registry-driven cases use the IANA HTTP Status Code Registry.