feat(core): HTTP/2 Phase 5 — frame layer

Implements HTTP/2 frame reading, validation, and writing: FrameType (the
10 RFC 9113 types + per-type validation descriptor), FrameFlags (with
the deliberate END_STREAM/ACK bit collision documented), FrameHeader (a
flyweight, never allocated per frame), Http2FrameReader (length-prefixed
reader over BufferedByteSource, mirroring RequestParser's buffer/
compaction discipline), FrameValidator (table-driven, specific RFC error
code per violation -- not a uniform code per type), Padding (RFC 9113
6.1/6.2), and FrameWriteBuffer (beginFrame/endFrame length back-patching
over Phase 4's ByteWriter).

All 10 frame types round-trip correctly; every RFC-mandated rejection
has its own test asserting the specific error code; the reader is
fuzz-tested against 10,000,000 random inputs (~14s). The zero-alloc
contract is measured, not asserted: reading + validating + consuming a
frame is 0.002 B/op, writing one is ~10^-4 B/op -- both indistinguishable
from zero (DEC-21).

Found and fixed EX-37 while writing Http2FrameReaderTest: BufferedByteSource's
deadline mechanism (EX-07's actual fix) NPE'd against a null socket, which
every isolated unit test in this codebase uses -- it had zero dedicated
test coverage of its own. Fixed to treat a null socket as "no OS-level
timeout to bound" rather than a misuse, and given BufferedByteSourceTest,
which did not exist before.

449/449 tests green, both with and without -Pjmh.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Zakaria El Orche
2026-08-13 14:25:12 +00:00
co-authored by Claude Sonnet 5
parent 704a00a551
commit 0e1bbed42c
18 changed files with 1665 additions and 23 deletions
+32
View File
@@ -733,3 +733,35 @@ once Phase 6 lands `Request`/`RequestBody` pooling — re-run this exact benchma
this entry (or add a new one) with the "after" number, closing the loop Phase 4 opened.
---
## DEC-21 — Phase 5's zero-alloc contract, measured
**Context.** Phase 5's plan states: "Reading, validating and discarding a frame: 0 B/op ...
Writing a frame header: 0 B/op." Measured with JMH `-prof gc` (JDK 21.0.11, JMH 1.37,
`FrameLayerBenchmark`, `src/jmh/java`) rather than left as an unverified assertion, per this
project's own standing practice of measuring every stated performance/allocation claim
(`DEC-09`, `DEC-20`).
**Measurement.** `readValidateAndDiscard` (`Http2FrameReader.readFrame` +
`FrameValidator.validate` + one byte read from the payload + `consumeFrame`, against a warm,
already-grown buffer, matching real keep-alive-connection steady state): 299.846 ± 19.722 ns/op,
**0.002 B/op** — indistinguishable from zero (compare `DEC-20`'s harness-floor discussion: even
this near-zero figure is most plausibly measurement noise around the true 0, not a real
allocation, since nothing in the read/validate/consume path can be shown by inspection to
allocate on the warm path). `writeFrame` (`FrameWriteBuffer.beginFrame` + one `writeBytes` call +
`endFrame`, against an already-grown `ByteWriter`): 14.262 ± 1.084 ns/op, **≈10⁻⁴ B/op** —
likewise indistinguishable from zero.
**Decision.** Contract verified as stated; no design change required. Both numbers are recorded
here as the baseline Phase 17's eventual CI allocation gate should hold this component to.
**Consequence.** None beyond the recorded numbers — this entry exists so a future regression
(e.g. a later phase accidentally introducing an allocation on this path while adding HPACK or
stream-state integration) has a concrete "was 0, now isn't" baseline to diff against, per this
project's standing insistence that every non-obvious performance claim trace to an actual number.
**Revisit when.** Not expected to be revisited; re-measure if `FrameHeader`, `Http2FrameReader`,
or `FrameWriteBuffer` are ever modified in a way that could plausibly affect their allocation
profile.
---
+148
View File
@@ -0,0 +1,148 @@
# The Frame Layer (Phase 5)
Audience: contributors. This is the design record for `dev.relism.flash.h2.frame`'s frame
reading, validation, and writing — the 9-byte header and payload boundary, with no connection
semantics, no streams, and no HPACK above it.
## Why this is simpler than the h1 parser
HTTP/1.1 request parsing must scan for `\r\n\r\n` (`RequestParser`, `ByteScan.indexOfCrLfCrLf`)
because nothing in the h1 wire format states the header block's length up front. HTTP/2 states
every frame's payload length in the first three bytes of its 9-byte header — nothing is ever
scanned for. `Http2FrameReader` is a length-prefixed reader and nothing more: read 9 bytes,
decode the length, ensure that many more bytes are available, done.
## The wire format
```
+-----------------------------------------------+
| Length (24) |
+---------------+---------------+---------------+
| Type (8) | Flags (8) |
+-+-------------+---------------+-------------------------------+
|R| Stream Identifier (31) |
+=+=============================================================+
| Frame Payload (0...) ...
+---------------------------------------------------------------+
```
`R` (RFC 9113 §4.1) is reserved and MUST be ignored on receipt — `FrameHeader.reset` masks it
out of `streamId()` once, so no caller has to remember to.
## Package layout
```
dev.relism.flash.h2.frame
├── FrameType the 10 known types + per-type validation descriptor (min/max length, stream-id rule)
├── FrameFlags END_STREAM/ACK/END_HEADERS/PADDED/PRIORITY bit constants + predicates
├── FrameHeader flyweight over a read buffer: length/type/flags/streamId/payloadOffset
├── Http2FrameReader length-prefixed reader, RequestParser's buffer/compaction discipline
├── FrameValidator table-driven per-type RFC validation, specific error code per rule
├── Padding RFC 9113 §6.1/§6.2 pad-length byte + trailing padding, DATA/HEADERS
├── FrameWriteBuffer beginFrame()/endFrame() length back-patching over a ByteWriter
├── Http2FrameWriter (Phase 3) the connection's single serialized writer — unchanged here
├── WriteIntent (Phase 3) unchanged
└── IntrusiveMpscQueue (Phase 3) unchanged
```
## The validation table
Every rule below is enforced by `FrameValidator.validate(FrameHeader, insideHeaderBlock)`, in
this order: unknown-type handling, `SETTINGS`' modulus-6 special case, the generic
min/max length bounds, the `MAX_FRAME_SIZE_LOCAL` ceiling, the stream-id rule, then
`PUSH_PROMISE`'s always-reject rule.
| Type | Code | Length | Stream id | Notes / RFC |
|---|---|---|---|---|
| DATA | 0x0 | 0..MAX_FRAME_SIZE | required (≠0) | §6.1. Padding via `Padding.unpad`. |
| HEADERS | 0x1 | 0..MAX_FRAME_SIZE | required (≠0) | §6.2. Padding + PRIORITY fields (Phase 7+ parses the latter). |
| PRIORITY | 0x2 | exactly 5 | required (≠0) | §6.3. Deprecated (§5.3.2) — parsed, discarded, never acted on. |
| RST_STREAM | 0x3 | exactly 4 | required (≠0) | §6.4. The 4 bytes are the error code. |
| SETTINGS | 0x4 | multiple of 6 | forbidden (=0) | §6.5. Modulus checked before the generic bounds. |
| PUSH_PROMISE | 0x5 | ≥4 | required (≠0) | §6.6. Always `PROTOCOL_ERROR` from a client — never sent by Flash. |
| PING | 0x6 | exactly 8 | forbidden (=0) | §6.7. Opaque 8-byte payload, echoed on ACK. |
| GOAWAY | 0x7 | ≥8 | forbidden (=0) | §6.8. Last-stream-id (4) + error code (4) + optional debug data. |
| WINDOW_UPDATE | 0x8 | exactly 4 | either | §6.9. 0 = connection window, ≠0 = one stream's window. |
| CONTINUATION | 0x9 | 0..MAX_FRAME_SIZE | required (≠0) | §6.10. Continues a header block; see the flood guard below. |
| *(unrecognised)* | >0x9 | — | — | §4.1: ignored outside a header block, `PROTOCOL_ERROR` inside one (§6.10). |
**The error code is not uniform per type** — a `SETTINGS` frame with a bad length is
`FRAME_SIZE_ERROR`; the same frame with a non-zero stream id is `PROTOCOL_ERROR`. Every violation
in the table above carries its own RFC citation and the specific code that citation mandates;
`FrameValidatorTest` has one test per row asserting the exact code, not merely "an exception".
## Ignore vs. reject policy
RFC 9113 §4.1 makes unknown frame types part of the protocol's extension mechanism: an endpoint
that does not recognise a type MUST read and discard its payload, never reject the connection for
it. `FrameType.fromCode` returns `null` for anything above `CONTINUATION` (0x9); `FrameHeader`
still exposes the raw `typeCode()` for logging even when `type()` is `null`.
The one exception (§6.10): if an unrecognised-type frame arrives **between** a HEADERS/
PUSH_PROMISE frame that lacked `END_HEADERS` and the CONTINUATION that eventually sets it, the
HPACK decoder's state has nowhere to put that frame's bytes without desynchronizing — so this one
case *is* a `PROTOCOL_ERROR`, tracked by `FrameValidator.validate`'s `insideHeaderBlock`
parameter (owned and threaded through by the Phase 8 connection loop, which is the only caller
that knows whether a header block is currently open).
`PRIORITY` frames are a different kind of "ignore": they are a recognised, well-formed type that
Flash chooses not to act on (RFC 9113 §5.3.2 deprecates priority signalling and permits an
implementation to disregard it) — they are still fully parsed and validated like any other frame,
just never influence scheduling. `PUSH_PROMISE` is the opposite: recognised, but **always**
rejected when received (Flash advertises `SETTINGS_ENABLE_PUSH=0` and never sends one itself), so
receiving one at all can only mean the peer has the client/server roles backwards.
## Buffer discipline and the frame-size defence
`Http2FrameReader` never grows its buffer to accommodate a declared length before checking that
length against `Http2Limits.MAX_FRAME_SIZE_LOCAL` — the check happens first, so a hostile 16 MB
declared length is rejected at the cost of reading 9 bytes, not at the cost of a 16 MB
allocation. This mirrors `RequestParser`'s own `EX-08` discipline (bound the request line before
trusting it) applied to the frame layer's own attack surface.
The buffer itself follows `RequestParser`'s compact-before-grow policy: unconsumed bytes slide to
offset 0 when there is room to do so without growing, and growth only happens when compaction
alone cannot make room — bounded, because the reader's own length check already rejected
anything that would require growing past `9 + MAX_FRAME_SIZE_LOCAL`.
## Padding
`Padding.unpad` locates the actual data range within a `PADDED` frame's payload: 1 byte of
pad-length, then data, then that many padding bytes (whose contents carry no meaning — they exist
only to obscure payload size from network observers). A pad length greater than or equal to the
whole payload length is `PROTOCOL_ERROR` (RFC 9113 §6.1), checked before any arithmetic that
could otherwise underflow. Flow-control accounting for padded DATA frames (RFC 9113 §6.9.1: the
*whole* payload counts against the window, not just the data) is Phase 11 scope — `Padding` only
locates the data range, it performs no window bookkeeping itself.
## Writing: `FrameWriteBuffer`'s back-patching
A frame's length is rarely known before its payload is serialized (an HPACK-encoded header block,
in particular, has no cheap way to be measured in advance). `FrameWriteBuffer.beginFrame` writes
a 9-byte header with a placeholder length; the caller writes the payload directly through the
same `ByteWriter`; `endFrame` computes the actual length from how far the writer has advanced and
rewrites the three length bytes in place. This is *why* `Http2FrameWriter` (Phase 3) serializes a
complete buffer before ever taking the connection lock, rather than streaming bytes as they are
produced — streaming would need the length upfront, which back-patching deliberately avoids
needing.
## `EX-37`, found while building this phase's tests
`BufferedByteSource`'s deadline mechanism (`EX-07`'s actual fix) turned out to have zero dedicated
unit tests and an unconditional `socket.setSoTimeout(...)` call that NPE'd against the `null`
socket every isolated unit test in this codebase uses. Found while writing
`Http2FrameReaderTest`, fixed, and given its own regression suite (`BufferedByteSourceTest`) —
full writeup in the plan's registry, `EX-37`.
## Testing
- `Http2FrameReaderTest` — round-trips every frame type, boundary lengths (0, 1, 16383, 16384,
16385), a frame split across three socket reads, a frame exactly filling the initial buffer,
multiple sequential frames, clean-EOF-vs-mid-frame-EOF, and reserved-bit masking.
- `FrameValidatorTest` — one test per RFC-mandated rejection above, asserting the specific
`Http2ErrorCode`.
- `Http2FrameReaderFuzzTest` — 10 000 000 random-length (064 byte), random-content inputs; only
`Http2Exception`, `EOFException`, or `SocketTimeoutException` may escape. Green, ~14s.
- `PaddingTest` — every boundary of the pad-length arithmetic, including the exact
`padLength == payloadLength - 1` (maximum valid) and `padLength >= payloadLength` (rejected)
cases.
+64 -19
View File
@@ -66,7 +66,7 @@ Status values: `not started` / `in progress` / `blocked` / `done`.
| 2 — Transport decomposition | done | `feature/core/http2` | `HttpServer.java` deleted; `transport`/`http1` packages + WS extraction (EX-01/06/11/12/13/14/15/16/32/34) done. Router `ThreadLocal` (EX-06 router half) deliberately deferred to Phase 4 per DEC-15. 311/311 tests green (run 3×). h1 benchmark check deferred — no JMH harness until Phase 3. |
| 3 — Serialized frame writer (GO/NO-GO gate) | done | `feature/core/http2` | `Http2FrameWriter`/`WriteIntent`/`IntrusiveMpscQueue` + `Http2FrameWriterTest`/`Http2FrameWriterStressTest` + `FrameWriterBenchmark` (JMH, `-Pjmh`, `src/jmh/java` — moved there from `src/test/java` after it broke default `mvn test`; see `DEC-17`). All 4 gate criteria met: N=1 0 B/op & 42.6 ns overhead (≤50 ns budget); N=64 65.5% throughput retention (≥60%) & 11.814.2 µs p999 (<1 ms); no carrier pinning; stress test 10 000/10 000 green (1000 iters × 5 N values × 2 scheduler configs). Full numbers in `WRITER.md`, `DEC-09`. 321/321 non-JMH tests green. |
| 4 — Byte-layer foundations | done | `feature/core/http2` | `dev.relism.flash.bytes` package (`ByteScan`+SWAR, `ArrayBackedByteView`, `SegmentedByteView`, `PooledSlice`/`SlicePool`, `ByteWriter`, `Pairs`) built. `EX-04`/`EX-05`/`EX-09`/`EX-19`/`EX-25`/`EX-26`/`EX-33` done, plus `EX-06`'s router half (plan correction, `DEC-19`) removing `FastPathRouterImpl`/`FastPathWsRouterImpl`'s `ThreadLocal`s via an opaque per-connection scratch (`AbstractRouter#newScratch`) instead of extending `ConnectionScratch` (would have created a `routing``transport` package cycle). `AbstractRouter`/`AbstractWsRouter.route()` gained a `scratch` param — all call sites updated. Measured (`DEC-20`): SWAR scan 35.4% faster (kept), `EX-04`'s word-path 32.1% faster at the mechanism level (kept; today's router doesn't route through it — `MethodPathByteView` stays non-array-backed by design). Router matching itself is ≈0 B/op including parametric routes. Full h1 pipeline is 120.008 B/op, 100% attributable to `Request`/`RequestBody`/`RequestLine` construction — explicitly Phase 6 scope, not a Phase 4 regression. Two documented (non-hot-path) anonymous-`ByteView` fallbacks remain in `QueryParams`/`PathParams.view`. `BYTES.md` written. 395/395 tests green (both with and without `-Pjmh`). |
| 5 — Frame layer | not started | — | — |
| 5 — Frame layer | done | `feature/core/http2` | `FrameType`/`FrameFlags`/`FrameHeader`/`Http2FrameReader`/`FrameValidator`/`Padding`/`FrameWriteBuffer` built. All 10 frame types read/validated/written; per-type RFC error codes verified individually (`FrameValidatorTest`); fuzz-tested 10M random inputs (~14s, green). Zero-alloc contract measured, not asserted: read+validate+consume 0.002 B/op, write ≈10⁻⁴ B/op (`DEC-21`). Found+fixed `EX-37` (`BufferedByteSource`'s deadline mechanism NPE'd against a `null` socket — zero prior test coverage of `EX-07`'s own fix; added `BufferedByteSourceTest`). `FRAMES.md` written. 449/449 tests green. |
| 6 — Request/Response model refactor | not started | — | — |
| 7 — HPACK decoder | not started | — | — |
| 8 — Connection state machine | not started | — | — |
@@ -630,6 +630,30 @@ RFC 9112 §5 gives no such leniency: a header field line without a colon is not
**Fix**: `colon == -1` now rejects the request with `400 Bad Request`.
**Phase**: 1.
### EX-37 — `BufferedByteSource`'s deadline mechanism NPEs against a `null` socket, so it was never actually testable in isolation
Found while writing `Http2FrameReaderTest` (Phase 5): `BufferedByteSource.clearDeadline()` and
`fillFromUnderlying()` both call `socket.setSoTimeout(...)` unconditionally. Every isolated unit
test in this codebase that constructs a `BufferedByteSource` directly (over a
`ByteArrayInputStream`, to test a parser/reader without a real connection) passes `null` for
`socket` — the codebase's own established idiom, used throughout `RequestParserTest`,
`ChunkedInputStreamTest`, `RequestParserSecurityTest`. That idiom works today only because none
of those tests ever call `setDeadline`/trigger a deadline-bounded read — `RequestParser` itself
never calls `setDeadline` (only `Http1Connection`, which always has a real socket, does). The
moment any code under test (here, `Http2FrameReader`, which correctly uses the deadline exactly
as `EX-07` designed it) sets a deadline and then performs a read against a `null`-socket source,
both methods threw `NullPointerException` instead of the intended `SocketTimeoutException`/
normal read. `BufferedByteSource` — the class that exists specifically to implement `EX-07`'s
slowloris defence — had **zero** dedicated unit tests (`BufferedByteSourceTest` did not exist);
its deadline mechanism was exercised only indirectly, end-to-end, via real-socket tests
(`HttpServerTimeoutTest`), which never hit this path.
**Fix**: both methods now skip the `socket.setSoTimeout(...)` call when `socket == null` — a
`null` socket means "no OS-level timeout to bound", not a misuse; the deadline-expiry check
itself (`remainingNanos <= 0` → `SocketTimeoutException`) is independent of the socket and keeps
working. Production always supplies a real socket, so no production behavior changes.
`BufferedByteSourceTest.java` added (previously absent) with direct coverage of the deadline
mechanism against a `null` socket, closing the actual test gap this bug lived in.
**Phase**: 5 (found and fixed while building `Http2FrameReaderTest`).
---
# PART III — The phases
@@ -1602,36 +1626,57 @@ Created:
payload copy at this layer (the payload stays in the read buffer; copies happen above, per
the layer that needs to retain it).
- Writing a frame header: 0 B/op (writes into the existing scratch).
- [x] **Measured**, not just asserted: `FrameLayerBenchmark` (`-prof gc`) — read+validate+consume
0.002 B/op, write 10⁻⁴ B/op, both indistinguishable from zero. `DECISIONS.md`, `DEC-21`.
### Safety checks
- [ ] Declared length checked against `SETTINGS_MAX_FRAME_SIZE` **before** any buffer growth
- [ ] Buffer growth bounded and monotonic (never shrink mid-connection; shrink only on release
to the pool if the high-water mark was pathological)
- [ ] Per-type length/stream-id/flag validation table complete for all 10 types
- [ ] Unknown types ignored; unknown types inside a header block rejected
- [ ] Reserved bit masked, not rejected
- [ ] Padding length validated against frame length
- [ ] Frame read is timeout-bounded (reuse `bodyReadTimeoutMs` semantics or add
`Http2Limits.FRAME_READ_TIMEOUT_MS`)
- [x] Declared length checked against `SETTINGS_MAX_FRAME_SIZE` **before** any buffer growth
`Http2FrameReader.readFrame` checks `declaredLength > MAX_FRAME_SIZE_LOCAL` immediately
after decoding the header, before the payload-sized `ensureAvailable` call that would grow
the buffer.
- [x] Buffer growth bounded and monotonic — grows only to accommodate `9 + declaredLength`,
itself already bounded by the check above; never shrinks (matches `RequestParser`'s own
buffer policy, not yet pool-released — no per-connection buffer pool exists before Phase 13).
- [x] Per-type length/stream-id/flag validation table complete for all 10 types — `FrameType`'s
constants + `FrameValidator`, one `FrameValidatorTest` case per RFC-mandated rejection.
- [x] Unknown types ignored; unknown types inside a header block rejected —
`FrameValidator.validate`'s `insideHeaderBlock` parameter,
`unknownType_outsideHeaderBlock_isIgnoredNotRejected`/`unknownType_insideHeaderBlock_isProtocolError`.
- [x] Reserved bit masked, not rejected — `FrameHeader.reset` masks it out of `streamId()`;
`reservedBitInStreamId_isMaskedNotRejected`.
- [x] Padding length validated against frame length — `Padding.unpad`, `PaddingTest`'s boundary
cases (`padLength == payloadLength - 1` valid, `padLength >= payloadLength` rejected).
- [x] Frame read is timeout-bounded — `Http2Limits.FRAME_READ_TIMEOUT_MS` (new constant, this
phase), enforced via `BufferedByteSource`'s existing deadline mechanism.
### Tests
- `Http2FrameReaderTest` — round-trip every frame type; boundary lengths 0, 1, 16383, 16384,
16385; a frame split across three socket reads; a frame exactly filling the buffer.
16385; a frame split across three socket reads; a frame exactly filling the buffer; multiple
sequential frames; reserved-bit masking.
- `FrameValidatorTest` — one test per RFC-mandated rejection, asserting the **specific** error
code, not merely that an error occurred.
- `Http2FrameReaderFuzzTest` — random bytes into the reader; assert only `Http2Exception` or
`Http2StreamException` escapes (never `ArrayIndexOutOfBoundsException`, `NegativeArraySizeException`,
`OutOfMemoryError`, or an infinite loop — enforce with a per-case timeout).
- `Http2FrameReaderFuzzTest` — 10 000 000 random-length, random-content inputs — **plan
correction**: asserts only `Http2Exception`, `EOFException`, or `SocketTimeoutException`
escapes, not `Http2Exception`/`Http2StreamException` as originally written here.
`Http2StreamException` is stream-scoped and this phase has no stream concept yet (Phase 10);
`EOFException`/`SocketTimeoutException` are the correctly-typed outcomes for a fuzz input that
truncates mid-frame or (in principle) times out — both legitimate, expected rejections of
malformed/incomplete input, not bugs. Any other exception type still fails the test. Green,
~14s.
- `PaddingTest`.
- `BufferedByteSourceTest` — new, not originally planned for this phase: regression coverage for
`EX-37`, a `NullPointerException` bug in `BufferedByteSource`'s deadline mechanism found while
writing `Http2FrameReaderTest` (see the registry entry for the full writeup — a plain bug fix,
not a design decision, so no `DECISIONS.md` entry).
### Docs
`flash/docs/http2/FRAMES.md` — the wire format, the validation table (as an actual table, one row per
frame type, with the RFC section for each rule), and the ignore-vs-reject policy.
- [x] `flash/docs/http2/FRAMES.md` — the wire format, the validation table (as an actual table,
one row per frame type, with the RFC section for each rule), and the ignore-vs-reject policy.
### DoD
- [ ] All 10 frame types read, validated, and written.
- [ ] Fuzz test green for 10 million random inputs.
- [ ] `flash/docs/http2/FRAMES.md` complete with the validation table.
- [x] All 10 frame types read, validated, and written — `roundTrip_everyFrameType`.
- [x] Fuzz test green for 10 million random inputs — `Http2FrameReaderFuzzTest`, ~14s.
- [x] `flash/docs/http2/FRAMES.md` complete with the validation table.
---