docs(testing): document flash-testing and the limits it deliberately keeps
README covers the application handle, requests and assertions, service replacement, multi-server wiring, scope, WebSockets, configuration and the teardown ordering. limits.md records the seven things the harness cannot do and what to use for each: TLS, HTTP/2, WebSocket over HTTP/2, malformed requests, response framing, the flash core module's dependency cycle, and scoped services. Each is a consequence of a real constraint rather than an unfinished feature, so writing them down stops the next person rediscovering them one at a time. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
7785712efe
commit
58bae41f7a
@@ -146,6 +146,7 @@ See extension-specific READMEs for full details:
|
|||||||
- [`flash-ext-mcp`](flash-extensions/flash-ext-mcp/docs/README.md)
|
- [`flash-ext-mcp`](flash-extensions/flash-ext-mcp/docs/README.md)
|
||||||
- [`flash-ext-view-jte`](flash-extensions/flash-ext-view-jte/README.md)
|
- [`flash-ext-view-jte`](flash-extensions/flash-ext-view-jte/README.md)
|
||||||
- [`flash-ext-view-thymeleaf`](flash-extensions/flash-ext-view-thymeleaf/README.md)
|
- [`flash-ext-view-thymeleaf`](flash-extensions/flash-ext-view-thymeleaf/README.md)
|
||||||
|
- [`flash-testing`](flash-testing/docs/README.md)
|
||||||
|
|
||||||
## Error handlers
|
## Error handlers
|
||||||
|
|
||||||
@@ -482,6 +483,9 @@ That is stock JUnit field semantics — the isolation switch is the keyword, not
|
|||||||
|
|
||||||
### Configuration
|
### Configuration
|
||||||
|
|
||||||
|
Full reference: [`flash-testing/docs`](flash-testing/docs/README.md), including the
|
||||||
|
[limits](flash-testing/docs/limits.md) the harness deliberately does not cross.
|
||||||
|
|
||||||
`profile` customises the `FlashConfiguration` — timeouts, HTTP/2 switches, buffer sizes. Host, port
|
`profile` customises the `FlashConfiguration` — timeouts, HTTP/2 switches, buffer sizes. Host, port
|
||||||
and the shutdown drain window are stamped afterwards, so a profile cannot break the harness;
|
and the shutdown drain window are stamped afterwards, so a profile cannot break the harness;
|
||||||
`listener(...)` and `tls(...)` are rejected because the harness owns the loopback listener it gives
|
`listener(...)` and `tls(...)` are rejected because the harness owns the loopback listener it gives
|
||||||
|
|||||||
@@ -0,0 +1,195 @@
|
|||||||
|
# flash-testing
|
||||||
|
|
||||||
|
JUnit 5 harness for testing Flash applications. Boots a real app on an OS-assigned port,
|
||||||
|
swaps services for fakes, and asserts on responses. No mocking library, no assertion library —
|
||||||
|
`flash` and `junit-jupiter-api`, nothing else.
|
||||||
|
|
||||||
|
## What it provides
|
||||||
|
|
||||||
|
| Component | Description |
|
||||||
|
|---|---|
|
||||||
|
| `FlashTest` | JUnit 5 extension — owns one app's lifecycle and hands you a client |
|
||||||
|
| `FlashRequest` | Header/body builder; the HTTP verb is terminal and sends |
|
||||||
|
| `FlashResponse` | Chainable assertions that report the real response on failure |
|
||||||
|
| `FlashWebSocket` | WebSocket client over `java.net.http`, with a queue and timeouts |
|
||||||
|
|
||||||
|
## Dependency
|
||||||
|
|
||||||
|
```xml
|
||||||
|
<dependency>
|
||||||
|
<groupId>dev.relism</groupId>
|
||||||
|
<artifactId>flash-testing</artifactId>
|
||||||
|
<version>${flash.version}</version>
|
||||||
|
<scope>test</scope>
|
||||||
|
</dependency>
|
||||||
|
```
|
||||||
|
|
||||||
|
## Quick start
|
||||||
|
|
||||||
|
```java
|
||||||
|
class UserRoutesTest {
|
||||||
|
|
||||||
|
@RegisterExtension
|
||||||
|
static FlashTest app = FlashTest.of(new BlogApp())
|
||||||
|
.mock(UserService.class, new InMemoryUserService());
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void listsUsers() {
|
||||||
|
app.get("/api/users")
|
||||||
|
.expectStatus(200)
|
||||||
|
.expectHeader("content-type", "application/json")
|
||||||
|
.expectBodyContains("alice");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## The application under test
|
||||||
|
|
||||||
|
`FlashTest.of` takes a `FlashApplication` — your app's routes, extensions and services declared
|
||||||
|
independently of the port they run on:
|
||||||
|
|
||||||
|
```java
|
||||||
|
public final class BlogApp implements FlashApplication {
|
||||||
|
@Override public void configure(FlashApp app) {
|
||||||
|
app.install(new JacksonExtension());
|
||||||
|
app.mount("/api", scope -> scope.scan("dev.blog.api"));
|
||||||
|
app.ws("/live", new FeedSocket());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
FlashApp.create(8080).apply(new BlogApp()).startAndBlock(); // production
|
||||||
|
```
|
||||||
|
|
||||||
|
It takes `FlashApp` rather than `FlashRegistrar` deliberately: `ws()` and `mount()` live there,
|
||||||
|
and an application that could not register a WebSocket route or a mounted namespace would be a
|
||||||
|
half-application.
|
||||||
|
|
||||||
|
It is a `@FunctionalInterface`, so a lambda and a named class are the same thing:
|
||||||
|
|
||||||
|
```java
|
||||||
|
FlashTest.of(app -> app.get("/ping", (req, res) -> "pong"))
|
||||||
|
```
|
||||||
|
|
||||||
|
## Requests
|
||||||
|
|
||||||
|
`get` and `delete` skip the builder when there is nothing to add. Everything else goes through
|
||||||
|
`request()`, where the HTTP verb sends:
|
||||||
|
|
||||||
|
```java
|
||||||
|
app.get("/api/users").expectStatus(200);
|
||||||
|
|
||||||
|
app.request()
|
||||||
|
.header("Authorization", "Bearer " + token)
|
||||||
|
.json("{\"name\":\"bob\"}")
|
||||||
|
.post("/api/users")
|
||||||
|
.expectStatus(201);
|
||||||
|
```
|
||||||
|
|
||||||
|
`json(...)` sets the body and `content-type: application/json`. `send(method, path)` reaches any
|
||||||
|
verb, including the ones Flash adds beyond RFC 9110 (`PURGE`, `QUERY`).
|
||||||
|
|
||||||
|
Assertions chain, and every failure message carries the request line, status and body:
|
||||||
|
|
||||||
|
```
|
||||||
|
expected: <200> but was: <404>
|
||||||
|
request: GET /api/users
|
||||||
|
status: 404
|
||||||
|
body: {"status":404,"error":"Not Found"}
|
||||||
|
```
|
||||||
|
|
||||||
|
`status()`, `body()`, `headers()` and `header(name)` are there for anything the assertions do
|
||||||
|
not cover.
|
||||||
|
|
||||||
|
## Replacing services
|
||||||
|
|
||||||
|
`mock` installs its replacements as the **last** extension, after everything the application and
|
||||||
|
its own extensions declare, so a fake always wins:
|
||||||
|
|
||||||
|
```java
|
||||||
|
FlashTest.of(new BlogApp())
|
||||||
|
.mock(UserService.class, new InMemoryUserService())
|
||||||
|
.mock(Clock.class, Clock.fixed(instant, ZoneOffset.UTC));
|
||||||
|
```
|
||||||
|
|
||||||
|
Any object is accepted — a hand-written fake, or a Mockito mock you created yourself. This module
|
||||||
|
depends on no mocking library.
|
||||||
|
|
||||||
|
Prefer a constructor parameter when the application already takes one:
|
||||||
|
`new AdminApi(upstreamUri)` needs no override at all. `mock` is for what you do not control —
|
||||||
|
services declared inside `configure`, or provided by an installed extension.
|
||||||
|
|
||||||
|
Calling `mock` after the server has started throws.
|
||||||
|
|
||||||
|
## More than one server
|
||||||
|
|
||||||
|
`FlashTest` is an ordinary object in a field, so a class can hold as many as it needs and wire one
|
||||||
|
from another in plain Java. Booting is lazy — reading `baseUri()` starts that server on the spot —
|
||||||
|
so declaration order does the wiring, with no `@Order` and no reliance on JUnit's extension
|
||||||
|
ordering:
|
||||||
|
|
||||||
|
```java
|
||||||
|
@RegisterExtension static FlashTest auth = FlashTest.of(new FakeOidcApp());
|
||||||
|
@RegisterExtension static FlashTest api = FlashTest.of(new BlogApp(auth.baseUri()));
|
||||||
|
```
|
||||||
|
|
||||||
|
Because binding happens when the app is created rather than when it starts, an application can
|
||||||
|
even be configured against its own address:
|
||||||
|
|
||||||
|
```java
|
||||||
|
FlashTest.of(app -> app.install(new McpExtension(
|
||||||
|
McpConfig.builder("srv").resourceIdentifier("http://127.0.0.1:" + app.port() + "/mcp").build())));
|
||||||
|
```
|
||||||
|
|
||||||
|
A server nothing touches is never booted.
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
A `static` field boots once for the test class; a non-static field boots a fresh app for every
|
||||||
|
test. That is stock JUnit field semantics — the isolation switch is the keyword, not an option:
|
||||||
|
|
||||||
|
```java
|
||||||
|
@RegisterExtension static FlashTest shared = FlashTest.of(new BlogApp()); // one boot per class
|
||||||
|
@RegisterExtension FlashTest fresh = FlashTest.of(new BlogApp()); // one boot per test
|
||||||
|
```
|
||||||
|
|
||||||
|
A class-scoped server shares state across its tests, including anything a `mock` fake accumulates.
|
||||||
|
Reset it in `@BeforeEach`, or use an instance field.
|
||||||
|
|
||||||
|
## WebSockets
|
||||||
|
|
||||||
|
```java
|
||||||
|
try (FlashWebSocket socket = app.ws("/live")) {
|
||||||
|
socket.sendText("hello");
|
||||||
|
assertEquals("echo:hello", socket.awaitText(Duration.ofSeconds(2)));
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Incoming text is queued as it arrives, so a message that lands before `awaitText` is called is not
|
||||||
|
lost. `awaitClose(timeout)` returns the server's close status code. `close()` never throws — a
|
||||||
|
connection the server already dropped must not mask the failure the test was reporting.
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
`profile` customises the `FlashConfiguration` — timeouts, HTTP/2 switches, buffer sizes:
|
||||||
|
|
||||||
|
```java
|
||||||
|
FlashTest.of(new BlogApp()).profile(cfg -> cfg.http2CleartextEnabled(true));
|
||||||
|
```
|
||||||
|
|
||||||
|
Host, port and the shutdown drain window are stamped **after** the profile runs, so it cannot
|
||||||
|
break the harness. `listener(...)` and `tls(...)` are rejected — see [limits](limits.md).
|
||||||
|
|
||||||
|
## Teardown
|
||||||
|
|
||||||
|
The client is cancelled before the server stops. `HttpClient` holds keep-alive sockets open and
|
||||||
|
`ServerLifecycle.stop()` spins until the last one closes or the drain window expires, so the
|
||||||
|
default 15s drain would otherwise be paid on every test class. The harness pins it to 250ms and
|
||||||
|
uses `shutdownNow()` rather than `close()`, which blocks until every operation completes and would
|
||||||
|
hang on a leaked WebSocket.
|
||||||
|
|
||||||
|
Service cleanup registered with `FlashContext.onClose` runs after the drain, so a pooled
|
||||||
|
`DataSource` is released between test classes rather than at JVM exit.
|
||||||
|
|
||||||
|
## See also
|
||||||
|
|
||||||
|
- [limits.md](limits.md) — what the harness deliberately cannot do, and what to use instead
|
||||||
@@ -0,0 +1,79 @@
|
|||||||
|
# Limits
|
||||||
|
|
||||||
|
What `flash-testing` deliberately cannot do, why, and what to use instead. Each of these is a
|
||||||
|
consequence of a real constraint, not an unfinished feature.
|
||||||
|
|
||||||
|
## TLS
|
||||||
|
|
||||||
|
`profile(cfg -> cfg.tls(...))` is rejected. The harness serves plaintext on loopback and hands you
|
||||||
|
a client bound to `http://127.0.0.1:<port>`; a TLS listener would leave that base URI pointing at
|
||||||
|
the wrong scheme, and the client would need the test certificate in a trust store.
|
||||||
|
|
||||||
|
**Instead:** build the app directly with `FlashApp.create(...)` and a raw `SSLSocket`, as
|
||||||
|
`HttpServerTlsTest` does. `port(0)` plus `FlashApp.port()` still removes the free-port dance.
|
||||||
|
|
||||||
|
Lifting this is the only limit here worth reconsidering, and only if application-level TLS testing
|
||||||
|
is actually wanted — transport-level TLS is already covered by the core suite.
|
||||||
|
|
||||||
|
## HTTP/2
|
||||||
|
|
||||||
|
`java.net.http` reaches cleartext h2 through an `Upgrade:` handshake. Flash implements HTTP/2
|
||||||
|
cleartext by **prior knowledge only** — a deliberate choice recorded in the root README — so the
|
||||||
|
harness client cannot negotiate h2 against a plaintext Flash listener. Over TLS it would work
|
||||||
|
through ALPN, but TLS is unavailable per the section above.
|
||||||
|
|
||||||
|
**Instead:** the `flash/src/test/.../http2/` suites drive h2 frames over raw sockets. That is the
|
||||||
|
right tool for protocol behaviour anyway.
|
||||||
|
|
||||||
|
## WebSocket over HTTP/2
|
||||||
|
|
||||||
|
`java.net.http.WebSocket` does not negotiate RFC 8441 extended CONNECT, so `FlashWebSocket` always
|
||||||
|
speaks the HTTP/1.1 upgrade. Flash supports both, but this client can only exercise one.
|
||||||
|
|
||||||
|
**Instead:** `WebSocketOverH2Test` and `WebSocketParityTest` frame extended CONNECT by hand.
|
||||||
|
|
||||||
|
## Malformed requests
|
||||||
|
|
||||||
|
Every request goes through `java.net.http`, which structurally cannot emit an invalid request
|
||||||
|
line, a bad header block, a smuggled `Content-Length`, or a chunked *request* body on demand. That
|
||||||
|
is a feature for application testing and a blocker for parser testing.
|
||||||
|
|
||||||
|
**Instead:** `RequestParserSecurityTest`, `RequestParserFuzzTest` and the raw-socket half of
|
||||||
|
`HttpServerTest` write bytes directly. A harness that could send malformed requests would just be
|
||||||
|
a socket.
|
||||||
|
|
||||||
|
## Response framing
|
||||||
|
|
||||||
|
`java.net.http` transparently decodes chunked responses and hides connection reuse, so
|
||||||
|
`Transfer-Encoding: chunked` and `Connection: keep-alive` are not observable through
|
||||||
|
`FlashResponse`.
|
||||||
|
|
||||||
|
**Instead:** `HttpServerTest` keeps raw sockets for exactly those assertions, taking its port from
|
||||||
|
a `FlashTest` field so the class still boots once. Mixing the two styles in one class is the
|
||||||
|
intended pattern, not a workaround.
|
||||||
|
|
||||||
|
## The `flash` core module
|
||||||
|
|
||||||
|
`flash-testing` depends on `flash`, so `flash`'s own tests cannot depend on `flash-testing` —
|
||||||
|
Maven rejects module cycles regardless of scope.
|
||||||
|
|
||||||
|
**Instead:** core tests use `FlashApp.create(...)` with `port(0)` and read `port()` back. Every
|
||||||
|
core suite already does this. Unblocking it would need a third module depending on both, which is
|
||||||
|
not worth it for the two suites that would benefit.
|
||||||
|
|
||||||
|
## Scoped services
|
||||||
|
|
||||||
|
`mock` writes to the app's root `FlashContext`. `FlashContext.require` checks its own bindings
|
||||||
|
before its parent's, so a service declared inside a `mount(...)` scope's child context shadows the
|
||||||
|
root and is **not** reachable from `mock`.
|
||||||
|
|
||||||
|
**Instead:** declare the service on the app rather than inside the scope, or assert against the
|
||||||
|
real one. Child-context targeting would be a small addition if a scoped service ever needs faking.
|
||||||
|
|
||||||
|
## Shutdown draining
|
||||||
|
|
||||||
|
The harness pins `shutdownDrainTimeoutMs` to 250ms after any profile runs, so a test cannot
|
||||||
|
exercise graceful-drain behaviour through it.
|
||||||
|
|
||||||
|
**Instead:** `ServerLifecycleGracefulShutdownTest` builds its app directly. A profile escape hatch
|
||||||
|
would be easy to add if this ever comes up twice.
|
||||||
Reference in New Issue
Block a user