docs: add justfile and rewrite README/CLAUDE.md for service architecture

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-05 15:34:21 +02:00
parent 86c4440576
commit 6babaa0166
4 changed files with 93 additions and 65 deletions
+42 -49
View File
@@ -4,64 +4,57 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
## What this is
A CLI that looks up Swedish phone numbers ("who is calling me?") by scraping
reverse-lookup sites. Old codebase: Rust edition 2018, reqwest 0.9 (synchronous
API), insta 0.11.
A self-hosted HTTP service that looks up Swedish phone numbers ("who is
calling me?") by scraping reverse-lookup sites. Providers are WASM components
(Component Model / WASI p2) loaded from a directory at startup; the host does
all fetching and caching. Design spec:
`docs/superpowers/specs/2026-06-05-wasm-provider-service-design.md`.
## Commands
```bash
cargo build
cargo run -- 0700000000 # query a number (hitta.se built in)
cargo run -- -d definitions/vem_ringde.toml 0700000000 # add TOML-defined probes
cargo run -- -o 0700000000 # open probe URLs in browser (macOS `open`)
cargo test # all tests (insta snapshot tests)
cargo test probe::hitta # one module
cargo test test_0104754350 # one test
cargo +nightly fmt # always nightly, not stable
cargo clippy
just test # build components + run all tests (preferred)
just run # build components + run the service
just build # release build of everything
cargo test -p whoareyou-provider-hitta # provider parser tests (native, no WASM)
cargo test -p whoareyou-server --test component # WIT-boundary integration test
cargo +nightly fmt # always nightly, not stable
cargo clippy --workspace
./fetch-fixture <number> # refresh an HTML fixture from hitta.se
```
Tests are inline-snapshot tests (`assert_yaml_snapshot!(..., @r###"..."###)`)
against checked-in HTML fixtures in `fixtures/<provider>/<number>.html` — no
network needed. Refresh/add fixtures with `./fetch-fixture <number>` (requires
`http`/httpie); it fetches the number from all five sites into `fixtures/`.
The integration test needs the component built first — run via `just test`,
or `cargo build --release --target wasm32-wasip2 -p whoareyou-provider-hitta`
before bare `cargo test`.
## Architecture
Everything revolves around the `Probe` trait (`src/probe.rs`): `provider()`,
`uri(number)`, `fetch(number)`, `parse(html) -> Result<Entry, ()>`.
Two kinds of probes:
1. **Hard-coded**: `Hitta` (`src/probe/hitta.rs`) — extracts the
`__NEXT_DATA__` JSON blob via regex and deserializes it with serde. Always
registered in `main.rs`.
2. **Declarative**: `Definition` (`src/definition.rs`) — generic scraper
configured by a TOML file (`definitions/*.toml`) with CSS selectors for
`messages`, `history`, and `comments` (each comment has optional
`date_time`/`title`/`message` sub-selectors). The URL `path` is a
tinytemplate string with `{ number }`. Loaded at runtime via `-d`.
Flow in `main.rs`: build probe list → for each probe, check the cache
(`Context` in `src/context.rs`, bincode files under the platform cache dir
with a 1-day TTL) → otherwise `fetch()` and cache → `parse()` into an `Entry`
(`src/entry.rs`) → `Display` it.
- `wit/provider.wit` — the provider contract (`metadata`/`requests`/`parse`).
Components are pure: no network, no filesystem. The HOST fetches URLs.
- `crates/providers/hitta` — parse logic in `parser.rs` is plain Rust,
unit-tested natively against `fixtures/hitta/*.html`; `component.rs` is
thin WIT glue, compiled only for `wasm32` (`cargo test` never touches WASM
here). hitta.se serves Next.js App Router pages — data lives in RSC flight
payloads (`self.__next_f.push`), NOT `__NEXT_DATA__` (that's the dead 2019
format kept in old fixtures as a Failed-path regression case).
- `crates/server` — lib + thin bin. `service.rs` holds the `ProviderHandle` +
`Fetch` traits and `LookupService` (moka cache, TTL 24h, key
`provider:number`; fetch failures are NOT cached). `wasm.rs` implements
`ProviderHandle` over wasmtime (fresh Store per call, epoch deadline ≈5s
`spawn_epoch_thread` must run once at startup or runaway guests hang
instead of trapping). `http.rs` is axum: `GET /api/v1/number/{number}`,
`GET /healthz`.
## Gotchas
- `src/probe/{eniro,konsument_info,telefonforsaljare,vem_ringde}.rs` are
**orphaned**`probe.rs` only declares `mod hitta;`. Those providers were
superseded by the TOML definitions in `definitions/`. Don't "fix" them or
expect them to compile; they're kept as reference.
- `_build.rs` is intentionally disabled (underscore prefix, not referenced in
Cargo.toml) — an abandoned attempt at generating fixture tests.
- `definitions/vem_ringde.yml` is an experimental YAML variant of the TOML
definition, but `main.rs` only parses TOML (`toml::from_slice`).
- The `Filter` enum in `src/definition.rs` has no variants yet — `filters` is
parsed from definitions but unimplemented (commented-out loops in `parse`).
- insta 0.11 is old: the macro is `assert_yaml_snapshot!` and inline-snapshot
updates need a matching old `cargo-insta`; it's usually easier to update the
inline `@r###"..."###` literals by hand.
- Components build with plain `cargo build --target wasm32-wasip2` — no
cargo-component. Output name uses underscores:
`whoareyou_provider_hitta.wasm`; the justfile copies it to
`components/hitta.wasm` (gitignored).
- One provider failing maps to a per-provider `status` in the JSON response —
never a non-200 for the whole lookup. `parse_failed` in logs (WARN) means a
site changed its markup: refresh a fixture with `./fetch-fixture` and fix
the parser.
- `ParseError::NoData` vs `Failed`: a fetched page with no phone data is
NoData (normal); a page that doesn't match the expected structure is Failed
(scraper rot). Don't conflate them.