minor: yugioh support added

docs/AGENTS.md

@@ -11,7 +11,8 @@ Long-form contributor documentation that lives outside the source tree.
 - `dow-doc-build-locally.md` — complete local build/setup reference for Windows and Linux, including dependency management and troubleshooting.
 - `intro-to-new-developers.md` — onboarding map for new contributors: architecture, folder responsibilities, guardrails, anti-patterns, and links to deeper docs.
 - `testing-and-test-code-of-conduct.md` — testing workflow plus expected standards for writing and maintaining deterministic, hermetic, behavior-focused tests.
-- `assets-and-info-apis.md` — reference for the external info APIs (set metadata) and asset APIs (card preview images) used by the Magic and Pokemon modules, plus the runtime flow through `SetService` / `CardPreviewService` and the error-surface conventions.
+- `assets-and-info-apis.md` — reference for the external info APIs (set metadata) and asset APIs (card preview images) used by the Magic, Pokémon, and Yu-Gi-Oh! modules, plus the runtime flow through `SetService` / `CardPreviewService`, shared HTTP defaults (`CprHttpClient`, `Accept: */*`), per-game card-back fallbacks (URLs + bundled `ygo_card_back.png`), and error-surface conventions.
+- `caching.md` — dedicated reference for preview-byte caching tiers (`CardPreviewService` LRU + `LocalPreviewByteCache`), cache keys and eviction, HTTP session reuse via `CprHttpClient`, and explicit non-goals (no error caching).
 - `README.md` — index page that clusters docs by area and links to all documents in this directory.
 
 ## Subdirectories
@@ -27,7 +28,7 @@ Long-form contributor documentation that lives outside the source tree.
 ## Required follow-ups
 
 - After changing per-game seams in `core/` (e.g. `IGameModule`, `ISetSource`, `ICardPreviewSource`, `CollectionService`, `SetService`, `CardPreviewService`, `ImageService`) you **must** update `adding-a-new-game.md` to keep the canonical procedure in sync. The same applies to the UI seams (`IGameView`, `BaseCardListPanel`, `BaseCardEditDialog`, `BaseSelectedCardPanel`) and the composition-root wiring in `app/main.cpp`.
-- After changing the Magic or Pokemon set/preview adapters (`MagicSetSource`, `MagicCardPreviewSource`, `PokemonSetSource`, `PokemonCardPreviewSource`) — endpoints, response parsing, name/number normalization, or the info-vs-asset split — you **must** update `assets-and-info-apis.md` so the API reference matches the live behavior.
+- After changing any game's set/preview adapters (`MagicSetSource`, `MagicCardPreviewSource`, `PokemonSetSource`, `PokemonCardPreviewSource`, `YuGiOhSetSource`, `YuGiOhCardPreviewSource`) — endpoints, response parsing, name/number normalization, or the info-vs-asset split — you **must** update `assets-and-info-apis.md` so the API reference matches the live behavior.
 - After bumping a key dependency (`nlohmann/json`, `cpr`, `wxWidgets`, `doctest`) in a way that changes a public API used in the guide's examples, update those examples.
 - After adding a new file under `docs/` (or a new entry under `docs/assets/images/`) you **must** add it to the file list above **and** to `README.md` so the index stays complete.
 - Do **not** rename, move, or split this file without first updating every other `AGENTS.md` that points at it (root, `core/`, `ui_wx/`, `app/`, `tests/`).

docs/README.md

@@ -22,5 +22,9 @@ This folder contains contributor documentation for Card Collection Manager 3. St
 
 - [adding-a-new-game.md](adding-a-new-game.md): canonical end-to-end procedure for adding a new game module across `core/`, `ui_wx/`, and `app/`.
 
-- [assets-and-info-apis.md](assets-and-info-apis.md): external info and asset APIs used by Magic/Pokemon modules and their runtime purpose.
+- [assets-and-info-apis.md](assets-and-info-apis.md): external info and asset APIs used by Magic, Pokémon, and Yu-Gi-Oh! modules, preview fallback URLs / bundled YGO card-back asset, and shared HTTP behavior (`CprHttpClient`).
+
+## Performance & Caching
+
+- [caching.md](caching.md): preview image caching (in-memory LRU, on-disk byte cache, HTTP connection reuse), lookup order, keys, eviction, and what is intentionally not cached.
 

docs/adding-a-new-game.md

@@ -37,6 +37,9 @@ A few traps to plan around now, before you write code:
 - **Lookup precision.** Some APIs return many ambiguous matches when you query by name only and require the set id (and sometimes the collector number) to disambiguate. Decide up front which fields make a search reliable enough to take the first result.
 - **URL encoding.** All query strings must be RFC 3986 percent-encoded before they reach `IHttpClient::get` (`cpr::Url` does **not** re-encode). The Magic/Pokemon implementations have a private `urlEncode` helper you can copy.
 - **Collector-number normalization.** Pokemon stores `4/102` but the API only accepts `4`. Whichever convention your domain type uses, normalize it inside `buildSearchUrl` so the wire format is whatever the API actually expects. Mismatches here produce empty result sets, which then look identical to "no preview available" and are very tedious to debug.
+- **Name-matching strictness.** Some APIs reject strict exact-name parameters for real-world card spelling variants (e.g. hyphenation/punctuation differences). If your provider supports fuzzy-name search, prefer that for the first request, then disambiguate in `parseResponse` using set/print metadata.
+- **400 fallback strategy.** If adding optional set filters can produce request validation errors (`HTTP 400`), add a second request path that retries without the risky filter and keeps disambiguation local in `parseResponse`.
+- **Image variant priority.** If the provider returns both cropped art and full-card images, prefer the full-card URL for selected-card preview. Use cropped variants only as fallback.
 
 ### 1.3 Flag icons
 
@@ -138,6 +141,8 @@ Mirror `core/include/ccm/games/pokemon/PokemonCardPreviewSource.hpp`. The header
 - `static std::string buildSearchUrl(std::string_view name, std::string_view setId, std::string_view setNo);`
 - `static Result<std::string> parseResponse(const std::string& body);`
 
+If your game benefits from edit-dialog metadata helpers (for example auto-detecting collector number / rarity), you can opt in to `ICardPreviewSource::detectFirstPrint(...)` and route it via `CardPreviewService::detectFirstPrint(...)`. If you need to enumerate multiple upstream printings (for example Yu-Gi-Oh! “Next” cycling between alternate `set_code` or `set_rarity` values), also override `ICardPreviewSource::detectPrintVariants(...)` and expose it through `CardPreviewService::detectPrintVariants(...)`. Keep both optional per game — default behavior should remain an explicit unsupported error.
+
 Both `buildSearchUrl` and `parseResponse` are static and pure on purpose: every URL-encoding and JSON-shape rule is testable without HTTP. Common edge cases your tests must cover:
 
 - Names with spaces, punctuation, or non-ASCII characters (percent-encoding correctness).
@@ -328,6 +333,10 @@ In the constructor:
 
 The reference implementation is `ui_wx/src/PokemonCardEditDialog.cpp`.
 
+When an extra field is from a controlled vocabulary (rarity tiers, print types, etc.), prefer a dropdown (`wxChoice`) over free text to keep list/filter values consistent and reduce user-input variants.
+
+If the external API expects a full print code (e.g. `LOB-001`) but users mostly edit only the numeric suffix, expose a numeric input plus a read-only derived preview (for example `(LOB-001)`) and compose/decompose the stored full value in `readExtraFromCard()` / `writeExtraToCard()`.
+
 ### 5.6 `<Name>GameView`
 
 This is the polymorphic glue between the new game's panels and the rest of the app. Create:
@@ -452,6 +461,8 @@ Run, in order, from the workspace root. Do not skip any step.
 These do not match a single seam in this guide but are worth calling out explicitly.
 
 - **Stale set caches.** Each `IGameView` caches `std::vector<Set> setsCache_`. After `onUpdateSets` succeeds, refresh the cache (assign the new vector). The reference implementations do this.
+- **Set ordering drift.** Keep set lists sorted by release date not only in `<Name>SetSource::parseResponse`, but also at UI consumption points (preloaded/cached vectors passed to `BaseCardEditDialog`). Older on-disk cache data or future parser changes can otherwise surface unsorted set pickers.
+- **Auto-detect feature scope.** Treat print auto-detect as a per-game capability. Do not assume every game supports it; gate UI affordances behind game-specific dialog logic and source opt-in.
 - **`signed_` / `signed`.** The C++ field is `signed_`; the JSON key is `"signed"`. This is intentional and must not be changed. The same convention applies to any new field where the natural name collides with a C++ keyword — pick a trailing-underscore C++ name and an unaliased JSON key.
 - **Spacer column index.** `BaseCardListPanel` reserves index `0` for a hidden zero-width spacer column (MSW comctl32 image-list gutter workaround). Real columns start at index `1`. If you ever need to call into `wxListCtrl` directly from a derived panel (you should not), remember this.
 - **Preview-fetch threading.** The async preview fetch in `BaseSelectedCardPanel` uses a `shared_ptr<State>` + `std::atomic alive` + `std::atomic currentGen` triple. Do not capture `this` raw in any background work you add to a new game's selected panel; copy that pattern verbatim.

docs/assets-and-info-apis.md

@@ -26,27 +26,92 @@ Used by `PokemonCardPreviewSource` to search by `name` plus optional `set.id` an
 
 The Pokemon source also normalizes collector numbers before request build. For example, `4/102` is reduced to `4` because the remote query expects only the printed number component.
 
+## Yu-Gi-Oh! APIs (Yugipedia + YGOPRODeck)
+
+Yu-Gi-Oh! splits its remote calls across two upstreams. **Yugipedia** is the primary preview source because it hosts actual per-printing card scans; **YGOPRODeck** continues to drive set listings and the auto-detect-first-print helper, plus a last-resort image fallback.
+
+Upstream documentation:
+- [Yugipedia MediaWiki API help](https://yugipedia.com/api.php?action=help) (standard MediaWiki action API; we only need `prop=imageinfo`).
+- [Yu-Gi-Oh! API Guide — YGOPRODeck](https://ygoprodeck.com/api-guide/). CCM3 uses **v7** endpoints only.
+
+### Info API: YGOPRODeck `cardsets.php`
+
+`https://db.ygoprodeck.com/api/v7/cardsets.php`  
+Used by `YuGiOhSetSource`. The response is a top-level JSON array. Each object maps `set_code` → internal `Set.id`, `set_name` → `Set.name`, and `tcg_date` → `Set.releaseDate` with `-` rewritten to `/` for consistency with other games’ date strings. Results are sorted ascending by `releaseDate`.
+
+### Asset API: Yugipedia `api.php` (primary)
+
+`https://yugipedia.com/api.php?action=query&prop=imageinfo&iiprop=url&titles=...`  
+Used by `YuGiOhCardPreviewSource::fetchImageUrl` for the actual per-printing card scan. Yugipedia is the only public source we have found that distinguishes art between same-passcode reprints (LOB Blue-Eyes vs SDK Blue-Eyes, for example), and uses a **deterministic file-name convention** of the shape `<Slug>-<SET>-<REGION>-<RARITY>-<EDITION>[-Misc].<png|jpg>` per [Yugipedia’s image policy](https://yugipedia.com/wiki/Yugipedia:Image_policy).
+
+The UI passes a positional tuple in `setNo` of the form `set_code||rarity||edition` (for example `SDK-001||Ultra Rare||UE`); the source splits on `||` before building filenames. Field meanings:
+
+- `set_code` — full code as printed (`LOB-005`, `SDK-001`, `RA04-EN001`). Everything before the first `-` becomes the Yugipedia `<SET>` slot (`LOB`, `SDK`, `RA04`).
+- `rarity` — full English rarity name from the edit dialog (`Ultra Rare` → `UR`, `Quarter Century Secret Rare` → `QCScR`, …). The mapping table lives in `rarityCodeFor(...)`. Unknown values fall back to the rarity-less filename pattern.
+- `edition` — `1E` when the user marked the card as 1st Edition, otherwise `UE` (Unlimited).
+
+`buildCandidateFilenames(...)` then produces a priority-ordered list:
+
+1. Printed edition first (`1E` then `UE`, or `UE` then `1E` for non-first), with `LE` last for promo-style prints.
+2. English regions only — `EN`, then `NA`, then `EU`, then `AU`. **Yugipedia is queried with English regions regardless of the card’s stored Language**, so a German-language card still shows the English scan; this matches the user-visible policy in the edit dialog and avoids querying region-specific scans that are sparser on Yugipedia.
+3. Both `.png` and `.jpg` extensions per combo (older LOB-era uploads are `.jpg`, modern reprints are `.png`).
+4. A rarity-less fallback round so cards with unknown rarities still resolve in single-rarity sets.
+
+`buildYugipediaQueryUrl(...)` joins all candidates into one MediaWiki **batch query** (`titles=File:A|File:B|...` URL-encoded), so the entire list resolves in a single HTTP call. `parseYugipediaResponse(...)` walks the candidate list in order and returns the URL of the first filename that came back with `imageinfo[0].url`; missing files come back with `"missing": ""` and are skipped.
+
+### Asset API: YGOPRODeck `cardinfo.php` (fallback + auto-detect)
+
+`https://db.ygoprodeck.com/api/v7/cardinfo.php?fname=...`  
+Used in two situations:
+
+1. **Last-resort preview fallback.** If Yugipedia returns no candidate match (cards without an English scan yet, transient API errors), `fetchImageUrl` falls through to `parseFallbackImageUrl(...)`, which prefers an exact-name match in YGOPRODeck’s `data[]`, otherwise the first row, and returns the first entry from `card_images[0]`. This is intentionally **not** filtered by `cardset=`: when YGOPRODeck applies that filter, it reorders `card_images` so alt-art passcodes are promoted ahead of the standard art, which would re-introduce the “wrong artwork” bug we fixed by switching to Yugipedia.
+
+2. **Auto-detect print (`detectFirstPrint` / `detectPrintVariants`, Yu-Gi-Oh! edit dialog).** Uses `fname=` plus **`cardset=`** set to the **display set name** from the picker (must match `card_sets[].set_name` in the payload). If that request fails (for example unknown set label), it retries with **`fname=` only** and still filters prints by preferred `set_name`. `YuGiOhCardPreviewSource::parsePrintVariants(...)` walks every `(set_code, set_rarity)` pair for rows whose **card name matches exactly** (case-insensitive) so the dialog can offer ring-buffer **Next** controls: one cycles distinct `set_code` values for that name+set (and resets rarity to the first upstream rarity for the newly selected code); another cycles distinct `set_rarity` values for the **current** `set_code` without changing the collector number. Shared HTTP and parsing rules live beside `parseFirstPrint`. When the dialog passes both an exact card name and a display `set_name`, an upstream miss on that label returns an error instead of falling back to unfiltered `card_sets[]` rows — otherwise unrelated products (same card name, different `set_name` on each printing) could be blended into one bogus variant list. The Yu-Gi-Oh! edit dialog additionally drops European alternate `set_code` rows that use the `-E###` pattern (single `E` before digits, e.g. `LOB-E003`) when the card language is **English**, because YGOPRODeck keeps those alongside NA numbering (`LOB-005`) under the same English `set_name`; it also collapses `LOB-005`-style and `LOB-EN005`-style codes to one **Next** slot via digit-tail matching (`ccm/util/YuGiOhPrintingSlot.hpp`). No image data is needed for this path, so Yugipedia is not consulted.
+
+YGOPRODeck publishes rate limits and asks clients to cache responses and avoid abusive hotlinking; treat failures after burst traffic as an upstream policy signal, not an app bug. Yugipedia’s MediaWiki API is similarly polite — one batched call per preview lookup keeps us well under any normal threshold.
+
 ## Runtime Flow In CCM3
 
-The app uses the same flow for both games:
+The app uses the same flow for every game that registers a module:
 
 - `SetService` asks the game's `ISetSource` (info API) for the latest set list.
 - `CardPreviewService` asks the game's `ICardPreviewSource` (asset API) for a preview image URL.
 - `CardPreviewService` performs a second HTTP GET to that URL and returns raw bytes to the UI layer.
-- If preview lookup fails (or returns empty bytes), the UI fetches a per-game fallback card-back image URL through `CardPreviewService::fetchImageBytesByUrl(...)` and shows that image in the selected-card preview panel.
+- If preview lookup fails (or returns empty bytes), the UI loads a **per-game card-back fallback** in `BaseSelectedCardPanel`: Magic / Pokémon call `CardPreviewService::fetchImageBytesByUrl(...)` against fixed HTTPS URLs. Yu-Gi-Oh! tries two Yugipedia URLs (thumbnail then full `Back-EN.png`), then reads **`assets/ygo_card_back.png`** next to the executable if both downloads fail (bundled asset; see `app/CMakeLists.txt`).
 
-Current fallback image URLs (kept in `BaseSelectedCardPanel` for CCM2 parity):
+### Caching And Connection Reuse
+
+See [caching.md](caching.md) for a dedicated reference on preview cache tiers, internal keys, eviction, clearing, and HTTP session reuse.
+
+Three mechanisms reduce preview latency for **all** games (Magic, Pokemon, Yu-Gi-Oh!). In addition, the shared HTTP session speeds **every** `IHttpClient::get` call (including set-list fetches), not only previews:
+
+- **In-memory preview LRU** (`CardPreviewService`). Successful `fetchPreviewBytes` results are cached keyed by `(game, name, setId, setNo)`; successful `fetchImageBytesByUrl` results are cached keyed by URL (used for the per-game card-back fallback). Re-selecting a previously viewed row is decode-only — no HTTP at all. The cache is bounded by `CardPreviewService::kCacheCapacity` (currently 128 entries) and uses a list+map LRU under a mutex (the preview pipeline is invoked from a worker thread in `BaseSelectedCardPanel`). **Source errors are split** by `PreviewLookupError::Kind`: `NotFound` (the upstream answered cleanly that the record has no image) is *negative-cached* in this tier so subsequent selections short-circuit without HTTP, while `Transient` (HTTP/network/parse failures) is **never** cached so a brief outage cannot permanently disable a card's preview.
+- **Persistent disk byte cache** (`LocalPreviewByteCache`, port `IPreviewByteCache`). Wraps the in-memory tier with an on-disk store under `<exeDir>/.cache/preview-cache/` — pinned **next to the executable**, in the same scope as `config.json`, **not** under the user-configurable `Configuration.dataStorage` path. The cache stays put when the user reconfigures or relocates their collection data, and it is not part of the user's data directory backups; it is install-scoped, not collection-scoped. Both positive previews and `NotFound` verdicts survive an app restart. Each entry is a mutually-exclusive `<hash>.bin` (positive payload) or `<hash>.neg` (negative marker) plus a `<hash>.idx` sidecar containing the original key — load-time mismatch on the sidecar treats the entry as a miss, so a hash collision degrades to a one-time HTTP refetch instead of serving the wrong card's bytes (or the wrong card's "no image" verdict). Hashing is FNV-1a 64-bit (no crypto dependency). The cache is bounded by total `.bin` payload bytes (default `kDefaultMaxBytes = 64 MiB`) and evicts oldest entries by mtime when a new write would exceed the cap; reading an entry touches its mtime so frequently-viewed cards survive eviction. Negative `.neg` markers are tiny and not counted against the cap — their count is naturally bounded by the user's actively-viewed records. Filesystem mutations route through `IFileSystem`; size and mtime queries (which the port does not expose) use `std::filesystem` directly inside the adapter. The persistent tier is **fire-and-forget on the way down** — every adapter operation swallows I/O errors so a flaky or full disk never breaks the preview path.
+- **Persistent HTTP session** (`CprHttpClient`). The adapter owns one long-lived `cpr::Session` (libcurl easy handle) for the lifetime of the app. Per-request configuration is limited to `SetUrl(...)`; headers, timeout, and redirect policy are configured once in the constructor. Default **`Accept: */*`** keeps JSON responses and raw image bodies working on the same session (avoid tying every GET to `application/json`). libcurl's connection pool keeps the TLS connection to each host warm, so repeat calls to `api.scryfall.com`, `api.pokemontcg.io`, `db.ygoprodeck.com`, `yugipedia.com`, and `ms.yugipedia.com` skip the TLS handshake. A `std::mutex` serializes callers — libcurl easy handles are not thread-safe, and the preview pipeline is single-flight per panel anyway.
+
+`CardPreviewService` consults the tiers in order **memory → disk → source/HTTP**. On a disk hit (positive *or* negative) the entry is promoted into the in-memory LRU so the next click on the same row never re-touches the disk cache. On HTTP success the bytes are written through to both tiers in one shot. On a `NotFound` source error the **negative** marker is written through to both tiers; on `Transient` source errors nothing is written, so the next selection retries cleanly.
+
+The combined effect on the preview path: first selection of a previously-unseen card pays one TLS handshake per *new* host this session (typically two hops for Yu-Gi-Oh!: `yugipedia.com` for the API, `ms.yugipedia.com` for the image), each subsequent fresh card on the same host skips the handshake, any re-selection of an already-viewed card is instant, after the first run with the disk cache populated **even a fresh app launch is decode-only for previously-seen cards** until eviction or a manual cache clear, and **records the upstream cleanly has no image for** stay "instant card-back" across restarts instead of re-paying the lookup every launch. Editing a lookup-relevant field of a record (name, set, setNo, or for Yu-Gi-Oh! the rarity / edition packed into setNo) changes the cache key automatically, so a fresh resolution attempt happens on the next click.
+
+To clear the persistent cache (for example to recover from a bad upstream image), delete the `<exeDir>/.cache/preview-cache/` subdirectory or the umbrella `<exeDir>/.cache/` folder. Note: the in-app "Reset" / data-storage-relocation flow does **not** touch this directory — the cache is install-scoped, not collection-scoped, so it is preserved across data-dir moves and only cleared by deleting the directory above explicitly (or by reinstalling / relocating the executable).
+
+For the full caching design and contributor rules see the dedicated [caching.md](caching.md).
+
+Fallback card-back sources (`BaseSelectedCardPanel`; Magic/Pokémon URLs match CCM2):
 
 - Magic: `https://gamepedia.cursecdn.com/mtgsalvation_gamepedia/f/f8/Magic_card_back.jpg`
-- Pokemon: `https://archives.bulbagarden.net/media/upload/1/17/Cardback.jpg`
+- Pokémon: `https://archives.bulbagarden.net/media/upload/1/17/Cardback.jpg`
+- Yu-Gi-Oh!: Yugipedia English TCG back — try `https://ms.yugipedia.com/thumb/e/e5/Back-EN.png/250px-Back-EN.png`, then `https://ms.yugipedia.com/e/e5/Back-EN.png`; if both fail, load `<exeDir>/assets/ygo_card_back.png` (shipped from `ui_wx/assets/ygo_card_back.png` at link time). `fallbackImageUrlForGame(Game::YuGiOh)` returns the thumbnail URL for helpers that only consult a single string.
 
 If a game module does not provide a preview source (`cardPreviewSource() == nullptr`), preview registration is skipped and the UI behaves as "no remote preview API available."
 
 ## Error Surface And Debugging Intent
 
-Both source types return `Result<T, std::string>` errors so failures cross boundaries without exceptions. In practice, this keeps failures debuggable by separating:
+All source types return `Result<T, std::string>` errors so failures cross boundaries without exceptions. In practice, this keeps failures debuggable by separating:
 
 - info API failures (bad set payload, schema mismatch, endpoint/network failure), and
 - asset API failures (query mismatch, no matching card, missing image fields, image download failure).
 
-When previews fail, verify request construction first (name sanitization, number normalization, percent encoding), then verify response shape assumptions (`data`, `image_uris`, `images.large`/`images.small`). If the fallback fetch succeeds, the panel intentionally shows the card-back image and the inline label `(image preview unavailable)`.
+When previews fail, verify request construction first (name sanitization, number normalization, percent encoding), then verify response shape assumptions: Scryfall (`data`, `image_uris`), Pokemon (`data`, `images.large`/`images.small`), Yu-Gi-Oh! Yugipedia (`query.pages.<id>.imageinfo[0].url` per filename, missing files tagged `"missing": ""`), Yu-Gi-Oh! YGOPRODeck fallback (`data`, `name`, `card_images`). If the UI fallback path succeeds (network card-back and/or bundled PNG), the panel shows the card-back image and the inline label `(image preview unavailable)`; only if every fallback fails does the preview stay empty with status text.
+
+For Yu-Gi-Oh! specifically, when a printing shows the wrong art compared with Yugipedia’s gallery, debug in this order: (1) verify the candidate list via `YuGiOhCardPreviewSource::buildCandidateFilenames(...)` against the actual file names on Yugipedia’s `Card_Gallery:<Card>` page; (2) confirm the dialog rarity name maps to the right code in `rarityCodeFor(...)` (extend the table when a new rarity surfaces); (3) confirm the `firstEdition` flag matches the printed edition stamp — the candidate ordering puts the printed edition first.

docs/caching.md

@@ -0,0 +1,167 @@
+#documentation #performance #network #ccm3
+
+# Caching In CCM3
+
+This document describes **what CCM3 caches**, **how lookups are ordered**, and **what is deliberately not cached**. It focuses on the card **preview image** path (remote APIs → raw bytes → UI decode), which is where most explicit caching lives. For upstream URL shapes and API roles, see [assets-and-info-apis.md](assets-and-info-apis.md).
+
+## Scope
+
+| Mechanism | What it stores | Survives restart? |
+|-----------|----------------|-------------------|
+| In-memory preview LRU (`CardPreviewService`) | Successful preview / fallback-image **bytes** + negative markers for "no upstream image" | No |
+| Disk preview byte cache (`LocalPreviewByteCache`) | Same bytes + negative markers, persisted under `<exeDir>/.cache/preview-cache/` | Yes |
+| HTTP session reuse (`CprHttpClient`) | libcurl **connections** (TLS + TCP keep-alive), not response bodies | No (process lifetime only) |
+
+Other persistence (for example `JsonSetRepository` after “Update sets”, `JsonCollectionRepository` for card JSON, `LocalImageStore` for **user-attached** scan files) is normal app data storage, not preview caching. Those layers are documented elsewhere via domain services; this page stays centered on **preview latency** and **repeat lookups**.
+
+### Cache directory layout
+
+The umbrella cache root is `<exeDir>/.cache/`, where `exeDir` is the directory containing the running `ccm3` executable — the same scope as `config.json`. It is reserved for any future computed-from-network caches (set-list response snapshots, etc.); the leading dot keeps it out of the way for users poking around the install folder. Today it contains a single subdirectory:
+
+- `<exeDir>/.cache/preview-cache/` — owned by `LocalPreviewByteCache`. Files inside are `<hash>.bin` / `<hash>.neg` / `<hash>.idx` triples (see below).
+
+The cache is deliberately **not** under the user-configurable `Configuration.dataStorage` path. The data-storage path is meant for the user's own data — collection JSON, attached scans, set lists — and is meant to be relocatable, syncable, and backup-friendly. Previews are downloaded artifacts from upstream APIs:
+
+- They must **not** follow the user's collection when the data-storage path is reconfigured at runtime (the cache would otherwise either rebuild from cold every time the user moves the dir, or pollute every chosen target with a recurring `.cache/` directory).
+- They must **not** be uploaded together with the user's collection if the user backs up / syncs / version-controls the data dir.
+- They are install-scoped, not collection-scoped: a fresh install elsewhere on disk should start cold, and uninstalling / moving the executable should leave nothing stale behind.
+
+Pinning the cache to `exeDir` is what gives all three properties without per-flow plumbing. The trade-off is that the cache is **not** wiped by the in-app data-storage reset / relocation flow; if you genuinely want a clean slate (e.g. you suspect cache corruption), delete `<exeDir>/.cache/` manually.
+
+## Preview lookup order
+
+For `CardPreviewService::fetchPreviewBytes(game, name, setId, setNo)`:
+
+1. **In-memory LRU** — O(1) lookup. A positive entry returns the bytes immediately; a **negative** entry returns an error immediately (no network call). Either kind of hit moves the entry to the most-recently-used position.
+2. **Disk cache** (`IPreviewByteCache`, production: `LocalPreviewByteCache`) — on memory miss, look up on disk. A positive disk hit is **promoted** into the in-memory LRU and returned; a negative disk hit is likewise promoted into memory as a negative entry and returned as an error. So the next selection of the same card stays in-memory only.
+3. **Network** — ask `ICardPreviewSource` for an image URL, then `IHttpClient::get(url)` for bytes. On success, write through to **both** memory and disk tiers as a positive entry. **`CardPreviewService::fetchAndCache`** treats an HTTP **2xx with an empty body** as an error (no cache write) so empty payloads cannot populate the LRU as false positives.
+
+For `fetchImageBytesByUrl(url)` (used for per-game **card-back fallbacks** when preview lookup fails):
+
+- Same three-tier pattern, but the cache key is derived only from the URL. There is no negative-cache analogue here: the URL is a fixed constant, so any failure is by definition transient. Empty bodies are rejected the same way as on the preview-image GET path.
+
+## Negative caching: transient vs. permanent failures
+
+`fetchPreviewBytes` distinguishes two failure kinds via `PreviewLookupError::Kind`:
+
+- **`NotFound`** — the upstream answered cleanly that this exact record has no preview image. Examples:
+  - Scryfall search returned `data: []`, or the matched card has no top-level `image_uris`.
+  - Pokémon TCG search returned `data: []`, or the card has no `images` object / no `large` / `small` URL.
+  - Yu-Gi-Oh!: **both** Yugipedia and YGOPRODeck answered cleanly with no match (Yugipedia tagged every candidate filename `missing`, *and* YGOPRODeck returned an empty `data` array or no usable image variants).
+
+  These are **negative-cached** in both tiers. Subsequent selections of the same record return an error instantly, without any HTTP call. The user-visible effect is that the per-game card-back fallback shows up immediately on every click.
+
+  The cache key is `(game, name, setId, setNo)` (with Yu-Gi-Oh! also packing rarity and edition into `setNo`). Any **edit to a lookup-relevant field** of the record changes the key automatically, which means the negative entry no longer matches and a fresh network resolution attempt runs the next time the user clicks the row. So if the user fixes a typo, changes the set, switches a YGO printing's edition or rarity, etc., the new fingerprint guarantees a re-fetch — no manual cache clear needed.
+
+- **`Transient`** — we couldn't tell whether the record has an image because the upstream couldn't speak. Examples:
+  - HTTP / network / TLS / DNS failure.
+  - Malformed JSON, missing top-level fields (schema deviation that suggests an outage page rather than a real "no match" response).
+  - Yu-Gi-Oh!: **either** Yugipedia or YGOPRODeck failed at the network/parse layer. The cautious rule is that as soon as one upstream couldn't speak, the overall outcome is transient — we cannot conclude the record has no image, only that we couldn't reach the place that would tell us.
+
+  These are **never cached** (positive or negative). The next selection of the same row retries cleanly. This is the property that keeps a temporary connection drop from semi-permanently breaking previews.
+
+This split is the reason the preview path doesn't keep retrying every click for cards whose printing genuinely has no upstream scan, *and* the reason a brief loss of connectivity doesn't poison the cache with bogus "no image" markers.
+
+## Updating cached entries
+
+There is no explicit "refresh" or "invalidate" API on `CardPreviewService` — by design. Every way an entry's state can change is driven by **what already happens** in the system, so contributors don't have to reason about a side-channel mutation API. The full set of transitions is:
+
+### 1. Edit-driven invalidation (record changed → fresh lookup, automatic)
+
+The cache key for the preview path is `(game, name, setId, setNo)`. For Yu-Gi-Oh! the third slot also encodes rarity and edition, packed by `YuGiOhSelectedCardPanel::previewKey()` as `<setNo>||<rarity>||<1E|UE>`. The user editing **any** lookup-relevant field of a card record produces a **different cache key** for the resulting selection, which means:
+
+- Memory and disk lookups for the new key **miss** the old entry (positive or negative).
+- A fresh `ICardPreviewSource::fetchImageUrl` call runs.
+- The result is cached under the new key, leaving the old key's entry untouched but unreachable from the UI (it ages out via LRU / mtime eviction).
+
+Concretely: fix a typo in the card name → re-fetch. Switch the printing's set → re-fetch. Toggle `1E` ↔ `UE` on a YGO card → re-fetch the per-printing scan. **No manual cache clear needed**; the test `editing a lookup-relevant field invalidates the negative entry automatically` pins this behavior.
+
+If you add a new disambiguator (say a future "art treatment" flag), the rule is to pack it into one of the existing key slots (`setNo`'s `||`-separated tuple is the established hook) so this auto-invalidation continues to apply. Adding it as a side parameter that the cache key *doesn't* see would silently break the update story.
+
+### 2. Same-key positive ↔ negative state transitions
+
+If a previously cached entry's verdict flips upstream (Yugipedia uploads a missing scan, a Scryfall printing's `image_uris` get fixed, etc.) **and** the user re-encounters it under the same key, the next fetch decides:
+
+- **Positive → negative.** Source returns `NotFound` for a key that previously cached a positive entry: `cacheStoreNegative(key)` overwrites the in-memory entry's bytes with an empty payload and flips `negative=true`; on disk, `LocalPreviewByteCache::storeNegative` removes the existing `<hash>.bin` (releasing its bytes from the size cap) and writes a `<hash>.neg` marker.
+- **Negative → positive.** Source returns a real URL, `IHttpClient` returns bytes, `cacheStore(key, payload)` overwrites the existing in-memory entry with the new bytes and flips `negative=false`; on disk, `LocalPreviewByteCache::store` removes the existing `<hash>.neg` and writes the new `<hash>.bin`.
+
+Both paths preserve a key invariant: **`.bin` and `.neg` for the same hash are never co-resident**. `LocalPreviewByteCache` tests pin this down (`a later positive store overwrites an earlier negative entry`, `storeNegative for an existing positive entry replaces the bytes`).
+
+The "trigger" for these transitions in production is one of: the user edits the record back to a previous key (so the still-cached old entry surfaces and a network attempt then re-reaches the upstream), or the in-memory tier was cleared by an app restart and the disk-tier verdict is now stale. There is no time-based revalidation today; the design relies on the upstream answer being stable enough that a stale verdict only hurts until the natural transitions above kick in.
+
+### 3. Eviction-based aging (passive)
+
+- **In-memory LRU.** Capacity is hard-capped at `CardPreviewService::kCacheCapacity = 128` entries (positive and negative share the count). When a new entry is inserted past the cap, the **least-recently-used** entry — the back of the list — is dropped. Any access (positive hit, negative hit, or store) moves the entry to the front, so heavily clicked cards are the last to go.
+- **Disk cache.** Capacity is hard-capped by total `.bin` payload bytes (`LocalPreviewByteCache::kDefaultMaxBytes = 64 MiB`). When a `store` would push the total past the cap, oldest-by-mtime `.bin` files (with their `.idx` sidecars) are deleted until the new write fits. A successful `load` touches the entry's mtime, so frequently viewed cards rarely become eviction victims. `.neg` markers are not counted against the cap and are not actively evicted; their count is naturally bounded by the number of records the user has viewed whose upstream cleanly reported "no image".
+
+Eviction is the only way an entry "ages out" without an explicit user action.
+
+### 4. Manual / external invalidation
+
+- **Delete the cache directory.** Removing `<exeDir>/.cache/preview-cache/` (or the umbrella `<exeDir>/.cache/`) is safe: `LocalPreviewByteCache` recreates the directory on the next store. The in-memory tier is unaffected by the disk delete during a running session, but a subsequent app restart starts cold.
+- **Reinstall / move the executable.** Because the cache is rooted at `<exeDir>`, a fresh install elsewhere on disk starts cold by construction, and uninstalling / moving the exe leaves no stray cache in the user's data directory. (Note: **resetting or moving the user's data-storage directory does NOT clear the preview cache** — that's intentional; the cache is install-scoped, not collection-scoped.)
+- **Tampering with sidecar files.** If a `<hash>.idx` is ever rewritten with a key that doesn't match the requested cache key (corruption, hash collision, filesystem hiccup), `LocalPreviewByteCache::load` reports `Miss` rather than serving the entry. A subsequent `store` / `storeNegative` overwrites the corrupted record cleanly. This is what makes FNV-1a (non-cryptographic) safe to use as the hash: the worst case is a one-time miss, never a wrong answer.
+
+### What does *not* trigger an update
+
+To keep the mental model crisp, the following cases **do not** invalidate or refresh anything:
+
+- **Re-selecting the same row repeatedly.** That's a hit by design — the whole point of the cache. The only thing that changes is the entry's LRU position / mtime.
+- **Transient errors on a row that is already negatively cached.** The negative entry is consulted first and short-circuits the call; the network is never touched, so a flaky network can't accidentally turn a `NotFound` verdict into a `Transient` outcome.
+- **Restart with a populated disk cache.** This is a *warm start*, not an update. Both positive and negative entries flow back into memory on first re-access via the disk tier. No upstream is consulted, no entries are rewritten.
+
+## In-memory LRU (`CardPreviewService`)
+
+- **Implementation:** Doubly linked list + hash map, guarded by a mutex. Each entry is `{key, payload, negative}`; positive entries hold the bytes, negative entries hold an empty payload and a `negative=true` flag. Hits move the entry to the front regardless of kind.
+- **Capacity:** `CardPreviewService::kCacheCapacity` (128 entries — positive and negative entries share this count).
+- **Threading:** Preview work can run on a worker thread from the UI layer; all cache access goes through the mutex.
+- **Keys:** Internal strings built in `CardPreviewService.cpp`:
+  - Preview path: prefix `'p'`, then NUL-separated fields: enum `game`, `name`, `setId`, `setNo`. The `setNo` string may embed game-specific disambiguators (for example Yu-Gi-Oh! packs rarity and edition into `setNo` before it reaches the service — see `YuGiOhSelectedCardPanel::previewKey()`).
+  - URL path: prefix `'u'` plus the full URL string.
+
+Callers should treat the key as opaque; **correctness** depends on passing stable `(game, name, setId, setNo)` (and stable URL for fallback fetches) so the same printing always maps to the same cache entry.
+
+## Disk byte cache (`LocalPreviewByteCache`)
+
+- **Root directory:** `<exeDir>/.cache/preview-cache/` (created on first store). Pinned next to the executable, **not** under the user-configurable `Configuration.dataStorage` path — see "Cache directory layout" above for the rationale.
+- **Files per logical entry:** mutually-exclusive `.bin` / `.neg`, plus an always-present `.idx` sidecar:
+  - `<hash>.bin` — raw image bytes (PNG/JPEG payload). **Positive** entry.
+  - `<hash>.neg` — zero-byte marker file. **Negative** entry (the upstream cleanly said "no image").
+  - `<hash>.idx` — text sidecar holding the **exact** cache key string used by `CardPreviewService`. Used to reject hash collisions on load — if `.idx` does not match the requested key, the entry is treated as a miss regardless of which marker file is present.
+
+  `store()` removes any existing `.neg` for the same hash; `storeNegative()` removes any existing `.bin`. The two states never co-exist. If they ever somehow did, `load()` prefers the `.bin` (more useful answer).
+
+- **Hash:** FNV-1a 64-bit over the key, rendered as 16 hex digits. Not cryptographic; the `.idx` sidecar is the safety net that prevents collisions from serving the wrong card's bytes or the wrong card's "no image" verdict.
+- **Size bound:** Default total payload cap `LocalPreviewByteCache::kDefaultMaxBytes` (64 MiB). Eviction removes **oldest by modification time** among `.bin` files (with their `.idx` sidecars) until the new write fits. **Negative entries** (`.neg` markers) are tiny and are not counted against the cap — their count is naturally bounded by the user's actively-viewed records.
+- **Recency on read:** A successful `load` updates the corresponding `.bin` or `.neg` file's mtime (“touch”) so frequently viewed cards are less likely to be evicted.
+- **Failure policy:** All adapter I/O failures are swallowed (miss on read, no-op on failed write). Preview still works from network; worst case is “cold” performance.
+
+### Clearing the disk preview cache
+
+- Delete the `<exeDir>/.cache/` folder (or just the `preview-cache/` subfolder inside it). Both options are safe: `LocalPreviewByteCache` recreates the directory on the next store.
+- The in-app data-storage reset/relocation flow does **not** touch this directory — the cache is install-scoped (sits next to the exe), not collection-scoped. If you need a clean slate for the cache, delete the directory above explicitly.
+
+## HTTP connection reuse (`CprHttpClient`)
+
+The app constructs **one** `CprHttpClient` and shares it across set sources, preview sources, and image downloads. It owns a single long-lived `cpr::Session` (one libcurl easy handle per process).
+
+- **Benefit:** Repeated HTTPS requests to the **same host** reuse TLS sessions / TCP connections where the server allows keep-alive, which materially reduces latency vs. a fresh session per GET (especially for Yu-Gi-Oh!, where preview resolution and the actual image often hit different hosts).
+- **Thread safety:** All `get()` calls are serialized with a mutex because libcurl easy handles are not thread-safe.
+
+This is **not** a response-body cache; it only amortizes connection setup.
+
+## Design constraints (for contributors)
+
+- **Classify source errors honestly.** A new game module's `ICardPreviewSource::fetchImageUrl` must return `PreviewLookupError::Kind::NotFound` only when the upstream answered cleanly (parsed response, no match / no image variants). Anything that could be the network — HTTP error, malformed body, schema deviation, timeout — is `Transient`.
+- **Do not cache transient errors.** That's the rule that keeps a flaky connection from permanently disabling previews. If you ever need to record a failure, route it through `IPreviewByteCache::storeNegative` only on a confirmed `NotFound`.
+- **Updates flow through the cache key, not a side channel.** Don't add a `clearCache(...)` / `invalidate(...)` API to `CardPreviewService` to "fix" a stale entry. The supported update mechanic is: edit-driven invalidation (key changes), same-key positive/negative replacement on the next successful resolution, and LRU/mtime eviction (see "Updating cached entries" above). A side-channel invalidation API would just be another way for callers to forget to keep the disk tier in sync with the memory tier.
+- **Extend cache keys** by packing new disambiguators into existing coordinates (typically `setNo` / tuple encoding) rather than bypassing `CardPreviewService`, so memory and disk tiers stay aligned and editing the record continues to invalidate the negative entry automatically.
+- **Tests:**
+  - `card_preview_service_tests.cpp` pins tier ordering and write-through using an in-memory `IPreviewByteCache` fake; it also exercises the negative-caching behavior end-to-end (NotFound is remembered, Transient is retried, edits invalidate the entry, warm-restart honors the disk negative entry, a later positive overwrites a previous negative).
+  - `local_preview_byte_cache_tests.cpp` exercises the real-disk adapter in isolated temp directories, including the `.bin`/`.neg`/`.idx` interactions (round-trip, restart, mutual replacement, sidecar collision rejection, eviction).
+
+## Related reading
+
+- [assets-and-info-apis.md](assets-and-info-apis.md) — external APIs and the same preview tiers in **runtime flow** context.
+- Root `AGENTS.md` — UI performance guardrails summary.
+- `core/AGENTS.md` — conventions for preview caching, source-error classification, and `CprHttpClient` session ownership.