finn-mcp/IMPLEMENTATION.md

# IMPLEMENTATION.md — Phase 2 build runbook

How to drive Phase 2 (the 12 steps in `PRD.md` §29) to completion using an AI agent. Each step has its own kickoff prompt, files affected, and "done" criteria. Run them in order. Each step is independently mergeable.

---

## 0. Pre-flight

Before starting step 1:

1. ls -la

2. **Venv is healthy.** From the project root:

   ```bash
   source .venv/bin/activate
   pytest -x                       # green except for any pre-existing FastMCP-related skips
   ruff check .                    # zero issues
   ```

3. **Docs are in place.** Re-confirm `PRD.md` §17 (code ownership) is current — every step below references it.

If any of these fail, stop and fix before proceeding.

---

## How to use this runbook

For each step:

1. Create a feature branch: `git checkout -b feat/phase2-step-<N>-<slug>` off `chore/cleanup-phase-2-prep`.
2. Open a fresh agent chat with repo access. Paste the kickoff prompt verbatim.
3. Let the agent propose, implement, and test. Push back where it skips tests or violates §17.
4. When all "done" boxes are checked, merge into `chore/cleanup-phase-2-prep`.
5. Move to the next step.

Each kickoff prompt assumes the agent reads PRD.md, AGENTS.md, and the relevant instruction files first — that's encoded in the prompt.

After step 12, merge `chore/cleanup-phase-2-prep` into `main`.

---

## Step 1 — Dev workflow already switched to local venv

This step is **done** by the time `CLEANUP.md` is merged. The instruction files and `AGENTS.md` already use local venv. Sanity check:

```bash
source .venv/bin/activate
which finn-eiendom 2>/dev/null || echo "expected: not yet installed; entry points come in steps 6 and 7"
ruff check .                 # zero issues
pytest -x                    # green (allow mcp_server failures)
```

Move on.

---

## Step 2 — Pydantic v2 cleanup

### Kickoff prompt

> Read **PRD.md** (especially §17 code ownership and A8 acceptance criterion), **`.github/instructions/python.instructions.md`**, and **`.github/instructions/clean-code.instructions.md`**.
>
> Implement Phase 2 step 2: convert every Pydantic model in `finn_eiendom/models.py` from v1 (`class Config:`) to v2 (`model_config = ConfigDict(...)`). Use `context7:query-docs` on `pydantic/pydantic` if you're not sure of the v2 syntax — don't guess.
>
> Add `tests/test_models.py` with a JSON roundtrip test per model.
>
> Run `ruff check .`, `ruff format .`, and `pytest tests/test_models.py -v` before declaring done.

### Files

* `finn_eiendom/models.py` (edit)
* `tests/test_models.py` (new)

### Done when

* `grep -rn "class Config:" finn_eiendom/` produces zero output.
* `pytest tests/test_models.py` is green.
* Existing tests still pass.

---

## Step 3 — Service layer

### Kickoff prompt

> Read **PRD.md** §16 (Service layer) and §17 (code ownership), **`.github/instructions/python.instructions.md`** and **`.github/instructions/clean-code.instructions.md`**.
>
> Create `finn_eiendom/service.py` with the public surface listed in PRD §16: `get_or_fetch_ad`, `get_or_fetch_eiendom_unit`, `get_or_fetch_similar_units`, `analyze_search`, `analyze_ad`, `analyze_ad_against_comps`, `find_similar_to_liked`, `compare_ads`, `resolve_eiendom_unit_from_finn_url`, `build_unit_vector_for_unit_code`, `decode_unit_vector_to_dict`, `save_feedback`, `get_shortlist`, `get_new_ads_since_last_run`.
>
> Each function:
> 1. Opens its own SQLite connection via `cache.init_db(FINN_CACHE_PATH)`.
> 2. Reads cache first with TTLs from `config.py`.
> 3. On miss or `force_refresh=True`, calls the fetcher in `ad.py` / `eiendom_no.py`.
> 4. Writes the fresh result back.
> 5. Returns a typed model or dict.
>
> Do not duplicate behavior from `analysis.py` — delegate to it. Add `tests/test_service.py` covering the five service tests listed in PRD §25.2.

### Files

* `finn_eiendom/service.py` (new)
* `tests/test_service.py` (new)
* `tests/conftest.py` (may need a `tmp_db` fixture if it doesn't exist)

### Done when

* `pytest tests/test_service.py` is green.
* `service.py` imports only from `models`, `config`, `cache`, `analysis`, `ad`, `eiendom_no`, `feedback`, `scoring`, stdlib.
* No `import httpx` or `import sqlite3` outside their owners.

---

## Step 4 — Formatting layer

### Kickoff prompt

> Read **PRD.md** §17.6 (shared formatting module) and §19 (output formats), **`.github/instructions/clean-code.instructions.md`**.
>
> Create `finn_eiendom/formatting.py` with these renderers (signatures in PRD §17.6): `render_ad`, `render_shortlist`, `render_comparison`, `render_diff`, `render_similar_units`, `render_unit`, `render_score_breakdown`, plus `render_cache_stats` for the CLI cache subcommand.
>
> Each renderer accepts `(payload, fmt: Literal["json","markdown","table"]) -> str`. Unsupported formats raise `ValueError` listing supported options. Table rendering only applies where it makes sense (shortlist, comparison, diff, similar-units).
>
> Add `tests/test_formatting.py` covering the three tests listed in PRD §25.5.

### Files

* `finn_eiendom/formatting.py` (new)
* `tests/test_formatting.py` (new)

### Done when

* `pytest tests/test_formatting.py` is green.
* `render_*` is the *only* place that formats output. No inline rendering anywhere else (verified by reading diffs of steps 6 and 7).

---

## Step 5 — HTTP retry on 5xx

### Kickoff prompt

> Read **PRD.md** A9 (acceptance criterion), **`.github/instructions/python.instructions.md`**.
>
> Extend `HTTPClient.get()` in `finn_eiendom/http.py` to retry on 5xx responses (500/502/503/504) with exponential backoff `1s, 2s, 4s`, up to `retries` attempts (default 3). Surface 4xx as `httpx.HTTPStatusError` immediately. Apply the existing `request_delay` between any two calls.
>
> If you're unsure about `httpx` retry semantics or `respx` test patterns, use `context7`.
>
> Add `tests/test_http.py` covering the three tests listed in PRD §25.6 using `respx`.

### Files

* `finn_eiendom/http.py` (edit)
* `tests/test_http.py` (new)

### Done when

* `pytest tests/test_http.py` is green.
* `httpx` imports remain confined to `http.py`.

---

## Step 6 — Replace FastAPI with FastMCP

### Kickoff prompt

> Read **PRD.md** §14 (MCP design — every tool and input schema), §17 (code ownership), and **`.github/instructions/mcp.instructions.md`** end-to-end.
>
> Rewrite `finn_eiendom/mcp_server.py` from scratch:
> - Use `from mcp.server.fastmcp import FastMCP`.
> - Configure stderr-only logging.
> - Register all 14 tools listed in PRD §14.1 with the `finn_` prefix.
> - Each tool body has the shape in `mcp.instructions.md` §"Tool body shape": one `service.<function>` call, one `formatting.render_*` call, try/except returning the JSON error envelope.
> - Input schemas as in PRD §14.2.
> - Annotations: `readOnlyHint=True` for all except `finn_save_feedback`.
> - `main()` calls `mcp.run(transport="stdio")`.
> - Add `finn-eiendom-mcp = "finn_eiendom.mcp_server:main"` to `[project.scripts]` in `pyproject.toml`.
>
> If unsure about FastMCP annotations or transport options, use `context7:query-docs` on the MCP Python SDK.
>
> Rewrite `tests/test_mcp_server.py` to cover the three tests in PRD §25.3. Use the SDK's testing helpers — do not spawn a subprocess.
>
> Verify: `finn-eiendom-mcp` boots over stdio, `mcp dev finn_eiendom/mcp_server.py` lists all 14 tools.

### Files

* `finn_eiendom/mcp_server.py` (full rewrite)
* `tests/test_mcp_server.py` (full rewrite)
* `pyproject.toml` (edit `[project.scripts]`)

### Done when

* `mcp_server.py` imports only `service`, `formatting`, `models`, `config`, stdlib, `mcp`, `pydantic`.
* All 14 tools registered.
* `pytest tests/test_mcp_server.py` is green.
* `grep -rn "FastAPI" finn_eiendom/` is empty.

---

## Step 7 — CLI

### Kickoff prompt

> Read **PRD.md** §15 (CLI design — every command and option) and **`.github/instructions/cli.instructions.md`** end-to-end.
>
> Create `finn_eiendom/cli.py` with a `typer.Typer` app exposing all commands in PRD §15.1, plus `finn_eiendom/__main__.py` that calls the app. Add to `pyproject.toml`:
> ```
> [project.scripts]
> finn-eiendom = "finn_eiendom.cli:app"
> ```
>
> Each command:
> - Translates options into a `service.<function>` call.
> - Calls `formatting.render_*(result, format)` and `typer.echo(...)`.
> - No business logic, no inline rendering.
> - Body under ~20 lines.
>
> Sub-app for `cache` (stats/clear/clear-html/clear-json) and `config` (show/path). `serve` accepts `--transport stdio|http` and dispatches to `mcp_server.main()` or the HTTP transport.
>
> If unsure about Typer sub-apps or `CliRunner`, use `context7`.
>
> Add `tests/test_cli.py` covering the five tests in PRD §25.4 using `typer.testing.CliRunner`. Mock `service.*` with `monkeypatch` — do not exercise the full stack here, that's `test_service.py`.

### Files

* `finn_eiendom/cli.py` (new)
* `finn_eiendom/__main__.py` (new)
* `tests/test_cli.py` (new)
* `pyproject.toml` (edit)

### Done when

* `finn-eiendom --help` lists every command in PRD §15.1.
* `cli.py` imports only `service`, `formatting`, `models`, `config`, stdlib, `typer`.
* `pytest tests/test_cli.py` is green.

---

## Step 8 — Diff workflow (new / removed / changed)

### Kickoff prompt

> Read **PRD.md** §10.8, §13 (search_runs table), workflow I in §18, and **`.github/instructions/clean-code.instructions.md`**.
>
> Implement:
> 1. `search_runs` and `scores` tables in `cache.py` (use existing migration pattern).
> 2. `service.get_new_ads_since_last_run(search_url)` that compares against the previous run for the same `normalized_url` and returns `{new_ads, removed_ads, changed_ads}` with price/common_costs/status diffs on changed.
> 3. `finn_get_new_ads_since_last_run` MCP tool.
> 4. `finn-eiendom diff <url>` CLI command.
> 5. `formatting.render_diff(result, fmt)`.
>
> Add tests covering: empty previous-run case, all-new case, mixed new+removed+changed case.

### Done when

* The three new tests pass.
* MCP and CLI both expose the same behavior with identical defaults.

---

## Step 9 — Compare workflow

### Kickoff prompt

> Read **PRD.md** workflow K in §18 and §14.2 (`CompareAdsInput`).
>
> Implement `service.compare_ads(finnkoder, include_eiendom_no=True, include_comps=True)` returning a comparison table + winners by category (best value / lifestyle / hybel / bargain / safest / highest risk / most overpriced).
>
> Wire `finn_compare_ads` MCP tool and `finn-eiendom compare <finnkode...>` CLI command. Add `formatting.render_comparison`. Tests for service and CLI.

### Done when

* `finn-eiendom compare 462400360 461153194 --format markdown` produces a readable comparison.
* Service test covers the winners-by-category logic.

---

## Step 10 — Similar-to-liked

### Kickoff prompt

> Read **PRD.md** workflow G in §18 and `FindSimilarToLikedInput` in §14.2.
>
> Implement `service.find_similar_to_liked(finnkode, mode, listing_status)`:
> 1. Load FinnAd; verify `feedback` has `verdict=liked` for this finnkode.
> 2. Ensure Eiendom.no enrichment + unit_vector exist.
> 3. Fetch similar-units (prefer `FOR_SALE` for recommendations, `RECENTLY_SOLD` for comps).
> 4. Score candidates against user preferences.
> 5. Return ranked recommendations.
>
> Wire MCP tool and CLI command. Tests covering: no liked feedback raises clear error; happy path returns ranked list.

### Done when

* `finn-eiendom similar-to-liked 462400360` returns ranked candidates when the listing has a liked verdict, and a clear error otherwise.

---

## Step 11 — Architecture tests

### Kickoff prompt

> Read **PRD.md** A10 (architecture acceptance criterion) and §17.3 (layering invariants).
>
> Create `tests/test_architecture.py` that walks every `.py` file under `finn_eiendom/` with `ast`, collects all `import` and `from X import Y` statements, and asserts the layering invariants in PRD A10:
> - No `httpx` outside `http.py`.
> - No `sqlite3` outside `cache.py`.
> - No `BeautifulSoup` outside `search.py` / `ad.py`.
> - No `msgpack` outside `eiendom_no.py`.
> - `mcp_server.py` and `cli.py` import only from the allowed set.
> - `service.py` never imports `mcp_server` or `cli`.
>
> Add a parametrize'd test per invariant so failures show which module violated which rule. Failures should print the offending import line and module.

### Done when

* `pytest tests/test_architecture.py` is green.
* Deliberately introducing a violation (e.g. `import httpx` in `service.py`) makes a test fail with a clear message.

---

## Step 12 — README + Claude Desktop config + final verification

### Kickoff prompt

> Read **PRD.md** §21 (deployment), §22 (MVP scope), §24 (all acceptance criteria), **README.md** and **USAGE.md**.
>
> Update `README.md` and `USAGE.md` so every command, env var, and Claude Desktop snippet matches what was actually built in steps 1–11. Verify with the user's exact paths.
>
> Run the full A1–A11 acceptance check:
>
> - A1: `finn-eiendom-mcp` boots over stdio; `mcp dev finn_eiendom/mcp_server.py` lists all 14 tools.
> - A2: `finn-eiendom --help` lists every §15.1 command; each command runs against fixtures.
> - A3 – A9: matching service tests pass.
> - A10: `pytest tests/test_architecture.py` is green.
> - A11: `ruff check .` is clean; `pytest` is fully green; `mypy --strict finn_eiendom` passes or is documented as a gap.
>
> Report any failures with specific file/line references — don't paper over them.

### Files

* `README.md` (edit to match reality)
* `USAGE.md` (edit to match reality)

### Done when

* All 11 acceptance criteria in PRD §24 pass.
* README + USAGE quickstart examples actually work end-to-end on a fresh clone.

---

## Definition of done for the whole phase

Merge `chore/cleanup-phase-2-prep` into `main` when **every** box is checked:

* [ ] All 12 steps merged in order.
* [ ] `finn-eiendom-mcp` boots over stdio with all 14 tools.
* [ ] `finn-eiendom --help` lists every command in PRD §15.1.
* [ ] `pytest` is green, including the new `test_service.py`, `test_cli.py`, `test_http.py`, `test_formatting.py`, `test_models.py`, `test_architecture.py`.
* [ ] `ruff check .` is clean.
* [ ] `mypy --strict finn_eiendom` passes or has a documented exception list.
* [ ] `README.md` and `USAGE.md` quickstart examples work on a fresh clone in under 5 minutes.
* [ ] Claude Desktop config in USAGE.md is verified to work against your installation.

---

## When a step blocks

If a step blocks on an unclear requirement:

1. Re-read the relevant PRD section.
2. Check `PRD.md` §28 (open questions) — the answer may be a deferred decision.
3. If still unclear, write the question down, pick the simplest interpretation, mark it `# TODO(<date>): revisit <question>` in code, and move on.

If a step blocks on a library question (FastMCP, Pydantic v2, Typer, httpx, msgpack, respx):

1. Use `context7` — see `.github/instructions/docs.instructions.md`.
2. If context7 returns nothing useful, write the smallest possible spike in `scratch/` (gitignored) to verify behavior.

If a step blocks on §17 (code ownership) — i.e. it feels like the right answer requires putting logic in the "wrong" place:

1. Stop.
2. Re-read PRD §17.2 (decision table) and §17.3 (layering invariants).
3. Ask whether the service layer is actually missing a function. Usually it is.
4. Add the missing service function instead of bending the layering.