ole/finn-mcp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

initial

Browse Source
This commit is contained in:
Ole
2026-05-16 06:54:17 +00:00
commit 1399f61c1a
44 changed files with 6746 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/copilot-instructions.md
+181
View File
@@ -0,0 +1,181 @@
# Copilot instructions for finn-eiendom-mcp
This project is a private, self-hosted Python platform for analyzing FINN real-estate listings. It exposes the same code through three coordinated front ends:
1. A **Python library** (`finn_eiendom`) — source of truth.
2. An **MCP server** (FastMCP, stdio + optional HTTP) over `finn_eiendom/mcp_server.py`.
3. A **CLI** (`finn-eiendom`) over `finn_eiendom/cli.py`.
All three share the same `service.py`, `formatting.py`, `cache.py`, and `models.py`. Code lives in exactly one place and is called from both front ends. See `PRD.md` §17 for the full ownership rules — that section is the constitution.
---
## Source of truth
Read in this order:
1. `PRD.md` — product and architecture, especially §17.
2. `PROJECT.md` — module map.
3. `AGENTS.md` — workflow.
4. `.github/instructions/*.md` — per-topic rules.
---
## Module layout
```
finn_eiendom/
config.py # env vars, defaults, TTLs
models.py # Pydantic v2 models
parser.py # number/area/date/URL/finnkode normalization
http.py # async HTTP (httpx) with delay + retry + user-agent
cache.py # SQLite (sqlite3) schema + persistence
search.py # FINN search HTML parsing + pagination
ad.py # FINN listing HTML parsing
eiendom_no.py # Eiendom.no unit search/detail, unit_vector, similar-units
scoring.py # score model + classifications
feedback.py # verdicts + soft preference signal
analysis.py # orchestration + shortlist + summary
service.py # get_or_fetch_* + thin facade for MCP and CLI
formatting.py # render_* helpers shared by MCP and CLI
mcp_server.py # FastMCP wrappers around service.py
cli.py # typer-based CLI wrappers around service.py
__main__.py # python -m finn_eiendom → CLI entry
```
---
## The five hard rules
Enforced by `tests/test_architecture.py`:
1. **`mcp_server.py` and `cli.py` are siblings.** They never import from each other. Both import only from `service`, `formatting`, `models`, `config`, stdlib, and their own framework (`mcp` / `typer`).
2. **`service.py` is the only orchestrator.** Nothing above it touches HTTP or SQLite directly.
3. **`httpx` lives only in `http.py`.**
4. **`sqlite3` lives only in `cache.py`.**
5. **Output formatting lives only in `formatting.py`.** Never inline in CLI or MCP tool bodies.
---
## Development workflow — local venv
Default runtime is a project-local virtualenv. Docker is supported for packaging but optional for development.
```bash
uv venv # or: python3.12 -m venv .venv
source .venv/bin/activate
uv pip install -e ".[dev]" # or: pip install -e ".[dev]"
# from now on:
pytest
ruff check .
ruff format .
mypy finn_eiendom
finn-eiendom --help
finn-eiendom-mcp # stdio MCP server
```
**Never** install packages globally. **Never** add a dependency without updating `pyproject.toml`.
---
## Coding rules
* Python 3.12+.
* Pydantic v2 with `model_config = ConfigDict(...)`. No v1 `class Config:` blocks.
* Type hints on every function signature.
* Async I/O for all network and DB code paths through `service.py`.
* Dependency injection for HTTP/cache clients in tests.
* Small, focused functions. One job per function. See `clean-code.instructions.md`.
* Errors raise with actionable messages; the MCP boundary translates them to `{"error": True, "code": ..., "message": ...}`.
* stdio MCP servers log to **stderr only**.
---
## Code ownership — the short version
| Concern | Lives in |
| -------------------------------------- | ------------------------------ |
| FINN search HTML parsing | `search.py` |
| FINN listing HTML parsing | `ad.py` |
| Norwegian number / area / URL regexes | `parser.py` |
| HTTP fetching + retry + delay | `http.py` |
| SQLite reads / writes | `cache.py` |
| Eiendom.no unit search/detail/comps | `eiendom_no.py` |
| `unit_vector` encode/decode (msgpack) | `eiendom_no.py` |
| Scoring + classification | `scoring.py` |
| Feedback storage | `feedback.py` |
| Cache-aware orchestration | `service.py` (`get_or_fetch_*`)|
| Shortlist + summary assembly | `analysis.py` |
| End-to-end runs | `service.py` (`analyze_search`)|
| MCP tool definitions | `mcp_server.py` |
| CLI command definitions | `cli.py` |
| Output rendering | `formatting.py` |
| Env-var defaults | `config.py` |
| Pydantic models | `models.py` |
Full table with "never lives in" column is in `PRD.md` §17.2.
---
## Adding a feature
1. Decide the home using the table above (and `PRD.md` §17.2).
2. Implement in `service.py` (or `analysis.py` if pure orchestration).
3. Add a service-level test.
4. Add a thin MCP tool — `response_format`-aware.
5. Add a thin CLI command — `--format`-aware.
6. Add a renderer in `formatting.py`.
7. Test MCP and CLI registration.
8. Update PRD and instruction docs.
If the MCP tool body or CLI command body grows past ~20 lines, push logic down to `service.py`.
---
## Documentation lookups — use context7
When uncertain about an external library API (FastMCP, Pydantic v2, Typer, httpx, msgpack, pytest-asyncio, respx, BeautifulSoup), call the **`context7` MCP server** *before* writing code. Don't rely on training-data memory.
```
context7:resolve-library-id → library_id
context7:query-docs(library_id, topic) → authoritative snippets
```
Details in `.github/instructions/docs.instructions.md`.
---
## Clean code is a hard requirement
See `clean-code.instructions.md`. DRY, single-responsibility, descriptive names, type hints, no dead code, comments explain why not what. If duplication slips in, the right answer is to extract it — not to copy the second instance.
---
## Product behavior
The MVP does one thing well:
```
FINN search URL in
→ relevant property candidates out
→ enriched with Eiendom.no estimates
→ similar-units / comps
→ explanations
→ risks
→ next steps
→ broker questions
```
Always explain:
* why a property is interesting,
* price vs estimate,
* price vs comparable sales,
* renovation upside,
* hybel / rental potential,
* technical / legal risks,
* uncertainty / confidence,
* next questions for the broker.
Scores and estimates are decision support, not advice. Surface uncertainty, never hide it.
.github/instructions/clean-code.instructions.md
+150
View File
@@ -0,0 +1,150 @@
---
name: Clean code rules
description: Best-practice standards for all production and test code
applyTo: "**/*.py"
---
# Clean code rules
These rules apply everywhere — every module, every function, every test. They are intentionally opinionated. If a rule conflicts with the architecture rules in `PRD.md` §17, the architecture rules win. If it conflicts with another best practice here, pick the one that produces the simpler, more readable result.
## Single responsibility
* One job per function. If a function name needs "and" to describe it, it's two functions.
* One job per module. `parser.py` parses. `cache.py` caches. `formatting.py` formats. Don't mix.
* One job per class. We rarely need classes outside Pydantic models, dataclasses, and the `HTTPClient`. Avoid OO for OO's sake.
## Function size
* Aim for under **30 lines** of body.
* Past **50 lines** it's a code smell — extract helpers.
* If you've got more than **3 levels of nesting**, the function wants splitting (extract the inner block into a helper named after what it does).
## Naming
* Names describe **intent**, not implementation. `get_or_fetch_ad`, not `process_ad`. `render_shortlist_markdown`, not `format2`.
* Verbs for actions (`fetch_`, `parse_`, `score_`, `render_`).
* Nouns for data (`FinnAd`, `EiendomUnit`, `shortlist`).
* Boolean variables / parameters read as predicates: `force_refresh`, `include_eiendom_no`, `is_recently_sold`. Not `flag`, not `do_thing`.
* Avoid abbreviations except those well-established in the domain (`url`, `ad`, `nok`, `bra`, `sqm`).
* Norwegian terms stay Norwegian when they're domain vocabulary (`hybel`, `fellesgjeld`, `finnkode`). Don't translate `finnkode` to `finn_code` — it's a proper noun.
## Type hints
Required on every function signature, including private helpers. Mypy in strict mode is the goal.
```python
# ❌
def parse(html, base_url=None):
...
# ✅
def parse(html: str, base_url: str | None = None) -> FinnAd | None:
...
```
Use modern syntax: `X | None` over `Optional[X]`, `list[int]` over `List[int]`, `dict[str, Any]` over `Dict[str, Any]`.
## Comments
* Comments explain **WHY**, never **WHAT**. The code already says what.
* If a comment is needed to explain *what* a line does, the line wants renaming or extracting.
* Use docstrings for public functions, classes, and modules. One-line summary, blank line, optional details and examples.
* No commented-out code. Delete it. Git remembers.
* No `# TODO` without a date or issue reference. `# TODO(2026-05): replace once Eiendom.no confirms ...` is fine.
## DRY — Don't Repeat Yourself
If you write the same logic, regex, SQL, or format string **twice**, extract it. The decision table in `PRD.md` §17.2 tells you where it belongs.
The pre-merge anti-duplication checklist (from `PRD.md` §17.4):
1. Is this logic already implemented somewhere? (`grep` the function name and obvious keywords.)
2. If I'm copy-pasting from another file, am I about to duplicate behavior that should live in one shared function?
3. Can a new caller use an existing `service.py` function instead of writing its own orchestration?
4. Is the same Pydantic field defined in two models? Factor out a base model.
5. Am I formatting output in two places (CLI + MCP)? Move it to `formatting.py`.
6. Am I opening a SQLite connection outside `cache.py`? Move it.
7. Am I building an httpx call outside `http.py`? Move it.
8. Am I writing a Norwegian-number / area / finnkode regex outside `parser.py`? Move it.
9. Am I adding an env-var lookup outside `config.py`? Move it.
10. Did I add a new behavior with only one front end (MCP or CLI)? If it should exist in both, the service function is missing.
A small amount of duplication is acceptable to keep boundaries clean — see `PRD.md` §17.8. Past a handful of lines, extract.
## Errors
* **Fail loudly** with actionable messages.
```python
# ❌
raise ValueError("bad input")
# ✅
raise ValueError(f"Unknown listing_status {status!r}; expected one of {VALID_LISTING_STATUSES}")
```
* **No silent failures.** `except Exception: pass` is forbidden. Catch the specific exception, log it, and either recover or re-raise.
* **Service raises; MCP wraps.** Service functions raise normal exceptions. The MCP tool boundary translates them into `{"error": True, "code": ..., "message": ...}`. CLI lets typer handle non-zero exits.
* **Graceful degradation is explicit.** If Eiendom.no enrichment fails, return a result with `eiendom_unit=None` and a warning, not a silently-missing field.
## State
* No global mutable state. The only module-level constants allowed are configuration values loaded from env in `config.py`.
* No module-level caches (dicts, lists) that mutate. Use `cache.py` if you need persistence.
* Pass dependencies in (HTTP clients, DB connections) for testability.
## Dead code
* No commented-out code.
* No unused imports (ruff catches these — fix them, don't add `# noqa`).
* No unused parameters (use `_` or remove).
* No `if False:` blocks "for later".
* Functions and classes that aren't called anywhere — delete them. Git keeps history.
## Magic numbers and strings
Anything that influences behavior and isn't self-explanatory belongs in `config.py` (env-controlled) or as a named module-level constant near the top of the file.
```python
# ❌
if days > 90:
confidence = "low"
# ✅
COMPS_STALE_AFTER_DAYS = 90
if days > COMPS_STALE_AFTER_DAYS:
confidence = "low"
```
URLs, timeouts, retries, TTLs, status codes — never inline.
## Imports
* Standard library first, third-party second, local last, separated by blank lines.
* Ruff's `I` rules sort and group these — run `ruff check . --fix`.
* No wildcard imports.
* No relative imports above one level (`from ..thing import x` is a smell; refactor).
* Each module's allowed import set is enforced by `tests/test_architecture.py`.
## Tests are first-class code
Same rules. Same type hints. Same naming. Same DRY. If a fixture is used in three test files, it goes in `conftest.py`. If three tests share a setup, factor it into a fixture.
## Reviewing your own change before commit
A 60-second self-review:
1. Did I add a function that already exists somewhere? (`grep` it.)
2. Did I bypass `service.py`, `http.py`, `cache.py`, or `formatting.py`?
3. Is everything typed?
4. Did I leave a `print()`, `breakpoint()`, or commented-out block behind?
5. Does the test for this change actually fail without the change?
6. Did I update `PRD.md` or the relevant instruction file if I changed an architectural rule?
## When in doubt about a library API
Use the `context7` MCP server instead of guessing. See `docs.instructions.md`. Training-data memory of `pydantic.field_validator`, `typer.Option`, `mcp.tool` annotations, or `httpx.AsyncClient` is unreliable — they all change between versions.
.github/instructions/cli.instructions.md
+158
View File
@@ -0,0 +1,158 @@
---
name: CLI rules
description: Rules for the typer-based finn-eiendom CLI
applyTo: "finn_eiendom/cli.py,finn_eiendom/__main__.py"
---
# CLI rules
The CLI is a **thin wrapper** over `service.py`. It is a sibling of `mcp_server.py` — they never call each other and they share the same underlying service functions. Every CLI command maps 1:1 to a service function with the same parameters and defaults.
## Framework
Built with [`typer`](https://typer.tiangolo.com/). One `typer.Typer` app:
```python
# finn_eiendom/cli.py
import asyncio, typer
from . import service, formatting
app = typer.Typer(no_args_is_help=True, add_completion=False)
```
Entry points in `pyproject.toml`:
```toml
[project.scripts]
finn-eiendom-mcp = "finn_eiendom.mcp_server:main"
finn-eiendom = "finn_eiendom.cli:app"
```
Plus `finn_eiendom/__main__.py`:
```python
from .cli import app
if __name__ == "__main__":
app()
```
So `python -m finn_eiendom ...` works without installation.
## Command body shape
```python
@app.command()
def analyze_search(
url: str,
max_pages: int = 3,
detail_limit: int = 20,
no_details: bool = typer.Option(False, "--no-details"),
no_eiendom: bool = typer.Option(False, "--no-eiendom"),
with_similar: bool = typer.Option(False, "--with-similar"),
format: str = typer.Option("json", "--format"),
) -> None:
"""Analyze a FINN search URL and return a ranked shortlist."""
result = asyncio.run(service.analyze_search(
search_url=url,
max_pages=max_pages,
detail_limit=detail_limit,
include_details=not no_details,
include_eiendom_no=not no_eiendom,
include_similar_units_for_shortlist=with_similar,
))
typer.echo(formatting.render_shortlist(result, format))
```
Rules:
* The command body has at most three sections: option parsing (handled by typer), one `service.<function>` call, one `typer.echo(formatting.render_<thing>(result, format))`.
* If the body has more than ~20 lines, the logic belongs in `service.py`.
* No `print()` — use `typer.echo()` for stdout, `typer.echo(..., err=True)` for stderr.
* No business logic, no rendering, no SQLite, no HTTP, no parsing.
## Formats
Every command that produces structured output accepts `--format`:
* `--format json` (default) — full structured output, pipeable into `jq`.
* `--format markdown` — human-readable.
* `--format table` — terminal table (only where it makes sense: `analyze-search`, `compare`, `shortlist`, `diff`).
All three render paths are produced by `formatting.py`. Never format inline in `cli.py`. Unsupported values raise `ValueError` with a list of supported formats — typer surfaces this as a non-zero exit.
## Commands
```text
finn-eiendom analyze-search <url> [--max-pages 3] [--detail-limit 20] [--no-details] [--no-eiendom] [--with-similar] [--format ...]
finn-eiendom get-ad <finnkode> [--force-refresh] [--no-eiendom] [--with-similar] [--format ...]
finn-eiendom compare <finnkode...> [--no-eiendom] [--no-comps] [--format ...]
finn-eiendom save-feedback <finnkode> <verdict> [--notes "..."]
finn-eiendom shortlist [--run-id ID] [--limit 10] [--format ...]
finn-eiendom diff <url> [--format ...]
finn-eiendom resolve-unit <finn_url>
finn-eiendom get-unit <unit_code> [--force-refresh]
finn-eiendom enrich-ad <finnkode> [--with-similar]
finn-eiendom build-vector <unit_code>
finn-eiendom decode-vector <unit_vector>
finn-eiendom similar-units <unit_vector> [--status RECENTLY_SOLD|FOR_SALE|CURRENT]
finn-eiendom similar-to-liked <finnkode> [--mode recommendations|comps] [--status ...]
finn-eiendom analyze-against-comps <finnkode>
finn-eiendom cache stats | clear | clear-html | clear-json
finn-eiendom serve [--transport stdio|http] [--host 127.0.0.1] [--port 8010]
finn-eiendom config show | path
finn-eiendom doctor
finn-eiendom version
```
Sub-command groups (`cache`, `config`) use `typer.Typer` sub-apps:
```python
cache_app = typer.Typer(help="Cache management")
app.add_typer(cache_app, name="cache")
@cache_app.command("stats")
def cache_stats() -> None:
typer.echo(formatting.render_cache_stats(service.get_cache_stats(), "json"))
```
## Async glue
Service functions are async; CLI commands are sync. Always use `asyncio.run(service.<function>(...))` at the call boundary. Don't sprinkle `async def` across CLI commands — typer expects sync handlers.
## Exit codes
* `0` — success.
* `1` — runtime error (raised exception in service).
* `2` — usage error (typer's default for bad options).
Let exceptions propagate from `service.py` and rely on typer's default handling. Only catch where you want a more specific exit code or message.
## What stays out of cli.py
* `import httpx`, `import sqlite3`, `import msgpack` — never.
* `from .ad import ...`, `from .search import ...`, `from .eiendom_no import ...`, `from .scoring import ...`, `from .cache import ...`, `from .http import ...` — never.
* Inline formatting logic — goes in `formatting.py`.
* MCP imports (no `from .mcp_server import ...`).
Allowed imports in `cli.py`:
```python
import asyncio, json, sys
import typer
from . import service, formatting, config
from .models import FinnAd, EiendomUnit, SimilarUnit # only for type hints
```
`tests/test_architecture.py` enforces this.
## When uncertain about typer
Use `context7` instead of guessing:
```
context7:resolve-library-id → "tiangolo/typer"
context7:query-docs(id, "Typer sub-apps and option groups")
```
See `docs.instructions.md`.
.github/instructions/docs.instructions.md
+118
View File
@@ -0,0 +1,118 @@
---
name: Documentation lookups via context7 MCP
description: How and when to use the context7 MCP server for library documentation
applyTo: "**/*.py,**/*.md,**/*.toml,**/*.yaml,**/*.yml"
---
# Documentation lookups — use context7
When you are uncertain about a library's API, **call the `context7` MCP server before writing code**. Do not rely on training-data memory. Pydantic, FastMCP, Typer, httpx, and pytest all evolve quickly; what was true two releases ago is often wrong now.
## When to use context7
Use it **before** writing code involving any of these:
* **FastMCP / MCP Python SDK** — `@mcp.tool()` signatures, `ToolAnnotations`, `mcp.run(transport=...)`, resource and prompt decorators, server lifecycle, streamable-HTTP setup.
* **Pydantic v2** — `BaseModel`, `Field`, `ConfigDict`, `model_validator`, `field_validator`, `model_dump` / `model_dump_json`, discriminated unions, `Annotated[...]` with validators.
* **Typer** — `Typer()` apps, `typer.Option`, `typer.Argument`, sub-apps via `add_typer`, callbacks, exit codes, testing with `CliRunner`.
* **httpx** — `AsyncClient`, timeouts, transports, retries, `Response` API.
* **respx** — mocking httpx, `respx.mock`, `route.mock`, match patterns.
* **msgpack** — packing/unpacking, type extensions, raw vs string mode.
* **base64** — `urlsafe_b64encode`, padding handling.
* **pytest** / **pytest-asyncio** — fixtures, parametrize, async tests, markers, `tmp_path`, `monkeypatch`.
* **BeautifulSoup** / **lxml** — selectors, parser flavors, element traversal.
* **typer.testing.CliRunner** — invoking apps, asserting on stdout/stderr/exit codes.
Use it **also** when:
* A test fails with an error like `AttributeError: 'BaseModel' object has no attribute 'dict'` (Pydantic v1 vs v2 confusion).
* You see a `DeprecationWarning` from a third-party library and aren't sure of the modern replacement.
* You're about to copy a code pattern from memory that feels "old".
## When NOT to use it
* Pure Python stdlib (`json`, `pathlib`, `dataclasses`, `typing`) — these are stable and well-known.
* Project-internal modules — read the source.
* Generic programming questions ("what's a list comprehension") — use your own knowledge.
* FINN / Eiendom.no API behavior — these are not in context7. Use fixtures from prior runs in `tests/fixtures/` and the endpoint notes in `PRD.md` §9.
## How to use it
Two-step pattern:
### 1. Resolve the library ID
```
context7:resolve-library-id(query="fastmcp")
context7:resolve-library-id(query="pydantic")
context7:resolve-library-id(query="typer")
```
Returns the canonical library ID (e.g. `pydantic/pydantic`, `fastapi/typer`). Pick the most-starred / official-looking match.
### 2. Query the docs
```
context7:query-docs(
context7CompatibleLibraryID="pydantic/pydantic",
topic="field validators v2 mode after",
tokens=3000,
)
```
* **Keep the topic focused.** "Pydantic v2 field validators with mode=after on Optional[str]" beats "Pydantic validation".
* **Cap tokens** to roughly what you need (1500–4000 is usually plenty). The default is fine for most calls.
* **Use library-specific terminology** in the topic — "discriminator field" for Pydantic, "tool annotations" for FastMCP, "sub-apps" for Typer.
### Worked examples
**Q: How do I declare a FastMCP tool with read-only annotations?**
```
context7:resolve-library-id(query="modelcontextprotocol python sdk")
context7:query-docs(context7CompatibleLibraryID="<resolved id>",
topic="FastMCP @mcp.tool ToolAnnotations readOnlyHint")
```
**Q: How do I write a Pydantic v2 model_validator that runs after field validation?**
```
context7:resolve-library-id(query="pydantic")
context7:query-docs(context7CompatibleLibraryID="pydantic/pydantic",
topic="model_validator mode='after' v2")
```
**Q: How do I mock an async httpx POST with respx?**
```
context7:resolve-library-id(query="respx")
context7:query-docs(context7CompatibleLibraryID="<resolved id>",
topic="respx mock async httpx POST json body")
```
**Q: How do I add a Typer sub-app for `cache` commands?**
```
context7:resolve-library-id(query="typer")
context7:query-docs(context7CompatibleLibraryID="<resolved id>",
topic="Typer add_typer sub-application command groups")
```
## After the lookup
* Cite or summarize what you found in a code comment **only when** the snippet documents a non-obvious API choice — otherwise the code is enough.
* If context7 returns nothing useful, fall back to:
1. The library's official docs site.
2. The library's repo `README` / `examples/`.
3. The smallest possible spike (a 5-line script in the venv) to verify behavior.
## Anti-patterns
* **Don't** invent a method signature from memory and hope. If you're not 100% sure of an API, look it up.
* **Don't** copy patterns from old Stack Overflow answers without verifying — Pydantic, FastMCP, and Typer all had breaking changes recently.
* **Don't** silence a warning instead of fixing the deprecation. Look up the modern API.
* **Don't** query context7 for FINN or Eiendom.no — those endpoints aren't in any public docs index. Use `tests/fixtures/` and `PRD.md` §9.
## Network configuration note
`context7` is configured as a connected MCP server in this environment. If a call fails with a connection error, surface it clearly — don't fall back to guessing.
.github/instructions/mcp.instructions.md
+192
View File
@@ -0,0 +1,192 @@
---
name: MCP rules
description: Rules for FastMCP tools, resources, and prompts
applyTo: "finn_eiendom/mcp_server.py,finn_eiendom/**/*mcp*.py"
---
# MCP server rules
The MCP server is a **thin wrapper** over `service.py`. It owns:
* Tool registration with `@mcp.tool()` and annotations.
* Pydantic input schemas (these double as tool documentation).
* Error wrapping at the protocol boundary.
* JSON / markdown response formatting via `formatting.py`.
It does **not** own:
* Parsing, scraping, scoring, cache, or HTTP fetching logic.
* SQLite or `httpx` access.
* Any orchestration of "check cache, else fetch, else save" — that's `service.py`.
## Server bootstrap
```python
# finn_eiendom/mcp_server.py
import sys, logging
from mcp.server.fastmcp import FastMCP
logging.basicConfig(stream=sys.stderr, level=logging.INFO,
format="%(asctime)s %(levelname)s %(name)s %(message)s")
mcp = FastMCP("finn_eiendom_mcp")
# ... tools registered here ...
def main() -> None:
mcp.run(transport="stdio")
if __name__ == "__main__":
main()
```
stdio servers **must** log to stderr only — anything on stdout breaks the JSON-RPC frame.
## Tool naming
All tools use the `finn_` prefix so they don't collide with other MCP servers running in the same Claude Desktop:
* `finn_analyze_search`
* `finn_get_ad`
* `finn_compare_ads`
* `finn_save_feedback`
* `finn_get_shortlist`
* `finn_get_new_ads_since_last_run`
* `finn_resolve_eiendom_unit`
* `finn_get_eiendom_unit`
* `finn_enrich_ad`
* `finn_build_unit_vector`
* `finn_decode_unit_vector`
* `finn_get_similar_units`
* `finn_find_similar_to_liked_ad`
* `finn_analyze_ad_against_comps`
## Tool body shape
Every tool body looks like this:
```python
@mcp.tool(
annotations=ToolAnnotations(
title="Analyze a FINN search URL",
readOnlyHint=True,
destructiveHint=False,
openWorldHint=True,
)
)
async def finn_analyze_search(input: AnalyzeSearchInput) -> str:
"""Analyze a FINN search URL and return a ranked shortlist."""
try:
result = await service.analyze_search(
search_url=input.search_url,
max_pages=input.max_pages,
detail_limit=input.detail_limit,
include_details=input.include_details,
include_eiendom_no=input.include_eiendom_no,
include_similar_units_for_shortlist=input.include_similar_units_for_shortlist,
)
return formatting.render_shortlist(result, input.response_format)
except Exception as e:
log.exception("finn_analyze_search failed")
return json.dumps({
"error": True,
"code": type(e).__name__,
"message": str(e),
})
```
Notes:
* Every tool delegates to `service.<function>` in one call.
* Every tool wraps in try/except and returns the error envelope as a JSON string.
* Output rendering goes through `formatting.py`, never inline.
* If the tool body needs more than ~20 lines, logic has leaked out of the service layer — push it back down.
## Input schemas
Every tool has a Pydantic v2 input model. Schemas live with the tool in `mcp_server.py` (they document the tool to LLM clients). Reuse from `models.py` only when the same shape is also a domain object — otherwise keep them as tool-local input types.
```python
class AnalyzeSearchInput(BaseModel):
search_url: str = Field(..., description="Full FINN search URL")
max_pages: int = Field(default=3, ge=1, le=10)
detail_limit: int = Field(default=20, ge=1, le=100)
include_details: bool = True
include_eiendom_no: bool = True
include_similar_units_for_shortlist: bool = False
response_format: Literal["json", "markdown"] = "json"
```
## Annotations
Set the right hints:
* Read-only tools (most of them): `readOnlyHint=True, destructiveHint=False, openWorldHint=True`.
* `finn_save_feedback`: `readOnlyHint=False, destructiveHint=False, idempotentHint=False`.
## Response format
Tools accept a `response_format` parameter (`"json"` or `"markdown"`):
* `"json"` — return `json.dumps(result_dict)`.
* `"markdown"` — return `formatting.render_<thing>(result, "markdown")`.
Errors are always returned as the JSON error envelope regardless of `response_format`.
## What stays out of mcp_server.py
* `import httpx` — never.
* `import sqlite3` — never.
* `from .ad import ...`, `from .search import ...`, `from .eiendom_no import ...`, `from .scoring import ...`, `from .cache import ...`, `from .http import ...` — never. Go through `service`.
* Output formatting logic — goes in `formatting.py`.
* Cache management — goes in `service.py`.
Allowed imports in `mcp_server.py`:
```python
import json, logging, sys
from typing import Literal, Optional
from mcp.server.fastmcp import FastMCP
from mcp.server.fastmcp.utilities import ToolAnnotations
from pydantic import BaseModel, Field
from . import service, formatting
from .models import FinnAd, EiendomUnit, SimilarUnit # only if needed for type hints
from . import config
```
`tests/test_architecture.py` enforces this.
## Resources and prompts
When you add resources or prompts, they follow the same rule: thin wrappers over `service.py` and `formatting.py`. Resources:
```
finn://preferences/current
finn://search-runs/latest
finn://search-runs/{id}
finn://ads/{finnkode}
finn://ads/{finnkode}/enriched
finn://shortlist/latest
finn://feedback/{finnkode}
finn://eiendom-units/{unitCode}
finn://eiendom-units/{unitCode}/similar/{listingStatus}
```
Prompts: `evaluate_property_for_user`, `compare_properties_for_user`, `refine_search_from_feedback`, `find_more_like_this`.
## When uncertain about FastMCP
Use `context7` for FastMCP / MCP SDK questions instead of guessing:
```
context7:resolve-library-id → "modelcontextprotocol/python-sdk" or similar
context7:query-docs(id, "FastMCP tool annotations") → snippets
```
See `docs.instructions.md`.
## Transports
* Default: stdio. `finn-eiendom-mcp` is the entry point.
* Optional: Streamable HTTP via `finn-eiendom serve --transport http --port 8010`. Path: `POST /mcp`. Operational endpoints: `GET /health`, `GET /version`, `GET /debug/config`.
* Keep tools transport-agnostic. No request/response shape depends on the transport.
.github/instructions/python.instructions.md
+80
View File
@@ -0,0 +1,80 @@
---
name: Python project rules
description: Python conventions for the FINN/Eiendom MCP server
applyTo: "**/*.py"
---
# Python conventions
## Runtime
* Python **3.12+**.
* Project-local virtualenv at `.venv/` (created by `uv venv` or `python3.12 -m venv .venv`).
* All commands run inside the activated venv.
* Editable install: `uv pip install -e ".[dev]"` (or `pip install -e ".[dev]"`).
* Never install packages globally; never use `sudo pip`; never mutate host Python.
* Add new dependencies to `pyproject.toml` in the same change that uses them.
## Language
* Use Python 3.12 syntax. Prefer `X | None` over `Optional[X]`, `list[int]` over `List[int]`, structural pattern matching where it actually helps.
* **Type hints on every function signature**, including private helpers. `mypy --strict finn_eiendom` is the target.
* Async-first for I/O. Sync code is fine for parsing, scoring, and cache access (SQLite).
* Pydantic v2 for all structured domain models, with `model_config = ConfigDict(...)`. No v1 `class Config:` blocks.
## Prefer
* Small, pure functions for parsing, normalization, and scoring.
* Explicit return types and explicit exceptions.
* Dependency injection for HTTP clients and DB connections in tests (pass `client` / `conn` as args; let services own the defaults).
* Domain names from the PRD (`FinnAd`, `EiendomUnit`, `SimilarUnit`, `analyze_search`, `get_or_fetch_ad`).
* `dataclass` for internal value objects that don't cross the API boundary; Pydantic for anything serialized or validated.
## Avoid
* Global mutable state (module-level dicts as caches, etc.). The only allowed module-level state is configuration loaded from env in `config.py`.
* Hardcoded URLs, credentials, paths, or magic numbers anywhere outside `config.py`.
* `httpx` imports anywhere except `finn_eiendom/http.py`.
* `sqlite3` imports anywhere except `finn_eiendom/cache.py`.
* `BeautifulSoup` imports anywhere except `finn_eiendom/search.py` and `finn_eiendom/ad.py`.
* `msgpack` imports anywhere except `finn_eiendom/eiendom_no.py`.
* Scraping, scoring, cache, or HTTP fetching logic inside MCP tool or CLI command bodies.
* Direct network calls in unit tests — use `respx` and fixtures.
* `print()` for logging — use the `logging` module. stdio MCP server logs go to **stderr only**.
* Bare `except:` or `except Exception: pass` — catch the specific exception or let it propagate.
## External fetches
All external fetches must support:
* Configurable request delay (`FINN_REQUEST_DELAY_SECONDS`, `EIENDOM_NO_REQUEST_DELAY_SECONDS`).
* Cache lookup before fetch.
* Retry on 5xx with exponential backoff (`1s, 2s, 4s`).
* Graceful failure that returns `None` or empty rather than raising, when the caller can degrade.
* Structured logging at INFO for success, WARNING for retry, ERROR for final failure.
## Best practices
* **Single responsibility per function.** If a function name needs "and" to describe it, it's two functions.
* **Function length:** aim for under 30 lines. Past 50 lines it's a code smell — extract helpers.
* **Cyclomatic complexity:** if you've got more than 3 levels of nesting, the function wants splitting.
* **Naming:** `get_or_fetch_ad`, not `process_ad`. Verbs for actions, nouns for data. Avoid abbreviations except those well-known in the domain (`url`, `ad`, `nok`).
* **DRY:** if you write the same logic, regex, SQL, or format string twice, extract it. The decision table in `PRD.md` §17.2 tells you where it belongs.
* **Comments explain WHY**, not WHAT. The code already says what.
* **Errors are loud:** raise with actionable messages (`f"Unknown listing_status {status!r}; expected one of {VALID_STATUSES}"`). The MCP boundary wraps them as `{"error": True, ...}`.
## When uncertain about a library API
Use the `context7` MCP server **before** writing code:
1. `context7:resolve-library-id` with the package name → canonical library ID.
2. `context7:query-docs` with that ID + focused topic.
See `docs.instructions.md`. Don't guess from training memory — Pydantic, FastMCP, and Typer all change.
## Tooling
* `ruff check .` — lint. Target Python 3.12. Active rules: `E F I UP B SIM`.
* `ruff format .` — format. Line length 100.
* `mypy --strict finn_eiendom` — type-check.
* `pytest` — run the full suite.
.github/instructions/tests.instructions.md
+199
View File
@@ -0,0 +1,199 @@
---
name: Test rules
description: Testing conventions for parser, cache, scoring, service, MCP, CLI, and architecture
applyTo: "tests/**/*.py"
---
# Test rules
## Runtime
Tests run in the project-local `.venv`. From the project root with the venv activated:
```bash
pytest # full suite
pytest tests/test_service.py -v # one file
pytest -k "shortlist" # one keyword
pytest --lf # rerun last failures
```
`pytest-asyncio` is in `[tool.pytest.ini_options]` with `asyncio_mode = "auto"` — `async def` tests run without an `@pytest.mark.asyncio` decorator.
## Never do live network calls
No real HTTP in unit tests. Mock with `respx` (sits in front of `httpx.AsyncClient`):
```python
import respx, httpx
from finn_eiendom import http as http_module
@respx.mock
async def test_finn_search_fetch_uses_user_agent():
route = respx.get("https://www.finn.no/realestate/homes/search.html").mock(
return_value=httpx.Response(200, html=SAMPLE_FINN_SEARCH_HTML)
)
client = http_module.HTTPClient(user_agent="test-agent")
resp = await client.get("https://www.finn.no/realestate/homes/search.html")
assert resp.status_code == 200
assert route.calls.last.request.headers["user-agent"] == "test-agent"
```
## Fixtures
Fixture-driven testing for parsers and APIs:
* FINN search HTML → `tests/fixtures/finn_search.html`.
* FINN listing HTML → `tests/fixtures/finn_ad_*.html`.
* Eiendom.no unit search JSON → `tests/fixtures/eiendom_unit_search.json`.
* Eiendom.no unit detail JSON → `tests/fixtures/eiendom_unit_detail.json`.
* Eiendom.no similar-units JSON → `tests/fixtures/eiendom_similar.json`.
Loader helpers in `tests/fixtures.py` (e.g. `SAMPLE_FINN_SEARCH_HTML`, `SAMPLE_EIENDOM_UNIT_JSON`). Add new fixtures here, don't inline large strings in test files.
## Test layout
```
tests/
fixtures/ # raw HTML / JSON inputs
fixtures.py # loader helpers
conftest.py # shared pytest fixtures (tmp DB, http client, etc.)
test_parser.py # number/area/date/URL/finnkode normalization
test_search.py # FINN search HTML → cards
test_ad.py # FINN listing HTML → FinnAd
test_eiendom_no.py # unit search/detail/similar JSON, unit_vector encode/decode
test_scoring.py # all scoring components + classifier
test_cache.py # SQLite read/write/TTL
test_http.py # retry on 5xx, raise on 4xx, delay applied (new)
test_service.py # get_or_fetch_*, analyze_* (new)
test_formatting.py # render_* json/markdown/table (new)
test_mcp_server.py # tool registration + error envelope (expanded)
test_cli.py # typer CliRunner (new)
test_architecture.py # import-graph invariants (new)
```
## What to test per category
### Parsers (`test_parser`, `test_search`, `test_ad`, `test_eiendom_no`)
* Missing fields → `None`, not exception.
* Norwegian number formats: `7 200 991 kr`, `kr 7 200 991`, `7.200.991`.
* URL normalization (relative → absolute).
* Finnkode extraction from various URL shapes.
* Area parsing: `77 m²`, `77m2`, `77 kvm`.
* Price parsing (asking vs total vs shared debt).
* Eiendom.no JSON edge cases: empty `units`, missing `valuation`, missing `latestMarketData`.
### Unit vectors (`test_eiendom_no`)
* msgpack encoding + base64url without padding.
* Decode roundtrip.
* Missing optional fields (floor, rooms, built).
* Both lon/lat orderings handled.
### Scoring (`test_scoring`)
* Each component in isolation.
* Total clamped to 0–100.
* Risk penalties applied (negative range).
* Bargain classification triggers on the expected signal mix.
* Hybel classification: documented / possible / unclear / not relevant.
* Explainability: explanation list non-empty when score is non-trivial.
### Cache (`test_cache`)
* Read after write returns same object.
* TTL expiry returns `None`.
* JSON roundtrip preserves all fields.
* `init_db` is idempotent on existing DBs.
### HTTP (`test_http`)
* Retries on 500/502/503/504 with backoff (count exactly N retries).
* Raises immediately on 404 / 4xx.
* Applies `request_delay` between calls.
* Honors `user_agent`.
### Service (`test_service`)
The service tests are the heart of the suite. They cover orchestration end-to-end against fixtures.
* `test_get_or_fetch_ad_uses_cache` — second call hits cache, no HTTP.
* `test_get_or_fetch_ad_fetches_when_cache_miss` — first call hits HTTP, then writes cache.
* `test_get_or_fetch_ad_force_refresh` — `force_refresh=True` bypasses cache.
* `test_analyze_search_with_fixtures` — full run from search HTML → shortlist.
* `test_find_similar_to_liked_uses_liked_feedback` — only seeds from `liked` verdicts.
Use a tmp SQLite DB via the `tmp_path` pytest fixture:
```python
@pytest.fixture
def tmp_db(tmp_path, monkeypatch):
db_path = tmp_path / "finn.sqlite"
monkeypatch.setenv("FINN_CACHE_PATH", str(db_path))
return db_path
```
### Formatting (`test_formatting`)
* `render_shortlist(result, "json")` is parseable JSON and roundtrips.
* `render_shortlist(result, "markdown")` contains the score and at least one risk.
* `render_<thing>(result, "xml")` raises `ValueError` listing supported formats.
### MCP (`test_mcp_server`)
* `test_mcp_server_has_correct_tools` — all 14 `finn_*` tool names registered.
* `test_finn_decode_unit_vector_returns_json` — happy path.
* `test_finn_analyze_search_handles_error` — error envelope shape: `{"error": True, "code": ..., "message": ...}`.
Use the `mcp` SDK's testing helpers; don't spawn a subprocess.
### CLI (`test_cli`)
Use Typer's `CliRunner`:
```python
from typer.testing import CliRunner
from finn_eiendom.cli import app
runner = CliRunner()
def test_cli_help():
result = runner.invoke(app, ["--help"])
assert result.exit_code == 0
assert "analyze-search" in result.stdout
```
Patch `service.<function>` with `monkeypatch` so CLI tests don't exercise the full stack — that's covered by `test_service.py`.
### Architecture (`test_architecture`)
Static checks of the module dependency graph:
* No `import httpx` outside `finn_eiendom/http.py`.
* No `import sqlite3` outside `finn_eiendom/cache.py`.
* No `BeautifulSoup` import outside `search.py` and `ad.py`.
* No `msgpack` import outside `eiendom_no.py`.
* `mcp_server.py` only imports from `service`, `formatting`, `models`, `config`, `mcp`, stdlib, `pydantic`.
* `cli.py` only imports from `service`, `formatting`, `models`, `config`, `typer`, stdlib.
* `service.py` does not import from `mcp_server` or `cli`.
Implementation: walk `.py` files under `finn_eiendom/` with `ast`, collect imports, assert allowed sets per module.
## Best practices
* One assertion per test (or per closely related group). Long tests die in painful ways.
* Test names describe the behavior: `test_get_or_fetch_ad_uses_cache_within_ttl`.
* Use `monkeypatch` for env vars and `tmp_path` for files. No `os.environ` mutation.
* No `time.sleep` — use `freezegun` if a test depends on time, or refactor the code under test to take a `now` parameter.
* No "smoke tests" that ping real servers — those go under a separately-marked `pytest -m live` suite and are not part of CI.
## When uncertain about test tooling
Use `context7` for pytest, respx, freezegun, or Typer testing:
```
context7:resolve-library-id → "pytest-dev/pytest" / "lundberg/respx"
context7:query-docs(id, "respx mock httpx async post")
```
See `docs.instructions.md`.