The placegraph contract

Every package builds against this. It is stable; if you need something that is not here, add it to placegraph-core and update this document in the same change.

1. Layout

placegraph/
  packages/placegraph-<name>/
    pyproject.toml                       # name = "placegraph-<name>"
    src/placegraph/<name>/__init__.py     # module = placegraph.<name>

PEP 420 implicit namespace packages: there is no src/placegraph/__init__.py in any package. Adding one breaks every other package's imports.

placegraph-core depends only on the standard library. Other packages may depend on placegraph-core and, where genuinely needed, httpx; placegraph-web adds fastapi, uvicorn, jinja2, python-multipart.

2. Configuration

There is no module-level config and no default site. Everything reads the active SiteConfig:

from placegraph.core import settings

cfg = settings.active()          # raises settings.NotConfigured if unset
cfg.slug, cfg.site_name, cfg.base_url, cfg.data_dir, cfg.db_path
cfg.target_lads, cfg.neighbour_lads, cfg.all_lads, cfg.listed_lads
cfg.postcode_areas, cfg.postcode_districts, cfg.towns
cfg.taxonomy, cfg.score_profile, cfg.headline_score
cfg.demo_mode, cfg.publish_live_data, cfg.publish_thresholds
cfg.env("SOME_EXTRA", default, cast=int)     # read {PREFIX}_SOME_EXTRA

Never read os.environ["KILN_..."] or any hardcoded prefix. Never cache settings.active() at import time in a module-level constant — tests swap the config with settings.using(...), and a captured value defeats that. Read it inside the function.

3. Core API

from placegraph.core.db import (
    connect, session, init_db,
    record_fact, facts_for, facts_by_key,
    find_entity, upsert_entity, attach_address, recompute_entity_lad,
    suppression_keys, is_suppressed, add_suppression,
    set_score, get_score, scores_for, record_signals, signals_for,
    rebuild_fts, start_run, finish_run, now_iso, hash_ip, slugify,
    LAD_ROLE_PRIORITY,
)
from placegraph.core.normalise import (
    now_iso, normalise_name, normalise_postcode, postcode_district,
    postcode_area, normalise_domain, normalise_phone, format_phone,
    clean_display_name, slugify, hash_ip, in_scope,
    has_legal_suffix, has_company_token, looks_personal_name,
    find_company_number,
)
from placegraph.core import joblog, migrate, sources as source_registry

Signatures you will use constantly:

upsert_entity(conn, *, display_name, source_id, source_ref=None, kind="unknown",
              company_number=None, website=None, phone=None, email=None,
              postcode=None, address_lines=None, lat=None, lng=None,
              category_hint=None, extra_facts=None, confidence=0.6,
              fetched_at=None, address_role="primary") -> (entity_id|None, created)

record_fact(conn, entity_id, key, value, *, source_id, source_ref=None,
            fetched_at=None, confidence=0.5, ttl_days=None) -> None

attach_address(conn, entity_id, *, postcode, lines=None, lat=None, lng=None,
               role="primary", source_id="crawl") -> address_id|None

set_score(conn, entity_id, name, value)
record_signals(conn, entity_id, score, [(signal, weight, detail), ...])

upsert_entity returns (None, False) for a suppressed record. Always check for None before using the id — a suppressed record silently skipped is the correct behaviour, an AttributeError is not.

4. Schema names

The graph is vertical-neutral. Use these, never craft/sector/sole-trader names:

Platform migrations are 0001–0999 in placegraph/core/migrations/. Site migrations start at 1000 and live in the site package; both streams merge by version number.

Do not add CREATE TABLE IF NOT EXISTS schema evolution anywhere. New tables are new numbered migrations.

5. Taxonomy (placegraph.scoring.taxonomy)

@dataclass(frozen=True)
class Category:
    code: str
    label: str
    sic: tuple[str, ...]        # codes SPECIFIC to this category, may be empty
    keywords: tuple[str, ...]
    blurb: str = ""

class Taxonomy:
    def __init__(self, categories, *, broad_sic={}, excluded_sic={},
                 reconciliation_name=""): ...
    categories: list[Category]
    def label_for(code) -> str
    def get(code) -> Category | None
    def sic_to_category(sic) -> str | None    # None unless SPECIFIC
    def is_broad_sic(sic) -> bool             # corroborates, never classifies
    def is_excluded_sic(sic) -> bool
    def is_in_scope_sic(sic) -> bool          # the single ingestion-time test
    def discovery_sic_codes() -> set[str]
    def classify(name, *texts) -> (code|None, confidence, hits)

The two rules that must survive: SIC lookup checks five digits then four (47781 art galleries vs 47789 everything else), and a broad code never assigns a category on its own.

6. Scores (placegraph.scoring.engine)

@dataclass(frozen=True)
class ScoreDefinition:
    name: str
    question: str                       # plain-English, shown on the profile
    weights: dict[str, float]           # signal name -> weight
    collector: Callable[[SignalContext], list[Signal]]
    squash: str = "logistic"            # logistic | clamp | raw
    gain: float = 3.0

class ScoreProfile:
    scores: list[ScoreDefinition]
    def run(conn, entity_id) -> dict[str, float]
    def run_all(conn) -> int

@dataclass
class SignalContext:
    conn; entity: sqlite3.Row; facts: dict[str, list[str]]; sources: set[str]
    audit: sqlite3.Row | None; pages: list[sqlite3.Row]; accounts: sqlite3.Row | None
    premises: list[sqlite3.Row]; taxonomy; config
    def signal(name, detail) -> Signal    # weight looked up from the definition

A collector returns only the signals that fired; the engine looks up weights, squashes, persists the value with set_score and the explanation with record_signals.

7. Publication

placegraph.scoring.publish.publish(conn) is the only thing that sets entity.is_public. It enforces, in order:

1. demo_mode publishes only synthetic records; publish_live_data publishes only real ones; with both off nothing is published.
2. Publishing live data with cfg.missing_controller_fields() non-empty raises SystemExit — a notice that cannot name the controller is not a notice.
3. Every threshold in cfg.publish_thresholds must be met (>= for a min_*-style score, <= for one prefixed max_).
4. status in ('candidate','verified','claimed'), a lad_code in cfg.listed_lads, and a category when cfg.require_category_to_publish.
5. status='claimed' overrides the thresholds but never the gates.
6. removed, suppressed, removal_pending are always hidden.

8. Sources

Each source module exposes SOURCE_ID and an ingest(...)/run(...) returning a JSON-serialisable dict of counts. Every one:

• opens a run with start_run / closes it with finish_run;
• commits incrementally (every ~250 rows) and prints progress;
• skips work it has already done, so re-running is cheap and safe;
• honours its cap from config in code, not by convention, so a loop bug costs nothing.

9. Web

placegraph.web.create_app(config=None) -> FastAPI. Templates resolve through ChoiceLoader([FileSystemLoader(cfg.theme_dir), FileSystemLoader(base)]), so a site overrides home.html by shipping its own and inherits everything else. Routes are registered by small register_*(app, deps) functions in placegraph/web/routes/, so a site can add its own without forking the factory.

10. House style

Match the code this was extracted from:

• Module docstring explaining why the module exists and what it got wrong before. Comments explain reasoning, not mechanism.
• from __future__ import annotations, modern typing, 4-space indent, ~88 cols.
• No new dependencies without a reason that survives being said out loud.
• Never log or store a raw IP; hash_ip exists for that.
• Every metered call is capped in code.
• British English in user-facing copy.