Architecture decisions

Five choices that shaped this platform and would otherwise be re-litigated every time someone new opens the repository and finds them surprising. Each entry says what the situation was, what was decided, what it costs as well as what it buys, and — the part that stops an ADR becoming dogma — what evidence would make us change our minds.

All five were taken during the August 2026 extraction of placegraph from Kiln / Makers Map. All are accepted and load-bearing today.

1. Namespace packages over one distribution

Context

Kiln was one importable package, kiln, holding everything: the graph, the sources, the web app, the posters, the deploy kit. Extracting it left a choice between one distribution called placegraph with optional extras, and several distributions sharing a namespace.

The forces:

• A site runs three or four sources, not thirteen. Installing an iXBRL parser and a headless-browser tier to load a council rating list is waste, and every dependency is something that can break a deploy at the wrong moment.
• The graph layer changes slowly and the web layer changes weekly. One version number over both means either the graph is versioned by the web's churn, or nobody bumps it and the number means nothing.
• placegraph-core genuinely depends on nothing but the standard library, and that is a property worth being able to state and enforce rather than intend.

Decision

Ten distributions — placegraph-core, -geo, -sources, -scoring, -web, -compliance, -outreach, -content, -metrics, -deploy — sharing the placegraph.* namespace as PEP 420 implicit namespace packages. There is no src/placegraph/__init__.py in any of them. Each has its own pyproject.toml, its own dependency list and its own version. Sites pin a git tag and upgrade deliberately; there is no PyPI release until a third consumer exists.

Consequences

Good:

• A site's dependency list is a statement about what it actually does. Circuit declares all nine; a hypothetical read-only mirror could declare core and web and get an install with four transitive dependencies.
• placegraph-core importing anything outside the standard library is now a visible change to a file, not a drift.
• Optional packages degrade rather than break. placegraph.web probes with importlib.util.find_spec and hides the guides section when placegraph-content is absent; the chart helpers return an empty string rather than raising, so a page says everything it said without the redundant copy.
• Lazy module loading falls out naturally. placegraph.sources.__getattr__ (PEP 562) means sources.nndr imports one module, not thirteen.

Costs, and they are real:

• The empty-__init__.py trap. Adding src/placegraph/__init__.py to any one package shadows the namespace and breaks every other package's imports, with an error that names the wrong thing. It is the first thing to check when imports fail, and it is checked in every package build.
• No workspace concept in plain pip. pip install -e . at the root cannot install ten editable packages, so scripts/dev-install.py exists and the root pyproject.toml deliberately lists no dependencies — otherwise pip goes looking on PyPI, where they are not published.
• Ten version numbers is ten things to keep coherent. Mitigated by the git-tag pinning: sites move all ten together, so the numbers document intent rather than gate compatibility.

What would change it

Collapse to one distribution with extras if, after three or four sites exist, every site installs all ten anyway and the __init__.py trap keeps firing in review. The split earns its keep only while somebody is genuinely installing a subset; if nobody is, it is ten times the packaging for a property nobody uses. The reverse move is also available and cheap: publishing a placegraph metapackage that depends on all ten costs one file and changes no imports.

2. SiteConfig injection over module globals

Context

Kiln's config.py was module-level globals read from KILN_* at import time:

DATA_DIR = Path(os.environ.get("KILN_DATA_DIR", ROOT / "data"))
SITE_NAME = os.environ.get("KILN_SITE_NAME", "Kiln")
TARGET_LADS = {"E06000045": "Southampton"}

Flat, boring, and completely adequate for one site. For two it is impossible on three counts: the values are frozen at first import, there is no way to hold two of them at once, and the prefix is part of the source code.

The obvious alternatives were a settings framework with layered files, or threading a config parameter through every function signature.

Decision

The same flat, boring set of values — no framework, no layering, no tree — but as a frozen SiteConfig dataclass resolved from a named environment prefix, constructed by the site and installed once:

CONFIG = SiteConfig.from_env(slug="circuit", env_prefix="CIRCUIT", ...)
settings.configure(CONFIG)

Everything downstream reads settings.active(). There is no default site: a package that runs before a site has configured one raises NotConfigured with a specific message rather than falling back to a half-built default.

Two supporting rules, both enforced by review rather than by machinery:

• Never capture settings.active() in a module-level constant. Read it inside the function.
• Facts about the product live in code; facts about a deployment — hostname, secrets, caps, gates — are overridable from the environment.

Consequences

Good:

• The env prefix is a parameter. KILN_DB and CIRCUIT_DB are the same setting for two sites, and neither name appears in platform code.
• settings.using(cfg.with_(demo_mode=True)) makes configuration-dependent behaviour testable without subprocesses or environment mutation. Most of the platform's more interesting tests are one with block.
• Failing loud beats a default. The failure that error prevents — writing to the wrong database, publishing under the wrong brand — is not one you want to discover in production, and there is no plausible default that is safer than refusing.
• Vertical-specific objects ride along in the same place: taxonomy, score_profile, headline_score, theme_dir, migrations_dirs, extra. Injecting them through config is why no platform module imports a site.

Costs:

• settings.active() is process-global. Two sites cannot be served from one process, and using() is not safe under concurrency. This is a deliberate ceiling, not an oversight: one droplet per site is the deployment model.
• The captured-constant rule is a convention with no compiler behind it. It has been broken once already, in a module that cached the taxonomy on an instance, and the symptom was a test that passed alone and failed in a suite.
• SiteConfig is wide — roughly sixty fields. Every attempt to organise it into sections has made it harder to answer "what is this actually set to on the box?", which is the question it exists to answer.

What would change it

Make _active a contextvars.ContextVar the moment a single process genuinely needs two live configs — a shared admin surface across sites, or a test runner choosing parallelism over isolation. That is a contained change: active(), configure() and using() are the only readers of the module global, and using() already has the right shape for it.

A settings framework would need a different justification entirely: layered files, schema validation and secret backends solve problems this does not have, and each one adds a place where the answer to "what is it set to" lives.

3. Scores as data over columns

Context

Kiln had two numbers per entity and made them columns:

sole_trader_likelihood REAL,   -- 0..1, explainable via entity_signal
craft_confidence       REAL,

with entity_signal(entity_id, signal, weight, detail) — no score column, because there was only one score to explain.

Circuit needs four: locally_present_likelihood, registered_office_only, contractor_likelihood, hiring. On the column model that is four migrations, four new WHERE clauses in publish, four more columns on a table that is already the widest in the schema, and a name in the platform's own DDL that means nothing to any other vertical.

Decision

Scores are rows:

CREATE TABLE entity_score (
    entity_id INTEGER, score TEXT, value REAL, computed_at TEXT,
    PRIMARY KEY (entity_id, score));
CREATE TABLE entity_signal (
    entity_id INTEGER, score TEXT, signal TEXT, weight REAL, detail TEXT, ...);

A vertical declares ScoreDefinition(name, question, weights, collector) objects and hands the platform a ScoreProfile. Publication thresholds are keyed by score name, with a max_ prefix inverting the comparison. entity.headline_score is a denormalised copy of the site's primary score, and entity_score is always the authority.

question is required and is plain English, shown on the profile beside the number: if a score cannot be written as a sentence a listed business would recognise, it is not one we should be publishing.

Consequences

Good:

• Adding a score is a definition, not a migration. Circuit added four without touching the schema.
• The explanation generalises with it. record_signals is per (entity, score), so each number carries its own working — which is the Article 14 requirement, not a nicety, and also the only reason weights ever got tuned on evidence.
• Thresholds compose. publish_thresholds is a dict, each entry becomes one EXISTS clause, and a record with no such score has not met it in either direction and stays internal — the conservative reading in both senses.
• The graph stops naming one vertical's question. entity.category_primary and entity_score say nothing about crafts or presence, which is what makes the same DDL serve both directories.

Costs:

• A score is a join. Ordering the directory by the headline score would be a join on every listing page, which is why entity.headline_score exists as a denormalised copy — and a denormalised copy is a thing that can go stale. db.set_score writes both in one call, and it is the only writer, but that is a discipline rather than a constraint.
• Reading five scores for one entity is five rows to pivot, and a report wanting scores as columns has to write the CASE WHEN itself.
• A typo in a score name is a silently absent threshold rather than a SQL error. Mitigated where it matters most: ctx.signal() raises on a signal name the definition has no weight for, because scoring a typo as zero would hide it behind a plausible number.
• One historical trap, now fixed and worth remembering: matching entity_score.score against the prefixed key max_registered_office_only found nothing, and a threshold that nothing can satisfy publishes an empty directory — every record failing a rule that does not exist.

What would change it

If a vertical ever needs a dozen scores filtered together in one query and the join cost shows up in a page render, add a generated view or promote the two or three hottest scores to generated columns. The storage model would not change; only the read path would gain a cache.

If the scores stopped being explainable sums — if one became a model output — the argument would need reopening entirely, because the whole design assumes that a score is a list of named signals and that the list is the point.

4. Shared libraries and shared data artifacts over shared runtime services

Context

Two directories, two £10 droplets, three heavyweight national datasets that are identical for both: a 5.5m-row Companies House snapshot, the ONS Postcode Directory, the ICO register. Building those on each box costs the same bandwidth and the same hour every month, per site, for byte-identical results.

The obvious modern answer is a service: one API that both sites query for company lookups, postcode attribution and matching. The obvious cheap answer is to keep doing it twice.

Decision

Neither. Shared libraries plus shared data artifacts, and no shared runtime services.

• The code is shared by pip install, pinned to a git tag.
• The data is shared as files: one host builds them, publishes manifest.json with checksums beside them, and each site pulls what has changed at ingest time only, verifies the checksum, and serves from local files thereafter.
• Each site keeps its own graph, its own suppression list and its own claims. Different publics, one controller.

The consumer contract is four rules — pull at ingest time only, fail soft, keep the last good copy, alarm on staleness — and the enforcement is structural: the web process is never given network access to the refinery, placegraph.web does not depend on placegraph-sources, and the pull is a separate systemd unit on an earlier timer. See refinery.md.

Consequences

Good:

• At runtime a site depends on nothing but its own box. Makers Map's droplet dying cannot take Circuit down; it can only make Circuit's reference data slowly older.
• The failure mode is bounded and boring. A month-old Companies House snapshot answers almost every question this month's would, so "the refinery is down" degrades to "the numbers are a bit old", visible as a staleness figure rather than as an outage.
• No service mesh, no service discovery, no shared authentication surface, no API versioning, no deployment coordination. Two sites do not need any of that and would have to operate all of it.
• The whole thing is a URL and a manifest, which is why it can move to a dedicated ingestion VM later by changing one line in one env file per site, with unchanged on every artifact as the proof it worked.

Costs:

• Storage is duplicated. Each site holds its own copy of a 1–2GB snapshot. At two sites that is cheaper than a service; at twenty it would not be.
• No cross-site queries. An agency listed in both directories is two entities with no link between them. Cross-site entity resolution is a Phase-2 nicety by explicit decision, not an accident.
• Sites can drift. Two sites pulling on different days hold different snapshots, so a count taken on one is not reconcilable with a count taken on the other unless both name their artifact's built_at.
• The contract is only real while it is enforced. "The refinery becomes a hidden runtime dependency" is on the risk register precisely because one convenient import would make it true.

What would change it

A genuinely interactive shared need — cross-site identity resolution answered at query time, a shared claims account across properties, or a fourth and fifth site where per-site copies of the national artifacts stop being cheap. Any one of those is a real service, and at that point the refinery host is already the obvious place to put it, with the manifest URL as the seam it grows from.

Note the asymmetry that makes this reversible: going from files to a service is an addition. Going from a service back to files means unpicking every call site that learned to assume the network was there.

5. Site migrations numbered from 1000 over a registry

Context

Migrations now come from more than one place: the platform ships 0001_graph through 0004_web_abuse, and each site ships its own. They have to apply in a single, well-defined order, and a site migration must be able to reference a platform table.

The usual answers are a registry (each package declares its migrations, a resolver orders them), or per-package migration tables, or timestamp-based version numbers.

Decision

One schema_migration table, one ordered stream, merged from several directories by version number. The platform occupies 0001–0999; sites start at 1000 — migrate.SITE_VERSION_FLOOR. A duplicate version across the directories is a hard SystemExit naming both files and reminding the reader where the floor is.

def migration_dirs() -> list[Path]:
    return [migrate.PLATFORM_MIGRATIONS, *[Path(p) for p in cfg.migrations_dirs]]

Everything else about the runner is ordinary: -- +up / -- +down sections, a -- description: header, checksums so an edited migration is detected rather than ignored, and per-migration transactions.

Consequences

Good:

• A site migration can safely reference any platform table, because every platform version is lower than every site version. No dependency declarations, no topological sort, no way to express a cycle.
• No registry to keep in sync, and therefore no failure mode where the registry and the filesystem disagree — which is the failure a registry is supposed to prevent and the one it most often causes.
• migrate.status() prints one list with an origin column, so "what is the schema of this database" is one question with one answer.
• The gap is deliberately enormous. 999 platform migrations at the current rate is decades, so the floor is not a constraint anybody will feel.
• It composes with baseline(), which is what made the Makers Map backport tractable at all: platform 0001–0004 are recorded as baselined against a database that already carries their effects under kiln's own version numbers, and only 1000 onwards actually runs. See backport-kiln.md.

Costs:

• Only two streams. If a package other than placegraph-core wanted to ship migrations — say placegraph-outreach owning venue and touch — there is no range for it. Today those tables live in the platform stream, which is a compromise: placegraph-core's migrations create tables that only other packages use.
• Two sites cannot share a migration number, even though they never share a database. Harmless, and mildly confusing the first time someone copies a migration between sites and has to renumber it.
• Version numbers are chosen by a human, so two branches can both add 1004. Caught at discovery time with a message naming both files, but caught at discovery time rather than at review time.
• Timestamp versions would avoid that collision. They were rejected because 1000, 1001, 1002 is readable in a status table and 20260813142211 is not, and because a monotonic small integer makes the platform/site split visible at a glance.

What would change it

Move to explicit per-package ranges — core 0001–0499, other platform packages 0500–0999, sites 1000+ — the first time a package other than core genuinely needs to own a table's lifecycle. That is a constant change plus a discover() assertion, and the existing databases would not notice.

Move to a full registry only if migrations ever need to interleave in an order that is not a total order by package — a site migration that has to run before a platform one. If that requirement appears, treat it as a design smell first: it usually means a platform migration is encoding a vertical's assumption.