sims-hq/docs/superpowers/specs/2026-07-17-module-field-schema-design.md

HQ Console — Module-Defined Field Schema (D21)

STATUS: DELIVERED 2026-07-17 (built additively over D20; suite 363 green). Approach: config on the module (the simplest of the three weighed). North star (founder): the app, the modules, and the client↔module mapping must stay flexible — a new module or a changed field must never need a code change or a DB migration.

Verified on real data (re-import into local Postgres): RTGS declares 13 fields, SMS 6, MobileApp 1; 53/53 RTGS links and 102/105 SMS links carry structured field values (IP, IP code, partner bank, IFSC, QR, SMS balance, reseller, alert types…) instead of a free-form blob. Owner defines a module's fields on the Modules page; every client on that module shows the form automatically; secret fields encrypt with an audited reveal.

Built additively (deliberate, for the confirm-after-build gate): D20's login trio (provider/username/portal-password) stays in its own columns/card; D21 adds the module-specific fields on top. Retiring the D20 columns into field_spec is a later cleanup once the direction is confirmed.

1. The problem

Per-client service data lives on the client_module link (good — never on the client record). But its shape is only half-structured: a few fixed columns (provider/username/password_enc) that are SMS/RTGS-shaped, plus a free-form details JSON list. The module does not declare what fields it has, so:

• every client's details are ad-hoc — no consistent labels, no validation, no required-ness;
• the service card cannot render a real form (it doesn't know what fields should exist);
• the importer maps by guesswork; a future module has no defined home for its fields.

2. The idea (config over code, dated — house rule)

The module declares its own operational fields as configuration. Per-client values are then organized by that module's schema. Adding a module = define it + its fields as DB data. Zero code, zero migration.

module.field_spec (JSON) = [
  { key, label, type, required?, options?, help?, sort? }
]  type ∈ text | secret | date | number | select | boolean

• Non-secret values live in client_module.details as [{ key, value }] (label/type come from the module, so values stay small and never drift from the definition).
• Secret values (type secret) live in ONE client_module.secrets_enc blob — AES-256-GCM (HQ_SECRET_KEY) over a JSON map { fieldKey: plaintext }. Generalizes today's single password_enc; N secrets, one encrypt/decrypt. Reveal endpoint decrypts and returns the map (or one key); every reveal audited, exactly as the DB password is today.

3. Data model (additive, both engines)

Change	Notes
module.field_spec TEXT NOT NULL DEFAULT '[]'	the field definitions (JSON)
client_module.secrets_enc TEXT	generalized secret store (replaces the single portal password path)
keep client_module.details	now the value store keyed by field key (validated against field_spec)

Migration from D20 (no data loss): each existing SMS/RTGS/ATM module gets a seeded field_spec describing the fields it already uses (provider, username, portal password as a secret, IP/IFSC/partner-bank, etc.); existing details values (already labeled) map to the matching keys; the existing password_enc folds into secrets_enc under portal_password. provider/username/password_enc columns are retired once values are migrated (or kept as convenience aliases — decided at build time; retiring is cleaner).

4. Behavior

• Service card (Client 360 → module): renders a form from the owning module's field_spec — text/date/number/select/boolean inputs; secret fields as •••• + Reveal (managerial, audited) + Set/Update; required fields validated. No hard-coded field names.
• Module editor (owner): a "Fields" section to define/reorder/type the field_spec — this is the surface where a module "defines itself." Config over code, so a new field appears on every client of that module immediately.
• Ticket + roster module pickers: already read the live catalog (fixed in D20) — a new module flows everywhere with no code change.
• Importer: maps APEX columns → defined field keys per module (targets keys instead of free labels); required fields absent → a warning note, never a blocking problem.

5. Flexibility guarantees (the founder mandate, made concrete)

1. New module = one catalog row + its field_spec. It instantly gets price book, roster (both directions), quote content, service card, renewals, revenue — no code, no migration.
2. New/changed field on an existing module = edit field_spec. Every client of that module picks it up; old values untouched; nothing to re-key.
3. client↔module mapping stays a first-class link (client_module) — reassign, add, deactivate, bulk-correct without touching client or module identity.
4. Secrets scale without new columns (one encrypted map per link).

6. Non-goals (deliberate, revisit later)

• Cross-client reporting/filtering by a module field (e.g. "every client's IFSC in one table") — that is the heavier "field tables" approach (option 2); add only if a dashboard needs it.
• Dating field definitions (a field's own effective-from history) — fields are config now, not price-like facts; add if a module's field set must change on a date.
• Bulk-edit UI across clients — separate small slice, add when cleanup friction is felt.

7. Build order (after the D20 trial import; suite green each phase)

1. Schema + field_spec/secrets_enc + generalized secret repo (encrypt/reveal map) + migration of D20 columns; tests.
2. API: field_spec on module CRUD; client_module value writes validated against field_spec; multi-secret reveal (audited).
3. Web: dynamic service-card form from field_spec; module "Fields" editor.
4. Importer: target defined keys; seed the five modules' field_spec from the real APEX columns.
5. Docs + D-record.