sims-hq/docs/superpowers/specs/2026-07-17-module-field-schema-design.md

# HQ Console — Module-Defined Field Schema (D21)

> **STATUS: DELIVERED 2026-07-17** (built additively over D20; suite 363 green). Approach:
> *config on the module* (the simplest of the three weighed). North star (founder): the app,
> the modules, and the **client↔module mapping** must stay flexible — a new module or a
> changed field must never need a code change or a DB migration.
>
> **Verified on real data** (re-import into local Postgres): RTGS declares 13 fields, SMS 6,
> MobileApp 1; 53/53 RTGS links and 102/105 SMS links carry structured field values (IP, IP
> code, partner bank, IFSC, QR, SMS balance, reseller, alert types…) instead of a free-form
> blob. Owner defines a module's fields on the Modules page; every client on that module
> shows the form automatically; secret fields encrypt with an audited reveal.
>
> **Built additively** (deliberate, for the confirm-after-build gate): D20's login trio
> (provider/username/portal-password) stays in its own columns/card; D21 adds the
> module-specific fields on top. Retiring the D20 columns into field_spec is a later cleanup
> once the direction is confirmed.

## 1. The problem

Per-client service data lives on the `client_module` link (good — never on the `client`
record). But its *shape* is only half-structured: a few fixed columns
(`provider`/`username`/`password_enc`) that are SMS/RTGS-shaped, plus a free-form `details`
JSON list. The module does not **declare** what fields it has, so:
- every client's `details` are ad-hoc — no consistent labels, no validation, no required-ness;
- the service card cannot render a real form (it doesn't know what fields *should* exist);
- the importer maps by guesswork; a future module has no defined home for its fields.

## 2. The idea (config over code, dated — house rule)

**The module declares its own operational fields as configuration.** Per-client values are
then organized by *that module's* schema. Adding a module = define it + its fields as DB
data. Zero code, zero migration.

```
module.field_spec (JSON) = [
  { key, label, type, required?, options?, help?, sort? }
]  type ∈ text | secret | date | number | select | boolean
```

- **Non-secret values** live in `client_module.details` as `[{ key, value }]` (label/type
  come from the module, so values stay small and never drift from the definition).
- **Secret values** (type `secret`) live in ONE `client_module.secrets_enc` blob —
  AES-256-GCM (HQ_SECRET_KEY) over a JSON map `{ fieldKey: plaintext }`. Generalizes today's
  single `password_enc`; N secrets, one encrypt/decrypt. Reveal endpoint decrypts and returns
  the map (or one key); **every reveal audited**, exactly as the DB password is today.

## 3. Data model (additive, both engines)

| Change | Notes |
|---|---|
| `module.field_spec TEXT NOT NULL DEFAULT '[]'` | the field definitions (JSON) |
| `client_module.secrets_enc TEXT` | generalized secret store (replaces the single portal password path) |
| keep `client_module.details` | now the value store keyed by field key (validated against field_spec) |

Migration from D20 (no data loss): each existing SMS/RTGS/ATM module gets a seeded
`field_spec` describing the fields it already uses (provider, username, portal password as a
secret, IP/IFSC/partner-bank, etc.); existing `details` values (already labeled) map to the
matching keys; the existing `password_enc` folds into `secrets_enc` under `portal_password`.
`provider`/`username`/`password_enc` columns are retired once values are migrated (or kept as
convenience aliases — decided at build time; retiring is cleaner).

## 4. Behavior

- **Service card (Client 360 → module):** renders a form *from the owning module's
  field_spec* — text/date/number/select/boolean inputs; secret fields as `••••` + Reveal
  (managerial, audited) + Set/Update; required fields validated. No hard-coded field names.
- **Module editor (owner):** a "Fields" section to define/reorder/type the field_spec —
  this is the surface where a module "defines itself." Config over code, so a new field
  appears on every client of that module immediately.
- **Ticket + roster module pickers:** already read the live catalog (fixed in D20) — a new
  module flows everywhere with no code change.
- **Importer:** maps APEX columns → defined field keys per module (targets keys instead of
  free labels); required fields absent → a warning note, never a blocking problem.

## 5. Flexibility guarantees (the founder mandate, made concrete)

1. **New module** = one catalog row + its field_spec. It instantly gets price book, roster
   (both directions), quote content, service card, renewals, revenue — no code, no migration.
2. **New/changed field on an existing module** = edit field_spec. Every client of that module
   picks it up; old values untouched; nothing to re-key.
3. **client↔module mapping stays a first-class link** (`client_module`) — reassign, add,
   deactivate, bulk-correct without touching client or module identity.
4. **Secrets scale** without new columns (one encrypted map per link).

## 6. Non-goals (deliberate, revisit later)

- Cross-client reporting/filtering *by* a module field (e.g. "every client's IFSC in one
  table") — that is the heavier "field tables" approach (option 2); add only if a dashboard
  needs it.
- Dating field definitions (a field's own effective-from history) — fields are config now,
  not price-like facts; add if a module's field set must change on a date.
- Bulk-edit UI across clients — separate small slice, add when cleanup friction is felt.

## 7. Build order (after the D20 trial import; suite green each phase)

1. Schema + `field_spec`/`secrets_enc` + generalized secret repo (encrypt/reveal map) + migration of D20 columns; tests.
2. API: field_spec on module CRUD; client_module value writes validated against field_spec; multi-secret reveal (audited).
3. Web: dynamic service-card form from field_spec; module "Fields" editor.
4. Importer: target defined keys; seed the five modules' field_spec from the real APEX columns.
5. Docs + D-record.