You cannot select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
sims-hq/docs/07-DB-AND-CONFIG.md

168 lines
10 KiB
Markdown

This file contains ambiguous Unicode characters!

This file contains ambiguous Unicode characters that may be confused with others in your current locale. If your use case is intentional and legitimate, you can safely ignore this warning. Use the Escape button to highlight these characters.

# 07 — Database Strategy & Config-Driven Design
Founder direction captured 2026-07-08: *"everything should be in DB — product codes,
notification messages, etc. Frontend should not need major changes; backend changes are
done by the team. Pricing/plans must be addable in DB so they reflect in the app."*
This doc turns that into a design — and gives the honest analysis of the Oracle 12c
question (tracked as decision **D12**).
## 1. The principle: the app is an engine, the DB is the product
Embraced. The frontend is built once as a stable rendering/billing engine; everything
that varies — per client, per plan, per language, per tax regime, per season — is **data**,
editable by your team from the back office with zero frontend release:
**Lives in the DB (change without any release):**
- All masters, obviously — items/product codes, parties, HSN, barcodes.
- **Message catalog** — every notification, error message, alert, WhatsApp/SMS/email
template, and UI label, in every language, keyed by code.
- **Settings hierarchy** — system → tenant → store → counter → user overrides, with
effective dates.
- **Plans, prices, feature matrix (D5)** — add a plan or change a price in the DB and
every client reflects it; nothing hard-coded.
- Tax rates & slabs (dated), rounding rules, document-numbering formats/series.
- Print/label/invoice templates (with visual preview editor).
- Scheme/offer definitions, loyalty rules.
- Menu/feature visibility per role and per edition (data evaluated by the engine).
- Keymaps and counter behavior profiles.
**Lives in code (versioned in git, tested, silently auto-updated):**
- Screen layouts and workflows, the billing/tax/rounding engine, validation logic,
the sync protocol.
**Why the boundary matters — the Forms lesson.** Putting *executable logic* in DB tables
is exactly how Oracle Forms systems became untestable, unversionable, and impossible to
hire for. Code in git gets code review, automated tests (the billing engine's
golden-file tests), instant rollback, and staged rollout. Data in the DB gets your
support team editing a message or adding a plan in a back-office screen, audit-logged.
Both change cheaply; neither contaminates the other.
**The real reason frontend changes used to hurt** — in the Classic era, a frontend change
meant touching every shop machine, so of course the instinct is "never change the
frontend." Silent auto-update (already in the platform checklist) removes that pain: an
app release costs one staged rollout, no shop visits. So the goal is restated precisely:
**no frontend release should ever be needed for content, config, pricing, tax, language,
or template changes** — and when a frontend release does happen, it costs nothing to ship.
## 2. Concrete shape (the tables your team will own)
```
message_catalog(code, channel, lang, template, placeholders, updated_by, updated_at)
e.g. ('BILL_ESHARE', 'whatsapp', 'hi', 'नमस्ते {name}, आपका बिल {amount} …')
setting(scope_type, scope_id, key, value, effective_from)
scope_type ∈ system|tenant|store|counter|user — nearest scope wins
plan(plan_code, name, status) -- Lite/Standard/Pro/Enterprise + future
plan_feature(plan_code, feature_code, limit_value) -- feature matrix as rows, not code
price_book(plan_code, billing_period, currency, amount, effective_from) -- D5: add/change anytime
tax_rate(hsn_or_class, rate, cess, effective_from, effective_to)
doc_series(store, counter, doc_type, prefix, next_no, fy)
template(kind, name, engine_version, body, preview_data) -- print/label/notification layouts
```
All of it edited through back-office screens with the same audit log as everything else.
The license token the client holds is just a signed snapshot of `plan_feature` rows —
which is why a new plan added in the DB reflects everywhere with no release.
## 3. The local DB engine — the Oracle 12c question (D12)
Two readings of "we will be using Oracle 12c for the local DB" — both addressed:
**Reading A — "Classic runs on Oracle 12c today, and we'll migrate from it."**
Perfect, no conflict: the migration tool reads Oracle 12c, and your existing schema
becomes the seed of the new data model (§4). Nothing else changes.
**Reading B — "the NEW system should also run Oracle 12c locally in each store."**
This one deserves the honest brainstorm, because it quietly decides the project's
economics and support load. What you actually want to preserve is real and preservable —
but the engine choice and the schema choice are separable.
### What carries over no matter which engine we pick
- **Your schema** — years of domain knowledge encoded in tables and relationships. It
ports nearly 1:1 (see §4).
- **Your team's skills** — SQL is SQL; PL/SQL maps to PostgreSQL's PL/pgSQL more closely
than to anything else on the market, and `ora2pg` automates most of the translation.
- **The store-server mental model** — the optional Store Hub in the architecture *is*
your current "Oracle server in the store" pattern, modernized.
### The comparison
| | Oracle 12c per store | Postgres Store Hub + SQLite counters | SQLite per counter only |
|---|---|---|---|
| License cost per store | Paid — and there was never a free 12c edition (free XE jumped 11g → 18c, and XE is capped ~12 GB data / 2 GB RAM) | ₹0 | ₹0 |
| Security patches | 12c left Premier Support in 20182020 (by release); in 2026 it gets no patches without costly extended contracts | Active, free | Active, free |
| Footprint on a shop PC | Multi-GB install, services, RAM-hungry | Hub only on one machine; counters are one file | One file per counter, zero admin |
| Silent auto-update | Very hard | Easy | Trivial |
| Power-cut behavior | DBA-grade recovery on a shop PC | Journaled, self-healing | Journaled, self-healing |
| Counter survives store-server death | ✗ (all counters die — today's failure mode) | ✓ (counters keep billing solo) | ✓ |
| Sync-engine ecosystem (PowerSync/ElectricSQL etc.) | None — everything hand-built | Native (both are the supported pair) | Native |
| Compliance optics | GST + personal data (DPDP) on an unpatched EOL DB | Clean | Clean |
### Recommendation (as originally analyzed)
**Keep the schema, retire the engine** — Postgres cloud + SQLite counters, porting via
`ora2pg`. (Kept on record for the revisit; superseded for now by the founder call below.)
### ✅ Founder decision (2026-07-09): Oracle 12c stays — adapted design
New fact that changes the economics: **the in-store Oracle 12c is shared infrastructure —
multiple other systems already run on it.** For the existing base the license and the DBA
burden are sunk costs, and ripping the engine out would disturb systems outside this
project's scope. Decision: Oracle 12c remains the store-tier DB *for now*.
The architecture adapts like this:
```
STORE CLOUD
┌────────────────────────────┐
│ Oracle 12c (existing box) │ sync agent ┌──────────────────┐
│ = store master DB │ ───────────────▶│ Postgres (multi- │
│ (our schema + other apps) │ outbox tables │ tenant, reports, │
└──────────▲─────────────────┘ drained when │ licensing, Inbox)│
│ LAN (write-through) online └──────────────────┘
┌──────────┴─────────────────┐
│ POS counters (Electron) │
│ each with local SQLite: │
│ • item cache (<150ms) │
│ • offline bill queue │
└────────────────────────────┘
```
Design guardrails that make "for now" honest:
1. **The counter's SQLite buffer is non-negotiable.** POS reads item data from its local
cache and writes bills locally first, forwarding to Oracle over LAN. If the Oracle box
or LAN dies, counters keep billing solo and drain the queue on recovery. The
"counter never stops" north star survives any store-server engine.
2. **Sync = custom outbox** (D3 → build): outbox tables in Oracle, written in the same
transaction as the business change; a store-side sync agent drains them to cloud with
idempotency keys. Off-the-shelf sync engines don't speak Oracle — accepted.
3. **Engine-portable code**: all new access goes through a repository/DAL layer with
portable SQL by default; Oracle-specific features require a written justification.
node-oracledb thin mode (supports 12.1+) keeps clients free of Oracle client installs.
4. **Risks stay on record, reviewed at each revisit trigger** (see D12 in
[06-DECISIONS.md](06-DECISIONS.md)): EOL patch status, new-customer licensing,
heavy footprint for greenfield installs. The biggest open product question: **what do
brand-new customers (no existing Oracle) get installed?** Options when we get there:
same 12c image as today (license question), SQLite-only store (Lite tier may need no
store server at all), or that's the moment the engine port happens.
## 4. Schema reuse plan (when you share the Classic structure)
When you provide the Oracle DB structure, we run a mapping exercise and produce
`12-DATA-MODEL.md`:
1. Table-by-table map: Classic table → Next entity (keep your codes, names, and domain
vocabulary — the team should feel at home).
2. Modernizations applied mechanically: client-generated UUIDv7 keys alongside legacy
codes, `tenant_id` on every row, effective-dating where Classic overwrote in place,
audit columns everywhere, soft historical states instead of deletes.
3. Explicit list of Classic quirks we deliberately do NOT carry (every legacy schema has
patched-around bugs baked into its shape — we list them, you veto).
4. The migration tool's field-level mapping falls straight out of this document.
Send the structure whenever ready — DDL export, or even a schema report from the tool of
your choice; format doesn't matter.