`npm run backup` ran scripts/backup.mjs, which read process.env.DATABASE_URL
directly and exited when it was unset. On the production box DATABASE_URL is NOT
exported -- the connection is resolved (D19) -- so the backup command could never
reach the real data.
The dump now lives in apps/hq (src/backup.ts + a thin scripts/backup.ts CLI, the
same split the other maintenance scripts use, so it is typechecked and unit
tested) and resolves its target through the shared resolveDbUrl. It logs the
resolved host/db -- never the password -- before pg_dump runs, and a SQLite
outcome is a loud error naming backup-hq.sh instead of a dump of nothing.
Behaviour otherwise unchanged: -Fc dump under ./backups (or HQ_BACKUP_DIR),
rotation to the newest HQ_BACKUP_KEEP (default 14), same pg_dump discovery.
The .mjs entry could not import the TS resolver, and duplicating the resolution
would have re-introduced the copy of the production connection string that
6f50799 deliberately removed -- hence the move to TS. `npm run backup` is
unchanged for the operator; prod already runs `npx tsx apps/hq/scripts/...` for
maintenance (see DEPLOY-HQ.md), so tsx is on the box.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
apply-onboarding-pipeline.ts documented its own command with no `DATABASE_URL=`
prefix -- the only hardened data script missing it. Followed literally on the
production host it resolves to SQLite, not the real database. Header corrected to
match its siblings.
The deeper problem is that landing on SQLite was a DEFAULT, not a decision: with no
DATABASE_URL, resolution quietly returned '' and the run was only saved by the
"never CREATE a database" guard -- which passes as soon as any stale ./data/hq.db
exists under the operator shell's cwd. requireDbUrlForEnv now refuses a SQLite
outcome unless it was explicitly asked for (HQ_DB_TARGET=sqlite, a --sqlite flag,
or HQ_DATA_DIR=:memory:), and the error names both escape hatches. The create guard
is unchanged and still applies after the opt-in. server.ts calls resolveDbUrl
directly, so first-boot DB creation is untouched.
All five maintenance-script headers now state the SQLite opt-in.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Same pre-ship audit fix as apply-onboarding-pipeline.ts and migrate-onboarding-
templates.ts: replace the raw `DATABASE_URL ?? ''` read with the shared
requireDbUrlForEnv() resolver, log the resolved engine/target before mutating,
hard-fail instead of silently opening a throwaway SQLite file in production, and
close the DB handle in a finally block.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
apply-onboarding-pipeline.ts and migrate-onboarding-templates.ts resolved the
engine as `DATABASE_URL ?? ''`, diverging from server.ts's production
fallback. On the prod box (DATABASE_URL unset) each script opened a brand-new
empty SQLite file, "succeeded" against it, and Postgres was never touched.
resolveDbUrl()/requireDbUrlForEnv()/pgTargetDescription() now live in
db-pg.ts as the one place this is decided; server.ts and both scripts call
it. The scripts additionally hard-fail (never fall back to SQLite) when
NODE_ENV=production resolves nothing, log their target engine before doing
any work, and close the DB handle in a finally.
Also lands client_module.created_at (nullable, additive) — schema groundwork
FIX D uses to measure a no-tick project's age from its real creation date
instead of treating a missing done_on as infinitely old.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Final whole-branch review FIX 2.
The live-deploy script upserted the onboarding template settings with raw
SQL (ON CONFLICT DO UPDATE), bypassing setSetting -> no audit row, violating
the append-only-audit rule -- and it unconditionally overwrote templates an
owner may have already edited in the Modules onboarding-template editor,
making the "Idempotent" header comment false and dangerous for a script that
runs against ~167 real projects.
Upserts now go through setSetting(db, 'system', key, value) (audited), and
only write a key that is still absent, unless run with an explicit --force
argument (documented). Header comment corrected to say idempotency holds
only while the templates haven't been owner-edited, and that --force
overwrites unconditionally.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Final whole-branch review FIX 3.
migrateOnboardingTemplates previously DELETEd every project_milestone row and
reinserted only template keys, so a project-specific step added via
addProjectMilestone ("+ Add step", custom_* keys) vanished with no trace if
un-ticked, or was miscounted as `dropped` if ticked.
Old rows are now partitioned before the DELETE into template-mapped vs
ad-hoc custom (not a template key, not in KEY_MAP, and not a known old
generic FRONT_FALLBACK/TAIL_FALLBACK key like amc_start — those keep their
existing drop behavior). Custom rows are re-inserted unchanged after the
template block, sort continuing past it, preserving
label/done/done_on/payment_id/document_id. The return type and the audit
`after` payload both gained a `preserved` count/list.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Idempotent migrateOnboardingTemplates(db): for every client_module whose module
has a per-module onboarding template (task 2), rebuilds its milestone rows from
that template, carrying mapped ticks (installed->installation, go_live->go_live,
advance_paid->advance_payment, balance_paid->balance_payment, setup_done->
setup_configuration when present) with done_on and payment_id intact. Unmapped
ticks (amc_start, or setup_done where the new template has no matching step) are
dropped and counted. Audited per project; a second run is a no-op via an
identical-key-set short-circuit. Adds the CLI wrapper (mirrors
apply-name-fixes.ts's open-DB idiom, Postgres or SQLite) for the real one-time
run against production data.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Appends ' LTD NO. <n>' where the Kerala co-op directory lookup found a registration
number (found=true). Guarded: only found rows, skips names that already contain the
number (space/punct-insensitive), appends the number only (no name/casing overwrite),
audited via updateClient. Applied on live data after backup: 91 names updated (K-Z half).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reads the reviewed name-review.json and applies each changed name via updateClient
(audited, guarded: only updates when the stored name still matches what was reviewed).
Ran on live data after a backup: 187 names corrected (co-op style unified to
CO-OPERATIVE, run-on splits, ARP00KARA->ARPOOKARA, double spaces), 0 skipped.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- staff_user gains must_change_password (SQLite migrate + PG migration 011): the
invited-employee indicator, cleared on the user's first self password-change.
- createEmployee takes phone/title/mustChangePassword; Employee carries mustChangePassword.
- scripts/import-employees.ts (tsx, Postgres- or SQLite-aware) seeds the 11 from
empmaster.xlsx: username = first-name slug, default password <username>@123, role staff,
active per ACTIVE_STAT, must-change flagged. Idempotent. Ran on live DB — 11 added.
- Employees list: 'Invited · must change pw' badge + a notice explaining the default
password pattern so the owner can share logins. Verified: adithya/adithya@123 → 200,
inactive raveena blocked. typecheck + employee tests green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Founder: there is no Tecnostac in this application. Replaced the seed default
company.name ('Tecnostac' -> 'SiMS'), demo-seed data, ~22 test fixtures/emails, and
docs (Tecnostac -> SiMS, @tecnostac.com -> @sims.com). Live DB company.name also
updated to 'SiMS' (shows on dashboard, documents, letterhead, reminder emails). Owner
email was already SiMS (simssoftware13@gmail.com). Full suite 416 green.
Bills are LINE ITEMS not documents: group by INVOICE_NO across three GST
eras (GST pseudo-line / 2026 columns-only / pre-GST zero-tax). 46 raw
ticket statuses -> 5 via ordered pattern map; OLD_ID re-key chains
collapse so superseded rows do not double-import. YES/NO flags respect
real vocabulary (GENERAL ENQUIRY != yes); ATM cell value becomes the
module provider. SID 329 test client + orphan refs skip (counted notes,
distinct from commit-blocking problems). District normalize, NO DATA
scrub, payment normalize, richer-row merge for duplicate service rows,
shared-number-space series seed. Dry run over the real export: 273
clients, 150 documents = Rs 33,16,800.24, 72 payments, 1,826 tickets,
667 interactions, INVOICE seed 10252 (matches APEX next_val). Suite 358
green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
parseWorkbookCsvs + stageApexFull (pure validation pass, zero writes) +
commitApexFull (one transaction): clients w/ encrypted DB passwords +
secretary contacts, YES-flags -> live client_module rows for the 5
modules, branches, bill_filed -> QT/PI/INV history with paisa-exact
CGST/SGST split + payment/allocation rows + series seed, 4.5k tickets
with status/date mapping, SMS/RTGS service data -> client_module
provider/username/password_enc/details, project lists -> interaction
notes with tolerant APEX date parsing. Plaintext passwords never stored
or audited; missing HQ_SECRET_KEY is a blocking problem. CLI:
npm run import:full -- --dir <folder> [--commit]. 7 new tests; suite
359 green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
P1: every repo, route, scheduler path and test now speaks one async DB
interface (all/get/run/exec/transaction). better-sqlite3 stays underneath
for dev/tests via SqliteDb (statement cache, manual BEGIN/SAVEPOINT
nesting); openDb stays sync so fixtures remain one-liners. Zero behavior
change: 346/346 tests, typecheck clean.
P2: migrations-pg.ts — numbered Postgres migration set (001-init: bigint
paise, integer flags, TEXT ISO dates, TEXT JSON; reminder CHECK born in
final widened form) + schema_migrations runner.
P3: db-pg.ts — pg Pool behind the same interface: ?->$n rewrite outside
string literals, transactions pinned to one pooled client via
AsyncLocalStorage with savepoint nesting, int8/numeric parsed to JS
numbers so paise math is identical on both engines.
Also: reminder upsert INSERT OR IGNORE -> ON CONFLICT DO NOTHING
(portable, same changes-count semantics).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Turnkey deployment for the AWS box: documented env vars, pm2/systemd run,
HTTPS reverse-proxy, nightly SQLite->S3 backup with rotation, and the Postgres
switch note. No secrets; unblocks go-live the moment the server/creds arrive.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Thomas Joise
Thomas Joise