sims-hq/docs/DEPLOY-HQ.md

Go-Live Runbook — HQ ops console (apps/hq)

How to put the internal console on your AWS box for real use. The store product is unaffected (it stays local, D14); this is the vendor-side cloud app doc 11 always prescribed. Engine: SQLite for dev, Postgres in production via DATABASE_URL (D19 — the dual-engine switch is delivered; see §7).

0. The decided target (D20)

Piece	Decision
URL	https://hq.simssoftware.com (subdomain of the company domain)
DNS	GoDaddy — simssoftware.com stays at the registrar; one A record added
Server	The existing company AWS EC2 — HQ gets its own section on the box (/opt/sims-hq), beside whatever else runs there
TLS	Caddy reverse proxy (auto Let's Encrypt) — or a server block in the box's existing nginx if one already fronts other sites
Process	pm2, started from the repo root
Prod data	Born in RDS Postgres — provision RDS first, set DATABASE_URL, THEN run the APEX cutover import straight into it. Sequencing: database first, app section second, DNS/TLS last.

GoDaddy DNS (5 minutes): My Products → simssoftware.com → DNS → Add record: Type A, Name hq, Value <EC2 Elastic IP>, TTL 600 (raise to 1h once stable). Nothing else on the domain changes; existing www/mail records are untouched.

What you provide (the only blockers)

1. A server — a small AWS EC2 instance (t3.small is plenty for 5 users), Ubuntu 22.04+, with a domain or Elastic IP and HTTPS (via a reverse proxy — step 4).
2. A GitHub repo — so the box can pull the code (or we copy a build artifact).
3. Later, per capability: a Google Cloud OAuth app (Gmail), a read-only AWS IAM user (cost data), and your APEX CSV export (the 300 clients).

1. Get the code on the box

sudo mkdir -p /opt/sims-hq && sudo chown $USER /opt/sims-hq
git clone <your-repo-url> /opt/sims-hq        # after `gh repo create sims-hq` + push
cd /opt/sims-hq
# Node 20+ (nvm or NodeSource). Then:
npm install

2. Configure

cp apps/hq/.env.example apps/hq/.env
# Fill at minimum SIMS_PIN_PEPPER and HQ_SECRET_KEY (generators are in the file).
# Set HQ_DATA_DIR to a path on a backed-up volume, e.g. /var/lib/sims-hq/data.

SIMS_PIN_PEPPER must be set before the first real login and never changed after (changing it invalidates every password). HQ_SECRET_KEY must be set before connecting Gmail.

3. Build + run under a process manager

cd apps/hq-web && npm run build && cd ../..     # builds the web UI the server serves
# First boot prints a one-time owner password — capture it from the logs.
env $(grep -v '^#' apps/hq/.env | xargs) node apps/hq/dist/server.cjs   # smoke test, Ctrl-C

# Keep it running with pm2 (or a systemd unit). Start it FROM THE REPO ROOT:
# the DB path resolves to ./data/hq.db against the process CWD, so a pm2 app
# started from apps/hq would boot an empty database in apps/hq/data instead.
npm i -g pm2
env $(grep -v '^#' apps/hq/.env | xargs) pm2 start apps/hq/dist/server.cjs --name sims-hq
pm2 save && pm2 startup

Owner login on first boot is admin@sims.com + the printed password (change it immediately). The Gmail-disconnected banner is expected until step 6.

4. Put HTTPS in front (never expose Node directly)

Reverse-proxy 443 → localhost:5182 with Caddy (simplest) or nginx + certbot:

# /etc/caddy/Caddyfile
hq.simssoftware.com {
  reverse_proxy localhost:5182
}
# If the box already runs nginx for other sites, use a server block instead:
#   server {
#     listen 443 ssl; server_name hq.simssoftware.com;
#     # certbot --nginx -d hq.simssoftware.com issues the cert
#     location / { proxy_pass http://127.0.0.1:5182; proxy_set_header Host $host;
#                  proxy_set_header X-Forwarded-For $remote_addr; }
#   }

Lock the EC2 security group so only 443 is public; 5182 stays localhost-only. The public /share/:token links are served through this same proxy — that's intended (they're the one unauthenticated route, token-guarded, rate-limited, revocable).

5. Backups (do this on day one)

chmod +x apps/hq/scripts/backup-hq.sh
# Nightly to S3 + 14-day local rotation:
crontab -e
0 2 * * * HQ_DATA_DIR=/var/lib/sims-hq/data S3_BUCKET=s3://your-bucket/hq-backups /opt/sims-hq/apps/hq/scripts/backup-hq.sh >> /var/log/sims-hq-backup.log 2>&1

Do a restore drill once: copy a backup to a scratch box and boot the app against it.

6. Connect the capabilities (each optional, each unlocks a feature)

• Gmail (sending): create a Google Cloud OAuth client, publish the app to Production, set GOOGLE_CLIENT_ID/SECRET + HQ_SECRET_KEY, then npx tsx apps/hq/scripts/gmail-connect.ts and grant consent as the company mailbox. The banner clears; auto-reminders start sending.
• AWS cost per client: create a read-only IAM user with Cost Explorer access, tag each client's cloud resources with a client cost-allocation tag, set AWS_*. The scheduler pulls monthly.
• Your 300 clients: export clients.csv + invoices.csv from APEX into import-drop/, then run the importer (verification report first, --commit after you approve).

6b. Funnel slice knobs (D16 — set these at go-live)

• share.base_url setting (required before quote follow-ups can email): set it to the public origin, i.e. https://hq.simssoftware.com, via the settings API or a one-line SQL insert into setting. The escalating quote follow-up refuses to send without it — a client email carrying a dead relative link is worse than no email.
• quote.followup.policy setting: manual (default — nudges queue on the dashboard and a person clicks Send) or auto (client emails go out unattended after each daily scan). Start on manual; flip to auto once the letter text has been seen in anger.
• Reminder cadences are dated rows, not env vars: reminder_schedule ships seeded with quote_followup at 3/7/14 days and invoice_overdue at 7/15/30 days past invoice date. To change a cadence (or the follow-up email wording), insert a new dated row — effective from a business date, no release, no restart.
• Employee accounts: after first boot, open Employees (owner-only nav) and create the real manager/staff logins. Staff see only their own pipeline and reminder queue; owner/manager see everything. Deactivating someone kills their session immediately.
• Account owners: assign an owner on each client (Client 360 → Account owner) so leads route to the right person's queue; unassigned rows stay visible to owner/manager.

7. Production database — RDS Postgres (do this FIRST; D19 delivered the switch)

The dual-engine work is done (D19): the app selects its engine from DATABASE_URL — unset → SQLite (dev), set → Postgres. Migrations apply automatically from migrations-pg.ts on boot. Go-live order (so data is born in Postgres, never migrated):

1. Provision RDS PostgreSQL 17 (db.t4g.micro is plenty), same VPC as the EC2, not publicly accessible; security group allows 5432 from the EC2's group only.
2. Create database hq + a scoped hq role; put DATABASE_URL=postgres://hq:<pw>@<rds-endpoint>:5432/hq in apps/hq/.env.
3. Boot once — migrations run, first-boot seed prints the owner password.
4. Then run the APEX cutover UI (stage → verify → commit): the 300 clients import directly into Postgres.

Known nuance (D19): client search is case-sensitive on Postgres until repos move to LOWER(...) LIKE LOWER(?). SQLite + data/hq.db remains the dev/test path; the §5 sqlite backup script is superseded in prod by RDS automated snapshots (enable 7-day retention) — keep the S3 dump cron as belt-and-braces via pg_dump if desired.

Health & operations

• GET /api/health → { ok: true } for your uptime monitor.
• Logs: pm2 logs sims-hq. The scheduler runs on boot and every 6h (reminders, bounces, AWS pull when configured).
• Every mutation is audit-logged in the DB; the public share route and previews write nothing.