# Go-Live Runbook — HQ ops console (`apps/hq`)

How to put the internal console on your AWS box for real use. The store product is
unaffected (it stays local, D14); this is the vendor-side cloud app doc 11 always
prescribed. Today the engine is SQLite; the Postgres switch is called out where it lands.

## What you provide (the only blockers)
1. **A server** — a small AWS EC2 instance (t3.small is plenty for 5 users), Ubuntu 22.04+,
   with a domain or Elastic IP and HTTPS (via a reverse proxy — step 4).
2. **A GitHub repo** — so the box can pull the code (or we copy a build artifact).
3. Later, per capability: a Google Cloud OAuth app (Gmail), a read-only AWS IAM user
   (cost data), and your **APEX CSV export** (the 300 clients).

## 1. Get the code on the box
```bash
sudo mkdir -p /opt/sims-hq && sudo chown $USER /opt/sims-hq
git clone <your-repo-url> /opt/sims-hq        # after `gh repo create sims-hq` + push
cd /opt/sims-hq
# Node 20+ (nvm or NodeSource). Then:
npm install
```

## 2. Configure
```bash
cp apps/hq/.env.example apps/hq/.env
# Fill at minimum SIMS_PIN_PEPPER and HQ_SECRET_KEY (generators are in the file).
# Set HQ_DATA_DIR to a path on a backed-up volume, e.g. /var/lib/sims-hq/data.
```
`SIMS_PIN_PEPPER` **must be set before the first real login** and never changed after
(changing it invalidates every password). `HQ_SECRET_KEY` must be set before connecting
Gmail.

## 3. Build + run under a process manager
```bash
cd apps/hq-web && npm run build && cd ../..     # builds the web UI the server serves
# First boot prints a one-time owner password — capture it from the logs.
env $(grep -v '^#' apps/hq/.env | xargs) node apps/hq/dist/server.cjs   # smoke test, Ctrl-C

# Keep it running with pm2 (or a systemd unit):
npm i -g pm2
cd apps/hq && pm2 start "npm start" --name sims-hq && pm2 save && pm2 startup
```
Owner login on first boot is `admin@tecnostac.com` + the printed password (change it
immediately). The Gmail-disconnected banner is expected until step 6.

## 4. Put HTTPS in front (never expose Node directly)
Reverse-proxy 443 → `localhost:5182` with Caddy (simplest) or nginx + certbot:
```
# Caddyfile
hq.yourdomain.com {
  reverse_proxy localhost:5182
}
```
Lock the EC2 security group so only 443 is public; 5182 stays localhost-only. The public
`/share/:token` links are served through this same proxy — that's intended (they're the
one unauthenticated route, token-guarded, rate-limited, revocable).

## 5. Backups (do this on day one)
```bash
chmod +x apps/hq/scripts/backup-hq.sh
# Nightly to S3 + 14-day local rotation:
crontab -e
0 2 * * * HQ_DATA_DIR=/var/lib/sims-hq/data S3_BUCKET=s3://your-bucket/hq-backups /opt/sims-hq/apps/hq/scripts/backup-hq.sh >> /var/log/sims-hq-backup.log 2>&1
```
Do a **restore drill** once: copy a backup to a scratch box and boot the app against it.

## 6. Connect the capabilities (each optional, each unlocks a feature)
- **Gmail** (sending): create a Google Cloud OAuth client, publish the app to Production,
  set `GOOGLE_CLIENT_ID/SECRET` + `HQ_SECRET_KEY`, then `npx tsx apps/hq/scripts/gmail-connect.ts`
  and grant consent as the company mailbox. The banner clears; auto-reminders start sending.
- **AWS cost per client**: create a read-only IAM user with Cost Explorer access, tag each
  client's cloud resources with a `client` cost-allocation tag, set `AWS_*`. The scheduler
  pulls monthly.
- **Your 300 clients**: export `clients.csv` + `invoices.csv` from APEX into `import-drop/`,
  then run the importer (verification report first, `--commit` after you approve).

## 7. The Postgres switch (before or at go-live)
Decision D15 locks Postgres as the production engine. It is a **dedicated engineering task,
not a config flag** — the repositories move from synchronous SQLite to async Postgres
behind the same interfaces, validated against your RDS instance. Provide the RDS endpoint +
a dedicated empty `hq` database and a scoped user (`HQ_PG_URL`), and it is built and tested
against that target, then the 300 clients import directly into Postgres — data never
migrates engines mid-life. Until then the app runs correctly on SQLite with the backups above.

## Health & operations
- `GET /api/health` → `{ ok: true }` for your uptime monitor.
- Logs: `pm2 logs sims-hq`. The scheduler runs on boot and every 6h (reminders, bounces,
  AWS pull when configured).
- Every mutation is audit-logged in the DB; the public share route and previews write nothing.