|
|
# Go-Live Runbook — HQ ops console (`apps/hq`)
|
|
|
|
|
|
> ⚠️ **SUPERSEDED (2026-07-21, D39).** HQ is live at `https://hq.simssoftware.in` — Docker on
|
|
|
> the existing EC2 box against the shared `accurate-postgres` container, **not** the RDS +
|
|
|
> GoDaddy + `.com` plan below. For the actual setup, commands, and paths, use
|
|
|
> **`deploy/PATHS-AND-NAMES.md`** instead. This file is kept for its Postgres-engine background
|
|
|
> (§7) but its go-live steps (§0–§6, §6b) do not reflect what shipped.
|
|
|
|
|
|
How to put the internal console on your AWS box for real use. The store product is
|
|
|
unaffected (it stays local, D14); this is the vendor-side cloud app doc 11 always
|
|
|
prescribed. Engine: SQLite for dev, **Postgres in production via `DATABASE_URL`**
|
|
|
(D19 — the dual-engine switch is delivered; see §7).
|
|
|
|
|
|
## 0. The decided target (D20)
|
|
|
|
|
|
| Piece | Decision |
|
|
|
|---|---|
|
|
|
| URL | **`https://hq.simssoftware.com`** (subdomain of the company domain) |
|
|
|
| DNS | **GoDaddy** — simssoftware.com stays at the registrar; one A record added |
|
|
|
| Server | The existing company **AWS EC2** — HQ gets its own section on the box (`/opt/sims-hq`), beside whatever else runs there |
|
|
|
| TLS | Caddy reverse proxy (auto Let's Encrypt) — or a server block in the box's existing nginx if one already fronts other sites |
|
|
|
| Process | pm2, started **from the repo root** |
|
|
|
| Prod data | **Born in RDS Postgres** — provision RDS first, set `DATABASE_URL`, THEN run the APEX cutover import straight into it. Sequencing: database first, app section second, DNS/TLS last. |
|
|
|
|
|
|
**GoDaddy DNS (5 minutes):** My Products → simssoftware.com → DNS → Add record:
|
|
|
Type `A`, Name `hq`, Value `<EC2 Elastic IP>`, TTL 600 (raise to 1h once stable).
|
|
|
Nothing else on the domain changes; existing www/mail records are untouched.
|
|
|
|
|
|
## What you provide (the only blockers)
|
|
|
1. **A server** — a small AWS EC2 instance (t3.small is plenty for 5 users), Ubuntu 22.04+,
|
|
|
with a domain or Elastic IP and HTTPS (via a reverse proxy — step 4).
|
|
|
2. **A GitHub repo** — so the box can pull the code (or we copy a build artifact).
|
|
|
3. Later, per capability: a Google Cloud OAuth app (Gmail), a read-only AWS IAM user
|
|
|
(cost data), and your **APEX CSV export** (the 300 clients).
|
|
|
|
|
|
## 1. Get the code on the box
|
|
|
```bash
|
|
|
sudo mkdir -p /opt/sims-hq && sudo chown $USER /opt/sims-hq
|
|
|
git clone <your-repo-url> /opt/sims-hq # after `gh repo create sims-hq` + push
|
|
|
cd /opt/sims-hq
|
|
|
# Node 20+ (nvm or NodeSource). Then:
|
|
|
npm install
|
|
|
```
|
|
|
|
|
|
## 2. Configure
|
|
|
```bash
|
|
|
cp apps/hq/.env.example apps/hq/.env
|
|
|
# Fill at minimum SIMS_PIN_PEPPER and HQ_SECRET_KEY (generators are in the file).
|
|
|
# Set HQ_DATA_DIR to a path on a backed-up volume, e.g. /var/lib/sims-hq/data.
|
|
|
```
|
|
|
`SIMS_PIN_PEPPER` **must be set before the first real login** and never changed after
|
|
|
(changing it invalidates every password). `HQ_SECRET_KEY` must be set before connecting
|
|
|
Gmail.
|
|
|
|
|
|
## 3. Build + run under a process manager
|
|
|
```bash
|
|
|
cd apps/hq-web && npm run build && cd ../.. # builds the web UI the server serves
|
|
|
# First boot prints a one-time owner password — capture it from the logs.
|
|
|
env $(grep -v '^#' apps/hq/.env | xargs) node apps/hq/dist/server.cjs # smoke test, Ctrl-C
|
|
|
|
|
|
# Keep it running with pm2 (or a systemd unit). Start it FROM THE REPO ROOT:
|
|
|
# the DB path resolves to ./data/hq.db against the process CWD, so a pm2 app
|
|
|
# started from apps/hq would boot an empty database in apps/hq/data instead.
|
|
|
npm i -g pm2
|
|
|
env $(grep -v '^#' apps/hq/.env | xargs) pm2 start apps/hq/dist/server.cjs --name sims-hq
|
|
|
pm2 save && pm2 startup
|
|
|
```
|
|
|
Owner login on first boot is `admin@sims.com` + the printed password (change it
|
|
|
immediately). The Gmail-disconnected banner is expected until step 6.
|
|
|
|
|
|
## 4. Put HTTPS in front (never expose Node directly)
|
|
|
Reverse-proxy 443 → `localhost:5182` with Caddy (simplest) or nginx + certbot:
|
|
|
```
|
|
|
# /etc/caddy/Caddyfile
|
|
|
hq.simssoftware.com {
|
|
|
reverse_proxy localhost:5182
|
|
|
}
|
|
|
# If the box already runs nginx for other sites, use a server block instead:
|
|
|
# server {
|
|
|
# listen 443 ssl; server_name hq.simssoftware.com;
|
|
|
# # certbot --nginx -d hq.simssoftware.com issues the cert
|
|
|
# location / { proxy_pass http://127.0.0.1:5182; proxy_set_header Host $host;
|
|
|
# proxy_set_header X-Forwarded-For $remote_addr; }
|
|
|
# }
|
|
|
```
|
|
|
Lock the EC2 security group so only 443 is public; 5182 stays localhost-only. The public
|
|
|
`/share/:token` links are served through this same proxy — that's intended (they're the
|
|
|
one unauthenticated route, token-guarded, rate-limited, revocable).
|
|
|
|
|
|
## 5. Backups (do this on day one)
|
|
|
```bash
|
|
|
chmod +x apps/hq/scripts/backup-hq.sh
|
|
|
# Nightly to S3 + 14-day local rotation:
|
|
|
crontab -e
|
|
|
0 2 * * * HQ_DATA_DIR=/var/lib/sims-hq/data S3_BUCKET=s3://your-bucket/hq-backups /opt/sims-hq/apps/hq/scripts/backup-hq.sh >> /var/log/sims-hq-backup.log 2>&1
|
|
|
```
|
|
|
Do a **restore drill** once: copy a backup to a scratch box and boot the app against it.
|
|
|
|
|
|
## 6. Connect the capabilities (each optional, each unlocks a feature)
|
|
|
- **Gmail** (sending): create a Google Cloud OAuth client, publish the app to Production,
|
|
|
set `GOOGLE_CLIENT_ID/SECRET` + `HQ_SECRET_KEY`, then `npx tsx apps/hq/scripts/gmail-connect.ts`
|
|
|
and grant consent as the company mailbox. The banner clears; auto-reminders start sending.
|
|
|
- **AWS cost per client**: create a read-only IAM user with Cost Explorer access, tag each
|
|
|
client's cloud resources with a `client` cost-allocation tag, set `AWS_*`. The scheduler
|
|
|
pulls monthly.
|
|
|
- **Your 300 clients**: export `clients.csv` + `invoices.csv` from APEX into `import-drop/`,
|
|
|
then run the importer (verification report first, `--commit` after you approve).
|
|
|
|
|
|
## 6b. Funnel slice knobs (D16 — set these at go-live)
|
|
|
- **`share.base_url` setting** (required before quote follow-ups can email): set it to the
|
|
|
public origin, i.e. `https://hq.simssoftware.com`, via the settings API or a one-line SQL
|
|
|
insert into `setting`. The escalating quote follow-up **refuses to send** without it —
|
|
|
a client email carrying a dead relative link is worse than no email.
|
|
|
- **`quote.followup.policy` setting**: `manual` (default — nudges queue on the dashboard
|
|
|
and a person clicks Send) or `auto` (client emails go out unattended after each daily
|
|
|
scan). Start on `manual`; flip to `auto` once the letter text has been seen in anger.
|
|
|
- **Reminder cadences are dated rows**, not env vars: `reminder_schedule` ships seeded with
|
|
|
`quote_followup` at 3/7/14 days and `invoice_overdue` at 7/15/30 days past invoice date.
|
|
|
To change a cadence (or the follow-up email wording), insert a **new dated row** —
|
|
|
effective from a business date, no release, no restart.
|
|
|
- **Employee accounts**: after first boot, open **Employees** (owner-only nav) and create
|
|
|
the real manager/staff logins. Staff see only their own pipeline and reminder queue;
|
|
|
owner/manager see everything. Deactivating someone kills their session immediately.
|
|
|
- **Account owners**: assign an owner on each client (Client 360 → Account owner) so
|
|
|
leads route to the right person's queue; unassigned rows stay visible to owner/manager.
|
|
|
|
|
|
## 7. Production database — RDS Postgres (do this FIRST; D19 delivered the switch)
|
|
|
The dual-engine work is **done** (D19): the app selects its engine from `DATABASE_URL` —
|
|
|
unset → SQLite (dev), set → Postgres. Migrations apply automatically from
|
|
|
`migrations-pg.ts` on boot. Go-live order (so data is born in Postgres, never migrated):
|
|
|
|
|
|
1. Provision RDS PostgreSQL 17 (db.t4g.micro is plenty), same VPC as the EC2, **not**
|
|
|
publicly accessible; security group allows 5432 from the EC2's group only.
|
|
|
2. Create database `hq` + a scoped `hq` role; put
|
|
|
`DATABASE_URL=postgres://hq:<pw>@<rds-endpoint>:5432/hq` in `apps/hq/.env`.
|
|
|
3. Boot once — migrations run, first-boot seed prints the owner password.
|
|
|
4. **Then** run the APEX cutover UI (stage → verify → commit): the 300 clients import
|
|
|
directly into Postgres.
|
|
|
|
|
|
Known nuance (D19): client search is case-sensitive on Postgres until repos move to
|
|
|
`LOWER(...) LIKE LOWER(?)`. SQLite + `data/hq.db` remains the dev/test path; the §5
|
|
|
sqlite backup script is superseded in prod by **RDS automated snapshots** (enable 7-day
|
|
|
retention) — keep the S3 dump cron as belt-and-braces via `pg_dump` if desired.
|
|
|
|
|
|
## Health & operations
|
|
|
- `GET /api/health` → `{ ok: true }` for your uptime monitor.
|
|
|
- Logs: `pm2 logs sims-hq`. The scheduler runs on boot and every 6h (reminders, bounces,
|
|
|
AWS pull when configured).
|
|
|
- Every mutation is audit-logged in the DB; the public share route and previews write nothing.
|