forge-orm

Appearance

Connection pooling ​

Sizing rules per dialect, per-runtime constraints (Lambda, Workers, Bun, long-running Node), pgbouncer / RDS Proxy caveats, and the pool failure modes that show up in production.

Why pooling matters ​

A connection is not free. The cost is paid in three places, and you eat all three on every fresh connect.

  • TCP + TLS handshake. Round-trip for TCP SYN/SYN-ACK/ACK, then 1–2 round-trips for TLS depending on protocol version and session resumption. On managed Postgres in another VPC or another region this is the dominant cost — 30–80 ms before a single byte of SQL crosses the wire. Cold-path requests under heavy load aren't slow because the database is slow; they're slow because they're waiting for a fresh handshake to a backend they didn't get to reuse.

  • Postgres backend process. Each pg connection forks a backend process on the server. That fork costs CPU and roughly 5–10 MB of resident memory on a fresh install — more once you've loaded a reasonable schema, extensions, and the planner's stats cache. A 100 vCPU box is happy serving thousands of in-flight queries; it is unhappy serving thousands of backend processes that have to be context-switched on and off the cores. Past a kernel-defined ceiling you start dropping queries before they're even parsed.

  • MySQL thread cost. MySQL spawns a thread per connection. Threads are lighter than processes — but the cost compounds anyway because every thread holds InnoDB buffers, MDL locks, and a stack. The practical ceiling on a managed MySQL is the same shape as Postgres: the box runs out of headroom long before the schedulable workload does.

A pool collapses all three. Connections are opened once, kept warm, shared across requests, and torn down only at shutdown. The numbers matter — the difference between "pool reuses a warm connection" and "pool can't grow fast enough so it dials a new one" is the difference between a 2 ms request and a 60 ms one.

For the higher-level architectural intent see BACKEND.md; this doc goes deeper on what the knobs actually do and why each runtime needs a different recipe.

Where the pool lives in forge ​

forge does not own a pool. The pool is owned by your driver wrapper. The forge executor only ever calls query(sql, params) on the PostgresDriver, MysqlDriver, or MongoDriver port — and when you wrote the driver you constructed the pool. That makes pooling a property of the client library, not the ORM:

ts
import { Pool } from 'pg';
import { createDb, pgDriver } from 'forge-orm';

// You own this pool. forge just talks to it.
const pool = new Pool({
  connectionString: process.env.DATABASE_URL!,
  max: 8,
  idleTimeoutMillis: 30_000,
  connectionTimeoutMillis: 5_000,
});

const db = await createDb({ schema, driver: pgDriver(pool) });

That decoupling is intentional. It means:

  • You can swap pg for postgres.js or @neondatabase/serverless without changing your schema, queries, or transaction code.
  • You can hand the same pool to two createDb calls (one with a tenant-scoped schema, one without) — both share connections.
  • On shutdown, db.$disconnect() defers to the pool you gave it. If you also call pool.end() directly it's a no-op the second time.

The two-call shape lives in DRIVERS.md: pass a url and forge constructs a default pool with the library's default knobs, or pass a driver you wrapped yourself and you own every knob. For anything serving real traffic, own the driver.

Pool parameters across drivers ​

The same five knobs exist almost everywhere; the names differ. This table is the one to print and put next to your IDE.

Conceptpgpostgres.jsmysql2mariadbmongodbtedious (MSSQL)
Max connectionsmaxmaxconnectionLimitconnectionLimitmaxPoolSizepool.max
Min / warm-poolminmin (via idle_timeout: 0)n/a — lazyn/a — lazyminPoolSizepool.min
Idle timeoutidleTimeoutMillisidle_timeout (s)idleTimeout (ms)idleTimeout (ms)maxIdleTimeMSpool.idleTimeoutMillis
Acquire wait timeoutconnectionTimeoutMillis (covers both)connect_timeout (s)connectTimeout (ms) for TCP; waitForConnections for queueingacquireTimeout (ms)waitQueueTimeoutMSpool.acquireTimeoutMillis
Eviction policyper-connection idleTimeoutMillissliding idle_timeoutidleTimeout + maxIdleidleTimeout + maxIdlemaxIdleTimeMS + pingpool.evictionRunIntervalMillis
Per-conn lifetime capn/a (use maxLifetimeSeconds on pool patches)max_lifetime (s)maxIdle; no native lifetimemaxIdlemaxConnecting, maxStalenessSecondsn/a
Statement timeoutstatement_timeout server optionstatement_timeout (s)connectTimeout, enableKeepAlivesocketTimeoutper-op maxTimeMSrequestTimeout
Keep-alivekeepAlive: truekeep_aliveenableKeepAlive, keepAliveInitialDelaysamealways onalways on

A couple of asymmetries to internalise:

  • pg collapses two timeouts into one. connectionTimeoutMillis governs both "time to open a new socket" and "time to wait for an in-pool connection if all are busy". Most other libraries split these. The practical consequence: if you raise it to give slow handshakes more room, you also raise the time a request stalls waiting for an in-use connection.

  • mysql2 defaults are unbounded. queueLimit: 0 means "infinite pending acquires". Under traffic this turns pool exhaustion into a memory leak instead of a fast 503. Always cap it (queueLimit: 50 for an 8-connection pool is a sane shape).

  • postgres.js uses seconds, not milliseconds. Easy mistake; the driver doesn't validate the magnitude, so idle_timeout: 30000 means thirty thousand seconds, not thirty seconds.

  • mongodb defaults maxPoolSize to 100 per MongoClient. A 10-replica deployment with default settings has 1,000 sockets pointed at a 3-node replica set. Bring it down hard.

  • MSSQL tedious defaults the request timeout to 15 seconds. Long reports silently truncate. Raise requestTimeout and set a per-query cap with OPTION (MAXDOP …) where it matters.

Sizing — the formula and the floor ​

The Postgres community's working formula, cited often enough that it's the only one most teams need:

connections = ((core_count * 2) + effective_spindle_count)

It comes from PgBouncer / pgcat lore, originally documented on the PostgreSQL wiki. core_count is the number of physical cores on the database server (not vCPUs, not threads); effective_spindle_count is 1 for a single disk, ~the disk count for RAID, and 0 for "all the working set fits in RAM". For a managed Postgres on a 4-core instance with NVMe storage that's (4*2) + 1 = 9 — call it 10 — per replica of the application.

That formula is the upper bound of useful concurrency. Past it, adding connections moves work from pg_stat_activity queueing to kernel context-switching, and your p99 gets worse.

The floor is set by something different: how many in-flight queries you actually have. If a request handler issues four serial queries and you serve 200 RPS, you need enough pool to absorb the worst case (~all 200 requests happen to be on their slowest query at the same moment). In practice the queue depth distribution is much narrower than that — sqrt(RPS × avg_query_ms / 1000) is a tighter back-of-envelope.

Per-replica budgeting ​

Your pool per process and your pool across the fleet are different numbers, and the second is the one that hits the database.

  • Single replica, single process: pool = the formula above.
  • N replicas, single process each: pool per process = formula / N.
  • N replicas, M PM2 workers each: pool per worker = formula / (N × M).

If your managed Postgres has a 100-connection ceiling, your fleet's total pool — summed across every container, every PM2 worker, and a small headroom for migrations and psql — needs to fit under that ceiling. The most common production incident is "we added two more replicas during a traffic spike and the database fell over because we quietly went over max_connections".

A simple worksheet:

pool_per_process × processes_per_pod × pods × replicas_per_AZ + headroom < max_connections

(example)
   pool = 8
×  PM2 workers = 4
×  pods = 12
×  AZs running simultaneously during deploy overlap = 2
+  headroom (admin / migrations / replicas / metrics exporter) = ~10
=  778

max_connections must be ≥ 778. Most managed PG defaults to 100–200.

When the worksheet doesn't fit, the fix is a pooler ("Connection-pool sidecars" below), not a bigger pool. You never want to raise the pool past the formula on the box itself.

Per-runtime patterns ​

The same Postgres pool with max: 8 behaves completely differently in each of these runtimes. Pick the recipe that matches yours.

Long-running Node server ​

A normal API process — hyper-express, Fastify, NestJS, Express. The pool's lifetime is the process's lifetime. Build it once at module load and share it across every request.

ts
// src/db.ts — one module, exported once, lives for the process.
import { Pool } from 'pg';
import { createDb, pgDriver } from 'forge-orm';

const pool = new Pool({
  connectionString: process.env.DATABASE_URL!,
  max: 8,
  min: 2,
  idleTimeoutMillis: 30_000,
  connectionTimeoutMillis: 5_000,
  keepAlive: true,
});

// Surface pool events so you can wire them into your metrics layer.
pool.on('error', (err) => console.error('[pg] idle client error', err));

export const db = await createDb({ schema, driver: pgDriver(pool) });
export const dbPool = pool; // for /healthz and shutdown
ts
// src/server.ts
import { db, dbPool } from './db';
import HyperExpress from 'hyper-express';

const app = new HyperExpress.Server();
app.get('/healthz', (_req, res) => {
  res.json({ ok: true, idle: dbPool.idleCount, total: dbPool.totalCount });
});

await app.listen(3000);

// SIGTERM drain.
const drain = async (sig: string) => {
  console.log(`[server] ${sig} — draining`);
  await app.close();
  await db.$disconnect();
  process.exit(0);
};
process.on('SIGTERM', () => drain('SIGTERM'));
process.on('SIGINT',  () => drain('SIGINT'));

What matters here:

  • The pool is module-scope, not request-scope. A common mistake is building a new pg.Pool inside a request handler or per-test fixture. The handshake cost is the entire pool benefit.
  • min: 2 keeps two warm sockets so the first request after an idle minute doesn't pay handshake latency.
  • SIGTERM drain — see Connection lifecycle and SIGTERM drain.

PM2 cluster / Node cluster module ​

PM2 in exec_mode: 'cluster' (or the bare node:cluster module) forks worker processes. Each worker has its own pool. PM2 does not share connections across workers; it can't — they're separate V8 isolates.

The math you need is multiplicative: total connections to the database = pool.max × workers × replicas. With max: 8, four PM2 workers, and ten Kubernetes replicas you've already hit 320 sockets. Drop pool.max accordingly. The Postgres formula gives a per-box ceiling; PM2 makes that the per-fleet ceiling because every worker counts.

ts
// pm2 ecosystem.config.js
module.exports = {
  apps: [{
    name: 'api',
    script: './dist/server.js',
    exec_mode: 'cluster',
    instances: 4,
    kill_timeout: 30_000, // give SIGTERM drain room to finish
  }],
};
ts
// src/db.ts — same shape as the long-running case, but with worker-aware sizing.
const workersPerHost = Number(process.env.WEB_CONCURRENCY ?? 4);
const replicasPerEnv = Number(process.env.REPLICA_COUNT ?? 1);
const fleetCeiling   = 80; // headroom under PG max_connections

const pool = new Pool({
  connectionString: process.env.DATABASE_URL!,
  max: Math.max(2, Math.floor(fleetCeiling / (workersPerHost * replicasPerEnv))),
  min: 0,
  idleTimeoutMillis: 15_000,
  connectionTimeoutMillis: 5_000,
});

The kill_timeout: 30_000 matters. PM2's default is 1.6 seconds, and on SIGTERM your worker has to finish in-flight requests, then db.$disconnect(), then exit. 1.6 s isn't enough; PM2 force-kills, and you get leaked sockets stuck on the database side until tcp_keepalive reaps them.

AWS Lambda + RDS Proxy ​

Lambda containers are ephemeral but the global scope persists across invocations. A pool you build at module load survives every warm invocation in the same container — and there can be thousands of warm invocations between cold starts. So the question is not "should Lambda pool?" but "how large is the pool inside one container?"

The answer is 1 — at most. A single container handles a single invocation at a time, so it never needs more than one open connection to do useful work. If you size max: 10 per Lambda container and AWS spins up 200 concurrent containers under load, you've taken 2,000 sockets — and your max_connections on RDS is 200.

ts
// handler.ts — module scope (runs once per container).
import { Pool } from 'pg';
import { createDb, pgDriver } from 'forge-orm';

const pool = new Pool({
  connectionString: process.env.DATABASE_URL!, // points at RDS Proxy
  max: 1,
  min: 1,
  idleTimeoutMillis: 60_000,    // keep the one connection warm across invocations
  connectionTimeoutMillis: 5_000,
});

const db = await createDb({ schema, driver: pgDriver(pool) });

// Lambda handler — uses the warm pool.
export const handler = async (event: any) => {
  const user = await db.user.findUnique({ where: { id: event.userId } });
  return { statusCode: 200, body: JSON.stringify(user) };
};

Put RDS Proxy in front. Without it, your Lambda fleet's connection count scales with concurrency: 500 hot containers = 500 sockets, and the database pays the handshake bill on every cold start. With RDS Proxy in front:

  • The Proxy multiplexes Lambda's many short-lived sockets onto a small fixed pool against the real database (default 100% of max_connections).
  • IAM auth means no long-lived password; the proxy injects credentials.
  • Idle connections close on the Lambda side fast (you can drop idleTimeoutMillis to ~5 s); the proxy keeps the real backend warm.

What you do not do with Lambda + RDS Proxy:

  • Build the pool inside the handler. That re-handshakes every invocation.
  • Set max > 1. You will never use the extra slots inside one container.
  • Skip the Proxy and hit RDS directly. Provisioned IOPS and connection limits are what burn first under concurrency.

The trade-off RDS Proxy imposes is the transaction-mode vs session-mode question — it pools in transaction mode by default, which has subtle implications for PREPARE, advisory locks, and SET LOCAL outside a transaction.

Cloudflare Workers + Hyperdrive ​

Workers have no persistent globals between requests. You can declare top-level constants, but you cannot rely on a TCP socket surviving across requests in the same isolate — and you cannot reach a private VPC from a stock Worker over plain pg.Pool at all. There are two options:

  • Hyperdrive — Cloudflare's first-party Postgres pooler. The Worker connects to Hyperdrive via TCP-over-HTTP, Hyperdrive holds the warm connection to your origin Postgres, and you size its pool against your database. The Worker side has no pool at all; the binding is the pool.
  • HTTP transports — @neondatabase/serverless (Neon-flavoured HTTP/WebSocket), Supabase's postgres.js with transform, or PlanetScale's @planetscale/database for MySQL.
ts
// worker.ts
import { Pool } from '@neondatabase/serverless';
import { createDb, pgDriver } from 'forge-orm';

export interface Env { HYPERDRIVE: Hyperdrive }

export default {
  async fetch(req: Request, env: Env): Promise<Response> {
    // Per-request pool. max:1, lifetime:request. Hyperdrive holds the warm
    // connection on the other side of the binding.
    const pool = new Pool({ connectionString: env.HYPERDRIVE.connectionString });
    const db = await createDb({ schema, driver: pgDriver(pool) });
    try {
      const users = await db.user.findMany({ take: 10 });
      return Response.json(users);
    } finally {
      await db.$disconnect();
    }
  },
};

What's different:

  • The pool is per request, not per process. There is no "process" in the Workers sense — the isolate's TCP state isn't yours to assume about.
  • max: 1 is the only correct value. Workers requests are single-flight.
  • Hyperdrive itself has a connection limit per Worker — defaults to 4 per request; raise via the dashboard if you fan out queries inside a single handler.

For Cloudflare D1, the driver is the binding directly — no pool at all, because D1 is SQLite serving HTTP. See Six worked wrappers → D1.

Vercel Edge / Vercel Functions ​

Vercel Edge runs on the same isolate-style runtime as Workers, with the same constraints: no persistent TCP, no pooling on the Edge side, must front with an HTTP transport.

Vercel Functions (the Node.js runtime, not the Edge runtime) behave like Lambda — global scope persists across warm invocations within a container. The same rules apply:

  • max: 1 per function instance.
  • Front with a managed pooler (Neon's pooled URL, Supabase's pgBouncer URL, RDS Proxy, or Hyperdrive if you're calling Cloudflare-side).
  • idleTimeoutMillis long enough to survive warm-invocation gaps.

The most common Vercel mistake is export default function handler at the top level and new Pool at the top level, with traffic shaped such that you have 200 hot containers — each holding 10 sockets — and a 100-connection Postgres. Cap it.

Bun runtime ​

Bun's runtime model is closer to Node than to Workers — long-running processes with module-scope globals that persist.

  • bun:sqlite — synchronous, in-process, no pool concept. One Database handle per process is correct. If you fan out work across Bun's Worker threads, each Worker needs its own handle; SQLite serialises writes at the file lock regardless.
  • Postgres on Bun — pg works under Bun's Node-compat shim, with the same pool semantics as Node. postgres.js runs natively. Pool sizing is the same long-running-server formula.
  • MySQL on Bun — mysql2 works under Node-compat, same caveats.

The one Bun-specific concern: Bun's process model can spawn child processes more cheaply than Node. If you fork child processes for worker pools, each child gets its own connection pool — same multiplicative math as PM2.

Deno ​

Deno's deno-postgres (https://deno.land/x/postgres) has its own Pool class with a near-identical surface to pg. The forge driver is a 20-line shim that conforms it to PostgresDriver. All sizing advice above applies — Deno is long-running, module-scope is your pool's lifetime, SIGTERM drains via the same handler shape.

Deno Deploy (the managed runtime) behaves like Cloudflare Workers — no persistent TCP, no native pool, must front with an HTTP transport or use Deno KV / Deno's own managed PG offering.

Connection-pool sidecars ​

When your fleet's connection demand exceeds your database's max_connections, the answer is a pooler in front of the database, not a larger pool inside your app. The three to know:

PgBouncer ​

The classic. Lightweight C process, three pooling modes (session, transaction, statement), no query awareness — it just shuttles bytes.

Session mode. Each client connection is bound to one backend connection for the client's lifetime. Behaves identically to direct Postgres — every session feature works. Pool size = number of client sessions, which is roughly your fleet's pool sum. This is the safe default; it's also the mode that gives the least multiplication benefit.

Transaction mode. Each transaction borrows a backend for its duration and returns it to the pool on COMMIT/ROLLBACK. Pool size = number of concurrent transactions, which can be much smaller than the fleet's pool sum. This is where the multiplication actually happens — 2,000 client sockets onto 50 backend sockets, easily.

The catch is what breaks in transaction mode:

  • No PREPARE — prepared statements live on the backend connection. The next transaction gets a different backend and the statement is gone. Disable client-side prepares (?prepare=false on postgres.js, or pgbouncer=true).
  • No server-side cursors outside a transaction. A DECLARE CURSOR outside BEGIN lives on the backend connection, which you don't get back. forge's streaming cursor is OK because it always wraps the cursor in BEGIN/COMMIT.
  • No LISTEN/NOTIFY on the pooled connection. Use a separate direct connection (or session-mode port) for pub/sub.
  • No SET outside a transaction. SET application_name = … on a pooled connection leaks to whichever next client gets that backend. Use SET LOCAL inside a transaction, or pass session parameters via the connection URI options.
  • No advisory locks across statements. pg_advisory_lock taken outside a transaction is held by the backend; the next transaction on a different backend doesn't see it. Use pg_advisory_xact_lock inside a BEGIN.

Statement mode is too restrictive for an ORM — even simple multi-step interactions (a SELECT then a follow-up UPDATE in the same JS function) can land on different backends. Don't use it.

PgCat ​

A multi-protocol pooler written in Rust; pgbouncer-compatible plus query-aware features (read/write split, shard routing). Sizing is the same as pgbouncer. Adds load-balancing across read replicas without needing the app to know about replicas.

RDS Proxy ​

AWS's managed pooler. Transaction-mode by default with pinning_filters for situations that require session affinity (prepared statements, advisory locks, temp tables — RDS Proxy pins the connection for the rest of the transaction when it sees one). IAM auth so secrets live in IAM, not env vars.

The right shape: Lambda → RDS Proxy → RDS. With RDS Proxy, raise your RDS-side max_connections modestly (it's how the Proxy avoids the pin-then-stall failure mode) but you don't need fleet-wide multiplication — the Proxy is the pool.

Hyperdrive ​

Cloudflare's pooler. TCP-over-HTTP transport from Worker to Hyperdrive; Hyperdrive holds a small warm Postgres pool to your origin. Sizing is on the Hyperdrive side, not the Worker side. Caching is offered as a separate layer; transaction-mode applies the same way.

Transaction-mode vs session-mode poolers — what breaks ​

The summary table is worth memorising.

FeatureSession modeTransaction modeStatement mode
Pool multiplication factorlowhighvery high
Prepared statements (PREPARE)worksbreaks — disable in clientbreaks
Server-side cursors outside BEGINworksbreaksbreaks
Server-side cursors inside BEGINworksworksbreaks (statements split)
LISTEN/NOTIFYworksbreaksbreaks
SET outside BEGINworksleaks — use SET LOCALleaks
SET LOCAL inside BEGINworksworksbreaks
Advisory locks via pg_advisory_lockworksbreaksbreaks
Advisory locks via pg_advisory_xact_lockworksworksbreaks
Temp tablesworksbreaks across transactionsbreaks
forge db.$transaction(fn)worksworksunsafe
forge cursor streamworksworksbreaks

forge uses no session-bound features by default. It does not issue client-side PREPARE; it does not LISTEN; it does not SET outside a transaction; it does not take long-lived advisory locks. So transaction mode is safe for forge's normal query and mutation surface — which is why BACKEND.md recommends it as the default in front of managed PG.

The two things you may add that break this:

  • Client-side prepared statements. postgres.js will prepare any query you call more than once unless you pass prepare: false (or pgbouncer: true, which is the same flag). Set it.
  • App-level advisory locks. If you use pg_advisory_lock for a cross-app mutex (queue leader election, periodic job dedup), it has to run on a session-mode port. Open a second small pool against the session-mode pooler endpoint for those calls only.

SQLite "pooling" ​

SQLite has no pool. There is one file, one writer at a time, and nothing the application can do about it — the lock lives in the file system, not the driver.

What the runtime can do:

  • WAL mode — PRAGMA journal_mode = WAL at connect time lets many readers operate concurrently with one writer, instead of the default "one operation at a time" rollback journal. This is the single setting that moves the throughput needle on SQLite.
  • One handle per process for writes — better-sqlite3 is sync; the calling code serialises naturally. With wasmSqliteDriver the handle queues internally through a promise chain. Either way, one handle is correct.
  • Many handles for reads — open additional handles for read-only workers. They contend at the file lock for reads but not against each other.
  • Connection per worker — if you fan out across Node worker_threads, each worker opens its own handle. SQLite serialises through the file lock regardless; the parallelism is "decoder + JSON.parse + JS work", not raw I/O.

Pragmas worth setting at boot:

ts
db.exec(`
  PRAGMA journal_mode = WAL;
  PRAGMA synchronous  = NORMAL;
  PRAGMA busy_timeout = 5000;
  PRAGMA cache_size   = -200000; -- ~200MB
  PRAGMA foreign_keys = ON;
`);

busy_timeout is the important one. The default is 0 — a write that hits the lock fails immediately with SQLITE_BUSY. Setting it to 5 seconds turns those failures into the writer simply waiting its turn, which is almost always the right behaviour.

For libsql / Turso the story is different: there's a network in between, and the server pools on the other side. Pool size on the client is the count of in-flight HTTP requests you allow — practically, let the HTTP client handle it.

For sqlite-wasm in the browser, see BROWSER.md — there's only ever one writer-tab anyway because OPFS is exclusive-locked.

MongoDB pooling ​

mongodb ships its own pool, internal to the MongoClient. forge doesn't see it; it just calls operations on the client.

Knobs to set explicitly (all on MongoClient options):

  • maxPoolSize — default 100. Bring it down to your concurrency ceiling per process. For a typical Node API, 10–20 is plenty.
  • minPoolSize — default 0. Set to 2–4 to keep warm sockets and avoid handshake-cost spikes on first request after idle.
  • maxIdleTimeMS — default 0 (never close). Set to 60 s so idle sockets get reaped instead of accumulating.
  • waitQueueTimeoutMS — default 0 (wait forever). Set to 5–10 s so pool exhaustion becomes a fast 503 instead of an unbounded hang.
  • serverSelectionTimeoutMS — default 30 s. Drop to 5 s for fast fail under partial cluster outage.
  • connectTimeoutMS — default 30 s. Drop to 5 s.
  • socketTimeoutMS — default 0. Set roughly to your slowest expected operation (10–30 s for OLTP).
  • heartbeatFrequencyMS — default 10 s; rarely worth touching.

Sizing for Atlas tiers:

TierTotal connection limitRecommended per-replica pool
M0 free500 cluster4
M2 / M5 shared500 cluster4
M101,500 cluster20
M203,000 cluster30
M30+3,000+depends — multiply across fleet

The cluster-wide cap is the one that bites. You can have unlimited replicas, but if they each open 100 sockets you hit M0's 500-limit at 5 replicas. Always set maxPoolSize explicitly; never let the default 100 propagate to production.

DocumentDB and Cosmos (Mongo API) shim onto the same client. Their ceilings differ — DocumentDB caps connections by instance class, Cosmos by RU/s tier. The connection-shape behaviour is the same.

Pool exhaustion — symptoms and observability ​

When the pool fails, it fails in one of three shapes.

1. Acquire timeouts. Requests wait the full connectionTimeoutMillis for a free connection that never frees, then throw. Symptoms:

  • p99 latency jumps to exactly connectionTimeoutMillis (a "wall").
  • p50 is unchanged because uncontested requests still go fast.
  • pool.waitingCount (pg) or pool.acquiringConnections (mysql2) > 0 sustainedly.
  • Logs show timeout exceeded when trying to connect (pg), acquireTimeout (mariadb), MongoWaitQueueTimeoutError (mongo).

2. Backend cap exhaustion. The pool grows past the database's max_connections; the database rejects new connections.

  • Logs show too many connections for role (PG), ER_CON_COUNT_ERROR (MySQL), MongoTopologyClosedError (Mongo).
  • New replicas come up but their pools can't fill — you see "ready" health checks while traffic still 503s.
  • pg_stat_activity shows usename slots full.

3. Stuck connections. Connections check out and never check back in — long queries, leaked transactions, deadlocks.

  • pool.idleCount drops to 0 and stays there.
  • pg_stat_activity shows queries with state='active' and query_start minutes in the past.
  • The fix is server-side — statement_timeout, idle-in-transaction timeout — because the app has already lost track of the connection.

Observability hooks ​

Pool stats are not pushed; you have to read them.

ts
// pg
function snapshot(pool: Pool) {
  return {
    total: pool.totalCount,     // currently open sockets
    idle:  pool.idleCount,      // available for acquire
    waiting: pool.waitingCount, // pending acquires
  };
}
setInterval(() => prom.gauge.set(snapshot(pool)), 10_000);
ts
// mysql2
function snapshot(pool: Pool) {
  // mysql2 doesn't expose stats directly; reach in via the raw pool.
  const raw = (pool as any).pool;
  return {
    total: raw._allConnections.length,
    idle:  raw._freeConnections.length,
    waiting: raw._connectionQueue.length,
  };
}
ts
// mongo
function snapshot(client: MongoClient) {
  const events = client.options.monitorCommands ? /* via CMAP events */ : null;
  // Easier: subscribe to connectionPoolEvents at boot.
}
client.on('connectionPoolReady', e => prom.gauge.set({ available: e.options.maxPoolSize }));
client.on('connectionCheckOutFailed', () => prom.counter.inc());

The metric to alert on is waiting > 0 sustained for > 30 s. That's the queue-depth signal that arrives before the wall does.

See METRICS.md for the wider observability recommendation — query latency histograms, error code histograms, pool gauges side-by-side.

Mitigations ​

  • Raise the pool — only if you're under the database's ceiling and the formula budget.
  • Lower the per-request query count — N+1 queries are pool multipliers.
  • Add a pooler in front (pgbouncer / RDS Proxy / Hyperdrive).
  • Add a statement_timeout server-side so leaked-but-stuck connections get reaped.
  • Switch heavy reports to a separate replica + separate pool so OLTP isn't starved by analytics. See BACKEND.md → Read replicas.

Multi-tenant pools ​

Two shapes, with very different operational characteristics.

Shared pool, scoped queries ​

One pool, one db handle, every query carries a tenant column / filter. This is forge's default shape — model definitions are tenant-agnostic; the application code adds where: { orgId } to every read.

Pros: pool sizing is one number. Cold-tenant cost is nothing. Tenant churn (new sign-ups, churned customers) costs the pool nothing.

Cons: tenant isolation is application-enforced. A forgotten orgId filter leaks data. The mitigation is the forge query-builder pattern documented in MULTI-TENANT.md: a scopedDb() helper that injects the tenant filter at the model level.

ts
// One pool, one db, tenant filter via a thin wrapper.
const db = await createDb({ schema, driver: pgDriver(pool) });

export function scopedDb(orgId: string) {
  return {
    user: {
      findMany: (args: any = {}) =>
        db.user.findMany({ ...args, where: { ...args.where, orgId } }),
      // …
    },
  };
}

Pool per tenant ​

One pool per tenant — usually because each tenant has a separate database or schema. Common in B2B-vertical SaaS with per-customer DBs for compliance reasons.

Pros: tenant isolation is physical. A query against tenant A literally cannot reach tenant B's data.

Cons: pool sizing is now tenants × max_per_pool. With 1,000 tenants and max: 2, you're at 2,000 sockets per replica. The fix is LRU pool eviction — keep a bounded cache of warm pools, evict the least recently used.

ts
// LRU pool cache.
import LRU from 'lru-cache';

const pools = new LRU<string, Pool>({
  max: 50,                  // max warm tenants
  ttl: 5 * 60_000,
  dispose: (pool) => pool.end().catch(() => {}),
});

function tenantPool(orgId: string): Pool {
  let pool = pools.get(orgId);
  if (!pool) {
    pool = new Pool({ connectionString: dsnFor(orgId), max: 2 });
    pools.set(orgId, pool);
  }
  return pool;
}

export async function tenantDb(orgId: string) {
  return createDb({ schema, driver: pgDriver(tenantPool(orgId)) });
}

The eviction dispose callback closes the pool when an LRU slot is reclaimed. Set the LRU ttl long enough that hot tenants stay resident but short enough that cold tenants don't pin sockets.

For very large tenant counts (>10k), pool-per-tenant is the wrong shape regardless of LRU — switch to shared pool + row-level filtering, and accept the application-enforced isolation cost.

Pool warm-up ​

On a fresh process, the first N requests pay handshake cost. Two ways to amortise.

Configured min ​

Most pools accept a min. Setting min: 2 tells the pool to open two connections at boot and keep them open even when idle.

ts
const pool = new Pool({
  connectionString: process.env.DATABASE_URL!,
  max: 8,
  min: 2,
  idleTimeoutMillis: 30_000,
});

pg honours min only on first use, not at construction. To force the connections open at boot, do the warm-up explicitly.

Explicit warm-up at boot ​

ts
async function warmPool(pool: Pool, n: number) {
  const clients = await Promise.all(
    Array.from({ length: n }, () => pool.connect()),
  );
  // Each connection is now open; release them back to the pool.
  for (const c of clients) c.release();
}

await warmPool(pool, 2);

Run this before your HTTP server starts listening — so the /readyz probe doesn't pass until the pool is warm. The first request after boot then never pays handshake cost.

For Mongo, client.connect() is the warm-up — it opens minPoolSize sockets. For mysql2, do the same explicit acquire-loop pattern as pg.

Connection lifecycle and SIGTERM drain ​

The lifecycle rule across every dialect:

  1. Connect lazily, then warm. Build the pool object at boot but let the warm-up step decide when sockets actually open. Eager connect-at-construction is fine in long-running processes; lazy is correct in Lambda.

  2. Open before serving. The HTTP server starts listening after the pool is ready. Otherwise the first request races with the handshake.

  3. Close after the server drains. SIGTERM triggers server.close() (stops accepting new connections, finishes in-flight ones), then db.$disconnect() (drains the pool).

ts
const shutdown = async (sig: string) => {
  console.log(`[server] received ${sig}`);
  // 1. Stop accepting new connections.
  await app.close();
  // 2. Give in-flight requests a moment to finish (their pool slots will
  //    release naturally on response.end).
  await new Promise(r => setTimeout(r, 1_000));
  // 3. Drain the pool. db.$disconnect() defers to pool.end().
  await db.$disconnect();
  console.log('[server] drain complete');
  process.exit(0);
};
process.on('SIGTERM', () => shutdown('SIGTERM').catch(err => {
  console.error('drain failed', err); process.exit(1);
}));
process.on('SIGINT',  () => shutdown('SIGINT').catch(err => {
  console.error('drain failed', err); process.exit(1);
}));

The Kubernetes / PM2 / systemd default kill timeout is short. On SIGTERM you get a fixed window before SIGKILL. Make sure the drain fits:

  • Kubernetes terminationGracePeriodSeconds: default 30 s; raise if your slowest request can exceed that.
  • PM2 kill_timeout: default 1.6 s — always raise this, often to 30 s.
  • systemd TimeoutStopSec: default 90 s; usually fine.

If SIGKILL hits mid-drain, you leak sockets on the database side. PG will clean them up via tcp_keepalive defaults, but the database can have ghosts for minutes.

Health checks and readiness probes ​

Two distinct probes, often confused.

  • Liveness (/healthz) — "is this process alive?" Answer "yes" as long as the event loop is responsive. Do not touch the database; a transient DB outage shouldn't restart your pod.

  • Readiness (/readyz) — "can this process serve traffic right now?" Answer "yes" only if the pool is warm and able to acquire a connection. A DB outage should take the pod out of load balancing.

ts
app.get('/healthz', (_req, res) => {
  res.json({ ok: true });
});

app.get('/readyz', async (_req, res) => {
  try {
    const client = await Promise.race([
      pool.connect(),
      new Promise<never>((_, reject) =>
        setTimeout(() => reject(new Error('acquire timeout')), 1_000)),
    ]);
    try {
      await client.query('SELECT 1');
    } finally {
      client.release();
    }
    res.json({ ok: true, idle: pool.idleCount, total: pool.totalCount });
  } catch (err: any) {
    res.status(503).json({ ok: false, error: err.message });
  }
});

Tune the probe interval to be longer than the timeout. A /readyz that hits the database every 2 seconds across 20 replicas at 8 connections per replica with a 1-second timeout = 80 RPS of SELECT 1 that competes for the same pool slots as your real traffic, plus a real risk of probes timing out under load and flapping the pod.

Probe every 10–15 s; timeout 1–2 s.

db.$doctor() is the deeper diagnostic — runs the adapter's full self-check (capabilities, extensions, system pragmas). See DOCTOR.md. Don't put it on a hot probe path; run it at boot and on demand.

Common errors and what they actually mean ​

Error messageReal causeFirst thing to check
timeout exceeded when trying to connect (pg)All pool slots are busy or TCP handshake stalledpool.waitingCount, network to DB
Connection terminated unexpectedly (pg)Backend died, network blip, or idle_in_transaction_session_timeout reaped your idle sessionServer logs around the same timestamp
Connection terminated due to connection timeout (pg)Idle-killed by a pooler or firewall in betweentcp_keepalive_*, pooler idle_timeout
ER_CON_COUNT_ERROR: Too many connections (MySQL)Fleet sum of connectionLimit > server max_connectionsFleet pool worksheet
PROTOCOL_CONNECTION_LOST (mysql2)Server closed the connection (idle, restart, replica failover) — mysql2 will reconnect lazilySet enableKeepAlive to detect deads faster
MongoWaitQueueTimeoutErrorPool exhausted; all maxPoolSize sockets are checked outLong ops in db.currentOp()
MongoNetworkErrorReplica set election in progress, or transient networkAtlas / cluster events around the same time
pool is draining (any)A new acquire arrived after $disconnect() startedA request handler is racing with shutdown — fix shutdown() ordering
RDS Proxy: connection pinned (CloudWatch)A statement triggered backend pinning (PREPARE, advisory lock, temp table)Remove that statement or accept the lower multiplication factor
socket hang up from HyperdriveWorker exited before request completed, or > 30 s queryCheck Hyperdrive analytics; raise Worker CPU limit
SQLITE_BUSYAnother writer holds the file lock and busy_timeout is 0Set PRAGMA busy_timeout = 5000
Pool was closed (postgres.js)sql.end() ran somewhere it shouldn't haveAudit shutdown order

The pattern: the error message is rarely the cause. Pool exhaustion shows up as "timeout connecting"; backend death shows up as "pool was closed"; pooler eviction shows up as "connection terminated". The pool gauges are the truth — total, idle, waiting at the moment of the failure.

Worked example A — hyper-express + pg pool with metrics ​

A long-running API serving a managed Postgres, with pool gauges exported to Prometheus and a SIGTERM drain.

ts
// src/db.ts
import { Pool } from 'pg';
import { createDb, pgDriver } from 'forge-orm';
import { Counter, Gauge, Histogram } from 'prom-client';
import { schema } from './schema';

export const pool = new Pool({
  connectionString: process.env.DATABASE_URL!,
  max: Number(process.env.PG_POOL_MAX ?? 8),
  min: Number(process.env.PG_POOL_MIN ?? 2),
  idleTimeoutMillis: 30_000,
  connectionTimeoutMillis: 5_000,
  statement_timeout: 10_000,
  keepAlive: true,
  application_name: 'api',
});

const gTotal   = new Gauge({ name: 'pg_pool_total',   help: 'open sockets in the pool' });
const gIdle    = new Gauge({ name: 'pg_pool_idle',    help: 'idle sockets in the pool' });
const gWaiting = new Gauge({ name: 'pg_pool_waiting', help: 'pending acquires' });
const cAcqFail = new Counter({ name: 'pg_pool_acquire_failed_total', help: 'acquire timeouts' });
const hQuery   = new Histogram({
  name: 'forge_query_duration_seconds',
  help: 'query duration', buckets: [0.001,0.005,0.01,0.05,0.1,0.5,1,5,10],
  labelNames: ['model', 'op'],
});

// Sample the pool gauges every 5 s.
setInterval(() => {
  gTotal.set(pool.totalCount);
  gIdle.set(pool.idleCount);
  gWaiting.set(pool.waitingCount);
}, 5_000).unref();

pool.on('error', (err) => console.error('[pg] idle client error', err));

// Warm the pool before listening.
async function warm(n: number) {
  const cs = await Promise.all(Array.from({ length: n }, () => pool.connect()));
  for (const c of cs) c.release();
}

await warm(Number(process.env.PG_POOL_MIN ?? 2));

export const db = await createDb({
  schema,
  driver: pgDriver(pool),
  events: {
    query: (e) => hQuery.observe({ model: e.model, op: e.op }, e.durationMs / 1000),
  },
});
ts
// src/server.ts
import HyperExpress from 'hyper-express';
import { register } from 'prom-client';
import { db, pool } from './db';

const app = new HyperExpress.Server();

app.get('/healthz', (_req, res) => res.json({ ok: true }));

app.get('/readyz', async (_req, res) => {
  try {
    const client = await Promise.race([
      pool.connect(),
      new Promise<never>((_, rej) =>
        setTimeout(() => rej(new Error('acquire timeout')), 1_000)),
    ]);
    try { await client.query('SELECT 1'); } finally { client.release(); }
    res.json({ ok: true, idle: pool.idleCount, total: pool.totalCount });
  } catch (e: any) {
    res.status(503).json({ ok: false, error: e.message });
  }
});

app.get('/metrics', async (_req, res) => {
  res.set('content-type', register.contentType);
  res.send(await register.metrics());
});

app.get('/users/:id', async (req, res) => {
  const user = await db.user.findUnique({ where: { id: req.params.id } });
  res.json(user);
});

await app.listen(Number(process.env.PORT ?? 3000));

const shutdown = async (sig: string) => {
  console.log(`[server] ${sig} — draining`);
  await app.close();
  await new Promise(r => setTimeout(r, 1_000));
  await db.$disconnect();
  process.exit(0);
};
process.on('SIGTERM', () => shutdown('SIGTERM'));
process.on('SIGINT',  () => shutdown('SIGINT'));

What's wired here:

  • Pool gauges (total, idle, waiting) sampled into Prometheus on a timer.
  • Query latency histogram bucketed by model and op — alert on the right tail moving without throughput changing.
  • /readyz does a real SELECT 1 with a 1-second acquire timeout.
  • SIGTERM drains the server, sleeps for in-flight responses, then closes the pool.
  • statement_timeout server-side so a runaway query can't pin a pool slot indefinitely.

Worked example B — Lambda + RDS Proxy ​

The pool is one connection, module-scope, surviving warm invocations. RDS Proxy is the actual pooler.

ts
// handler.ts
import { Pool } from 'pg';
import { createDb, pgDriver } from 'forge-orm';
import { schema } from './schema';

// Built at container init. Survives across every warm invocation.
const pool = new Pool({
  // RDS Proxy endpoint (not the RDS endpoint directly).
  connectionString: process.env.DATABASE_URL!,
  max: 1,
  min: 1,
  // Lambda containers go idle between bursts; keep the one socket warm.
  idleTimeoutMillis: 60_000,
  connectionTimeoutMillis: 5_000,
  // Statements that pin the RDS Proxy connection cost multiplication.
  // Disable client-side prepares if you ever switch to postgres.js.
});

const db = await createDb({ schema, driver: pgDriver(pool) });

export const handler = async (event: any) => {
  // Every invocation reuses the one warm connection if available.
  // Cold starts open it; subsequent invocations skip the handshake.
  try {
    const user = await db.user.findUnique({ where: { id: event.userId } });
    return { statusCode: 200, body: JSON.stringify(user) };
  } catch (err: any) {
    // Distinguish pool/connection errors from query errors for retries.
    if (err.code === '57P01' /* admin_shutdown */ ||
        err.code === '57P02' /* crash_shutdown */ ||
        err.message?.includes('timeout exceeded when trying to connect')) {
      // RDS Proxy rolled the underlying connection; retry once.
      const user = await db.user.findUnique({ where: { id: event.userId } });
      return { statusCode: 200, body: JSON.stringify(user) };
    }
    throw err;
  }
};

Configuration to match on the AWS side:

  • RDS Proxy: enable IAM auth, set IdleClientTimeout to ~30 min, set MaxConnectionsPercent to 90 (leaves 10% for admin / migration / psql).
  • Lambda: ProvisionedConcurrency if you need predictable cold starts (also reduces handshake spike at deploy).
  • VPC: Lambda in the same VPC as RDS, with VPC endpoints for Secrets Manager and IAM. The Proxy itself fronts the RDS endpoint.
  • Timeouts: Lambda's function timeout > the slowest query + buffer. statement_timeout server-side as a hard brake.

See LAMBDA.md for the wider deployment story.

Worked example C — Workers + Hyperdrive ​

The Worker has no pool. Hyperdrive is the pool, lives on Cloudflare's edge, and connects back to your origin Postgres.

toml
# wrangler.toml
name = "api"
main = "src/worker.ts"
compatibility_date = "2025-01-01"
compatibility_flags = ["nodejs_compat"]

[[hyperdrive]]
binding = "HYPERDRIVE"
id = "<hyperdrive-id>"
ts
// src/worker.ts
import { Pool } from '@neondatabase/serverless';
import { createDb, pgDriver } from 'forge-orm';
import { schema } from './schema';

export interface Env { HYPERDRIVE: Hyperdrive }

export default {
  async fetch(req: Request, env: Env): Promise<Response> {
    // Per-request pool. max:1; Hyperdrive does the real pooling.
    const pool = new Pool({
      connectionString: env.HYPERDRIVE.connectionString,
      max: 1,
    });
    const db = await createDb({ schema, driver: pgDriver(pool) });
    try {
      const url = new URL(req.url);
      if (url.pathname === '/users') {
        const users = await db.user.findMany({ take: 50 });
        return Response.json(users);
      }
      return new Response('not found', { status: 404 });
    } finally {
      // Always disconnect — the isolate may be reused for any future request.
      await db.$disconnect();
    }
  },
};

Configuration to match on the Cloudflare side:

  • Hyperdrive pool size: configured in the Cloudflare dashboard. Default is a small fixed pool; size to your origin's connection budget. This is the only pool that matters for connection-counting against Postgres.
  • Hyperdrive caching: optional. Enable for read-heavy workloads; invalidate from the Worker via the API.
  • Connections per Worker request: default 4. If you fan out parallel queries inside a single request, raise it.
  • TLS: Hyperdrive handles TLS to the origin. The Worker sees a local-shaped connection.

For MySQL via PlanetScale, the equivalent shape uses @planetscale/database directly — it's HTTP-only with no pool concept on the Worker side; PlanetScale's edge does the pooling.

See WORKERS.md for the full edge-deployment story.

Cross-references ​

  • DRIVERS.md — driver ports, pool knob mappings, edge runtime driver matrix
  • METRICS.md — pool gauges, query histograms, error counters
  • BACKEND.md — framework wiring, request-scoped transactions, server lifecycle
  • MULTI-TENANT.md — shared pool vs pool-per-tenant patterns
  • DEPLOYMENT.md — packaging, environment, ops
  • POSTGRES.md — Postgres-specific tuning, including max_connections and statement_timeout
  • MYSQL.md — MySQL-specific tuning, wait_timeout
  • MONGO.md — Mongo-specific options, Atlas tier ceilings
  • LAMBDA.md — AWS Lambda deployment, RDS Proxy
  • WORKERS.md — Cloudflare Workers deployment, Hyperdrive, D1

MIT licensed. Built from the repository's own markdown — README.md, CHANGELOG.md and docs/.

© johnsonfash