MongoDB
forge-orm's Mongo adapter maps the same IR (models, fields, indexes, relations) onto collections, documents, and the aggregation pipeline. This page documents what that mapping looks like in practice, plus Mongo-specific surfaces (change streams, Atlas Search, vector, transactions, sharding) and the operational concerns that touch the forge layer.
If you have not read the surface tour, start with DRIVERS for the driver port shape and the Reading data chapter for the call vocabulary shared with every other adapter.
Contents
- Mongo as a dialect
- Driver — the
mongodbpeer dep - Relational to document mapping
- Schema validation —
$jsonSchema - Index types forge emits
- Atlas-specific features
- The aggregation pipeline
- Relation filters throw (behaviour change in 2.18.0)
- Transactions
- Change streams
- Read concerns and write concerns
- Sharding
- Connection pool
- One client per
createDb() - BSON types — round-trip with forge
dividecompiles to a pipeline update- The
updatepath and BSON coercion - Common errors
- Three worked examples
- Where to look in the source
Mongo as a dialect
There is no SQL dialect for Mongo. The adapter still goes through the same IR every other adapter uses — SelectNode, InsertNode, UpdateNode, DeleteNode, CountNode, GroupByNode. Compilation emits a MongoArtifact (src/compile.ts) instead of a SQL string:
interface MongoArtifact {
kind: 'mongo';
collection: string;
op: 'find' | 'findOne' | 'insertOne' | 'insertMany' |
'updateMany' | 'findOneAndUpdate' | 'deleteMany' |
'findOneAndDelete' | 'aggregate' | 'countDocuments';
args: { filter?: any; options?: any; document?: any; documents?: any[];
update?: any; pipeline?: any[] };
hydration?: RelationPlan[];
}That artifact is what the executor passes to the driver. The kind-level contract is identical to every other adapter (same wrapper API, same event payloads, same error envelope), and the only place the IR has to fork is when a feature has no direct Mongo analog — for instance, where: { posts: { some: {...} } } (relation predicate) compiles to a {} match on Mongo today (see Aggregation pipeline).
The adapter's capabilities flags state where the dialect line is drawn:
// src/adapters/mongo/adapter.ts
const CAPS: AdapterCapabilities = {
nativeCascades: false, // emulated by the JS cascade walker
nativeUpsert: true, // findOneAndUpdate({ upsert: true })
nullsOrdering: false, // Mongo $sort orders by BSON type
jsonPath: true, // dotted keys are native
transactionsRequireReplicaSet: true, // mongod --replSet rs0 or mongos
};The model declaration site does not branch on dialect. The schema you write for Postgres works on Mongo subject to: no bigserial ids, no expression indexes (idx.expression), and the per-feature footnotes below.
Driver — the mongodb peer dep
The driver is the official mongodb npm package, declared as an optional peer dep so SQL-only installs do not pay the cost of pulling it. Every Mongo file goes through a lazy accessor in src/adapters/mongo/bson.ts:
let _m: typeof import('mongodb') | undefined;
export function mongo() {
if (!_m) {
try { _m = require('mongodb'); }
catch {
throw new Error(
"[forge] the 'mongodb' driver is required for Mongo operations " +
'but is not installed.\n Install: npm install mongodb',
);
}
}
return _m!;
}A top-level import { ObjectId } from 'mongodb' would emit a module-load require('mongodb') and crash a Postgres-only consumer at boot. Anywhere forge needs ObjectId, MongoClient, or ClientSession, it goes through mongo() instead.
The driver shape is the simplest of the six ports — just "bring your own configured MongoClient" (see DRIVERS for the full interface):
export interface MongoDriver {
readonly kind: 'mongo';
client: any; // a MongoClient (or a shim that quacks like one)
dbName?: string; // defaults to the URI's default DB
}MongoAdapter.connect(url) calls client.connect() (which is idempotent on the driver, safe whether the caller already connected) and caches the Db handle on the client this adapter owns — one per createDb() since 2.19.0; see One client per createDb(). The url is passed in rather than read from the environment, which is the load-bearing part of that fix. Anything that implements .db(), .startSession(), and the standard collection methods qualifies — DocumentDB, Cosmos (Mongo API), and FerretDB all fit the slot but with the feature-flag caveats below.
Relational to document mapping
The IR carries a relational shape; Mongo stores documents. The adapter bridges the two in three places: the field-name remap, the relation hydrator, and the cascade walker.
Tables and collections
Every model declares a collection name in its ModelDef. forge maps 1:1 — model('users', {...}) becomes db.users. View-models compile to read-only Mongo views via createCollection(name, { viewOn, pipeline }); materialised view-models populate a real collection by running the pipeline with a trailing $out, refreshed on demand via db.<model>.refresh() (src/adapters/mongo/adapter.ts:refreshView).
Ids and the id ↔ _id rename
App code always reads and writes the field id. Mongo stores it as _id. The remap is centralised in coerce.ts:
// src/adapters/mongo/coerce.ts
export function appKeyToDbKey(key: string): string {
return key === 'id' ? '_id' : key;
}
export function dbKeyToAppKey(key: string): string {
return key === '_id' ? 'id' : key;
}f.id() defaults to ObjectId (BSON), f.id({ type: 'uuid' }) produces a string UUID, and f.id({ type: 'bigserial' }) is rejected at push time — there is no auto-incrementing scalar on Mongo. The default autoId is generated client-side by applyCreateDefaults:
if (def.default.kind === 'autoId') out[name] = new (mongo().ObjectId)();This is the same shape Mongo itself would assign at the server, just done early enough that insertOne and the return-row code path see the same id without a round trip.
On read, every ObjectId is converted back to its 24-char hex string before app code sees it. This is the rule that lets the rest of the codebase treat ids as opaque strings.
Relations via lookup vs. application-side hydration
forge's relations compile to a RelationPlan (see RELATIONS). On Mongo, the executor does not emit $lookup for hydration. It runs a separate batched fetch ({ <fk>: { $in: refs } }) per relation level and joins the result client-side:
// src/adapters/mongo/execute.ts
const node: SelectNode = {
...subNode, kind: 'select', model: rel.target,
cardinality: 'many',
where: { kind: 'leaf', field: rel.refs, op: 'in', value: coercedRefs },
};
const found = await executeSelect(node, targetModel, { session });Two reasons for the choice:
$lookupis restricted on the protocol-compatible servers. Cosmos rejects it; DocumentDB supports a limited form. The batched-fetch path runs everywhere afindruns.- Selection / projection /
_countcompose more cleanly. Each relation level is just anotherfindwith its ownselect,include,orderBy,limit. The pipeline form would need a sub-pipeline per relation and a final$projectto peel it apart.
The single batched fetch shown above is what runs when the nested include has no paging of its own. Add a nested take or skip and the hydration step switches to one find per parent, because a nested page is per parent — the shared logic lives in src/ir/hydrate-many.ts and behaves identically on all six adapters. See N-PLUS-ONE for the cost and the reasoning.
If you need a true server-side join (avoiding the round-trip overhead on a fan-out of thousands), reach for db.<model>.aggregate(...) directly — that surface is unfiltered and you can write a $lookup stage yourself. See Aggregation pipeline.
Embeds — sub-documents, no serialisation
f.embed() and f.embedMany() are first-class on Mongo: an embed becomes a sub-document and embedMany becomes a BSON array of sub-documents. There is no JSON-string serialisation step; the storage shape matches the in-memory shape.
const Order = model('orders', {
id: f.id(),
customer_id: f.objectId(),
items: f.embedMany(() => LineItem),
}, { /* ... */ });
// On disk:
// { _id: ObjectId('...'), customer_id: ObjectId('...'),
// items: [ { sku: 'BOOK-01', qty: 2 }, { sku: 'PEN-04', qty: 1 } ] }Embed access compiles to dotted-key paths (items.0.sku) for read and $set: { 'items.0.qty': 3 } for write. Mongo's positional $ and $[] operators are available via raw aggregate — the typed surface exposes update on a parent and some on embedMany (compiled to $elemMatch). See EMBED for the full surface.
Cascades, application-side
Mongo has no foreign-key enforcement. onDelete: 'Cascade' and 'SetNull' are walked in src/adapters/mongo/cascade.ts:
export async function applyCascadesForDelete(parentModel, parentDocs, ctx) {
const children = findChildRelations(parentModel);
for (const { childModel, rel } of children) {
const onDelete = rel.onDelete || 'NoAction';
if (onDelete === 'NoAction' || onDelete === 'Restrict') continue;
// ... resolve parent refs, coerce to ObjectId, fetch children, recurse, delete ...
}
}The walker recurses (Business → videos → likes/comments → replies) with a visited set guarding pathological loops. Leaves are deleted before parents so a concurrent reader does not see a dangling FK mid-walk.
Restrict is honoured by surfacing the cascade as a no-op — the parent delete proceeds but children stay. If you want strict enforcement (refuse the parent delete when any child exists), wrap the call in a $transaction and run a pre-flight count on the child collection.
Schema validation — $jsonSchema
Mongo 3.6+ supports collection-level JSON Schema validation (validator: { $jsonSchema: ... }, validationLevel: 'strict').
forge does not emit $jsonSchema validators by default.
The decision is deliberate. Three reasons:
- Two-source-of-truth risk. The TypeScript model is the schema. Emitting
$jsonSchemaon top duplicates that, and the duplication drifts the moment someone adds a field in code without re-pushing. - Migration friction.
validationLevel: 'strict'rejects writes that do not match — including writes from a service running an older version of the schema during a rolling deploy. The JS-side coercion incoerce.tsalready does the equivalent shape check before the write hits Mongo, so the runtime validation is redundant. - Embedded schema is full-featured.
$jsonSchemadoes not cover the same surface as forge field defs (no automatic ObjectId casting, noupdatedAttriggers, no defaults). Half-validating is worse than not validating.
If you want collection-level validation as a belt and braces second line (mostly to catch writes from another service that bypasses forge), the supported path is:
// One-off ops script. Run after forge push.
await db.command({
collMod: 'orders',
validator: { $jsonSchema: {
bsonType: 'object',
required: ['_id', 'customer_id', 'items'],
properties: {
_id: { bsonType: 'objectId' },
customer_id: { bsonType: 'objectId' },
items: { bsonType: 'array', minItems: 1 },
},
}},
validationLevel: 'moderate', // skip the validator on updates that don't touch validated fields
validationAction: 'warn', // log-only, never reject
});validationLevel: 'moderate' + validationAction: 'warn' is the only setting that survives a rolling deploy without dropping writes. Anything stricter needs a coordinated stop-the-world migration.
Index types forge emits
forge's IndexDef is a single shape that covers every dialect's index family. The Mongo emitter in src/adapters/mongo/apply-indexes.ts maps each field to the matching createIndex call.
| IndexDef field | Mongo emit |
|---|---|
keys: { col: 1 } | createIndex({ col: 1 }) — ASC b-tree |
keys: { col: -1 } | createIndex({ col: -1 }) — DESC b-tree |
keys: { col: 'text' } | createIndex({ col: 'text' }) — text index |
keys: { col: '2dsphere' } | createIndex({ col: '2dsphere' }) — geo (sphere) |
keys: { col: '2d' } | createIndex({ col: '2d' }) — legacy planar geo |
keys: { col: 'hashed' } | createIndex({ col: 'hashed' }) — for shard keys |
keys: { '$**': 1 } | wildcard — needs wildcardProjection |
unique: true | { unique: true } |
sparse: true | { sparse: true } (implicit on .unique() of optional field) |
expireAfterSeconds: 86400 | TTL — Mongo expires docs after N seconds past the indexed Date |
partialFilterExpression: {...} | { partialFilterExpression: {...} } |
where: {...} (object) | aliased to partialFilterExpression |
where: '...' (string) | SQL-only — skipped on Mongo |
collation: { locale, strength } | { collation: {...} } |
wildcardProjection: {...} | { wildcardProjection: {...} } |
method: 'spatial' | resolved to 2dsphere per key |
method: 'fulltext' | n/a — use .searchable() instead |
method: 'vector' | warned + skipped — Atlas Vector Search uses a different API |
method: 'gin' / 'gist' / 'brin' / 'hash' | warned + skipped (SQL-only) |
include: [...] | warned + skipped (SQL-only covering index) |
expression: '...' | warned + skipped (Mongo has no expression indexes) |
visible: false | warned + skipped |
Push is idempotent. pushAllIndexes calls listIndexes() once per collection, fingerprints the spec, and only createIndex's the differences. When the spec drifted (Mongo error 85/86 — "index already exists with different options"), forge drops and recreates the index with a ↻ log line:
📦 orders
⚡ idx_orders_customer_id (already up-to-date)
↻ idx_orders_status_uq (rebuilt — spec drifted)
✓ idx_orders_items_sku_text (new)The fingerprint in apply-indexes.ts preserves byte equality with the pre-2.2 form, which is why a pure-version-bump push does not unnecessarily rebuild your indexes.
Apply them from the app, not the CLI. db.$migrate() runs this same pass against the connection you already hold (2.20.5), so a server can guarantee its indexes before it accepts traffic:
const db = await createDb({ url: process.env.DATABASE_URL!, type: 'mongo', schema });
await db.$migrate(); // idempotent; writes nothing when in sync
await db.$migrate({ dryRun: true }); // plan only
await db.$migrate({ prune: true }); // also drop undeclared indexesDo not write your own boot-time createIndex loop — see INDEXES.md § 13a for the three ways that goes wrong.
Undeclared indexes. Anything live that the schema does not declare is reported (report.pending from $migrate(), a trailing list from the CLI) and left alone. prune drops it. _id_ and the *_fts shadow never count.
TTL drift is visible since 2.20.5. expireAfterSeconds is read back by introspect(), so forge diff / db.$diff() now report a changed retention window — and, more usefully, a TTL that has silently gone missing, which otherwise shows up only as documents that stop expiring.
Searchable fields collapse into one text index. Mongo allows exactly one text index per collection, so every field marked .searchable() lands in a combined { a: 'text', b: 'text', ... } index named forge_<collection>_fts. Per-field weights are set via { weights: { title: 10, body: 1 } } on the index. See FTS for the query side.
Atlas-specific features
MongoDB Atlas exposes several features that are not in the open-source server.
| Feature | Where forge supports it |
|---|---|
$search (Atlas Search) | Via raw db.<model>.aggregate([{ $search }, ...]) — no typed surface yet. Index lives in the Atlas Search index API, not createIndex. |
$vectorSearch (Atlas Vector Search) | First-class — orderBy: { embedding: { nearTo: { vector: [...] } } } compiles to a $vectorSearch pipeline. See VECTOR and the executor at executeSelectWithVectorSearch. |
| Atlas Data API | Deprecated by MongoDB in 2024. The pattern still works via a custom HTTP proxy — see DRIVERS. |
| Triggers | Out of scope — Atlas Triggers are a server-side function runtime. Use change streams (below) if you need in-process reaction to writes. |
| Charts | Read-only against the collection. Nothing for forge to do. |
| Online Archive | Tier collection writes; reads stay transparent. Push warns if it cannot create an index on the archive tier (you create archive indexes via the Atlas UI). |
Atlas Search ($search)
forge does not yet have a typed surface for $search. The Atlas Search index is created via the Search Index API (createSearchIndex in the driver, or the Atlas UI), and queries go through raw aggregation:
const hits = await db.posts.aggregate({
pipeline: [
{ $search: {
index: 'posts_default',
compound: {
must: [{ text: { query: 'mongodb', path: ['title', 'body'] } }],
should: [{ text: { query: 'atlas', path: 'title', score: { boost: { value: 3 } } } }],
},
}},
{ $limit: 50 },
{ $project: { title: 1, body: 1, score: { $meta: 'searchScore' } } },
],
});The collection wrapper's aggregate runs the pipeline through the driver with extended-JSON coercion ({$oid} / {$date} markers in your pipeline are converted to native types — see coerceExtendedJSON in coerce.ts).
Atlas Vector Search ($vectorSearch)
This one is in the typed surface. A f.vector(N) field plus orderBy: { embedding: { nearTo: { vector: [...] } } } on the query compiles to a $vectorSearch stage:
// src/adapters/mongo/execute.ts
const stage: any = {
$vectorSearch: {
index: `${dbField}_vector_index`,
path: dbField,
queryVector: v.vector,
numCandidates: Math.max(limit * 10, 50),
limit,
},
};
if (Object.keys(filter).length > 0) stage.$vectorSearch.filter = filter;
const pipeline = [
stage,
{ $set: { _distance: { $meta: 'vectorSearchScore' } } },
];The search index itself is not created by forge push — vector indexes live in the Atlas Search index API (createSearchIndex({ mappings: { fields: { embedding: { type: 'knnVector', dimensions, similarity }}}})). Push warns and skips when it encounters method: 'vector' on Mongo with a pointer to the Atlas docs. See VECTOR for the full story.
The aggregation pipeline
Several IR shapes compile directly to a pipeline.
| IR | Pipeline emit |
|---|---|
groupBy | [{$match: filter}, {$group: {...}}, {$match: havingMatch}, {$sort}, {$skip}, {$limit}] |
near filter combined with nearTo orderBy | [{$geoNear: {...query, key, distanceField, spherical}}, {$skip}, {$limit}, {$project}] |
nearTo on a vector field | [{$vectorSearch: {...}}, {$set: {_distance: {$meta}}}, {$project}] |
_count on hydration | [{$match: {<fk>: {$in: refs}}}, {$group: {_id: '$<fk>', c: {$sum: 1}}}] |
count({ distinct: [...] }) | [{$match: filter}, {$group: {_id: <by>}}, {$count: 'n'}] |
db.<model>.aggregate({ pipeline }) | the pipeline you passed, through coerceExtendedJSON |
Stage-by-stage notes:
$match— both pre-group (inexecuteGroupBy) and post-group (having). Thewherecompiler incompile-from-ir.tsis the same onefinduses, so$expr,$and,$or,$nor, regex flags, and every operator listed in QUERIES translate identically.$group—_count,_sum,_avg,_min,_maxbecome flat aliases (__agg_count_<field>, etc.) so the reshaper can rebuild the nested{ _count: { ... }, _sum: { ... } }payload regardless of bucket count._count: { _all: true }is{ $sum: 1 }.$lookup— not emitted by the typed surface for hydration (see Relations). Use raw aggregate when you need it.$facet— not emitted; raw aggregate is the path. Useful for multi-metric dashboards (see worked examples).$project— appended forselect/omitprojections. Mongo cannot mix0and1in one$projectexcept for_id: 0, so the compiler picks "all 1s" (exclusive) or "all 0s" (omit-only) — never a mix.$unwind— not emitted; raw aggregate. Useful when you need to group by an element of anembedManyarray.
Relation filters throw (behaviour change in 2.18.0)
A relation filter — where: { author: { is: { … } } }, where: { posts: { some: { … } } } — cannot be compiled for Mongo, because a find has no join. Up to 2.17.0 compileWhereNode returned {} for it: match-all. That is the most dangerous answer available, because the same compiler builds the filter for reads and for writes:
// On 2.17.0 and earlier this compiled to deleteMany({})
// and emptied the collection.
await db.post.deleteMany({ where: { author: { is: { email: 'x' } } } });A read was no better — it returned every row in the collection instead of the matching ones, with no error and nothing in the logs.
From 2.18.0 it throws, naming the relation and its mode:
[forge:mongo] a relation filter on 'author' (is) cannot be compiled for
Mongo — a find has no join. Resolve it in two steps:
const ids = (await db.author.findMany({ where: { email: 'x' }, select: { id: true } })).map(a => a.id);
await db.post.findMany({ where: { author_id: { in: ids } } });
Or use db.<model>.aggregate([{ $lookup: … }]) for a single round trip.So the rewrite is two queries — resolve the related ids, then filter on the foreign key:
const ids = (await db.author.findMany({
where: { email: 'x' },
select: { id: true },
})).map((a) => a.id);
await db.post.findMany({ where: { author_id: { in: ids } } });Or one round trip, with the join written out by hand:
await db.post.aggregate([
{ $lookup: { from: 'authors', localField: 'author_id', foreignField: '_id', as: 'author' } },
{ $match: { 'author.email': 'x' } },
]);This is a behaviour change. Code written against 2.17.0 or earlier that leant on the match-all — knowingly or not — now errors instead of running. That is the point: an error is recoverable, and a deleteMany that ignored its filter was not.
aggregate() is two methods behind one name, and only one of them is Mongo-specific. Forge dispatches on the arguments: any of the five bucket keys (_count, _sum, _avg, _min, _max) picks the portable form, anything else goes to the pipeline.
The bucket form — aggregate({ where, _count, _sum, _avg, _min, _max }) — is groupBy({ by: [] }) with the one-element array unwrapped. It runs on every dialect and ports without change.
The pipeline form — aggregate({ pipeline }) / aggregate([...]) — is Mongo-only. It hands the pipeline straight to the driver's collection.aggregate (src/builder/collection.ts), so there is no usable equivalent on a SQL adapter; a query written this way is a query that stays on Mongo. That is a fair trade for $lookup, $facet and $setWindowFields, as long as it is a decision rather than an accident. See AGGREGATIONS.md.
Transactions
db.$transaction(fn) opens a ClientSession and runs fn inside session.withTransaction(). The session is what the IR threads through ExecOpts.session to the executor, so every read and write inside fn participates:
// src/adapters/mongo/client.ts
async transaction<T>(fn: (session: ClientSession) => Promise<T>): Promise<T> {
const session = this.client.startSession();
try {
let result!: T;
await session.withTransaction(async () => {
result = await fn(session);
});
return result;
} finally {
await session.endSession();
}
}withTransaction is the driver's built-in retry loop — it automatically retries on TransientTransactionError and UnknownTransactionCommitResult. You get that retry for free; the SQL adapters have to implement it themselves (see TRANSACTIONS).
Replica set requirement
Multi-document transactions require a replica set or mongos. A default-config single-node mongod will reject startSession for transactions with:
Transaction numbers are only allowed on a replica set member or mongosMongoAdapter.capabilities.transactionsRequireReplicaSet = true surfaces this in the doctor report. The local-dev path is:
mongod --replSet rs0 --port 27017 --bind_ip localhostthen in the mongo shell once:
rs.initiate();That's a single-node replica set — no extra processes, no coordination, just the right semantics. The mongo:6 Docker image in forge's integration-mongo.ts does exactly this.
Single-document writes are always atomic
You do not need a transaction to atomically update one document. updateOne, findOneAndUpdate, insertOne, and deleteOne either commit fully or not at all. Reach for $transaction only when you need cross-document or cross-collection atomicity.
Compatibility footguns
- AWS DocumentDB. Multi-document transactions since 4.0 but with operator restrictions (
$lookup,$outare limited inside a tx). Test the actual statements forge emits. - Azure Cosmos DB (Mongo API). No multi-document transactions.
withTransactionthrows at runtime. Use the native Cosmos SDK if you need cross-document atomicity on Cosmos. - FerretDB. Best-effort, version-gated.
The fallback for these targets is to lean on single-document atomicity plus an outbox collection for fan-out semantics.
See TRANSACTIONS for the full discussion including read/write concern composition.
Change streams
Change streams (collection.watch()) emit one event per insert, update, replace, or delete (plus invalidate / rename / drop). The adapter exposes them through the WATCH pattern; the raw surface is:
const cursor = db.collection('orders').watch([
{ $match: { 'fullDocument.status': 'PAID' } },
], { fullDocument: 'updateLookup' });
for await (const change of cursor) {
// { operationType: 'insert'|'update'|'replace'|'delete',
// fullDocument: {...}, documentKey: { _id }, ns, clusterTime, ... }
}Three things to know:
- Replica set or
mongosrequired. Same constraint as transactions — change streams use the oplog. Cosmos (Mongo API) and FerretDB do not support them; DocumentDB does. fullDocument: 'updateLookup'triggers a separate read of the current document on every update event. Without it the change event carries only the delta. The lookup is at the cluster's read concern (not the change'sclusterTime), so the document you receive may already reflect a later write.- Resume tokens survive restarts. Persist
change._idto a cursor-state collection and pass{ resumeAfter: token }on reconnect to pick up exactly where you left off. The oplog rolls, so resume tokens have a finite lifetime — keep your watcher within the oplog window or you lose events.
forge surfaces watch via the runtime collection wrapper: db.orders.watch([pipeline], { fullDocument: 'updateLookup' }) returns the driver cursor unchanged — no decoding, no projection, no hydration. If you want forge-shaped documents in the event payload, run decodeRow(model, change.fullDocument) yourself.
Read concerns and write concerns
The four read concerns and their forge-relevant trade-offs:
| Read concern | Reads | Use when |
|---|---|---|
local (default) | last write that reached the queried node | the OLTP default — fast, possibly stale on secondaries |
available | same as local on replica sets; ignores orphans on sharded clusters | shard reads where orphans are tolerable |
majority | only writes acknowledged by a majority of the replica set | reads inside a tx; cross-shard consistency |
linearizable | reads serialised through the primary; reflects every preceding write | strictly avoid; expensive |
snapshot | reads from a consistent snapshot at a cluster time | inside transactions on 4.2+ |
Write concerns:
| Write concern | Ack on |
|---|---|
{ w: 0 } | client send (no ack) — never for production data |
{ w: 1 } (default) | primary applied |
{ w: 'majority' } | majority of voting members acked |
{ w: 'majority', j: true } | majority acked AND journaled |
{ w: <tag> } | custom — e.g. one node per datacentre |
forge does not set a read or write concern itself — it uses the client's defaults. To override, construct the client at the right level:
const client = new MongoClient(uri, {
readConcern: { level: 'majority' },
writeConcern: { w: 'majority', j: true, wtimeout: 5_000 },
});
export const db = await createDb({ schema, driver: mongoDriver(client) });The transaction's read/write concern comes from defaultTransactionOptions on the client — set it there if you want transactions to default to 'majority' while regular reads stay on 'local'.
Sharding
Forge does not pick your shard key. The choice belongs in the Mongo shell at sharding-enable time. The relevant principles:
- Pick a key that distributes writes evenly. A monotonically increasing key (
createdAt,ObjectId) sends every new write to the same chunk — a single hot shard. The default_idis monotonic enough that a high-write collection should not shard on it directly; use{ _id: 'hashed' }instead. - Include the shard key in every targeted query. A query that does not name the shard key fans out to every shard (
mongosscatter-gather). The performance gap is order-of-magnitude on a sharded cluster. - Compound shard keys. Tenant + time is the standard pattern —
{ tenant_id: 1, created_at: 1 }. Writes spread across tenants; per-tenant queries hit one chunk.
A hashed-id shard key on a collection forge manages:
sh.enableSharding('app');
sh.shardCollection('app.events', { _id: 'hashed' });The 'hashed' index has to exist before shardCollection. Declare it in the schema:
const Event = model('events', { /* ... */ }, {
indexes: [{ keys: { id: 'hashed' } }],
});forge push creates the hashed index. Sharding is enabled separately in your ops scripts — forge does not run sh.shardCollection for you, because the timing (early in the collection's life, before it grows) is a deployment decision.
Transactions across shards work but cost more (mongos coordinates). Change streams across shards work transparently — mongos merges them.
Connection pool
The defaults forge picks when no driver is injected (src/adapters/mongo/client.ts):
this._client = new (mongo().MongoClient)(uri, {
maxPoolSize: 50,
minPoolSize: 5,
connectTimeoutMS: 10_000,
serverSelectionTimeoutMS: 10_000,
retryWrites: true,
retryReads: true,
});When to override:
- Higher
maxPoolSizefor a Node webserver doing fan-out work (hydration loads N relations per request) — set it above the request concurrency floor or you head-of-line under load. maxPoolSize: 1on edge / Lambda where each invocation is its own process and pooling would just queue inside one cold start.retryWrites: falseon a sharded write workload that hates duplicates —retryWritesis safe for idempotent operations (insert, update by id) but creates double-writes on non-idempotent pipelines. Forge's typed mutations are idempotent; raw aggregate pipelines are your responsibility.socketTimeoutMSis not set by default. The driver leaves it off, which is correct for long-running aggregates and change streams. Set it (60_000+) only if you have a known runaway-query story to truncate.
Inject your own client when you need any non-default setting — forge never re-instantiates a client you handed it:
const client = new MongoClient(uri, {
maxPoolSize: 200,
writeConcern: { w: 'majority' },
});
export const db = await createDb({ schema, driver: mongoDriver(client) });Register db.$disconnect() from your SIGTERM handler. close() is idempotent on the driver.
close() only closes what forge opened. A client built from a url is closed. An injected one is released: the handle stops working (its db throws again) and your MongoClient stays open, because closing it is the caller's call. That is what makes the shared-client, database-per-tenant pattern in MULTI-TENANT.md safe — an evicted handle no longer takes every other tenant's connection with it. Close the client you created, once, at shutdown.
One client per createDb()
Each MongoAdapter owns its own DatabaseClient, which owns its own MongoClient and its own Db. Two createDb() calls in one process are two connections to two databases, and closing one does not touch the other.
const live = await createDb({ url: 'mongodb://host/app', schema });
const arch = await createDb({ url: 'mongodb://host/app_2023', schema });
await live.order.count(); // app.orders
await arch.order.count(); // app_2023.orders
await arch.$disconnect(); // `live` is still usableOne thing that is still shared: the active schema is module-level, so relation targets resolve against whichever schema was registered last. Register the same schema in both handles (as above), or keep the second one to raw statements.
That is new in 2.19.0, and the previous behaviour is worth stating because it was silent.
What the singleton broke
dbClient used to be the only client there was — a module-level singleton every Mongo code path reached for directly. Two things compounded:
connect()early-returned when it already held aDb, so the secondcreateDb()kept the first one's connection.- it read
process.env.DATABASE_URLrather than the URL it was handed, so even without the early return it would have resolved to whichever URL reached the environment first.
So createDb({ url: B }) wrote to A. No error, no warning, no log line other than the first connect's. And because both handles shared one client, $disconnect() on either closed the connection under both, so the surviving handle's next query failed for a reason with no obvious connection to the code that caused it.
The three shapes that hit this are exactly the ones where it hurts most: a migration or backfill tool holding source and destination, a test harness holding a fixture database and the app database, and a database-per-tenant deployment. For the last one it is a cross-tenant data write, which is the kind of bug you find in a support ticket.
adapter.db and adapter.mongoClient
The adapter exposes its own handles, which is what raw access should use now:
const coll = db.adapter.db.collection('audit_events'); // the Db
const session = db.adapter.mongoClient.startSession(); // the MongoClientBoth throw a clear error if read before connect() has resolved, rather than returning undefined and failing later. The session that $transaction opens comes from this client, so it is valid against this database and no other — see TRANSACTIONS.md for why the ambient session is keyed by adapter.
The exported dbClient still works
dbClient is public API — plenty of code uses it for raw collection access — so it was kept, and it is not deprecated. It forwards to the first client created, which in a single-database app is the only one there is:
import { dbClient } from 'forge-orm';
await dbClient.db.collection('orders').watch(); // unchanged behaviourA second createDb() gets its own client and does not disturb that default. Closing the first releases the default, so a later connect starts clean — and close() only closes a client forge opened, never one you injected (see Connection pool). The rule of thumb: dbClient for a single-database app, db.adapter.db when there is more than one handle in the process — with two handles, dbClient silently means "the first one", which is the ambiguity you are trying to get away from.
One deliberate leftover: connect() still populates process.env.DATABASE_URL from the first URL if it is not already set, because the CLI scripts (forge push and friends) connect without an adapter and read it from there.
BSON types — round-trip with forge
The Mongo wire shape is BSON, not JSON. Five BSON types matter for the forge round-trip; the rest pass through untouched.
ObjectId
f.id() / f.objectId(). App-side: a 24-char hex string. DB-side: a 12-byte BSON ObjectId. Coercion is in both directions:
// inbound — app → db
if (typeof v === 'string' && mongo().ObjectId.isValid(v)) {
return new (mongo().ObjectId)(v);
}
// outbound — db → app
if (value instanceof mongo().ObjectId) return value.toString();Filter values, FK refs, and $in: [...] arrays all go through the coercer (coerceFieldValue), so where: { author_id: 'abc123...' } works the same way where: { author_id: new ObjectId('abc123...') } does. The app code does not need to know.
A BSON instance in a filter is always a value, never a container of operators. That holds for ObjectId, Decimal128, Long, Binary, UUID, a raw Buffer and Date, in a bare comparison, inside in: [...], and under not::
where: { author_id: new ObjectId(id) } // eq
where: { author_id: { not: new ObjectId(id) } } // ne
where: { author_id: { in: ids } } // inBefore 2.20.2 these threw unknown operator 'buffer' — forge read Object.keys() off the instance and treated its internal fields as operator names. Only a plain object is parsed for operators now.
Date
f.dateTime(). App-side: a JS Date (or an ISO string / epoch number on input). DB-side: BSON Date (UTC, ms precision). The inbound coercer parses strings and numbers:
if (typeof v === 'string' || typeof v === 'number') {
const d = new Date(v);
return isNaN(d.getTime()) ? v : d;
}Mongo stores ms-precision UTC. Sub-millisecond precision is lost. If you need ns timestamps, use a string field with a normalised ISO form.
Decimal128
f.decimal(p, s). App-side: a string (forge's portable shape). DB-side: Decimal128. The driver handles the conversion on write when you pass a string that parses; on read forge converts back to string. Do not round-trip through number — JavaScript's float arithmetic loses precision on values past 2^53.
Binary
f.bytes() — new in 2.18.0, see BINARY. App-side: a Node Buffer or Uint8Array. DB-side: BSON Binary (subtype 0 by default). The driver round-trips both directly. ThumbHash payloads, image blobs, and signed tokens all fit here.
Unlike the SQL adapters, Mongo coerces and size-checks an f.bytes() value on the update path as well as on create, because both run through remapAndCoerce — see The update path and BSON coercion.
Regex and Long
Neither has a typed field; both pass through. f.string() with a regex stored in it appears as a plain string on read. Long (64-bit ints — useful for legacy data) requires the bsonRegExp: false, useBigInt64: true options on the driver if you want native bigints on read.
BigInt
Not a BSON type — forge does not auto-convert. Use Decimal128 for exact arithmetic or Long for 64-bit-int compatibility.
divide compiles to a pipeline update
Mongo has no $div update operator, so an exact division cannot be written as an update document at all. Before 2.18.0 the IR papered over that by rewriting the operator before any adapter saw it:
{ price: { divide: 3 } } // became $mul: 0.3333333333333333That is not a division. On an exact f.decimal() money column it drifts a little on every update, and the drift compounds — divide a balance by three often enough and the ledger no longer adds up. On an f.int() column it rounds, and turns the field into a double on the way.
From 2.18.0 a divide compiles to an aggregation-pipeline update (compileUpdateAsPipeline), which has a real $divide:
await db.product.update({
where: { id: 'p1' },
data: { price: { divide: 3 } },
});
// → updateOne(filter, [{ $set: { price: { $divide: [{ $ifNull: ['$price', 0] }, 3] } } }])This needs MongoDB 4.2 or newer — pipeline updates do not exist before that.
Every other operator in the same data is translated into the matching pipeline expression, so the whole update stays one atomic statement rather than splitting into two writes:
| Operator | Update document | Pipeline expression |
|---|---|---|
increment | $inc | $add |
multiply | $mul | $multiply |
divide | — (none exists) | $divide |
max / min | $max / $min | $max / $min |
push | $push | $concatArrays |
addToSet | $addToSet | $setUnion |
pull | $pull | $filter |
unset | $unset | $unset stage |
Literals are wrapped in $literal
Inside a pipeline, a bare string that begins with $ is a field reference, not text. So a plain $set stage would read the wrong thing entirely:
data: { name: '$5 off', price: { divide: 2 } }
// without the wrap: { $set: { name: '$5 off' } }
// → stores the value of a field called "5 off" (usually missing → the key is dropped)
// with it: { $set: { name: { $literal: '$5 off' } } }
// → stores the text "$5 off"Forge wraps every literal in the $set stage in $literal, so a value that happens to start with $ — a price, a template token, a shell snippet — is stored as you wrote it.
divide + upsert throws
A pipeline update has no insert-only branch: $setOnInsert is an update-document operator and a pipeline cannot carry one. So "divide on conflict, seed the row on insert" cannot be expressed as one statement, and forge says so instead of silently dropping half of it:
[forge:mongo] upsert cannot combine `create` with a `divide` update.
Mongo needs an aggregation-pipeline update for an exact divide, and a
pipeline has no $setOnInsert. Split it: upsert the row first, then
update with the divide.Split it into two calls, as the message says:
await db.product.upsert({
where: { sku: 'A1' },
create: { sku: 'A1', price: '90.00' },
update: {}, // no-op on hit
});
await db.product.update({
where: { sku: 'A1' },
data: { price: { divide: 3 } },
});Those are two statements, so they are not atomic together. Wrap them in db.$transaction(...) if that matters — which needs a replica set, see Transactions.
One narrow exception to the throw: if create seeds the same field you are dividing, the wrapper never reaches the pipeline compiler. That overlap already has its own Mongo fallback — update-then-create with a duplicate-key retry — so you get two statements rather than an error. The outcome is the same either way: two statements, not one. See UPSERT — Atomic ops in the update branch.
The rest of the adapters
divide: 0 never reaches an adapter: it is refused when the IR is built (src/ir/build/data.ts) with update.<field>.divide: cannot divide by zero. Every SQL dialect emits a native col = col / $n, and IndexedDB does the division in JS. Mongo is the only adapter that needs the pipeline detour.
The update path and BSON coercion
Every value forge writes has to be turned into the right BSON type first — an id string into an ObjectId, an ISO string into a BSON Date. Three code paths need that, and until 2.18.0 only two of them did it.
| Path | 2.17.0 and earlier | 2.18.0 |
|---|---|---|
create | coerced (coerceCreatePayload) | coerced |
where | coerced (compileLeaf) | coerced |
update / updateMany | keys renamed, values untouched | coerced |
update called remapKeys, which renamed app field names to db column names and passed the values through as they arrived. So the same field written two ways ended up with two different BSON types in one collection:
await db.shop.create({ data: { location_id: '652f…' } }); // → ObjectId('652f…')
await db.shop.update({ where: { id }, data: { location_id: '652f…' } });
// → "652f…" a String!And because where does coerce, every later read went looking for an ObjectId:
await db.shop.findMany({ where: { location_id: '652f…' } });
// compiles to { location_id: ObjectId('652f…') } — never matches the String rowsThose rows were invisible, not missing. The same held for dates: an ISO string handed to update stayed a string, so gt / lt range filters silently stopped matching it.
Nothing errored at any point. The damage was permanent, and it stayed invisible until something counted the rows and the number came back wrong — which is how this was found. Undoing it needs a repair script; forge cannot fix it for you on read, because a string and an ObjectId are two different values as far as the server is concerned.
If you are on 2.17.0 or earlier and you have ever written an f.objectId() or f.dateTime() field through update, assume you have mixed-type rows. Upgrading stops new ones appearing; it does not convert the ones already there. Check per field with a $type probe and convert what you find:
// How many rows hold a String where an ObjectId belongs?
await db.shop.aggregate([
{ $match: { location_id: { $type: 'string' } } },
{ $count: 'broken' },
]);What coerces now
$set, $push, $addToSet and $pull all go through remapAndCoerce, so a value written or pushed lands with the field's declared BSON type. Pushing an id string onto an array of ObjectId would otherwise have parked a String next to ObjectIds in the same array, which breaks every $in against it.
The atomic numeric operators — $inc, $mul, $max, $min — are left alone. They carry numbers, and a number needs no coercion.
Common errors
| Code / shape | Cause | Where to fix |
|---|---|---|
E11000 duplicate key (code 11000) | unique index hit | rethrown as DbKnownError('P2002', ...) — same shape as the SQL UniqueViolation (src/adapters/mongo/errors.ts) |
Transaction numbers are only allowed on a replica set member or mongos | mongod started without --replSet | start as mongod --replSet rs0 and rs.initiate() |
TransientTransactionError (code 251) | tx aborted, safe to retry | withTransaction retries automatically; the typed surface bubbles the final failure |
NotPrimary (code 10107) | wrote to a stepped-down secondary | client reconnects on its own; the write is retried only if retryWrites: true |
MongoServerSelectionError | no node reachable inside serverSelectionTimeoutMS | check the URI's replica-set name and TLS settings; forge defaults to 10s |
WriteConflict (code 112) | optimistic-concurrency conflict inside a tx | retry the whole $transaction(fn) — Mongo's tx model is OCC, not pessimistic |
IndexOptionsConflict (codes 85, 86) | createIndex with a name that exists but with different options | forge push detects and rebuilds; if you call createIndex yourself, drop first |
cannot create unique index over … with shard key (code 67) | unique index without prefixing the shard key | include the shard key in the index, or make the index partialFilterExpression-scoped |
path collision at … with conflicting path | $set and another update operator both target the same field (or one is a prefix of the other) | forge's compileUpdate drops the colliding entry from $setOnInsert; if you build raw updates, mirror that rule |
BSONObj size … is invalid (10334) | a document exceeded the 16 MB limit | move the offending field out — typically an embedMany that should be a child collection |
Cursor not found (43) | cursor timed out (10 min idle by default) | use cursor.batchSize or stream with { noCursorTimeout: true } (and clean up explicitly) |
MongoNetworkError: connection X to … closed | replica-set primary stepped down mid-query | driver retries reads on retryReads; idempotent writes on retryWrites |
The dup-key rethrow path is the only error forge translates by code. Everything else surfaces verbatim — the calling code can match on err.code or err.name for finer routing.
Three worked examples
1. Atlas Search hybrid query
Lexical + vector + filter in one pipeline. Atlas Search defines two indexes — a "default" text index on title / body, and a vector index on embedding. The hybrid query runs both through $rankFusion (Atlas-only stage) and applies a price filter:
const results = await db.products.aggregate({
pipeline: [
{ $rankFusion: {
input: {
pipelines: {
lexical: [
{ $search: {
index: 'products_default',
text: { query: req.query.q, path: ['title', 'body'] },
}},
{ $limit: 100 },
],
semantic: [
{ $vectorSearch: {
index: 'embedding_vector_index',
path: 'embedding',
queryVector: req.query.embedding,
numCandidates: 500,
limit: 100,
}},
],
},
},
combination: { weights: { lexical: 0.4, semantic: 0.6 } },
}},
{ $match: { price: { $gte: 10, $lte: 500 } } },
{ $project: {
title: 1, body: 1, price: 1,
score: { $meta: 'searchScore' },
}},
{ $limit: 20 },
],
});Two things to note. First, $rankFusion is Atlas-only — community Mongo will error at the server. Second, the filter goes after the two search pipelines, not inside them. Mongo's $search and $vectorSearch both accept their own filter clause if you want the filter to apply before scoring (cheaper but only over fields included in the search index's filter mapping).
Pair this with f.vector(N) on the embedding field and create the indexes via the Atlas UI — see VECTOR.
2. Change-stream watcher
A worker that reacts to every paid order, dedupes by _id, and survives restarts via resume tokens stored in a cursor_state collection.
This uses the exported dbClient, which is the right choice for a single-database app. In a process with more than one createDb() handle, use db.adapter.db instead — dbClient means "the first client created" — see One client per createDb().
import type { ChangeStreamDocument, ResumeToken } from 'mongodb';
import { dbClient } from 'forge-orm';
const CURSOR_NS = 'cursor_state';
const CURSOR_KEY = 'paid_orders_watcher';
async function loadResume(): Promise<ResumeToken | undefined> {
const state = await dbClient.db.collection(CURSOR_NS).findOne({ _id: CURSOR_KEY as any });
return state?.token;
}
async function saveResume(token: ResumeToken) {
await dbClient.db.collection(CURSOR_NS).updateOne(
{ _id: CURSOR_KEY as any },
{ $set: { token, ts: new Date() } },
{ upsert: true },
);
}
export async function watchPaidOrders() {
const resumeAfter = await loadResume();
const cursor = dbClient.db.collection('orders').watch(
[{ $match: {
operationType: { $in: ['insert', 'update'] },
'fullDocument.status': 'PAID',
} }],
{ fullDocument: 'updateLookup', ...(resumeAfter ? { resumeAfter } : {}) },
);
for await (const change of cursor as AsyncIterable<ChangeStreamDocument>) {
if (change.operationType !== 'insert' && change.operationType !== 'update') continue;
const order = (change as any).fullDocument;
if (!order) continue; // updateLookup races a delete
try {
await chargeWebhook(order);
await saveResume((change as any)._id);
} catch (err) {
// Surface but don't save the token — we'll reprocess on restart.
console.error('[paid_orders_watcher] dispatch failed', err);
throw err;
}
}
}The pattern lives or dies on the resume token. Save it only after the side effect succeeds — if the worker crashes between the side effect and the save, the next restart replays the event. That's at-least-once semantics; the consumer (chargeWebhook here) has to be idempotent.
3. $facet for a multi-metric dashboard
One round trip, four facets — pipeline runs the same $match once and fans out into per-bucket aggregations.
const dashboard = await db.orders.aggregate({
pipeline: [
{ $match: {
org_id: org.id,
created_at: { $gte: range.from, $lte: range.to },
}},
{ $facet: {
// Revenue by day
revenueByDay: [
{ $group: {
_id: { $dateTrunc: { date: '$created_at', unit: 'day' } },
total: { $sum: '$amount' },
n: { $sum: 1 },
}},
{ $sort: { _id: 1 } },
],
// Top customers by revenue
topCustomers: [
{ $group: { _id: '$customer_id', total: { $sum: '$amount' } } },
{ $sort: { total: -1 } },
{ $limit: 10 },
{ $lookup: {
from: 'customers', localField: '_id', foreignField: '_id', as: 'customer',
}},
{ $unwind: '$customer' },
{ $project: { _id: 1, total: 1, customer: { name: 1, email: 1 } } },
],
// Status distribution
statusDistribution: [
{ $group: { _id: '$status', n: { $sum: 1 } } },
],
// Headline numbers — one row, no further grouping
headline: [
{ $group: {
_id: null,
total: { $sum: '$amount' },
orders: { $sum: 1 },
uniqueCustomers: { $addToSet: '$customer_id' },
}},
{ $project: {
_id: 0, total: 1, orders: 1,
uniqueCustomers: { $size: '$uniqueCustomers' },
}},
],
}},
],
});
// dashboard is one document:
// { revenueByDay: [...], topCustomers: [...], statusDistribution: [...], headline: [...] }Why $facet over four parallel queries: the $match runs once, the indexes are walked once, and the network round-trip is single. For a dashboard endpoint hit on every page load, the difference is measurable.
The $lookup inside the topCustomers facet is the same case mentioned under Relations — forge does not emit it from the typed surface, but the raw aggregate path is unfiltered. Use it when the call site does not compose well into typed include.
Where to look in the source
| Concern | File |
|---|---|
| Adapter wiring + capabilities + doctor | src/adapters/mongo/adapter.ts |
Driver port (MongoDriver) | src/adapters/mongo/driver.ts |
| MongoClient lifecycle + transactions, one client per adapter | src/adapters/mongo/client.ts |
Lazy access to the mongodb peer dep | src/adapters/mongo/bson.ts |
| IR → driver-call compiler | src/adapters/mongo/compile-from-ir.ts |
| Args → IR compile API | src/adapters/mongo/compile.ts |
| Inbound / outbound coercion (ObjectId, Date, embeds, defaults, extended-JSON) | src/adapters/mongo/coerce.ts |
executeSelect + $geoNear + $vectorSearch bridges + hydration | src/adapters/mongo/execute.ts |
| Relation hydration — batched vs. per-parent paging (shared by all six adapters) | src/ir/hydrate-many.ts |
Cascade walker (onDelete: Cascade / SetNull) | src/adapters/mongo/cascade.ts |
Error translation (P2002, P2025) | src/adapters/mongo/errors.ts |
forge push — collect specs, fingerprint, ensureIndex | src/adapters/mongo/scripts/push.ts |
See also
- DRIVERS — the driver port shape; how Cosmos / DocumentDB / FerretDB fit (and where they don't).
- QUERIES — operator surface every adapter shares. The Mongo per-operator emits in §99.
- INDEXES — the full
IndexDefshape and the per-dialect emit matrix that the table in §5 is drawn from. - EMBED — embed surface,
$elemMatch,$push/$pullpaths, embed-vs-child-collection decisions. - FTS —
$textqueries, weighting, language handling, and when to graduate to Atlas Search. - VECTOR —
$vectorSearchpipeline, Atlas index creation, hybrid query patterns. - GEO — 2dsphere,
$geoNearrewrite for cross-fieldnearTo, TTL collections for trailing positions. - WATCH — change-stream surface, resume tokens, oplog window.
- TRANSACTIONS — full transaction discussion, read/write concern composition, cross-database limits.