Wire protocol

Clients subscribe to database queries over a WebSocket and receive the results, then fresh results whenever the data changes. Every message is a JSON object in a text frame.

• Overview
• Client → server
• Subscribing
• Scopes and payloads
• Server → client
• A list subscription, step by step
• Incremental mode
• Timing
• Queries
• Session messages
• Change events (for backend authors)
• Compatibility

Overview

The server keeps one client object per socket. A socket can hold any number of subscriptions, each named by a target chosen by the browser (for example openTasks). The target is the key for everything that follows: re-subscribing with the same target replaces its query, results arrive under that key, and unsubscribe takes it as its argument.

Each subscription names an observe channel, which is a collection or table the server has registered, and a scope: a single document, a list, or a count. The server runs the query, sends the result, and runs it again when a relevant change arrives from the database.

Client → server

Subscribing

{
	"type": "subscribe",
	"target": "openTasks",
	"scope": "many",
	"observe": "tasks",
	"config": {
		"subscriptionId": "5f1c…",
		"query": {"status": "open"},
		"sort": {"dueAt": 1},
		"fields": ["title", "dueAt"],
		"page": 1,
		"pageSize": 20
	}
}

Sending the same config again (ignoring subscriptionId) changes nothing. Any other change re-runs the query immediately.

Scopes and payloads

Results arrive as update messages whose payload holds the data under the target's name:

A dotted target such as dashboard.tasks nests the data: payload.dashboard.tasks.

Server → client

Messages produced by a subscription also carry subscriptionId, target, execution_time (such as "3.12ms"), and the effective query, sort, fields, paging, populates, virtuals as JSON strings, which are useful when debugging.

A list subscription, step by step

These messages are recorded from the reference implementation, with timings left out.

1. Subscribe

→ {"type": "subscribe", "target": "list", "scope": "many", "observe": "items",
   "config": {"subscriptionId": "s1", "query": {"status": "open"}, "sort": {"n": 1}}}

← {"type": "debug", "payload": {"event": "updateSubscription", "target": "list", "availableTargets": "list"}, …}
← {"type": "update", "subscriptionId": "s1", "target": "list",
   "payload": {"list": [{"_id": "a", …}, {"_id": "b", …}, {"_id": "d", …}], "_listRecounting": true}, …}
← {"type": "total", "subscriptionId": "s1", "target": "list", "total": 3, …}

The first result is sent before the total is known, so the browser can render immediately.

2. Someone inserts a matching document

← {"type": "update", "payload": {"list": [ …4 documents… ], "_listCount": 3}, …}
← {"type": "total", "total": 4, …}

3. Someone edits a document outside the query

Nothing is sent: the changed document doesn't match {"status": "open"} and the change doesn't touch a queried field.

4. Unsubscribe

→ {"type": "unsubscribe", "target": "list"}
← {"type": "debug", "payload": {"event": "removeSubscription", "target": "list", "availableTargets": ""}, …}

Incremental mode

With "incremental": true a many subscription sends increment messages:

• The first load and full reloads carry the whole array in payload[target] (without _<target>Count).
• An inserted or updated document that matches arrives alone, as payload[target] = {document}: insert or replace it by _id.
• An updated document that no longer matches arrives as a delete message with its id.
• Deletions in the database trigger a full reload.

The browser is responsible for sort order and page size when merging single documents.

Timing

• Change bursts: per subscription, the first change is handled at once; further changes within delay ms collapse into one more reload at the end of that window.
• Totals: after the query changes, total follows the result immediately. Otherwise totals are limited to one every 5 seconds: the first is sent at once, and the latest is delivered when the 5 seconds are up.
• Session: after authenticate, the server checks the session and sends refresh if it has one, then re-checks at 95% of refresh_in (default 5 minutes).
• Ping: when enabled on the server, ping is sent on connect and every 60 seconds.

Queries

Queries use MongoDB syntax. The server checks changed documents against your query to decide whether to reload, using this operator set:

$eq $ne $gt $gte $lt $lte $in $nin $exists $not $regex $options $and $or $nor

Plain values mean equality; an array field matches if it contains the value; a missing field equals null. With other operators the server can't decide, so it reloads on every change.

On the SQL backends $ne, $nin, $not and $nor follow MongoDB, so they also match NULL columns.

Session messages

Authentication is up to the server. It supplies a connection manager that receives authenticate, location, ping round-trip times and disconnects, and can return a refresh message. Data middlewares registered per observe channel can rewrite each result for the connected user before it is sent, for example to strip fields.

Change events (for backend authors)

Backends report changes in the shape of MongoDB change events. The SQL backends build the same shape from triggers:

{
	"operationType": "update",
	"documentKey": {"_id": "42"},
	"updateDescription": {"updatedFields": {"status": "done"}, "removedFields": []},
	"fullDocument": {"id": 42, "title": "Ship it", "status": "done"}
}

An empty object {} means "something may have changed, reload everything". Backends send it after reconnecting to the database.

Compatibility

The TypeScript packages (v3) and the Rust crates speak the same protocol. The Rust implementation is tested against message sequences recorded from @owservable/core 3.2.0. Known differences:

• error messages carry the error text in Rust; v3 usually sent {}.
• For $exists, Rust treats an explicit null as missing, matching the SQL backends. A fix for v3 is planned.