Live queries

A live query sends your application a notification each time a record in a table is created, updated or deleted. This page shows how to start a live query with the Python SDK, read its notifications and stop it, and what happens to it when the connection drops. It is for readers who already connect to SurrealDB over a WebSocket, as described in Connecting to SurrealDB.

API references

Starting a live query

live() starts a live query on a table and returns a LiveSubscription, or an AsyncLiveSubscription on an asynchronous connection. Iterate the subscription to read its notifications. Each pass of the loop waits for the next change, for as long as the live query runs.

Use the subscription as a context manager to stop the live query when the block ends. In the example below, a second connection makes the change, as another part of your application or another client would.

from surrealdb import RecordID, Surreal

with Surreal("ws://localhost:8000") as db:
    db.signin({"username": "root", "password": "secret"})
    db.use("surrealdb", "docs")
    db.query("DEFINE TABLE IF NOT EXISTS person").execute()

    with db.live("person") as people:
        with Surreal("ws://localhost:8000") as writer:
            writer.signin({"username": "root", "password": "secret"})
            writer.use("surrealdb", "docs")
            writer.create(RecordID("person", "tobie"), {"name": "Tobie"})

        for change in people:
            print(change["action"], change["result"])
            break

The loop prints the first notification and then stops with break:

CREATE {'id': RecordID(table_name=person, record_id='tobie'), 'name': 'Tobie'}

The subscription listens from the moment live() returns, so the change made before the loop starts is kept for it. On an asynchronous connection live() is awaited, which is why the block starts with async with await.

From SurrealDB 3.0, the table must exist before a live query can start on it, so the example defines it first. Without the table, live() raises NotFoundError: The table 'person' does not exist. SurrealDB 2.x starts the live query anyway.

Reading a notification

The examples in the rest of this page use two open connections, db and writer, each signed in and using the surrealdb namespace and the docs database.

Each notification is a dict with five keys: action ("CREATE", "UPDATE" or "DELETE"), result (the record after the change, or before it for a DELETE), record (the RecordID of the record), id (the id of the live query) and session (the id of the session that started it). The notification reference describes each key. Printing a whole notification shows them all:

from pprint import pprint

with db.live("person") as people:
    writer.create(RecordID("person", "jaime"), {"name": "Jaime"})

    for change in people:
        pprint(change)
        break

{'action': 'CREATE',
 'id': UUID('86f54469-db1b-48a3-8df8-b616afbdddb0'),
 'record': RecordID(table_name=person, record_id='jaime'),
 'result': {'id': RecordID(table_name=person, record_id='jaime'),
            'name': 'Jaime'},
 'session': UUID('6b8e6c2a-cbb3-4354-a547-cab5ce667608')}

SurrealDB 2.x sends no session key. To read only some kinds of change, filter on the server with a LIVE SELECT statement, as shown in Live queries from SurrealQL, rather than checking action in the loop.

Stopping a live query

There are three ways to stop a live query, and each one ends every loop over its subscription, in this thread or another:

• •

  Leave the with or async with block of the subscription.

• •

  Call kill() on the subscription. On an asynchronous connection, write await people.kill().

• •

  Call db.kill() with the subscription or its id. A subscription is also the UUID of its live query, so db.kill(people) and db.kill(people.id) stop the same query.

people = db.live("person")
print(people.is_alive)

people.kill()
print(people.is_alive)

True
False

Calling kill() on a subscription that has already ended does nothing.

When a loop ends

A loop over a subscription ends without an error when one of these things happens:

• •

  This connection kills the live query, with kill(), db.kill() or the end of a with block.

• •

  Another connection kills it with KILL. The server announces this from SurrealDB 3.0. SurrealDB 2.x does not tell the subscriber, so the loop keeps waiting.

• •

  The connection closes, with close() or at the end of its with block.

• •

  The session that started the live query is reset with reset() or closed with close_session().

The server marks the end of a live query with a notification whose action is KILLED. The SDK ends the loop on it and does not yield it.

A loop raises ConnectionUnavailableError when the connection was lost and the live query was not started again, as described in Reconnecting. Every later read raises the same error, so a lost subscription never looks like one that finished.

A break does not stop the live query. The next loop over the same subscription continues from the first notification that the last loop did not read, so changes made in between are not missed. A loop over a subscription that has ended stops at once, and notifications that arrived before the end but were not read are dropped.

Receiving changes as JSON Patch

Pass diff=True to receive each change as a list of JSON Patch operations in result, in place of the whole record:

with db.live("person", diff=True) as people:
    writer.create(RecordID("person", "chris"), {"name": "Chris"})
    writer.update(RecordID("person", "chris")).merge({"age": 30})
    writer.delete(RecordID("person", "chris")).execute()

    for change in people:
        print(change["action"], change["result"])
        if change["action"] == "DELETE":
            break

CREATE [{'op': 'replace', 'path': '', 'value': {'id': RecordID(table_name=person, record_id='chris'), 'name': 'Chris'}}]
UPDATE [{'op': 'add', 'path': '/age', 'value': 30}]
DELETE [{'op': 'replace', 'path': '', 'value': None}]

An update sends only the fields that changed. SurrealDB 2.x sends the whole deleted record for a DELETE, in place of a patch.

Live queries from SurrealQL

A LIVE SELECT statement gives a live query the full language: a WHERE clause, a projection of chosen fields, and FETCH. Send it with query(), whose .first() returns the id of the new live query as a uuid.UUID, then attach a subscription to that id with live_of():

live_id = db.query("LIVE SELECT * FROM person WHERE age > 18").first()

with db.live_of(live_id) as adults:
    writer.create(RecordID("person", "lucy"), {"age": 12})
    writer.create(RecordID("person", "martin"), {"age": 40})

    for change in adults:
        print(change["action"], change["record"])
        break

CREATE person:martin

The record person:lucy does not match the WHERE clause, so it sends no notification. live_of() is not awaited on an asynchronous connection, so the block there is async with db.live_of(live_id) as adults:.

To receive only some kinds of change, filter on $event: LIVE SELECT * FROM person WHERE $event = 'CREATE' sends a notification for each created record and none for updates or deletes. From SurrealDB 3.0, a LIVE SELECT can also take query variables, such as query("LIVE SELECT * FROM person WHERE age > $min", {"min": 18}).

A subscription from live_of() has managed set to False. The connection did not start the query and does not know its statement, so it cannot start the query again after a reconnect.

Using subscribe_live()

subscribe_live() returns a generator of the notifications for a live query id, from live() or from a LIVE SELECT. It yields the same full notifications as a loop over a subscription. In SDK 2.x it yielded only the record, which is now notification["result"], so see Migrating to 3.x before you upgrade code that uses it.

live_id = db.query("LIVE SELECT * FROM person").first()

for change in db.subscribe_live(live_id):
    print(change["action"], change["result"])

On an asynchronous connection, subscribe_live() is awaited, and the generator it returns is read with async for. Without the await, the loop raises TypeError.

The generator ends in the same cases as a loop over a subscription. To stop the live query, pass its id to db.kill(). Each call to subscribe_live() registers a reader of its own, and every reader receives every notification of the live query.

Blocking connections and threads

A blocking connection has no background reader. It reads notifications from the socket while a loop over a subscription waits, and while any other request on the connection waits for its reply. Each notification goes to the subscription it belongs to, so a request made between two loops does not lose one.

One blocking connection can serve several subscriptions at once, each read by its own thread, and a write on the same connection while a loop runs also works. An asynchronous connection reads the socket in a background task, so notifications arrive while your code awaits something else.

Reconnecting

When the WebSocket drops, the connection opens a new one and starts every live query from live() again, under a new id on the server. Notifications keep arriving on the same subscription, and their id is still the id that the subscription was given. Changes made while the connection was down are not delivered, because the server ended the old live query when the socket closed. Automatic reconnection describes how the connection reconnects and how to configure it.

A live query is not started again, and reading its subscription raises ConnectionUnavailableError, in these cases:

• •

  The subscription came from live_of().

• •

  The session has signed in as a different user since the live query started.

• •

  The connection was made with reconnect=False, or used up its reconnect attempts.

To keep a LIVE SELECT running across reconnects, send the statement again when its subscription raises, and attach to the new id:

from surrealdb import ConnectionUnavailableError

statement = "LIVE SELECT * FROM person WHERE age > 18"

while True:
    adults = db.live_of(db.query(statement).first())
    try:
        for change in adults:
            print(change["action"], change["result"])
        break
    except ConnectionUnavailableError:
        continue

The break runs when the loop ends without an error, for example after a kill(), so the example stops instead of starting the query again.

Learn more

• •

  LiveSubscription for the properties, methods and notifications of a subscription

• •

  Surreal API reference for live(), live_of(), subscribe_live() and kill()

• •

  LIVE SELECT for the SurrealQL statement and its consistency guarantees

• •

  Multiple sessions for live queries started on a session