SurrealDB provides a number of different embeddings features that can be used to manage your data. This walkthrough shows how to embed your text with OpenAI, store the vectors in SurrealDB, and run fast k‑nearest‑neighbour (KNN) searches—all from Python. It follows the same flow you might have seen for Qdrant, but swaps in SurrealDB’s native vector‑search features so you can keep documents, graphs, and embeddings in one place.

Install the two clients

OpenAI calls the API that turns text into a 1 536‑dimensional vector, while SurrealDB lets your Python code connect over WebSocket or HTTP to any SurrealDB server.

If you do not have SurrealDB running already you can spin up an in‑memory node in one line:

pip install openai surrealdb

SurrealDB must be running somewhere. For local testing you can spin it up quickly:

docker run -p 8000:8000 surrealdb/surrealdb:latest start --user root --pass secret memory

Set up the OpenAI & SurrealDB clients

import asyncio
import openai
from surrealdb import Surreal

openai_client = openai.Client(api_key="YOUR_OPENAI_KEY")

TEXTS = [
    "SurrealDB is a multi-model database you can embed anywhere!",
    "Loved by devs who need graphs, documents **and** vector search in one box.",
]

async def init_db() -> Surreal:
    db = Surreal("ws://localhost:8000/rpc")          # change URL if remote
    await db.signin({"username": "root", "password": "secret"})
    await db.use("demo_ns", "demo_db")
    return db

Create the embeddings with OpenAI

text‑embedding‑3‑small is compact, cheap, and surprisingly accurate for semantic search. If you need more nuance (or plan to quantise heavily) you can upgrade to text‑embedding‑3‑large (3 072 dimensions) at roughly twice the cost.

EMBED_MODEL = "text-embedding-3-small"

emb_resp = openai_client.embeddings.create(
    input=TEXTS,
    model=EMBED_MODEL
)
emb_vectors = [row.embedding for row in emb_resp.data]   # list[list[float]]

Insert the documents into SurrealDB

SurrealDB is schemaless by default, so you can drop JSON directly into a table named article:

(table article with a vector field called embedding)

async def load_documents(db: Surreal):
    docs = [
        {"text": txt, "embedding": vec}
        for txt, vec in zip(TEXTS, emb_vectors)
    ]
    await db.create("article", docs)

asyncio.run(load_documents(asyncio.run(init_db())))

Add a vector index (HNSW)

The Hierarchical Navigable Small World (HNSW) index trades a tiny bit of accuracy for dramatic speed‑ups compared with brute‑force scanning—ideal once your table grows beyond a few thousand rows.Because HNSW lives in RAM today, index creation is instant but counts against your memory budget.

SurrealQL lets you add the index once; after that inserts are indexed automatically.

async def add_index(db: Surreal):
    await db.query("""
        DEFINE INDEX article_embedding_hnsw 
        ON article
        FIELDS embedding
        HNSW DIMENSION 1536;
    """)

asyncio.run(add_index(asyncio.run(init_db())))

The DIMENSION must match the length of the OpenAI vectors (1536 in this model). ([SurrealDB][1])

Search for the most relevant documents

async def semantic_search(question: str, k: int = 3):
    query_vec = openai_client.embeddings.create(
        input=[question],
        model=EMBED_MODEL
    ).data[0].embedding

    db = await init_db()

    # KNN search: <|k, ef|> – ef is “search breadth”. 100 is a decent default.
    result = await db.query(`
        LET $q := {query_vec};

        SELECT id, text,
               vector::distance::knn() AS distance
        FROM article
        WHERE embedding <|{k},100|> $q
        ORDER BY distance;
    """)
    return result[0]['result']

hits = asyncio.run(semantic_search("What’s the best database for unified search?"))
for hit in hits:
    print(hit["distance"], hit["text"])

The vector::distance::knn() helper returns the exact distance that the KNN operator just computed, so SurrealDB can immediately sort or filter without recalculating.

Two knobs to remember:

• k – the number of neighbours you want back.
• ef (the second number) – search breadth. Higher values spend more CPU for slightly better accuracy; 100–200 is a safe default.

(Optional) Binary-quantise the vectors

SurrealDB stores vectors as arrays of floats (F32 by default) and doesn’t yet expose built-in binary quantisation the way Qdrant does. If you need ultra-compact storage you can:

1. Quantise offline (e.g. convert to int8 with numpy).
2. Store the compressed list in a separate field (embedding_q8) and build an index on that instead:

DEFINE INDEX article_embedding_q8_mt
ON article
FIELDS embedding_q8
MTREE TYPE I8 DIMENSION 1536;

This keeps the SurrealDB side simple while you experiment with different quantisers. For more information about vector search in SurrealDB, see the Vector Search Reference guides.

Semantic search using SurrealDB

Intro

This guide demonstrates how to store OpenAI embeddings as SurrealDB vectors via the Rust SDK for the purposes of semantic search.

Setup

First set up a new Cargo project with cargo new project_name and add the following dependencies to Cargo.toml:

anyhow = "1.0.98"
async-openai = "0.28.3"
serde = "1.0.219"
surrealdb = { version = "2.3", features = ["kv-mem"] }
tokio = "1.45.0"

Inside main(), call the connect function with "memory" to instantiate an embedded database in memory.

use anyhow::Error;
use surrealdb::engine::any::connect;

#[tokio::main]
async fn main() -> Result<(), Error> {
    let db = connect("memory").await?;
    Ok(())
}

If you have a Cloud or local instance to connect to, you can pass that path into the connect function instead.

// Cloud address
let db = connect("wss://myinstance-06a4h41t12rtj7lsg45m3prm1k.aws-use1.surreal.cloud").await?;

// Local address
let db = connect("ws://localhost:8000").await?;

After connecting, select a namespace and database name, such as ns and db.

db.use_ns("ns").use_db("db").await?;

Create a vector table

Create a table to store documents and embeddings, along with an index for the embeddings:

DEFINE TABLE document;
DEFINE FIELD text ON document TYPE string;
DEFINE FIELD embedding ON document TYPE array<float>;
DEFINE INDEX hnsw_embed ON document FIELDS embedding HNSW DIMENSION 1536;

These can be called via a single .query() method.

let mut res = db
    .query(
        "DEFINE TABLE document;
DEFINE FIELD text ON document TYPE string;
DEFINE FIELD embedding ON document TYPE array<float>;
DEFINE INDEX hnsw_embed ON document FIELDS embedding HNSW DIMENSION 1536;",
    )
    .await?;

The size of the vector (1536 here) represents the number of dimensions in the embedding. Since OpenAI’s text-embedding-3-small model in this example uses 1536 as its default length, the vector size must be set to 1536.

The HNSW index is not strictly necessary to use the KNN operator (<||>) to find an embedding’s closest neighbours, and for our small sample code we will use the simple brute force method which chooses an algorithm such as Euclidean, Hamming, and so on. The following is the code that we will use, which uses the cosine of an embedding to find the four closest neighbours.

SELECT 
    text,
    vector::distance::knn() AS distance FROM document
    WHERE embedding <|2,COSINE|> $embeds
    ORDER BY distance;

As the dataset grows, if some loss of accuracy is acceptable then the syntax can be changed to use the HNSW index, by replacing an algorithm with a number that represents the size of the dynamic candidate list.

SELECT 
    text,
    vector::distance::knn() AS distance FROM document
    WHERE embedding <|2,40|> $embeds
    ORDER BY distance;

Another option is to use the MTREE index method.

Generate OpenAI embeddings

At this point, you will need an OpenAI API key to interact with the OpenAI API.

The best way to set the key is as an environment variable, OPENAI_API_KEY in this case. Using a LazyLock will let us call it via std::env::var() function the first time it is accessed. You can of course simply put it into a const for simplicity when first testing, but always remember to never hard-code API keys in your code in production.

static KEY: LazyLock<String> = LazyLock::new(|| {
    std::env::var("OPENAI_API_KEY").unwrap()
});

And then run the code like this:

OPENAI_API_KEY=whateverthekeyis cargo run

Or like this if you are using PowerShell on Windows.

$env:OPENAI_API_KEY = "whateverthekeyis"
cargo run

Inside main(), create a client from the async-openai crate holding this config inside main().

let config = OpenAIConfig::new().with_api_key(KEY);
let client = Client::with_config(config);

Then that to generate an OpenAI embedding using text-embedding-3-small that can be seen using a println! statement.

let input = "Octopuses solve puzzles and escape enclosures, showing advanced intelligence.";

let request = CreateEmbeddingRequestArgs::default()
        .model("text-embedding-3-small")
        .input(input)
        .dimensions(1536u32)
        .build()?;
let result = client.embeddings().create(request).await?;
println!("{result:?}");

Store embeddings in database

With the embeddings returned from the OpenAI client, they can be stored in the database. The response returned from the async-openai crate looks like this, with a Vec of Embedding structs that hold a Vec<f32>.

pub struct CreateEmbeddingResponse {
    pub object: String,
    pub model: String,
    pub data: Vec<Embedding>,
    pub usage: EmbeddingUsage,
}

pub struct Embedding {
    pub index: u32,
    pub object: String,
    pub embedding: Vec<f32>,
}

This simple request only returned a single embedding, so .remove(0) will do the job. In a more complex codebase you would probably opt for a match on .get(0) to handle any possible errors.

let embeds = result.data.remove(0).embedding;

Two structs can be put together here: one that implements Serialize to serve as the input put in a .create() statement, and another that implements Deserialize to show the result with the RecordId included.

#[derive(Serialize)]
struct DocumentInput {
    text: String,
    embedding: Vec<f32>,
}

#[derive(Debug, Deserialize)]
struct Document {
    id: RecordId,
    embedding: Vec<f32>,
    text: String,
}

Once that is done, we can print out the created documents as a Document struct.

let in_db = db
    .create::<Option<Document>>("document")
    .content(DocumentInput {
        text: input.into(),
        embedding: embeds.to_vec()
    })
    .await?;
println!("{in_db:?}");

We should now add some more document records. To do this, we’ll move the logic to create them inside a function of its own:

async fn create_embed(
    input: &str,
    db: &Surreal<Any>,
    client: &Client<OpenAIConfig>,
) -> Result<(), Error> {
    let request = CreateEmbeddingRequestArgs::default()
        .model("text-embedding-3-small")
        .input(input)
        .dimensions(1536u32)
        .build()?;
    let result = client.embeddings().create(request).await?;

    let embeds = &result.data.get(0).unwrap().embedding;

    let _in_db = db
        .create::<Option<Document>>("document")
        .content(DocumentInput {
            text: input.into(),
            embedding: embeds.to_vec(),
        })
        .await?;
    Ok(())
}

And then call it a few times inside main(). See if you can guess the answers yourself!

for input in [
    "Octopuses solve puzzles and escape enclosures, showing advanced intelligence.",
    "Sharks are primarily driven by instinct, but are capable of learning.",
    "Sea cucumbers lack a brain and show minimal cognitive response.",
    "Clams have simple nervous systems with no known intelligent behavior.",
    //
    "Seoul is South Korea’s capital and a global tech hub.",
    "Sejong is South Korea’s planned administrative capital.",
    "Busan a major South Korean port located in the far southeast.",
    "Tokyo is Japan’s capital, known for innovation and dense population.",
    //
    "Wilhelm II was Germany’s last Kaiser before World War I.",
    "Cyrus the Great founded the Persian Empire with tolerant rule.",
    "Napoleon Bonaparte was a French emperor and brilliant military strategist.",
    "Aristotle was a Greek philosopher who shaped Western intellectual thought.",
    //
    "Venus’s atmosphere ranges from scorching surface to Earth-like upper clouds.",
    "Mars has a thin, cold atmosphere with seasonal dust storms.",
    "Ceres has a tenuous exosphere with sporadic water vapor traces.",
    "Saturn’s atmosphere spans cold outer layers to a deep metallic hydrogen interior",
] {
    create_embed(input, &db, &client).await?
}

Semantic search

Finally, let’s perform semantic search over the embeddings in our database. Here is the query again:

SELECT 
    text,
    vector::distance::knn() AS distance FROM document
    WHERE embedding <|2,COSINE|> $embeds
    ORDER BY distance;

We will then put this into a separate function called ask_question() which uses the embedding retrieved from OpenAI to query the database against existing documents.

async fn ask_question(
    input: &str,
    db: &Surreal<Any>,
    client: &Client<OpenAIConfig>,
) -> Result<(), Error> {
    println!("{input}");
    let request = CreateEmbeddingRequestArgs::default()
        .model("text-embedding-3-small")
        .input(input)
        .dimensions(1536u32)
        .build()?;
    let mut result = client.embeddings().create(request).await?;

    let embeds = result.data.remove(0).embedding;

    let mut response = db.query("SELECT text, vector::distance::knn() AS distance FROM document WHERE embedding <|2,COSINE|> $embeds ORDER BY distance;").bind(("embeds", embeds)).await?;
    let as_val: Value = response.take(0)?;
    println!("{as_val}\n");
    Ok(())
}

You can now call this function a few times inside main() to confirm that the results are what we expect them to be.

ask_question("Which Korean city is just across the sea from Japan?", &db, &client).await?;
ask_question("Who was Germany's last Kaiser?", &db, &client).await?;
ask_question("Which sea animal is most intelligent?", &db, &client).await?;
ask_question("Which planet's atmosphere has a part with the same temperature as Earth?", &db, &client).await?;

The output shows that the closest documents to our question do indeed show up first.

Which Korean city is just across the sea from Japan?
[{ distance: 0.4879310782198243f, text: 'Busan a major South Korean port located in the far southeast.' }, { distance: 0.572999190509329f, text: 'Seoul is South Korea’s capital and a global tech hub.' }]

Who was Germany's last Kaiser?
[{ distance: 0.3236345624131668f, text: 'Wilhelm II was Germany’s last Kaiser before World War I.' }, { distance: 0.7554141523606017f, text: 'Napoleon Bonaparte was a French emperor and brilliant military strategist.' }]

Which sea animal is most intelligent?
[{ distance: 0.45382501257446206f, text: 'Octopuses solve puzzles and escape enclosures, showing advanced intelligence.' }, { distance: 0.4951347026545868f, text: 'Clams have simple nervous systems with no known intelligent behavior.' }]

Which planet's atmosphere has a part with the same temperature as Earth?
[{ distance: 0.4445578407153489f, text: 'Venus’s atmosphere ranges from scorching surface to Earth-like upper clouds.' }, { distance: 0.5039940919211086f, text: 'Saturn’s atmosphere spans cold outer layers to a deep metallic hydrogen interior' }]

At this point, you could give the HNSW index a try by changing the <|2,COSINE|> in the query to something like <|2,40|>. The distance numbers will end up looking quite different, but the ordering of the closest neighbours will probably be the same in this small example.

Here is the final code:

use std::sync::LazyLock;

use anyhow::Error;
use async_openai::{Client, config::OpenAIConfig, types::CreateEmbeddingRequestArgs};
use serde::{Deserialize, Serialize};
use surrealdb::{
    RecordId, Surreal, Value,
    engine::any::{Any, connect},
};

static KEY: LazyLock<String> = LazyLock::new(|| std::env::var("OPENAI_API_KEY").unwrap());

#[derive(Serialize)]
struct DocumentInput {
    text: String,
    embedding: Vec<f32>,
}

#[derive(Debug, Deserialize)]
struct Document {
    id: RecordId,
    embedding: Vec<f32>,
    text: String,
}

async fn create_embed(
    input: &str,
    db: &Surreal<Any>,
    client: &Client<OpenAIConfig>,
) -> Result<(), Error> {
    let request = CreateEmbeddingRequestArgs::default()
        .model("text-embedding-3-small")
        .input(input)
        .dimensions(1536u32)
        .build()?;
    let mut result = client.embeddings().create(request).await?;

    let embeds = result.data.remove(0).embedding;

    let _in_db = db
        .create::<Option<Document>>("document")
        .content(DocumentInput {
            text: input.into(),
            embedding: embeds.to_vec(),
        })
        .await?;
    Ok(())
}

async fn ask_question(
    input: &str,
    db: &Surreal<Any>,
    client: &Client<OpenAIConfig>,
) -> Result<(), Error> {
    println!("{input}");
    let request = CreateEmbeddingRequestArgs::default()
        .model("text-embedding-3-small")
        .input(input)
        .dimensions(1536u32)
        .build()?;
    let mut result = client.embeddings().create(request).await?;

    let embeds = result.data.remove(0).embedding;

    let mut response = db.query("SELECT text, vector::distance::knn() AS distance FROM document WHERE embedding <|2,COSINE|> $embeds ORDER BY distance;").bind(("embeds", embeds)).await?;
    let as_val: Value = response.take(0)?;
    println!("{as_val}\n");
    Ok(())
}

#[tokio::main]
async fn main() -> Result<(), Error> {
    let db = connect("memory").await?;

    db.use_ns("ns").use_db("db").await?;

    let mut res = db
        .query(
            "DEFINE TABLE document;
             DEFINE FIELD text ON document TYPE string;
             DEFINE FIELD embedding ON document TYPE array<float>;
             DEFINE INDEX hnsw_embed ON document FIELDS embedding HNSW DIMENSION 1536 DIST COSINE;",
        )
        .await?;
    for (index, error) in res.take_errors() {
        println!("Error in query {index}: {error}");
    }

    let config = OpenAIConfig::new().with_api_key(&*KEY);

    let client = Client::with_config(config);

    for input in [
        "Octopuses solve puzzles and escape enclosures, showing advanced intelligence.",
        "Sharks are primarily driven by instinct, but are capable of learning.",
        "Sea cucumbers lack a brain and show minimal cognitive response.",
        "Clams have simple nervous systems with no known intelligent behavior.",
        //
        "Seoul is South Korea’s capital and a global tech hub.",
        "Sejong is South Korea’s planned administrative capital.",
        "Busan a major South Korean port located in the far southeast.",
        "Tokyo is Japan’s capital, known for innovation and dense population.",
        //
        "Wilhelm II was Germany’s last Kaiser before World War I.",
        "Cyrus the Great founded the Persian Empire with tolerant rule.",
        "Napoleon Bonaparte was a French emperor and brilliant military strategist.",
        "Aristotle was a Greek philosopher who shaped Western intellectual thought.",
        //
        "Venus’s atmosphere ranges from scorching surface to Earth-like upper clouds.",
        "Mars has a thin, cold atmosphere with seasonal dust storms.",
        "Ceres has a tenuous exosphere with sporadic water vapor traces.",
        "Saturn’s atmosphere spans cold outer layers to a deep metallic hydrogen interior",
    ] {
        create_embed(input, &db, &client).await?
    }

    ask_question(
        "Which Korean city is just across the sea from Japan?",
        &db,
        &client,
    )
    .await?;

    ask_question("Which Korean city is just across the sea from Japan?", &db, &client).await?;
    ask_question("Who was Germany's last Kaiser?", &db, &client).await?;
    ask_question("Which sea animal is most intelligent?", &db, &client).await?;
    ask_question("Which planet's atmosphere has a part with the same temperature as Earth?", &db, &client).await?;

    Ok(())
}