Skip to content

Python API reference

Everything below is importable from lance_csr. The classes are thin Python wrappers around the Rust engine, and this page is generated from their docstrings and type hints, so it matches the installed package. The guides link here for exact signatures and link back for examples.

Class Purpose
GraphConfig, GraphConfigBuilder Map tables to node labels and relationship types
CypherEngine Run many Cypher queries over registered datasets
CypherQuery Run one Cypher query
SqlEngine, SqlQuery Run SQL through DataFusion
VectorSearch, DistanceMetric Vector similarity search and reranking
DirNamespace Resolve table names to Lance datasets in a directory
ExecutionStrategy, SqlDialect Enums

lance_csr.__version__ is the package version.

This page covers the Python API. Rust users should read Using from Rust and the Rust API reference on docs.rs.

Type checking

Every public class is type-annotated, and the package ships a py.typed marker. Editors and type checkers can follow builder methods, parameters, query results and statistics. The annotations describe the API; they don't change runtime validation or exceptions.

PyArrow distributes its type information separately. For strict checks of Arrow inputs and results, add pyarrow-stubs to your development dependencies:

Bash
uv add --dev pyright pyarrow-stubs
uv run pyright your_script.py
Python
from typing import assert_type

import pyarrow as pa
from lance_csr import CypherEngine, GraphConfig

config = GraphConfig.builder().node_table("Person", "id").build()
engine = CypherEngine(config, {"Person": pa.table({"id": [1, 2]})})
result = engine.execute("MATCH (p:Person) RETURN p.id ORDER BY p.id")
assert_type(result, pa.Table)
print(result.to_pylist())  # [{'p.id': 1}, {'p.id': 2}]

Datasets

Methods that take datasets take a dict[str, object] mapping each node label or relationship type (or SQL table name) to one of:

  • lance.LanceDataset: read lazily and version-pinned by CypherEngine; read into memory by the other classes;
  • pyarrow.Table, pyarrow.RecordBatch, pyarrow.RecordBatchReader, or any object exporting the Arrow C stream interface (__arrow_c_stream__).

Keys match labels case-insensitively.

Exceptions

Exception When
ValueError Invalid query, incompatible known operand types (with expression and type details), unknown label, missing dataset or column, unsupported parameter type, empty datasets
NotImplementedError A Cypher feature that parses but isn't supported
RuntimeError Execution or I/O failures
TypeError A dataset value of an unsupported type, such as a LanceDB Table, or an argument of the wrong class, such as a string where a GraphConfig is expected

Timeouts

CypherEngine.execute, CypherEngine.execute_batches and CypherQuery.execute accept keyword-only timeout in seconds (None means no deadline). Zero expires immediately; negative, nonfinite or unrepresentable durations raise ValueError. One deadline includes planning, every batch and pauses between reads. Before returning a reader, expiry raises TimeoutError. Reading an expired native Arrow reader raises pyarrow.ArrowInvalid containing Query timeout exceeded. Earlier batches remain valid. Expiry or reader close releases the stream even while the caller is idle; the engine remains usable.

Cancellation is cooperative. Traversal loops yield at bounded work intervals, and timed cross joins yield between output batches. Timed nested-loop joins check the deadline while evaluating candidates, including batches that produce no matches. Synchronous parsing, Python data providers and foreign blocking calls cannot be forcibly preempted. Deadlines are checked between planning stages and before accepting completed results; this is not a hard real-time limit.

Path values

Cypher named paths can be returned as Arrow structs containing length, nodes, node_records and relationships. They retain their values through collect() and UNWIND; nodes(path) returns integer keys, and relationships(path) returns typed relationship records. startNode(r) and endNode(r) return the stored endpoint node records. Their occurrence identities are query-local. See path values and their memory cost.

GraphConfig

Python
GraphConfig()

An immutable mapping from tables to a property graph.

Create one with GraphConfig.builder(); there is no public constructor. See Graph schema.

builder

Python
builder() -> GraphConfigBuilder

Start a new, empty builder.

Returns:

Type Description
GraphConfigBuilder

The same as GraphConfigBuilder().

node_labels

Python
node_labels() -> list[str]

The configured node labels, lower-cased, in no particular order.

relationship_types

Python
relationship_types() -> list[str]

The configured relationship types, lower-cased, in no particular order.

GraphConfigBuilder

Python
GraphConfigBuilder()

Builds a GraphConfig from node and relationship tables.

GraphConfigBuilder() is the same as GraphConfig.builder(). Every method returns a new builder and leaves this one unchanged.

node_table

Python
node_table(label: str, primary_key: str) -> Self

Add a node table.

The nodes labelled label are the rows of the dataset named label, each identified by its primary_key column.

Parameters:

Name Type Description Default
label str

The node label, and the name of its dataset (for example "Person").

required
primary_key str

The column that identifies each node.

required

Returns:

Type Description
GraphConfigBuilder

A new builder with the node table added.

rel_table

Python
rel_table(rel_type: str, from_table: str, to_table: str, src: str, dst: str) -> Self

Add a relationship table.

The edges of type rel_type are the rows of the dataset named rel_type, from from_table nodes to to_table nodes. Both node tables must be added by the time build runs.

Parameters:

Name Type Description Default
rel_type str

The relationship type, and the name of its dataset (for example "KNOWS").

required
from_table str

The node table at the source end.

required
to_table str

The node table at the target end.

required
src str

The column holding the source node's primary key.

required
dst str

The column holding the target node's primary key.

required

Returns:

Type Description
GraphConfigBuilder

A new builder with the relationship table added.

build

Python
build() -> GraphConfig

Validate the tables and return the config.

Returns:

Type Description
GraphConfig

The configured graph.

Raises:

Type Description
ValueError

If the configuration is invalid, for example when a relationship's endpoint label has no node table.

with_node_label

Python
with_node_label(label: str, id_field: str) -> Self

Deprecated: use node_table.

Emits DeprecationWarning.

with_relationship

Python
with_relationship(rel_type: str, source_field: str, target_field: str, source_label: str | None = None, target_label: str | None = None) -> Self

Deprecated: use rel_table.

rel_table also takes the node tables at both ends. Emits DeprecationWarning.

Raises:

Type Description
ValueError

If only one of source_label and target_label is given.

CypherEngine

Python
CypherEngine(config: GraphConfig, datasets: dict[str, DatasetT], use_csr: bool = False, *, target_partitions: int | None = None)

Run many Cypher queries over the same datasets.

The engine registers datasets once and reuses the catalog, the DataFusion session and (with use_csr=True) the CSR index across queries. It is safe to share between threads.

Lance datasets are read lazily, pinned to the version passed in: later writes, by this or another process, aren't visible until refresh. Each dataset is pinned separately, so versions across tables are not a transactional snapshot. PyArrow inputs are held in memory and never change; create a new engine to replace them.

execute and execute_batches reuse up to 128 unoptimized query templates per snapshot, keyed by the exact query text and the names and types of scalar parameters. Values are bound before optimization on every call. Container parameters, rich path values, CALL subqueries, parameterized SKIP/LIMIT and unsupported template shapes use ordinary planning. explain always plans directly, and a successful refresh clears the cache. See Repeated queries.

Parameters:

Name Type Description Default
config GraphConfig

The graph mapping.

required
datasets dict[str, Dataset]

The tables, keyed by node label or relationship type (see Datasets). Lance datasets are pinned to the version passed in and not read here.

required
use_csr bool

Build the experimental CSR index for relationship types with integer endpoints, and node indexes for labels with unique integer keys.

False
target_partitions int or None

Keyword-only. A target from 1 to 1024 for DataFusion query partitioning; None keeps DataFusion's default. refresh preserves it. It is not a thread or memory limit. See Query parallelism.

None

Raises:

Type Description
ValueError

If datasets is empty, a configured key or endpoint column is missing, a Lance dataset was opened through a namespace client and can't be reopened by URI, or target_partitions is outside 1 to 1024.

TypeError

If config isn't a GraphConfig, a dataset has an unsupported type, or target_partitions is a boolean or another non-integer other than None.

RuntimeError

If a dataset can't be opened.

Examples:

Python Console Session
>>> import pyarrow as pa
>>> from lance_csr import CypherEngine, GraphConfig
>>> config = (
...     GraphConfig.builder()
...     .node_table("Person", "id")
...     .rel_table("KNOWS", "Person", "Person", "src_id", "dst_id")
...     .build()
... )
>>> engine = CypherEngine(config, {
...     "Person": pa.table({"id": [1, 2], "name": ["Alice", "Bob"]}),
...     "KNOWS": pa.table({"src_id": [1], "dst_id": [2]}),
... })
>>> engine.execute(
...     "MATCH (a:Person)-[:KNOWS]->(b:Person) RETURN a.name, b.name"
... ).to_pylist()
[{'a.name': 'Alice', 'b.name': 'Bob'}]

execute

Python
execute(query: str, params: dict[str, ParameterT] | None = None, *, timeout: float | None = None) -> Table

Run a Cypher query and return the whole result.

The result is held in memory; use execute_batches for large outputs. Direct counts from size() of an unmapped list comprehension have a non-nullable Arrow field; see Cypher functions.

Parameters:

Name Type Description Default
query str

The Cypher query.

required
params dict[str, Parameter] or None

Values for $name parameters: int, float, str, bool, None, lists, string-keyed nested maps, datetime.datetime and datetime.date (compared as timestamps and dates). Integers must fit in a signed 64-bit value.

None
timeout float or None

Keyword-only. Seconds for the whole query; None means no deadline. See Timeouts.

None

Returns:

Type Description
Table

The query result.

Raises:

Type Description
ValueError

If the query is invalid, refers to an unknown label, compares or combines incompatible types, or a parameter is unsupported, nested beyond 128 levels or cyclic.

NotImplementedError

If the query uses a Cypher feature that parses but isn't supported.

TimeoutError

If the query runs past timeout.

RuntimeError

If execution fails.

execute_batches

Python
execute_batches(query: str, params: dict[str, ParameterT] | None = None, *, timeout: float | None = None) -> RecordBatchReader

Run a Cypher query and stream the result.

Batches are computed as the reader is consumed. Scans, filters and projections stream with bounded memory; joins, sorts and aggregations may still buffer. The reader keeps its snapshot alive, so a concurrent refresh doesn't affect it. Planning errors raise immediately; execution errors raise while reading.

Parameters:

Name Type Description Default
query str

The Cypher query.

required
params dict[str, Parameter] or None

Values for $name parameters, as for execute.

None
timeout float or None

Keyword-only. Seconds for the whole query, including pauses between reads; None means no deadline. See Timeouts.

None

Returns:

Type Description
RecordBatchReader

The result, one batch at a time.

Raises:

Type Description
ValueError

If the query is invalid.

TimeoutError

If the deadline passes before the reader is returned. Reading after it passes raises pyarrow.ArrowInvalid containing Query timeout exceeded.

RuntimeError

If planning fails.

explain

Python
explain(query: str, params: dict[str, ParameterT] | None = None) -> str

The plans for a query, without running it.

Parameters:

Name Type Description Default
query str

The Cypher query.

required
params dict[str, Parameter] or None

Values for $name parameters, as for execute.

None

Returns:

Type Description
str

The graph logical plan, the DataFusion logical plan and the physical plan, including pushed-down scans.

Raises:

Type Description
ValueError

If the query is invalid.

refresh

Python
refresh() -> None

Move every Lance dataset to its latest version.

Updates the CSR, node indexes and statistics to match. Datasets whose version and commit time are unchanged keep their open handle. The replacement is validated in full before it's swapped in: on error nothing changes and the previous snapshot stays usable. Queries already running finish on the snapshot they started with. PyArrow inputs are unchanged. Concurrent calls run one at a time, and target_partitions is preserved. See Lance datasets.

node_index_stats

Python
node_index_stats() -> dict[str, NodeIndexStats]

Statistics for each node index (experimental; needs use_csr=True).

A label is missing when its keys aren't unique integers; relationships that end at it then use a join.

Returns:

Type Description
dict[str, NodeIndexStats]

Per node label: rows, version and build_ms, as in csr_stats.

csr_stats

Python
csr_stats() -> dict[str, CsrStats]

Statistics for each CSR adjacency index (experimental; needs use_csr=True).

An identical result after refresh means the snapshot itself didn't change. See Inspecting the CSR.

Returns:

Type Description
dict[str, CsrStats]

Per relationship type with a CSR, empty without use_csr=True:

  • edges: live indexed edges (null endpoints are skipped).
  • delta_edges, tombstones: edges appended and removed since the last full build, held outside the base CSR.
  • update: how the last refresh produced it: "build" (first), "rebuild" (full), "delta" (changes applied) or "reused" (no edge change).
  • version: the relationship dataset version it reflects (None for PyArrow).
  • build_ms: wall time of that build or delta, both directions.

dataset_versions

Python
dataset_versions() -> dict[str, int]

The pinned version of each Lance dataset, keyed by dataset name.

PyArrow inputs aren't listed.

config

Python
config() -> GraphConfig

The engine's graph config.

execute_with_vector_rerank

Python
execute_with_vector_rerank(query: str, vector_search: VectorSearch) -> Table

Run a Cypher query, then rank its rows by vector similarity.

Like CypherQuery.execute_with_vector_rerank, with the registered datasets. See Vector search.

Parameters:

Name Type Description Default
query str

The Cypher query. It must return the column vector_search ranks by.

required
vector_search VectorSearch

The ranking.

required

Returns:

Type Description
Table

At most top_k rows, closest first.

Raises:

Type Description
ValueError

If the query is invalid or the vector search is misconfigured.

RuntimeError

If execution fails.

CypherQuery

Python
CypherQuery(query_text: str)

One parsed Cypher query, run against the datasets passed to each call.

Builder methods return a new query. Datasets are read into memory for each call; use CypherEngine to register them once and run many queries.

Parameters:

Name Type Description Default
query_text str

The Cypher query.

required

Raises:

Type Description
ValueError

If query_text doesn't parse.

with_config

Python
with_config(config: GraphConfig) -> Self

Set the graph config. Required before running the query.

Parameters:

Name Type Description Default
config GraphConfig

The graph mapping.

required

Returns:

Type Description
CypherQuery

A new query with the config set.

with_parameter

Python
with_parameter(key: str, value: Parameter) -> Self

Bind $key to value.

Parameters:

Name Type Description Default
key str

The parameter name, without $.

required
value Parameter

The value, of a type that CypherEngine.execute accepts in params.

required

Returns:

Type Description
CypherQuery

A new query with the parameter bound.

query_text

Python
query_text() -> str

The query text.

parameters

Python
parameters() -> dict[str, Scalar]

The bound parameters.

execute

Python
execute(datasets: dict[str, DatasetT], strategy: ExecutionStrategy | None = None, *, timeout: float | None = None) -> Table

Run the query.

Parameters:

Name Type Description Default
datasets dict[str, Dataset]

The tables, keyed by node label or relationship type (see Datasets). They are read into memory for this call.

required
strategy ExecutionStrategy or None

How to run the query; None means ExecutionStrategy.DataFusion.

None
timeout float or None

Keyword-only. Seconds for the whole query; None means no deadline. See Timeouts.

None

Returns:

Type Description
Table

The query result.

Raises:

Type Description
ValueError

If the query is invalid or a dataset is missing.

NotImplementedError

If the query uses an unsupported Cypher feature, or strategy is ExecutionStrategy.LanceNative.

TimeoutError

If the query runs past timeout.

RuntimeError

If execution fails.

execute_with_namespace

Python
execute_with_namespace(namespace: DirNamespace, strategy: ExecutionStrategy | None = None) -> Table

Run the query, reading each label from <base_uri>/<label>.lance.

Parameters:

Name Type Description Default
namespace DirNamespace

Where the datasets are.

required
strategy ExecutionStrategy or None

How to run the query; None means ExecutionStrategy.DataFusion.

None

Returns:

Type Description
Table

The query result.

Raises:

Type Description
ValueError

If the query is invalid.

RuntimeError

If a dataset can't be read or execution fails.

explain

Python
explain(datasets: dict[str, DatasetT]) -> str

The plans for the query, without running it.

Parameters:

Name Type Description Default
datasets dict[str, Dataset]

The tables, keyed by node label or relationship type.

required

Returns:

Type Description
str

The graph logical plan, the DataFusion logical plan and the physical plan.

Raises:

Type Description
ValueError

If the query is invalid or a dataset is missing.

RuntimeError

If planning fails.

to_sql

Python
to_sql(datasets: dict[str, DatasetT], dialect: SqlDialect | None = None) -> str

Export the query as SQL joins.

Only supported expressions are exported. See From Cypher to SQL.

Parameters:

Name Type Description Default
datasets dict[str, Dataset]

The tables, keyed by node label or relationship type; their schemas shape the SQL.

required
dialect SqlDialect or None

The SQL dialect; None means SqlDialect.Default.

None

Returns:

Type Description
str

The SQL query.

Raises:

Type Description
NotImplementedError

If a runtime-only Cypher function has no faithful SQL translation.

RuntimeError

If SQL generation fails.

execute_with_vector_rerank

Python
execute_with_vector_rerank(datasets: dict[str, DatasetT], vector_search: VectorSearch) -> Table

Run the query, then rank its rows by vector similarity.

This is the common GraphRAG pattern: find candidates by graph traversal, then rerank them by similarity. With VectorSearch.use_lance_index(True) and Lance datasets, the Lance index is searched first instead. See Vector search.

Parameters:

Name Type Description Default
datasets dict[str, Dataset]

The tables, keyed by node label or relationship type.

required
vector_search VectorSearch

The ranking. Its column is a returned column, such as "d.embedding".

required

Returns:

Type Description
Table

At most top_k rows, closest first.

Raises:

Type Description
ValueError

If the query is invalid or the vector search is misconfigured.

RuntimeError

If execution fails.

variables

Python
variables() -> list[str]

The variables the query uses, including UNION branches, unordered.

node_labels

Python
node_labels() -> list[str]

The node labels the query uses, including UNION branches, unordered.

relationship_types

Python
relationship_types() -> list[str]

The relationship types the query uses, with UNION branches, unordered.

SqlEngine

Python
SqlEngine(datasets: dict[str, DatasetT])

Run many SQL queries against the same datasets.

The datasets are registered once, under lower-cased names, in a DataFusion session that every query reuses.

Parameters:

Name Type Description Default
datasets dict[str, Dataset]

The tables, keyed by the names queries use.

required

Raises:

Type Description
ValueError

If datasets is empty.

Examples:

Python Console Session
>>> import pyarrow as pa
>>> from lance_csr import SqlEngine
>>> engine = SqlEngine({
...     "person": pa.table({"id": [1, 2], "name": ["Alice", "Bob"]}),
...     "knows": pa.table({"src": [1], "dst": [2]}),
... })
>>> engine.execute(
...     "SELECT p.name FROM person p JOIN knows k ON p.id = k.src"
... ).to_pylist()
[{'name': 'Alice'}]

execute

Python
execute(sql: str) -> Table

Run a SQL query against the registered datasets.

Parameters:

Name Type Description Default
sql str

The SQL query.

required

Returns:

Type Description
Table

The query result.

Raises:

Type Description
RuntimeError

If the query fails.

SqlQuery

Python
SqlQuery(sql: str)

One SQL query, run against the datasets passed to each call.

No GraphConfig is needed: you write the joins yourself. Table names in the query match the dataset keys case-insensitively. See SQL queries.

Parameters:

Name Type Description Default
sql str

The SQL query.

required

Examples:

Python Console Session
>>> import pyarrow as pa
>>> from lance_csr import SqlQuery
>>> person = pa.table({"name": ["Alice", "Bob"], "age": [28, 34]})
>>> query = SqlQuery("SELECT name FROM person WHERE age > 30")
>>> query.execute({"person": person}).to_pylist()
[{'name': 'Bob'}]

sql

Python
sql() -> str

The query text.

execute

Python
execute(datasets: dict[str, DatasetT]) -> Table

Register datasets and run the query.

Parameters:

Name Type Description Default
datasets dict[str, Dataset]

The tables, keyed by the names the query uses. Lance datasets are read into memory for this call.

required

Returns:

Type Description
Table

The query result.

Raises:

Type Description
RuntimeError

If the query fails.

explain

Python
explain(datasets: dict[str, DatasetT]) -> str

DataFusion's logical and physical plans for the query.

Parameters:

Name Type Description Default
datasets dict[str, Dataset]

The tables, keyed by the names the query uses.

required

Returns:

Type Description
str

The plans.

Raises:

Type Description
RuntimeError

If planning fails.

VectorSearch

Python
VectorSearch(column: str)

An immutable builder for vector similarity search.

Every method except search returns a new builder. Pass the builder to search to rank an Arrow table, or to execute_with_vector_rerank on CypherEngine or CypherQuery to rerank a Cypher result. This is separate from the vector_distance() and vector_similarity() Cypher functions. See Vector search.

Parameters:

Name Type Description Default
column str

The vector column. When reranking a Cypher result, use the returned column name, alias.property (for example "d.embedding").

required

Examples:

Python Console Session
>>> import pyarrow as pa
>>> from lance_csr import DistanceMetric, VectorSearch
>>> table = pa.table({"id": [1, 2], "embedding": [[1.0, 0.0], [0.0, 1.0]]})
>>> search = (
...     VectorSearch("embedding")
...     .query_vector([1.0, 0.1])
...     .metric(DistanceMetric.Cosine)
...     .top_k(1)
... )
>>> search.search(table)["id"].to_pylist()
[1]

query_vector

Python
query_vector(vector: Sequence[float]) -> Self

Set the query vector. Required before searching.

Parameters:

Name Type Description Default
vector Sequence[float]

The vector to compare each row with.

required

Returns:

Type Description
VectorSearch

A new builder with the query vector set.

metric

Python
metric(metric: DistanceMetric) -> Self

Set the distance metric. Default DistanceMetric.L2.

Parameters:

Name Type Description Default
metric DistanceMetric

The distance to rank by.

required

Returns:

Type Description
VectorSearch

A new builder with the metric set.

top_k

Python
top_k(k: int) -> Self

Set how many rows to return. Default 10.

Parameters:

Name Type Description Default
k int

The number of closest rows to keep.

required

Returns:

Type Description
VectorSearch

A new builder with top_k set.

include_distance

Python
include_distance(include: bool) -> Self

Set whether results get a distance column. Default True.

Parameters:

Name Type Description Default
include bool

Add the distance column when True.

required

Returns:

Type Description
VectorSearch

A new builder with the setting applied.

distance_column_name

Python
distance_column_name(name: str) -> Self

Set the name of the distance column. Default "_distance".

Parameters:

Name Type Description Default
name str

The column name.

required

Returns:

Type Description
VectorSearch

A new builder with the column name set.

use_lance_index

Python
use_lance_index(enabled: bool) -> Self

Set whether reranking queries a Lance ANN index first. Default False.

When the datasets are Lance datasets, reranking searches the Lance approximate-nearest-neighbour index first and then runs the Cypher query on the top-k rows. This can be much faster on large datasets, but it can change the results when the query also filters, because the filter then applies to the top-k rows only.

Parameters:

Name Type Description Default
enabled bool

Use the Lance index when possible.

required

Returns:

Type Description
VectorSearch

A new builder with the setting applied.

search

Python
search(table: Table | RecordBatch | RecordBatchReader | ArrowStream) -> Table

Brute-force search table.

Parameters:

Name Type Description Default
table (Table, RecordBatch, RecordBatchReader or ArrowStream)

The rows to search, with the vector column.

required

Returns:

Type Description
Table

At most top_k rows, sorted by distance, closest first.

Raises:

Type Description
ValueError

If no query vector is set or the column doesn't exist.

RuntimeError

If the search fails.

DistanceMetric

Bases: IntEnum

The distance VectorSearch ranks rows by; smaller is closer.

Attributes:

Name Type Description
L2

Euclidean distance.

Cosine

1 - cosine similarity.

Dot

Negative dot product.

DirNamespace

Python
DirNamespace(base_uri: str)

Resolves each table name to the Lance dataset <base_uri>/<name>.lance.

Use it with CypherQuery.execute_with_namespace.

Parameters:

Name Type Description Default
base_uri str

The directory that holds the datasets.

required

base_uri

Python
base_uri: str

The directory that holds the datasets.

ExecutionStrategy

Bases: IntEnum

How CypherQuery.execute runs a query.

Attributes:

Name Type Description
DataFusion

Plan and run the query with DataFusion: the default and only implemented one.

LanceNative

Reserved; raises NotImplementedError.

SqlDialect

Bases: IntEnum

The SQL dialect CypherQuery.to_sql writes.

Attributes:

Name Type Description
Default

Generic DataFusion SQL.

Spark

Spark SQL.

PostgreSql

PostgreSQL.

MySql

MySQL.

Sqlite

SQLite.