Python API reference¶
Everything below is importable from lance_csr. The classes are thin Python wrappers around the Rust engine,
and this page is generated from their docstrings and type hints, so it matches the installed package. The
guides link here for exact signatures and link back for examples.
| Class | Purpose |
|---|---|
GraphConfig, GraphConfigBuilder |
Map tables to node labels and relationship types |
CypherEngine |
Run many Cypher queries over registered datasets |
CypherQuery |
Run one Cypher query |
SqlEngine, SqlQuery |
Run SQL through DataFusion |
VectorSearch, DistanceMetric |
Vector similarity search and reranking |
DirNamespace |
Resolve table names to Lance datasets in a directory |
ExecutionStrategy, SqlDialect |
Enums |
lance_csr.__version__ is the package version.
This page covers the Python API. Rust users should read Using from Rust and the Rust API reference on docs.rs.
Type checking¶
Every public class is type-annotated, and the package ships a py.typed marker. Editors and type checkers
can follow builder methods, parameters, query results and statistics. The annotations describe the API;
they don't change runtime validation or exceptions.
PyArrow distributes its type information separately. For strict checks of Arrow inputs and results, add
pyarrow-stubs to your development dependencies:
from typing import assert_type
import pyarrow as pa
from lance_csr import CypherEngine, GraphConfig
config = GraphConfig.builder().node_table("Person", "id").build()
engine = CypherEngine(config, {"Person": pa.table({"id": [1, 2]})})
result = engine.execute("MATCH (p:Person) RETURN p.id ORDER BY p.id")
assert_type(result, pa.Table)
print(result.to_pylist()) # [{'p.id': 1}, {'p.id': 2}]
Datasets¶
Methods that take datasets take a dict[str, object] mapping each node label or relationship type (or SQL
table name) to one of:
lance.LanceDataset: read lazily and version-pinned byCypherEngine; read into memory by the other classes;pyarrow.Table,pyarrow.RecordBatch,pyarrow.RecordBatchReader, or any object exporting the Arrow C stream interface (__arrow_c_stream__).
Keys match labels case-insensitively.
Exceptions¶
| Exception | When |
|---|---|
ValueError |
Invalid query, incompatible known operand types (with expression and type details), unknown label, missing dataset or column, unsupported parameter type, empty datasets |
NotImplementedError |
A Cypher feature that parses but isn't supported |
RuntimeError |
Execution or I/O failures |
TypeError |
A dataset value of an unsupported type, such as a LanceDB Table, or an argument of the wrong class, such as a string where a GraphConfig is expected |
Timeouts¶
CypherEngine.execute, CypherEngine.execute_batches and CypherQuery.execute accept keyword-only
timeout in seconds (None means no deadline). Zero expires immediately; negative, nonfinite or
unrepresentable durations raise ValueError. One deadline includes planning, every batch and pauses
between reads. Before returning a reader, expiry raises TimeoutError. Reading an expired native Arrow
reader raises pyarrow.ArrowInvalid containing Query timeout exceeded. Earlier batches remain valid.
Expiry or reader close releases the stream even while the caller is idle; the engine remains usable.
Cancellation is cooperative. Traversal loops yield at bounded work intervals, and timed cross joins yield between output batches. Timed nested-loop joins check the deadline while evaluating candidates, including batches that produce no matches. Synchronous parsing, Python data providers and foreign blocking calls cannot be forcibly preempted. Deadlines are checked between planning stages and before accepting completed results; this is not a hard real-time limit.
Path values¶
Cypher named paths can be returned as Arrow structs containing length, nodes, node_records and
relationships. They retain their values through collect() and UNWIND; nodes(path) returns integer
keys, and relationships(path) returns typed relationship records. startNode(r) and endNode(r) return
the stored endpoint node records. Their occurrence identities are query-local. See
path values and their memory cost.
GraphConfig
¶
An immutable mapping from tables to a property graph.
Create one with GraphConfig.builder(); there is
no public constructor. See Graph schema.
builder
¶
builder() -> GraphConfigBuilder
Start a new, empty builder.
Returns:
| Type | Description |
|---|---|
GraphConfigBuilder
|
The same as |
node_labels
¶
The configured node labels, lower-cased, in no particular order.
relationship_types
¶
The configured relationship types, lower-cased, in no particular order.
GraphConfigBuilder
¶
Builds a GraphConfig from node and relationship
tables.
GraphConfigBuilder() is the same as GraphConfig.builder(). Every method returns
a new builder and leaves this one unchanged.
node_table
¶
Add a node table.
The nodes labelled label are the rows of the dataset named label, each
identified by its primary_key column.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
label
|
str
|
The node label, and the name of its dataset (for example |
required |
primary_key
|
str
|
The column that identifies each node. |
required |
Returns:
| Type | Description |
|---|---|
GraphConfigBuilder
|
A new builder with the node table added. |
rel_table
¶
Add a relationship table.
The edges of type rel_type are the rows of the dataset named rel_type, from
from_table nodes to to_table nodes. Both node tables must be added by the
time build runs.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
rel_type
|
str
|
The relationship type, and the name of its dataset (for example |
required |
from_table
|
str
|
The node table at the source end. |
required |
to_table
|
str
|
The node table at the target end. |
required |
src
|
str
|
The column holding the source node's primary key. |
required |
dst
|
str
|
The column holding the target node's primary key. |
required |
Returns:
| Type | Description |
|---|---|
GraphConfigBuilder
|
A new builder with the relationship table added. |
build
¶
build() -> GraphConfig
Validate the tables and return the config.
Returns:
| Type | Description |
|---|---|
GraphConfig
|
The configured graph. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the configuration is invalid, for example when a relationship's endpoint label has no node table. |
with_node_label
¶
Deprecated: use node_table.
Emits DeprecationWarning.
with_relationship
¶
with_relationship(rel_type: str, source_field: str, target_field: str, source_label: str | None = None, target_label: str | None = None) -> Self
Deprecated: use rel_table.
rel_table also takes the node tables at both ends. Emits DeprecationWarning.
Raises:
| Type | Description |
|---|---|
ValueError
|
If only one of |
CypherEngine
¶
CypherEngine(config: GraphConfig, datasets: dict[str, DatasetT], use_csr: bool = False, *, target_partitions: int | None = None)
Run many Cypher queries over the same datasets.
The engine registers datasets once and reuses the catalog, the DataFusion session
and (with use_csr=True) the CSR index across queries. It is safe to share between
threads.
Lance datasets are read lazily, pinned to the version passed in: later writes, by
this or another process, aren't visible until
refresh. Each dataset is pinned separately, so
versions across tables are not a transactional snapshot. PyArrow inputs are held in
memory and never change; create a new engine to replace them.
execute and
execute_batches reuse up to 128
unoptimized query templates per snapshot, keyed by the exact query text and the
names and types of scalar parameters. Values are bound before optimization on every
call. Container parameters, rich path values, CALL subqueries, parameterized
SKIP/LIMIT and unsupported template shapes use ordinary planning.
explain always plans directly, and a successful
refresh clears the cache. See Repeated queries.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
config
|
GraphConfig
|
The graph mapping. |
required |
datasets
|
dict[str, Dataset]
|
The tables, keyed by node label or relationship type (see Datasets). Lance datasets are pinned to the version passed in and not read here. |
required |
use_csr
|
bool
|
Build the experimental CSR index for relationship types with integer endpoints, and node indexes for labels with unique integer keys. |
False
|
target_partitions
|
int or None
|
Keyword-only. A target from 1 to 1024 for DataFusion query partitioning; |
None
|
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
TypeError
|
If |
RuntimeError
|
If a dataset can't be opened. |
Examples:
>>> import pyarrow as pa
>>> from lance_csr import CypherEngine, GraphConfig
>>> config = (
... GraphConfig.builder()
... .node_table("Person", "id")
... .rel_table("KNOWS", "Person", "Person", "src_id", "dst_id")
... .build()
... )
>>> engine = CypherEngine(config, {
... "Person": pa.table({"id": [1, 2], "name": ["Alice", "Bob"]}),
... "KNOWS": pa.table({"src_id": [1], "dst_id": [2]}),
... })
>>> engine.execute(
... "MATCH (a:Person)-[:KNOWS]->(b:Person) RETURN a.name, b.name"
... ).to_pylist()
[{'a.name': 'Alice', 'b.name': 'Bob'}]
execute
¶
execute(query: str, params: dict[str, ParameterT] | None = None, *, timeout: float | None = None) -> Table
Run a Cypher query and return the whole result.
The result is held in memory; use
execute_batches for large outputs.
Direct counts from size() of an unmapped list comprehension have a
non-nullable Arrow field; see Cypher functions.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
query
|
str
|
The Cypher query. |
required |
params
|
dict[str, Parameter] or None
|
Values for |
None
|
timeout
|
float or None
|
Keyword-only. Seconds for the whole query; |
None
|
Returns:
| Type | Description |
|---|---|
Table
|
The query result. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the query is invalid, refers to an unknown label, compares or combines incompatible types, or a parameter is unsupported, nested beyond 128 levels or cyclic. |
NotImplementedError
|
If the query uses a Cypher feature that parses but isn't supported. |
TimeoutError
|
If the query runs past |
RuntimeError
|
If execution fails. |
execute_batches
¶
execute_batches(query: str, params: dict[str, ParameterT] | None = None, *, timeout: float | None = None) -> RecordBatchReader
Run a Cypher query and stream the result.
Batches are computed as the reader is consumed. Scans, filters and projections
stream with bounded memory; joins, sorts and aggregations may still buffer. The
reader keeps its snapshot alive, so a concurrent
refresh doesn't affect it. Planning errors
raise immediately; execution errors raise while reading.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
query
|
str
|
The Cypher query. |
required |
params
|
dict[str, Parameter] or None
|
Values for |
None
|
timeout
|
float or None
|
Keyword-only. Seconds for the whole query, including pauses between reads;
|
None
|
Returns:
| Type | Description |
|---|---|
RecordBatchReader
|
The result, one batch at a time. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the query is invalid. |
TimeoutError
|
If the deadline passes before the reader is returned. Reading after it
passes raises |
RuntimeError
|
If planning fails. |
explain
¶
The plans for a query, without running it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
query
|
str
|
The Cypher query. |
required |
params
|
dict[str, Parameter] or None
|
Values for |
None
|
Returns:
| Type | Description |
|---|---|
str
|
The graph logical plan, the DataFusion logical plan and the physical plan, including pushed-down scans. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the query is invalid. |
refresh
¶
Move every Lance dataset to its latest version.
Updates the CSR, node indexes and statistics to match. Datasets whose version
and commit time are unchanged keep their open handle. The replacement is
validated in full before it's swapped in: on error nothing changes and the
previous snapshot stays usable. Queries already running finish on the snapshot
they started with. PyArrow inputs are unchanged. Concurrent calls run one at a
time, and target_partitions is preserved. See
Lance datasets.
node_index_stats
¶
Statistics for each node index (experimental; needs use_csr=True).
A label is missing when its keys aren't unique integers; relationships that end at it then use a join.
Returns:
| Type | Description |
|---|---|
dict[str, NodeIndexStats]
|
Per node label: |
csr_stats
¶
Statistics for each CSR adjacency index (experimental; needs use_csr=True).
An identical result after refresh means the
snapshot itself didn't change. See
Inspecting the CSR.
Returns:
| Type | Description |
|---|---|
dict[str, CsrStats]
|
Per relationship type with a CSR, empty without
|
dataset_versions
¶
The pinned version of each Lance dataset, keyed by dataset name.
PyArrow inputs aren't listed.
execute_with_vector_rerank
¶
execute_with_vector_rerank(query: str, vector_search: VectorSearch) -> Table
Run a Cypher query, then rank its rows by vector similarity.
Like CypherQuery.execute_with_vector_rerank, with the registered datasets.
See Vector search.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
query
|
str
|
The Cypher query. It must return the column |
required |
vector_search
|
VectorSearch
|
The ranking. |
required |
Returns:
| Type | Description |
|---|---|
Table
|
At most |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the query is invalid or the vector search is misconfigured. |
RuntimeError
|
If execution fails. |
CypherQuery
¶
One parsed Cypher query, run against the datasets passed to each call.
Builder methods return a new query. Datasets are read into memory for each call; use
CypherEngine to register them once and run many queries.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
query_text
|
str
|
The Cypher query. |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
with_config
¶
with_config(config: GraphConfig) -> Self
Set the graph config. Required before running the query.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
config
|
GraphConfig
|
The graph mapping. |
required |
Returns:
| Type | Description |
|---|---|
CypherQuery
|
A new query with the config set. |
with_parameter
¶
Bind $key to value.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
str
|
The parameter name, without |
required |
value
|
Parameter
|
The value, of a type that
|
required |
Returns:
| Type | Description |
|---|---|
CypherQuery
|
A new query with the parameter bound. |
execute
¶
execute(datasets: dict[str, DatasetT], strategy: ExecutionStrategy | None = None, *, timeout: float | None = None) -> Table
Run the query.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
datasets
|
dict[str, Dataset]
|
The tables, keyed by node label or relationship type (see Datasets). They are read into memory for this call. |
required |
strategy
|
ExecutionStrategy or None
|
How to run the query; |
None
|
timeout
|
float or None
|
Keyword-only. Seconds for the whole query; |
None
|
Returns:
| Type | Description |
|---|---|
Table
|
The query result. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the query is invalid or a dataset is missing. |
NotImplementedError
|
If the query uses an unsupported Cypher feature, or |
TimeoutError
|
If the query runs past |
RuntimeError
|
If execution fails. |
execute_with_namespace
¶
execute_with_namespace(namespace: DirNamespace, strategy: ExecutionStrategy | None = None) -> Table
Run the query, reading each label from <base_uri>/<label>.lance.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
namespace
|
DirNamespace
|
Where the datasets are. |
required |
strategy
|
ExecutionStrategy or None
|
How to run the query; |
None
|
Returns:
| Type | Description |
|---|---|
Table
|
The query result. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the query is invalid. |
RuntimeError
|
If a dataset can't be read or execution fails. |
explain
¶
The plans for the query, without running it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
datasets
|
dict[str, Dataset]
|
The tables, keyed by node label or relationship type. |
required |
Returns:
| Type | Description |
|---|---|
str
|
The graph logical plan, the DataFusion logical plan and the physical plan. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the query is invalid or a dataset is missing. |
RuntimeError
|
If planning fails. |
to_sql
¶
to_sql(datasets: dict[str, DatasetT], dialect: SqlDialect | None = None) -> str
Export the query as SQL joins.
Only supported expressions are exported. See From Cypher to SQL.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
datasets
|
dict[str, Dataset]
|
The tables, keyed by node label or relationship type; their schemas shape the SQL. |
required |
dialect
|
SqlDialect or None
|
The SQL dialect; |
None
|
Returns:
| Type | Description |
|---|---|
str
|
The SQL query. |
Raises:
| Type | Description |
|---|---|
NotImplementedError
|
If a runtime-only Cypher function has no faithful SQL translation. |
RuntimeError
|
If SQL generation fails. |
execute_with_vector_rerank
¶
execute_with_vector_rerank(datasets: dict[str, DatasetT], vector_search: VectorSearch) -> Table
Run the query, then rank its rows by vector similarity.
This is the common GraphRAG pattern: find candidates by graph traversal, then
rerank them by similarity. With
VectorSearch.use_lance_index(True)
and Lance datasets, the Lance index is searched first instead. See
Vector search.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
datasets
|
dict[str, Dataset]
|
The tables, keyed by node label or relationship type. |
required |
vector_search
|
VectorSearch
|
The ranking. Its column is a returned column, such as |
required |
Returns:
| Type | Description |
|---|---|
Table
|
At most |
Raises:
| Type | Description |
|---|---|
ValueError
|
If the query is invalid or the vector search is misconfigured. |
RuntimeError
|
If execution fails. |
variables
¶
The variables the query uses, including UNION branches, unordered.
node_labels
¶
The node labels the query uses, including UNION branches, unordered.
relationship_types
¶
The relationship types the query uses, with UNION branches, unordered.
SqlEngine
¶
Run many SQL queries against the same datasets.
The datasets are registered once, under lower-cased names, in a DataFusion session that every query reuses.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
datasets
|
dict[str, Dataset]
|
The tables, keyed by the names queries use. |
required |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Examples:
>>> import pyarrow as pa
>>> from lance_csr import SqlEngine
>>> engine = SqlEngine({
... "person": pa.table({"id": [1, 2], "name": ["Alice", "Bob"]}),
... "knows": pa.table({"src": [1], "dst": [2]}),
... })
>>> engine.execute(
... "SELECT p.name FROM person p JOIN knows k ON p.id = k.src"
... ).to_pylist()
[{'name': 'Alice'}]
execute
¶
Run a SQL query against the registered datasets.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
sql
|
str
|
The SQL query. |
required |
Returns:
| Type | Description |
|---|---|
Table
|
The query result. |
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If the query fails. |
SqlQuery
¶
One SQL query, run against the datasets passed to each call.
No GraphConfig is needed: you write the joins yourself.
Table names in the query match the dataset keys case-insensitively. See
SQL queries.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
sql
|
str
|
The SQL query. |
required |
Examples:
>>> import pyarrow as pa
>>> from lance_csr import SqlQuery
>>> person = pa.table({"name": ["Alice", "Bob"], "age": [28, 34]})
>>> query = SqlQuery("SELECT name FROM person WHERE age > 30")
>>> query.execute({"person": person}).to_pylist()
[{'name': 'Bob'}]
execute
¶
Register datasets and run the query.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
datasets
|
dict[str, Dataset]
|
The tables, keyed by the names the query uses. Lance datasets are read into memory for this call. |
required |
Returns:
| Type | Description |
|---|---|
Table
|
The query result. |
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If the query fails. |
explain
¶
DataFusion's logical and physical plans for the query.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
datasets
|
dict[str, Dataset]
|
The tables, keyed by the names the query uses. |
required |
Returns:
| Type | Description |
|---|---|
str
|
The plans. |
Raises:
| Type | Description |
|---|---|
RuntimeError
|
If planning fails. |
VectorSearch
¶
An immutable builder for vector similarity search.
Every method except search returns a new builder.
Pass the builder to search to rank an Arrow table, or to
execute_with_vector_rerank on CypherEngine or
CypherQuery to rerank a Cypher result. This is separate
from the vector_distance() and vector_similarity() Cypher functions. See
Vector search.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
column
|
str
|
The vector column. When reranking a Cypher result, use the returned column name,
|
required |
Examples:
>>> import pyarrow as pa
>>> from lance_csr import DistanceMetric, VectorSearch
>>> table = pa.table({"id": [1, 2], "embedding": [[1.0, 0.0], [0.0, 1.0]]})
>>> search = (
... VectorSearch("embedding")
... .query_vector([1.0, 0.1])
... .metric(DistanceMetric.Cosine)
... .top_k(1)
... )
>>> search.search(table)["id"].to_pylist()
[1]
query_vector
¶
Set the query vector. Required before searching.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
vector
|
Sequence[float]
|
The vector to compare each row with. |
required |
Returns:
| Type | Description |
|---|---|
VectorSearch
|
A new builder with the query vector set. |
metric
¶
metric(metric: DistanceMetric) -> Self
Set the distance metric. Default DistanceMetric.L2.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
metric
|
DistanceMetric
|
The distance to rank by. |
required |
Returns:
| Type | Description |
|---|---|
VectorSearch
|
A new builder with the metric set. |
top_k
¶
Set how many rows to return. Default 10.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
k
|
int
|
The number of closest rows to keep. |
required |
Returns:
| Type | Description |
|---|---|
VectorSearch
|
A new builder with |
include_distance
¶
Set whether results get a distance column. Default True.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
include
|
bool
|
Add the distance column when |
required |
Returns:
| Type | Description |
|---|---|
VectorSearch
|
A new builder with the setting applied. |
distance_column_name
¶
Set the name of the distance column. Default "_distance".
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
The column name. |
required |
Returns:
| Type | Description |
|---|---|
VectorSearch
|
A new builder with the column name set. |
use_lance_index
¶
Set whether reranking queries a Lance ANN index first. Default False.
When the datasets are Lance datasets, reranking searches the Lance approximate-nearest-neighbour index first and then runs the Cypher query on the top-k rows. This can be much faster on large datasets, but it can change the results when the query also filters, because the filter then applies to the top-k rows only.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
enabled
|
bool
|
Use the Lance index when possible. |
required |
Returns:
| Type | Description |
|---|---|
VectorSearch
|
A new builder with the setting applied. |
search
¶
Brute-force search table.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
table
|
(Table, RecordBatch, RecordBatchReader or ArrowStream)
|
The rows to search, with the vector column. |
required |
Returns:
| Type | Description |
|---|---|
Table
|
At most |
Raises:
| Type | Description |
|---|---|
ValueError
|
If no query vector is set or the column doesn't exist. |
RuntimeError
|
If the search fails. |
DistanceMetric
¶
Bases: IntEnum
The distance VectorSearch ranks rows by; smaller is closer.
Attributes:
| Name | Type | Description |
|---|---|---|
L2 |
Euclidean distance. |
|
Cosine |
|
|
Dot |
Negative dot product. |
DirNamespace
¶
Resolves each table name to the Lance dataset <base_uri>/<name>.lance.
Use it with CypherQuery.execute_with_namespace.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
base_uri
|
str
|
The directory that holds the datasets. |
required |
ExecutionStrategy
¶
Bases: IntEnum
How CypherQuery.execute runs a query.
Attributes:
| Name | Type | Description |
|---|---|---|
DataFusion |
Plan and run the query with DataFusion: the default and only implemented one. |
|
LanceNative |
Reserved; raises |
SqlDialect
¶
Bases: IntEnum
The SQL dialect CypherQuery.to_sql writes.
Attributes:
| Name | Type | Description |
|---|---|---|
Default |
Generic DataFusion SQL. |
|
Spark |
Spark SQL. |
|
PostgreSql |
PostgreSQL. |
|
MySql |
MySQL. |
|
Sqlite |
SQLite. |