Metadata Filtering

CyborgDB supports metadata tagging and filtering, enabling fine-grained search filters in addition to encrypted vector search. Metadata, like vectors, are end-to-end encrypted and never exposed in plaintext outside of the client.

Supported Metadata

The following metadata types are supported:

• Boolean
• String
• List of strings
• Numbers (will be stored as fp64)

Adding Items with Metadata

To add items with metadata tags, you can pass a dictionary of key-value pairs to the metadata field during upsert:

Python

# Example data in the format:
# [{id: int, vector: List[float], metadata: Dict)]
data = [
    {"id": "item_1", "vector": [0.1, 0.1, 0.1, 0.1], "metadata": {"category": "dog"}},
    {"id": "item_2", "vector": [0.2, 0.2, 0.2, 0.2], "metadata": {"category": "cat"}}
]

# Upsert data with metadata
index.upsert(data)

This metadata will be encrypted and stored in the index.

Metadata field names and string values are case-sensitive. Ensure that the metadata field names and values are consistent when filtering queries.

Filtering Queries with Metadata

CyborgDB supports a subset of MongoDB’s Query and Projection Operators. For more details, see Metadata Query Operators

To filter a query with metadata, you can pass a dictionary of filters to the filters field during query:

Python

# Example query
query_vectors = [0.5, 0.9, 0.2, 0.7]
top_k = 10

# Example filters
filters = {
    "category": {"$in": ["dog", "cat"]} # Will match either 'dog' or 'cat'
}

# Perform query
results = index.query(query_vectors=query_vectors, top_k=top_k, filters=filters)

Metadata Indexing

All metadata fields are indexed using encrypted indexing. This allows CyborgDB to securely exclude clusters which don’t match the provided metadata filters. As a result, searches with metadata filter should be as fast or faster than those without.

Metadata Query Operators

CyborgDB supports a subset of MongoDB’s Query and Projection Operators. Specifically, the following operators are supported:

Filter	Types	Description
$exists	any	Matches vectors that have this field
$eq	Boolean, Number, String	Matches vectors with the metadata that is equal to the filter value
$ne	Boolean, Number, String	Matches vectors with the metadata that is not equal to the filter value
$in	Number, String	Matches vectors with metadata that is in the filter array
$nin	Number, String	Matches vectors with metadata that is not in the filter array
$gt	Number	Matches vectors with metadata that is greater than the filter value
$gte	Number	Matches vectors with metadata that is greater than or equal to the filter value
$lt	Number	Matches vectors with metadata that is less than the filter value
$lte	Number	Matches vectors with metadata that is less than or equal to the filter value