Skip to content

Add Cassandra 5 SAI index and vector search support - #30

Open
pchlupacek wants to merge 1 commit into
series/0.5from
cassandra5-sai-vector-support
Open

Add Cassandra 5 SAI index and vector search support#30
pchlupacek wants to merge 1 commit into
series/0.5from
cassandra5-sai-vector-support

Conversation

@pchlupacek

Copy link
Copy Markdown
Member

Summary

  • Add SAI (Storage Attached Index) support: indexBySAI, indexBySAIVector, indexBySAICollection on TableBuilder
  • Add ANN vector search: orderByAnn, orderByAnnShared on QueryBuilder with ORDER BY col ANN OF :param
  • Add similarity functions: similarityCosine, similarityEuclidean, similarityDotProduct via new CQLFunction2 trait
  • Add CONTAINS, CONTAINS_KEY comparisons and byIndexIn for SAI query operators
  • Add index migration support: detect missing/removed/changed indexes and generate DROP/CREATE DDL
  • CI updated to test against Cassandra 5.0.7

Changes

New types

  • CollectionIndexTarget (KEYS/VALUES/ENTRIES/FULL) for SAI collection indexing
  • SimilarityFunction (COSINE/DOT_PRODUCT/EUCLIDEAN) enum
  • CQLFunction2[I1, I2, O] trait for two-argument CQL functions

QueryBuilder additions

  • orderByAnn / orderByAnnShared — ANN vector ordering with type-safe parameter binding
  • function2At / function2AtShared — similarity functions in SELECT with shared parameter support
  • byIndexIncol IN :paramList with List[V] parameter type

Migration

  • migrateIndexes compares desired vs current indexes from TableMetadata
  • Generates DROP INDEX for removed/changed indexes, CREATE INDEX for new/changed indexes

Test plan

  • 7 unit tests for SAI index DDL generation (TableBuilderSpec)
  • 10 unit tests for ANN/similarity/IN/CONTAINS CQL generation (QueryBuilderSpec)
  • 8 integration tests against Cassandra 5.0.7: SAI create, EQ query, range query, ANN OF, ANN with partition key, cosine/euclidean/dot_product similarity (SAISpec)
  • 3 integration tests for index migration: add missing, drop removed, recreate on option change (MigrationsSpec)
  • All 148 tests pass on Cassandra 5.0.7, all existing tests remain green

SAI indexes (indexBySAI, indexBySAIVector, indexBySAICollection),
ANN queries (orderByAnn), similarity functions (cosine, euclidean,
dot_product), CONTAINS/CONTAINS_KEY comparisons, IN operator,
and index migration support (add/drop/recreate on option change).
CI updated to test against Cassandra 5.0.7.

@AdamChlupacek AdamChlupacek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants