feat(indexes): add algorithm and probe_fraction support

CHANGELOG.md

@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Changed
 
+- feat(indexes): add algorithm and probe_fraction support
 - chore: add 403 forbidden response to endpoints
 - feat(jobs): add database_fork job type
 

docs/CreateIndexRequest.md

@@ -6,6 +6,7 @@ Request body for POST .../indexes  One constraint spans the whole table rather t
 
 Name | Type | Description | Notes
 ------------ | ------------- | ------------- | -------------
+**algorithm** | **str** | How a vector index organises the vectors it searches. Omit this field for `hnsw`, which is the default.  `hnsw` — builds a graph of the vectors and keeps it in memory. Searches are very fast, and the memory a search needs grows with the whole table, so a large enough table cannot be served at all.  `ivf` — groups the vectors into clusters and reads only the clusters nearest the search. Searches are considerably slower than `hnsw`, and the memory a search needs follows how much of the index it reads rather than the size of the table, so a table far too large for `hnsw` can still be searched. It keeps a copy of the table's rows beside the vectors so a search is answered without reading the table; that copy is extra storage, and how much depends on `vector_precision`, which decides how compactly the copied vectors are held. Available for columns that already hold vectors, with the `l2` and `cosine` metrics. | [optional] 
 **var_async** | **bool** | When true, create the index as a background job and return a job ID for polling. | [optional] [default to False]
 **async_after_ms** | **int** | If set (requires `async` = true), wait up to this many milliseconds for the index build to finish: if it completes in time the index is returned (201), otherwise a 202 with a job ID to poll. Must be between 1000 and the server maximum; a value out of that range, or set without `async` = true, is rejected with 400. | [optional] 
 **columns** | **List[str]** | Columns to index. Required for all index types. | 
@@ -15,8 +16,10 @@ Name | Type | Description | Notes
 **index_name** | **str** |  | 
 **index_type** | **str** | Index type. `sorted` supports range queries, `bm25` full-text search, and `vector` similarity search. | [optional] [default to 'sorted']
 **metric** | **str** | Distance metric for vector indexes: \"l2\", \"cosine\", or \"dot\". When omitted, defaults to \"l2\" for float array columns or the provider's preferred metric for text columns with auto-embedding. | [optional] 
+**nlist** | **int** | Number of clusters an `ivf` index divides the vectors into. More clusters means each one holds fewer vectors, so a search of the same effort reads less data. Omit this to let the number be chosen from the table's size. | [optional] 
 **output_column** | **str** | Custom name for the generated embedding column. Defaults to `{column}_embedding`. | [optional] 
-**vector_precision** | **str** | How precisely a vector index stores each number of a vector. Lower precision shrinks the index so a larger table can be indexed within the same memory, and lets searches run on a smaller instance. Omit this field to store vectors at the same precision as the column, which is the default.  The quality figures below come from one benchmark — 1536-dimension text embeddings, cosine distance, default search settings — and are a guide, not a guarantee. Other models, dimensions, distance metrics and data distributions behave differently, so measure on your own data before moving a production index to a lower precision.  `float32` — on a `float64` column this halves the index. Widely used embedding models emit 32-bit values, so for those nothing is lost; vectors that genuinely carry more than 32 bits of precision will lose some.  `float16` — half the memory of `float32`. In that benchmark its results matched `float32` to within 0.1 percentage points.  `float8` — a quarter of the memory of `float32`. In that benchmark it scored about 4 percentage points below `float32`, and raising the search effort did not close the gap, so treat the reduction as permanent for a given index.  `float64` — accepted only for a column that already holds double-precision values; it cannot add precision the stored data does not have.  Changing this means dropping the index and creating it again. It affects only the index: the table's own values are never altered, and text columns indexed with a generated embedding are not re-embedded. | [optional] 
+**probe_fraction** | **float** | How much of an `ivf` index a search reads, as a fraction greater than 0 and at most 1. Higher finds more of the true nearest neighbours and takes longer. This is a fraction rather than a number of clusters on purpose: the same number of clusters is a different share of the index whenever `nlist` changes, and results would quietly get worse. Omit this for the server's default. | [optional] 
+**vector_precision** | **str** | How precisely a vector index stores each number of a vector. Lower precision shrinks the index so a larger table can be indexed within the same memory, and lets searches run on a smaller instance. For an `ivf` index it also shrinks what every search reads, because a search reads part of that stored copy.  Omit this field to get each algorithm's own default: an `hnsw` index stores vectors at the same precision as the column, and an `ivf` index stores them as `int8`.  The quality figures below come from one benchmark — 1536-dimension text embeddings, cosine distance, default search settings — and are a guide, not a guarantee. Other models, dimensions, distance metrics and data distributions behave differently, so measure on your own data before moving a production index to a lower precision.  `float32` — on a `float64` column this halves the index. Widely used embedding models emit 32-bit values, so for those nothing is lost; vectors that genuinely carry more than 32 bits of precision will lose some.  `float16` — half the memory of `float32`. In that benchmark its results matched `float32` to within 0.1 percentage points.  `float8` — a quarter of the memory of `float32`. In that benchmark it scored about 4 percentage points below `float32`, and raising the search effort did not close the gap, so treat the reduction as permanent for a given index.  `float64` — accepted only for a column that already holds double-precision values; it cannot add precision the stored data does not have.  `int8` — for an `ivf` index only, and its default. A quarter of the size of `float32`, which is a quarter of the bytes every search reads. On the benchmark this index was designed against it found about 99.5% of the neighbours an exact search finds. With `cosine` that accuracy holds however widely your vectors vary in magnitude; with `l2` it falls as they spread — around 93% of the neighbours once the largest magnitude is about 16 times the smallest, and lower beyond that. Use `float32` instead to store the column as written, at four times the size and four times the bytes per search.  An `ivf` index accepts `int8` and `float32` only: it stores its copy as a table, and the remaining values have no column type to be stored in or are no smaller than `int8`. An `hnsw` index accepts everything except `int8`; `float8` is its 8-bit option.  Changing this means dropping the index and creating it again. It affects only the index: the table's own values are never altered, and text columns indexed with a generated embedding are not re-embedded. | [optional] 
 
 ## Example
 

docs/IndexEntryResponse.md

@@ -6,11 +6,13 @@ One index in a cross-table listing: the index itself plus the connection, schema
 
 Name | Type | Description | Notes
 ------------ | ------------- | ------------- | -------------
+**algorithm** | **str** | How this vector index organises the vectors it searches: `hnsw` or `ivf`. Absent for BM25 and sorted indexes. | [optional] 
 **columns** | **List[str]** |  | 
 **created_at** | **datetime** |  | 
 **index_name** | **str** |  | 
 **index_type** | **str** |  | 
 **metric** | **str** | Distance metric this index was built with. Only present for vector indexes. | [optional] 
+**probe_fraction** | **float** | How much of an `ivf` index a search reads, as a fraction greater than 0 and at most 1, when it was created with an explicit one. Absent means the server's default. Also absent for every other kind of index. | [optional] 
 **source_column** | **str** | Source text column for an embedding-backed vector index. A query searches it via `vector_distance(<source_column>, …)`; the indexed `columns` hold the generated embedding column instead. Absent for BM25, sorted, and direct (existing-column) vector indexes. | [optional] 
 **status** | [**IndexStatus**](IndexStatus.md) |  | 
 **updated_at** | **datetime** |  | 

docs/IndexInfoResponse.md

@@ -6,11 +6,13 @@ Result payload for a `create_index` job, and response for index endpoints.
 
 Name | Type | Description | Notes
 ------------ | ------------- | ------------- | -------------
+**algorithm** | **str** | How this vector index organises the vectors it searches: `hnsw` or `ivf`. Absent for BM25 and sorted indexes. | [optional] 
 **columns** | **List[str]** |  | 
 **created_at** | **datetime** |  | 
 **index_name** | **str** |  | 
 **index_type** | **str** |  | 
 **metric** | **str** | Distance metric this index was built with. Only present for vector indexes. | [optional] 
+**probe_fraction** | **float** | How much of an `ivf` index a search reads, as a fraction greater than 0 and at most 1, when it was created with an explicit one. Absent means the server's default. Also absent for every other kind of index. | [optional] 
 **source_column** | **str** | Source text column for an embedding-backed vector index. A query searches it via `vector_distance(<source_column>, …)`; the indexed `columns` hold the generated embedding column instead. Absent for BM25, sorted, and direct (existing-column) vector indexes. | [optional] 
 **status** | [**IndexStatus**](IndexStatus.md) |  | 
 **updated_at** | **datetime** |  | 

docs/JobResult.md

@@ -6,11 +6,13 @@ What a finished background job produced. Absent while the job is `pending` or `r
 
 Name | Type | Description | Notes
 ------------ | ------------- | ------------- | -------------
+**algorithm** | **str** | How this vector index organises the vectors it searches: `hnsw` or `ivf`. Absent for BM25 and sorted indexes. | [optional] 
 **columns** | **List[str]** |  | 
 **created_at** | **datetime** |  | 
 **index_name** | **str** |  | 
 **index_type** | **str** |  | 
 **metric** | **str** | Distance metric this index was built with. Only present for vector indexes. | [optional] 
+**probe_fraction** | **float** | How much of an `ivf` index a search reads, as a fraction greater than 0 and at most 1, when it was created with an explicit one. Absent means the server's default. Also absent for every other kind of index. | [optional] 
 **source_column** | **str** | Source text column for an embedding-backed vector index. A query searches it via `vector_distance(<source_column>, …)`; the indexed `columns` hold the generated embedding column instead. Absent for BM25, sorted, and direct (existing-column) vector indexes. | [optional] 
 **status** | [**IndexStatus**](IndexStatus.md) |  | 
 **updated_at** | **datetime** |  |