update sdk with new adds

CHANGELOG.md

@@ -5,6 +5,31 @@ All notable changes to the [Nucleus Python Client](https://github.com/scaleapi/n
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.19.0](https://github.com/scaleapi/nucleus-python-client/releases/tag/v0.19.0) - 2026-07-10
+
+### Added
+- **Benchmarks.** Full support for benchmark-paradigm evaluation: `NucleusClient.create_benchmark()` (members from `item_ids`, `(dataset_id, ref_id)` `items` pairs, a `slice_id`, or a `dataset_id`; membership frozen at creation), `list_benchmarks()`, `get_benchmark()`, `update_benchmark()`, `delete_benchmark()`, and `list_benchmark_items()`, plus the new `Benchmark` resource with `refresh()` / `update()` / `delete()` / `items()` / `create_evaluation_v2()`.
+- **Benchmark evaluations.** `create_benchmark_evaluation_v2(benchmark_id, model_run_id, ...)` evaluates a model run against every benchmark item (uncovered items score as false negatives, keeping leaderboard scores comparable). Accepts `rollup_groups`, legacy `allowed_label_matches` / `allowed_label_matches_id`, `exclusion_rules`, and `preset`. `EvaluationV2` responses now expose `benchmark_id` and `rollup_groups`.
+- **Rollup groups.** The new `RollupGroup` type (`class_name` + `labels`) is the primary label configuration: each group evaluates a set of raw labels as one class. Presets support it end to end — `create_evaluation_v2_preset()` / `update_evaluation_v2_preset()` accept `rollup_groups` (mutually exclusive with `allowed_label_matches`), and `EvaluationV2Preset` exposes the field.
+- **Benchmark leaderboards.** `leaderboard_ranking(metric_type, benchmark_ids, ...)` ranks model runs on one or more benchmarks (metrics: `MAP_50`, `MAP_50_95`, `AP_SMALL`, `AP_MEDIUM`, `AP_LARGE`, `PRECISION`, `RECALL`, `F1`; `scope` / `collapse` controls), and `leaderboard_f1_curve(benchmark_ids, ...)` returns F1-vs-confidence curves for the top runs. Requires a scaleapi server with the REST leaderboard endpoints deployed.
+- **Filter schema discovery.** `EvaluationV2.filter_schema()` / `NucleusClient.get_evaluation_v2_filter_schema()` return the evaluation's filter vocabulary (`gt_labels`, `pred_labels`, and item-metadata fields with inferred value types) — the valid inputs for `EvaluationV2FilterArgs`. Requires the same server deployment as the leaderboard endpoints.
+
+## [0.18.9](https://github.com/scaleapi/nucleus-python-client/releases/tag/v0.18.9) - 2026-06-25
+
+### Added
+- **Evaluations V2 slice scoping and exclusion rules.** `create_evaluation_v2()` accepts `slice_id` (restrict the evaluation to a slice's items) and `exclusion_rules` (drop items/annotations before metrics are computed) via the new `MetadataExclusionRule`, `LabelExclusionRule`, and `BoxAreaExclusionRule` types (or equivalent dicts). The `EvaluationV2` resource exposes `slice_id`, `exclusion_rules`, and `exclusion_stats`. `EvaluationV2FilterArgs` gains `gt_area_range` (filter by ground-truth box area, e.g. COCO small/medium/large bands) and `slice_ids`, applied by both `charts()` and `examples()`.
+- **Evaluation V2 presets.** Save and reuse evaluation configurations (`name` + `allowed_label_matches` + `exclusion_rules`) via `NucleusClient.list_evaluation_v2_presets()`, `create_evaluation_v2_preset()`, `update_evaluation_v2_preset()`, and `delete_evaluation_v2_preset()`, plus the new `EvaluationV2Preset` resource (with `update()` / `delete()`). Apply a preset directly when creating an evaluation: `create_evaluation_v2(model_run_id, preset=preset)` seeds the matches and rules (explicit arguments override the preset).
+- `create_evaluation_v2()` accepts `only_items_with_predictions` to restrict the evaluation to items that have at least one prediction.
+- **Batch create.** `create_evaluations_v2_batch()` creates one evaluation per `(model_run_id, slice_id)` pair with a shared configuration, running concurrently and returning a `BatchEvaluationResult` per job (capturing the created evaluation or the per-job error).
+- **Cancel & retry.** `EvaluationV2.cancel()` stops a running evaluation; `EvaluationV2.retry()` re-runs a failed one, reusing its slice/matches/exclusion rules.
+- `Dataset.evaluation_label_schema()` returns the dataset's ground-truth and prediction label vocabularies (`gt_labels` / `prediction_labels`) for building label matches and label exclusion rules.
+
+### Changed
+- `EvaluationV2.examples()` now treats `match_type` as optional — omit it to return examples of all match types.
+
+### Fixed
+- `EvaluationV2.charts()` issues a `POST` (matching the backend route) instead of a `GET` with a query string, which did not reach the server.
+
 ## [0.18.8](https://github.com/scaleapi/nucleus-python-client/releases/tag/v0.18.8) - 2026-06-17
 
 ### Fixed

docs/index.rst

@@ -12,36 +12,6 @@ Scale Nucleus helps you:
 
 Nucleus is a new way—the right way—to develop ML models, helping us move away from the concept of one dataset and towards a paradigm of collections of scenarios.
 
-.. _evaluations-v2:
-
-Evaluations V2
---------------
-
-Evaluation V2 measures how well a **model run** matches ground-truth annotations.
-Create a run with :meth:`NucleusClient.create_evaluation_v2`, wait with
-:meth:`nucleus.evaluation_v2.EvaluationV2.wait_for_completion`, then read summary metrics with
-:meth:`nucleus.evaluation_v2.EvaluationV2.charts` or individual matches with
-:meth:`nucleus.evaluation_v2.EvaluationV2.examples`.
-
-.. code-block:: python
-
-   import nucleus
-
-   client = nucleus.NucleusClient(api_key="YOUR_API_KEY")
-   evaluation = client.create_evaluation_v2(
-       model_run_id="run_xxx",
-       name="my-eval",
-       allowed_label_matches=[
-           nucleus.AllowedLabelMatch(
-               ground_truth_label="car",
-               model_prediction_label="vehicle",
-           ),
-       ],
-   )
-   evaluation.wait_for_completion()
-   charts = evaluation.charts(iou_threshold=0.5)
-   fps = evaluation.examples(match_type="FP", limit=20)
-
 .. _installation:
 
 Installation
@@ -56,8 +26,8 @@ To use Nucleus, first install it using `pip`:
 
 .. _api:
 
-Sections
---------
+API Reference
+-------------
 
 .. toctree::
    :maxdepth: 4