Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 5 additions & 9 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ name: CI
on:
push:
branches: [main]
# Child PRs target another stack branch and need the same checks.
pull_request:
branches: [main]
workflow_dispatch:

# Least privilege: enough to clone, nothing else. Checking out a private
Expand All @@ -24,10 +24,6 @@ concurrency:
group: ci-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: ${{ github.event_name == 'pull_request' }}

env:
# Fail rather than silently re-resolving when uv.lock is stale.
UV_FROZEN: "1"

jobs:
static:
name: Lint, format, docs and licences
Expand All @@ -37,7 +33,7 @@ jobs:

- uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
version: "latest"
version-file: "pyproject.toml"
python-version: "3.12"
enable-cache: true

Expand Down Expand Up @@ -67,13 +63,13 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
python-version: ["3.11", "3.12", "3.13", "3.14"]
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
version: "latest"
version-file: "pyproject.toml"
python-version: ${{ matrix.python-version }}
enable-cache: true

Expand Down Expand Up @@ -101,7 +97,7 @@ jobs:

- uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
version: "latest"
version-file: "pyproject.toml"
python-version: "3.12"
enable-cache: true

Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/health-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ jobs:

- uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
version: "latest"
version-file: "pyproject.toml"
python-version: "3.12"
enable-cache: true

Expand Down
6 changes: 1 addition & 5 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,6 @@ concurrency:
group: release-${{ github.ref }}
cancel-in-progress: false

env:
# Fail rather than silently re-resolving when uv.lock is stale.
UV_FROZEN: "1"

jobs:
build:
name: Build and check
Expand All @@ -45,7 +41,7 @@ jobs:

- uses: astral-sh/setup-uv@c18668ad3cf93ea998bef934396af7bb5c839dc7 # v10.2.0
with:
version: "latest"
version-file: "pyproject.toml"
python-version: "3.12"
# A cache written by another run could otherwise reach the
# published files.
Expand Down
1 change: 1 addition & 0 deletions .python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.12
22 changes: 19 additions & 3 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,13 +90,28 @@ passes the check automatically.

## Getting set up

Requires Python 3.10 to 3.14 and [uv](https://docs.astral.sh/uv/).
Supports Python 3.11 to 3.14. The default development interpreter is Python 3.12
from `.python-version`; CI tests every supported version. Start with an existing
[uv](https://docs.astral.sh/uv/) installation, then install the exact
`required-version` from `pyproject.toml`:

```bash
uv sync # installs the dev group: pytest, ruff, jupyter
make bootstrap # installs the declared uv; ensure its bin directory is on PATH
make install-dev # locked dev dependencies and pre-commit hook
uv run usersim smoke # offline self-check; no API key, no network
```

`make check-tools` rejects another uv version. `make check-lock` checks that
`uv.lock` agrees with dependency metadata without rewriting it. Both setup
targets use `--locked`, so stale metadata fails rather than updating the lock.
After a deliberate dependency change, run `uv lock`, review `uv.lock`, and
commit it with the metadata change. CI reads the same uv version declaration;
update it deliberately and validate the lock before committing a tool upgrade.

`make install-ci` performs locked setup without installing hooks. In a Git
worktree, `make install-dev` finds the hook directory through Git rather than
assuming `.git` is a directory.

`smoke` is the fastest way to confirm an install is sound. It resolves the
model catalog, every probe's assets, and the plugin entry points without
making a single model call.
Expand Down Expand Up @@ -219,7 +234,8 @@ Your PR title must follow [Conventional Commits](https://www.conventionalcommits

Every job in `.github/workflows/ci.yml` calls a `make` target, so anything CI
rejects reproduces locally with the same command. Tests and the dependency
licence check run on Python 3.10 to 3.14.
licence check run on Python 3.11 to 3.14. Pull-request checks run for every
base branch so stacked PRs receive CI; push checks remain scoped to `main`.

No job triggered by a pull request has access to a credential, so a PR from a
fork runs the complete suite. The consequence is that CI cannot tell you
Expand Down
25 changes: 20 additions & 5 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
SHELL := /bin/bash

UV ?= uv
UV_VERSION := $(shell sed -n 's/^required-version = "==\([^"]*\)"/\1/p' pyproject.toml)

# A top-level `_`-prefixed directory holds material that is not part of the
# distribution and may carry its own targets. Absent one, this expands to
Expand Down Expand Up @@ -57,7 +58,7 @@ COV_FAIL_UNDER ?= 84

.DEFAULT_GOAL := help

.PHONY: help install-dev install-ci install-kernel health-check check-notebooks-clean lint lint-fix format format-check check-all \
.PHONY: help bootstrap check-tools check-lock install-dev install-ci install-kernel health-check check-notebooks-clean lint lint-fix format format-check check-all \
check-all-fix test test-fast coverage smoke dist check-dist check-release-tag check-extensions check-doc-links clean-notebooks check-license-headers update-license-headers check-dependency-licenses clean

help: ## Show the available targets
Expand All @@ -67,11 +68,25 @@ help: ## Show the available targets

# --- setup -----------------------------------------------------------------

# Requires an existing uv installation; installs the declared release on PATH.
bootstrap: ## Install the repository's required uv version as a standalone tool
$(UV) tool install --force "uv==$(UV_VERSION)"

check-tools: ## Reject an unsupported uv version
@actual=$$($(UV) --version 2>/dev/null | awk '{print $$2}'); \
if [ -z "$(UV_VERSION)" ] || [ "$$actual" != "$(UV_VERSION)" ]; then \
echo "Expected uv $(UV_VERSION), found $${actual:-none}. Install uv, then run make bootstrap and ensure its bin directory is on PATH." >&2; \
exit 1; \
fi

check-lock: check-tools ## Check dependency metadata against uv.lock without rewriting it
$(UV) lock --check

# Reinstalling the project restores any file that removing another distribution
# took with it, such as the `usersim` command an earlier name also installed.
install-dev: ## Sync the workspace with dev tooling and install pre-commit hooks
$(UV) sync --reinstall-package usersim
@if [ ! -f .git/hooks/pre-commit ]; then \
install-dev: check-tools ## Sync the workspace with dev tooling and install pre-commit hooks
$(UV) sync --locked --reinstall-package usersim
@if [ ! -f "$$(git rev-parse --git-path hooks/pre-commit)" ]; then \
echo "Installing pre-commit hooks..."; \
$(UV) run pre-commit install; \
else \
Expand All @@ -81,7 +96,7 @@ install-dev: ## Sync the workspace with dev tooling and install pre-commit hook
# --locked installs strictly from uv.lock and refuses to re-resolve, so
# dependency drift becomes a build failure rather than a silent
# re-resolution. No pre-commit hooks -- CI has no working tree to guard.
install-ci: ## Sync exactly what CI needs, enforcing the lockfile
install-ci: check-tools ## Sync exactly what CI needs, enforcing the lockfile
$(UV) sync --locked

# --- lint / format ---------------------------------------------------------
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

[![CI](https://github.com/NVIDIA-NeMo/UserSim/actions/workflows/ci.yml/badge.svg)](https://github.com/NVIDIA-NeMo/UserSim/actions/workflows/ci.yml)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](https://github.com/NVIDIA-NeMo/UserSim/blob/main/LICENSE)
[![Python](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12%20%7C%203.13%20%7C%203.14-blue.svg)](https://github.com/NVIDIA-NeMo/UserSim/blob/main/pyproject.toml)
[![Python](https://img.shields.io/badge/python-3.11%20%7C%203.12%20%7C%203.13%20%7C%203.14-blue.svg)](https://github.com/NVIDIA-NeMo/UserSim/blob/main/pyproject.toml)

> Simulate a statistically representative population of users, sampled from
> persona datasets (e.g., census-grounded), having multi-turn conversations with
Expand Down Expand Up @@ -98,7 +98,7 @@ plug in through entry points without forking the repository.

## Setup

Python 3.10+, in a virtual environment.
Python 3.11–3.14, in a virtual environment.

```bash
python -m venv .venv && source .venv/bin/activate
Expand Down
3 changes: 1 addition & 2 deletions dependency-license-policy.toml
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,9 @@ reason = "Unmodified character-detection library installed as a separate transit
reports = [{ version = "5.2.0", license = "GNU Lesser General Public License v2 or later (LGPLv2+)" }]

[exceptions.numpy]
reason = "Core numerical dependency, BSD-3-Clause overall. The additional identifiers cover vendored components and are each permissive or public-domain-equivalent. Python 3.10 resolves 2.2.6, which reports the same licence as the generic BSD classifier."
reason = "Core numerical dependency, BSD-3-Clause overall. The additional identifiers cover vendored components and are each permissive or public-domain-equivalent."
reports = [
{ version = "2.4.3", license = "BSD-3-Clause AND 0BSD AND MIT AND Zlib AND CC0-1.0" },
{ version = "2.2.6", license = "BSD License" },
]

[exceptions.pathspec]
Expand Down
12 changes: 4 additions & 8 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name = "usersim"
version = "0.2.0rc2.dev0"
description = "Improve models and agents with population-grounded simulated users, even before production traffic exists."
readme = "README.md"
requires-python = ">=3.10"
requires-python = ">=3.11,<3.15"
keywords = [
"llm", "agents", "evaluation", "synthetic-data", "simulation", "user-simulation", "personas", "nemotron",
]
Expand All @@ -13,7 +13,6 @@ classifiers = [
"Intended Audience :: Developers",
"Intended Audience :: Science/Research",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
Expand Down Expand Up @@ -71,8 +70,6 @@ dependencies = [
# Reads credentials from a .env file so a run does not depend on what is
# exported in the calling shell. No required dependencies of its own.
"python-dotenv>=1.0,<2",
# Parses the models TOML on Python 3.10, which has no tomllib.
"tomli>=1.1.0; python_version < '3.11'",
]
[project.urls]
Homepage = "https://github.com/NVIDIA-NeMo/UserSim"
Expand Down Expand Up @@ -107,9 +104,6 @@ dev = [
# here. conftest.py fails the run if it is absent, because without it every
# async test is skipped with a warning rather than failing.
"pytest-asyncio>=1.0,<2",
# tests/test_network_guard.py unwraps exception groups, which Python 3.10
# has only through this backport.
"exceptiongroup>=1.0; python_version < '3.11'",
]
# Empty on purpose: the gate runs a pinned pip-licenses through `uv tool run`,
# so this group exists only to scope an isolated environment holding the
Expand All @@ -124,6 +118,8 @@ conversation-simulator = "usersim.engine.plugin:conversation_simulator"
trajectory-evaluator = "usersim.engine.plugin:trajectory_evaluator"

[tool.uv]
# Shared developer and CI version; setup-uv reads this declaration directly.
required-version = "==0.13.1"
override-dependencies = [
# Keep in sync with the ``dependencies`` pin above.
"data-designer>=0.9.5,<0.10",
Expand Down Expand Up @@ -192,7 +188,7 @@ asyncio_mode = "auto"
[tool.ruff]
line-length = 120
indent-width = 4
target-version = "py310"
target-version = "py311"
# Ruff lints .ipynb by default. The notebooks are user-facing walkthroughs with
# committed outputs, so linting them means noise and rewritten cells. Excluding
# them keeps a bare `ruff check` equivalent to `make lint`.
Expand Down
6 changes: 1 addition & 5 deletions scripts/check_dependency_licenses.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,11 @@
import re
import subprocess
import sys
import tomllib
from dataclasses import dataclass
from pathlib import Path
from typing import Any

if sys.version_info >= (3, 11):
import tomllib
else:
import tomli as tomllib

PIP_LICENSES_VERSION = "5.5.5"
POLICY_PATH = Path(__file__).resolve().parents[1] / "dependency-license-policy.toml"

Expand Down
7 changes: 1 addition & 6 deletions src/usersim/cli/_models.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,18 +54,13 @@

import logging
import os
import sys
import tomllib
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Iterable

from usersim.cli._errors import ConfigError

if sys.version_info >= (3, 11):
import tomllib
else:
import tomli as tomllib

logger = logging.getLogger(__name__)


Expand Down
2 changes: 1 addition & 1 deletion tests/fixtures/extension_package/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ build-backend = "setuptools.build_meta"
name = "usersim-extension-fixture"
version = "0.1.0"
description = "Exercises every published extension seam against a real install."
requires-python = ">=3.10"
requires-python = ">=3.11,<3.15"

# One entry per published group. If a seam stops working, installing this
# package and asking the CLI what it can see is what notices.
Expand Down
34 changes: 34 additions & 0 deletions tests/test_ci_workflows.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,12 @@
from __future__ import annotations

import re
import tomllib
from pathlib import Path

import pytest
import yaml
from packaging.specifiers import SpecifierSet

_WORKFLOWS = Path(__file__).resolve().parents[1] / ".github" / "workflows"

Expand Down Expand Up @@ -304,3 +306,35 @@ def test_the_signed_commit_message_uses_only_substituted_placeholders(self) -> N
assert not unsubstituted, (
f"signed-commit-message uses {sorted(unsubstituted)}, which the action never replaces."
)


class TestSharedToolSetup:
def test_ci_checks_every_pull_request_base(self) -> None:
doc = _load(_WORKFLOWS / "ci.yml")
triggers = doc.get("on", doc.get(True))
assert "pull_request" in triggers
assert not triggers["pull_request"]
assert triggers["push"] == {"branches": ["main"]}

@pytest.mark.parametrize("name", ["ci.yml", "release.yml", "health-checks.yml"])
def test_uv_setup_uses_the_repository_declaration(self, name: str) -> None:
doc = _load(_WORKFLOWS / name)
setups = [
step
for job in doc["jobs"].values()
for step in job.get("steps", [])
if step.get("uses", "").startswith("astral-sh/setup-uv@")
]
assert setups
for setup in setups:
assert setup["with"]["version-file"] == "pyproject.toml"
assert "version" not in setup["with"]
assert "UV_FROZEN" not in (_WORKFLOWS / name).read_text()


def test_native_matrix_matches_package_python_support() -> None:
project = tomllib.loads((_WORKFLOWS.parents[1] / "pyproject.toml").read_text())["project"]
supported = SpecifierSet(project["requires-python"])
expected = [f"3.{minor}" for minor in range(30) if f"3.{minor}" in supported]
matrix = _load(_WORKFLOWS / "ci.yml")["jobs"]["test"]["strategy"]["matrix"]
assert matrix["python-version"] == expected
4 changes: 0 additions & 4 deletions tests/test_network_guard.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,9 @@

import json
import socket
import sys

import pytest

if sys.version_info < (3, 11):
from exceptiongroup import BaseExceptionGroup

# TEST-NET-3 (RFC 5737): reserved for documentation and guaranteed not to be
# routable. Using a literal address keeps DNS out of the picture, so the guard
# is what fails rather than name resolution.
Expand Down
10 changes: 2 additions & 8 deletions tests/test_packaging_contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,20 +15,14 @@
import ast
import importlib
import sys
import tomllib
from pathlib import Path

import pytest

if sys.version_info >= (3, 11):
import tomllib
else:
import tomli as tomllib

_REPO_ROOT = Path(__file__).resolve().parents[1]

#: The running Python's standard library, plus ``tomllib``, which is new in
#: 3.11: on 3.10 the package imports the declared ``tomli`` backport instead.
_STDLIB = sys.stdlib_module_names | {"tomllib"}
_STDLIB = sys.stdlib_module_names


class TestPluginEntryPoints:
Expand Down
Loading
Loading