Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/ci_run.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ on:
required: true
type: string
profile:
description: 'Dependency profile declared in ci/dependency-profiles.json'
description: 'Dependency profile declared in dependencies.toml'
required: true
type: string
enable_reporting:
Expand Down Expand Up @@ -133,7 +133,7 @@ jobs:
id: dependencies
run: |
METADATA="$RUNNER_TEMP/ci-dependencies.json"
python3 scripts/resolve-ci-dependencies.py resolve \
python3 scripts/resolve-dependencies.py resolve \
--profile '${{ inputs.profile }}' \
--output "$METADATA" \
--github-output "$GITHUB_OUTPUT" \
Expand Down Expand Up @@ -214,7 +214,7 @@ jobs:
- name: "CHECK: {Verify dependency checkouts}"
id: verified-dependencies
run: |
python3 scripts/resolve-ci-dependencies.py verify \
python3 scripts/resolve-dependencies.py verify \
--metadata "$CI_DEPENDENCY_METADATA" \
--github-output "$GITHUB_OUTPUT"

Expand Down
12 changes: 7 additions & 5 deletions README_ADVANCED.md
Original file line number Diff line number Diff line change
Expand Up @@ -284,12 +284,14 @@ branch = "main"

### How Defaults Work

Defaults are defined in code (see [`src/config.rs`](src/config.rs) `Config::default()`) and written to `config.toml` during `init`. This means:
Dependency defaults are defined in [`dependencies.toml`](dependencies.toml) and
embedded into the binary at build time. They are written to `config.toml` during
`init`. This means:

- **First-time setup:** Running `foc-devnet init` creates `config.toml` with current defaults from code
- **First-time setup:** Running `foc-devnet init` creates `config.toml` with current defaults from the embedded manifest
- **Updating defaults:** When a new version of `foc-devnet` includes updated defaults (e.g., newer Lotus version), run `foc-devnet clean --all` then `foc-devnet init` to regenerate `config.toml` with the new defaults
- **Preserving config across re-init:** Running `foc-devnet clean` (without `--all`) preserves your `config.toml`, so a subsequent `init` reuses your existing settings
- **Source of truth:** The code defines what defaults are available; `config.toml` stores your specific configuration
- **Source of truth:** `dependencies.toml` defines dependency defaults; `config.toml` stores your specific configuration

### Editing Config

Expand Down Expand Up @@ -820,7 +822,7 @@ port_range_count = 100
- **[multicall3](https://github.com/mds1/multicall3)** - Multicall3 contract
### Dependent Version Strategy

Default versions for these repositories are defined in code (see [`src/config.rs`](src/config.rs) `Config::default()`).
Default versions for these repositories are defined in [`dependencies.toml`](dependencies.toml).

**Version specification methods:**
- **Latest tag** (`latesttag`, `latesttag:<selector>`, `latesttag:<url>:<selector>`): Resolved once at `init` time via `git ls-remote` and pinned as a concrete `GitTag` in `config.toml`. Use a glob selector to scope which tags are considered, e.g. `latesttag:v*` or `latesttag:pdp/v*`. Bare `latesttag` matches all tags.
Expand Down Expand Up @@ -1330,7 +1332,7 @@ Reports are written to `~/.foc-devnet/state/latest/scenario_report.md`.

Scenarios run automatically in CI after the devnet starts. On nightly runs (or manual dispatch with `reporting` enabled), failures automatically create a GitHub issue with a full report.

CI resolves compatibility-sensitive dependencies from `ci/dependency-profiles.json`.
CI resolves compatibility-sensitive dependencies from `dependencies.toml`.
Pull requests use the pinned `default` profile, while nightly `stability` runs use
the latest final releases and nightly `frontier` runs pin current development
branch heads to immutable commits. Nightly CI also runs manifest-declared mixed
Expand Down
1 change: 1 addition & 0 deletions build.rs
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ fn main() {
// Re-run if git info changes
println!("cargo:rerun-if-changed=.git/HEAD");
println!("cargo:rerun-if-changed=.git/refs/heads/");
println!("cargo:rerun-if-changed=dependencies.toml");

// Re-run if MockUSDFC contract files change
println!("cargo:rerun-if-changed=contracts/MockUSDFC/src/MockUSDFC.sol");
Expand Down
133 changes: 46 additions & 87 deletions ci/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# CI Dependency Profiles

`dependency-profiles.json` is the central manifest for CI dependency selection.
Its resolver is located in `scripts/resolve-ci-dependencies.py`.
`dependencies.toml` is the central manifest for runtime defaults and CI
dependency selection. Its resolver is located in `scripts/resolve-dependencies.py`.

## Profiles

Expand All @@ -19,22 +19,18 @@ The manifest declares valid profiles in its top-level `profiles` object:
- `stability-frontier-pdp`: used by nightly CI to test stable releases except
PDP, which is resolved from `frontier`.

Each component must define a selection for every component profile referenced by
the top-level profile definitions. Today those component selections are
`default`, `stability`, and `frontier`.
Components can define `default`, `stability`, and `frontier` selections. When a
profile selects a component profile that the component does not define, the
resolver uses that component's `default` selection unless the profile explicitly
overrides that component.

Top-level profile definitions have a `base` component profile and can override
specific components:

```json
{
"stability-frontier-curio": {
"base": "stability",
"components": {
"curio": "frontier"
}
}
}
```toml
[profiles.stability-frontier-curio]
base = "stability"
curio = "frontier"
```

In that example, Curio resolves from its `frontier` selection while every other
Expand All @@ -46,45 +42,38 @@ exist unless added there.

Top-level component fields:

- `repository`: Git repository URL.
- `git`: Git repository URL.
- `npm_package`: npm package name, for components that are resolved through npm
metadata.
- `default`, `stability`, `frontier`: component profile selections.

Profile selections always have a `strategy`. Some strategies require additional
fields.
Profile selections are inline TOML tables. The resolver infers the strategy from
the keys present in each selection.

### `config_default`
### `bundled`

Use the compiled `Config::default()` value and pass no runtime override to
`foc-devnet init`.
Use the component bundled by another dependency and pass no runtime override to
`foc-devnet init`. PDP uses this in the `default` profile so the runtime default
continues to use filecoin-services' bundled submodule.

```json
{
"strategy": "config_default"
}
```toml
default = { bundled = true }
```

### `git_commit`

Use an exact Git commit SHA.

```json
{
"strategy": "git_commit",
"commit": "fadc836e65804311aca3bd2276861acabe42313f"
}
```toml
default = { commit = "fadc836e65804311aca3bd2276861acabe42313f" }
```

### `git_branch`

Resolve a branch head to an immutable commit SHA before the run starts.

```json
{
"strategy": "git_branch",
"branch": "master"
}
```toml
frontier = { branch = "master" }
```

The resolved metadata records both the branch name and the exact commit.
Expand All @@ -93,70 +82,49 @@ The resolved metadata records both the branch name and the exact commit.

Resolve a Git tag to an immutable commit SHA. `tag` can be an exact tag:

```json
{
"strategy": "git_tag",
"tag": "v1.2.3"
}
```toml
default = { tag = "v1.2.3" }
```

`tag` can also be a pattern. Pattern selections choose the latest matching tag:

```json
{
"strategy": "git_tag",
"tag": "v*"
}
```toml
stability = { tag_pattern = "v*" }
```

By default, pattern selections exclude prerelease tags such as `-rc`, `-alpha`,
`-beta`, and development tags. Set `include_prereleases` to include them:

```json
{
"strategy": "git_tag",
"tag": "v*",
"include_prereleases": true
}
```toml
stability = { tag_pattern = "v*", include_prereleases = true }
```

### `git_submodule`

Resolve a git submodule gitlink from a tag or tag pattern in another repository.

```json
{
"strategy": "git_submodule",
"repository": "https://github.com/FilOzone/filecoin-services.git",
"tag": "v*",
"path": "service_contracts/lib/pdp"
}
```toml
stability = { submodule_git = "https://github.com/FilOzone/filecoin-services.git", tag_pattern = "v*", path = "service_contracts/lib/pdp" }
```

The resolver first resolves `repository` and `tag` with the same rules as
`git_tag`, then reads `path` from that tree and records the submodule gitlink SHA
as the selected component commit. PDP uses this to pin the same bundled PDP
gitlink as the selected filecoin-services stability tag, even in mixed profiles
that override filecoin-services itself.
The resolver first resolves `submodule_git` and `tag_pattern` with the same
rules as `git_tag`, then reads `path` from that tree and records the submodule
gitlink SHA as the selected component commit. PDP uses this to pin the same
bundled PDP gitlink as the selected filecoin-services stability tag, even in
mixed profiles that override filecoin-services itself.

### `npm_version`
### `npm`

Resolve an npm version, range, or dist-tag to a concrete package version.

```json
{
"strategy": "npm_version",
"version": "1.0.1"
}
```toml
default = { npm = "1.0.1" }
```

The `version` field can also be an npm dist-tag:
The `npm` field can also be an npm dist-tag:

```json
{
"strategy": "npm_version",
"version": "latest"
}
```toml
stability = { npm = "latest" }
```

The resolver records the concrete package version selected at resolution time
Expand All @@ -168,17 +136,8 @@ Some profile selections can include an optional `overrides` object. Each entry
maps a package name to a `version` and a `reason` explaining why the override
exists:

```json
{
"strategy": "git_tag",
"tag": "synapse-sdk-v1.0.1",
"overrides": {
"nanoid": {
"version": "3.3.13",
"reason": "nanoid 5.x is ESM-only and breaks the CJS build"
}
}
}
```toml
default = { tag = "synapse-sdk-v1.0.1", overrides = { nanoid = { version = "3.3.13", reason = "nanoid 5.x is ESM-only and breaks the CJS build" } } }
```

Overrides are explicit profile policy. Both `version` and `reason` are required
Expand All @@ -189,7 +148,7 @@ override is applied. The resolver does not infer overrides from package metadata
Overrides are currently allowed only for:

- `synapse-sdk`, because scenario setup controls its pnpm install.
- `filecoin-pin` selections using `npm_version`, because those install into a
- `filecoin-pin` selections using `npm`, because those install into a
temporary npm project controlled by the scenario.

Current consumers:
Expand All @@ -201,7 +160,7 @@ Current consumers:

## Current Boundary

`resolve-ci-dependencies.py` resolves metadata. It does **not** install
`resolve-dependencies.py` resolves metadata. It does **not** install
components.

Installation currently lives in three places (which consume the resolved
Expand Down
Loading