Skip to content

Add elfuse-oci pull into a local image store - #341

Open
henrybear327 wants to merge 2 commits into
sysprog21:mainfrom
henrybear327:oci/pull
Open

Add elfuse-oci pull into a local image store#341
henrybear327 wants to merge 2 commits into
sysprog21:mainfrom
henrybear327:oci/pull

Conversation

@henrybear327

@henrybear327 henrybear327 commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

Summary by cubic

Adds elfuse-oci, a Go CLI that pulls OCI images into a local store, keeping elfuse an OCI-agnostic runtime.

New Features

  • pull fetches an image via go-containerregistry into an OCI layout, pins it in index.json by canonical ref and platform, and refuses a legacy refs.json store.
  • A single flock serializes layout init and pin-table updates; the pin table is rewritten via a synced temp file and rename, so a crash can't publish a truncated table.
  • --timeout bounds the whole pull including the lock wait; a lock freed after the deadline is released, not used. --platform only accepts linux/arm64 and linux/amd64.
  • A bare manifest or index child that states no platform is pinned only when its config's platform matches; a child that is itself an index is refused.
  • Credentials come from Docker config inline auths or helpers, with the pull anonymous when no entry matches.
  • mk/oci.mk adds elfuse-oci, oci-test, and oci-lint targets; CI runs race tests and lint on Linux.
  • Documents the store, pull flow, and validation in docs/oci-images.md and the user guide.

Written for commit 1b858e2. Summary will update on new commits.

Review in cubic

@henrybear327
henrybear327 requested a review from jserv August 29, 2026 01:12
@henrybear327 henrybear327 self-assigned this Aug 29, 2026
cubic-dev-ai[bot]

This comment was marked as resolved.

cubic-dev-ai[bot]

This comment was marked as resolved.

cubic-dev-ai[bot]

This comment was marked as resolved.

cubic-dev-ai[bot]

This comment was marked as resolved.

cubic-dev-ai[bot]

This comment was marked as resolved.

@henrybear327
henrybear327 marked this pull request as draft August 29, 2026 22:50
@henrybear327
henrybear327 marked this pull request as ready for review August 29, 2026 23:44
cubic-dev-ai[bot]

This comment was marked as resolved.

@henrybear327

Copy link
Copy Markdown
Collaborator Author

Final switch of the open source library after trying out different projects and attempting to make it maintainable in the long run

@henrybear327
henrybear327 force-pushed the oci/pull branch 3 times, most recently from a5146cf to b96a08d Compare August 30, 2026 20:48
@jserv
jserv requested a review from maxliu0 August 31, 2026 12:56
Comment thread .github/workflows/build.yml Outdated
@jserv

jserv commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Use shorter and still meaningful paths. e.g., cmd/oci/.

Comment thread docs/oci-images.md

@jserv jserv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked and found sound, recording them so they do not get re-raised: --timeout really does bound the lock wait plus the whole transfer, because the deadline context is built before withLock and remote.Get threads it into the descriptor; authn.DefaultKeychain propagates config-load and helper errors instead of silently going anonymous, so the anonymous path only triggers on a genuinely absent entry; and v1.Hash.parse rejects any non-hex body and any algorithm other than sha256, so no reference or digest can walk out of filepath.Join.

The store design itself reads well. The pin table as a nested index annotated with org.opencontainers.image.ref.name is the right data structure, and it keeps elfuse itself free of OCI knowledge.

Comment thread mk/oci.mk Outdated
Comment thread cmd/elfuse-oci/pull.go Outdated
Comment thread cmd/elfuse-oci/store.go Outdated
Comment thread cmd/oci/store.go
Comment thread cmd/elfuse-oci/store.go Outdated
Comment thread cmd/elfuse-oci/store.go Outdated
Comment thread cmd/oci/store.go
if err != nil {
return v1.Descriptor{}, err
}
r, err := layer.Compressed()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A pull that fails partway leaves every layer already written in the store with nothing referencing it, and no subcommand removes it. Retry against a moved tag and the previous layers stay behind too, so the store grows without bound.

Either record what publishImage wrote and unwind on error, or land a prune subcommand and say so in docs/oci-images.md.

@henrybear327 henrybear327 Aug 31, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we have agreed in the previous iteration that we would not be handling OCI image lifecycle, this is intentionally left to the clean command to deal with (it's in the subsequent PR). WDYT?

Comment thread cmd/elfuse-oci/store.go Outdated
Comment thread cmd/elfuse-oci/store.go Outdated
Comment thread cmd/elfuse-oci/store.go Outdated
@henrybear327

Copy link
Copy Markdown
Collaborator Author

Use shorter and still meaningful paths. e.g., cmd/oci/.

Renamed.

Publish canonical references and per-platform manifests through standard
OCI indexes so layout readers can resolve the stored image. Use
go-containerregistry for reference parsing and registry access.
Describe the OCI store, reference normalization, platform selection,
credentials, timeouts, and pull validation. Add the command and
environment surface to the user guide.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants