Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
259 changes: 259 additions & 0 deletions .bob/rules/dependency-hygiene.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,259 @@
# Dependency Hygiene β€” Automatic Vulnerability Check

## Trigger: when this rule applies

When you write or modify **any file** inside a subdirectory that contains a
dependency manifest, you MUST perform a vulnerability check as part of that
same task β€” without waiting to be asked. Check the submodule directory **and**
its parents up to the repo root.

| Manifest file | Ecosystem |
|---|---|
| `go.mod` | Go |
| `package.json` | Node / npm |
| `requirements.txt` | Python / pip |

---

## Step 1 β€” Check for open Dependabot alerts first

Always consult GitHub before running local tools β€” it gives the authoritative
patched version and avoids redundant work.

```bash
# Requires: gh auth login --hostname github.com (scope: repo)
gh api --hostname github.com \
"/repos/IBM/CodeEngine/dependabot/alerts?per_page=100&state=open" \
| jq '[.[] | {
number,
severity: .security_advisory.severity,
cvss: .security_advisory.cvss.score,
package: .dependency.package.name,
ecosystem: .dependency.package.ecosystem,
manifest: .dependency.manifest_path,
patched_version: .security_vulnerability.first_patched_version.identifier,
summary: .security_advisory.summary
}] | sort_by(.severity, -.cvss)'
```

Cross-check the alert against the actual lock-file version β€” an alert may
already be resolved in the lock file even though still open on GitHub:

```bash
# npm β€” check transitive version in lock file
jq '(.packages // {}) | to_entries[]
| select(.key | test("<package>"))
| {pkg: .key, version: .value.version}' package-lock.json

# Go β€” check resolved version
grep "<module>" go.mod go.sum | head -5
```

---

## Step 2 β€” Assess risk before acting

| Category | Action |
|---|---|
| Patch/minor bump of an indirect dep | Apply automatically |
| Patch/minor bump within same major, no API change | Apply automatically |
| Major version bump | Explain breaking-change risk, ask user for approval |
| No patched version available (`patched_version` is null) | Document in PR under "Deferred"; do not block |
| Lock-file version already β‰₯ patched version | Mark resolved; no action needed |

### Previously deferred alerts β€” re-verify before treating as still blocked

Some alerts were previously deferred because no fix was available or because
upgrading was a breaking change. **Do not assume these are still blocked.**
When a task touches one of the listed submodules, re-check the current state
live before deciding to defer again:

```bash
# Re-check a specific previously-deferred alert by number
gh api --hostname github.com \
"/repos/IBM/CodeEngine/dependabot/alerts/<number>" \
| jq '{state, patched_version: .security_vulnerability.first_patched_version.identifier}'

# For npm: check whether the package's latest version is still ESM-only or has a CJS compat release
npm show <package>@latest main exports type 2>/dev/null

# For pip: check whether a patched version now exists
pip index versions <package> 2>/dev/null | head -3
```

Use the following as **background hints only** β€” they describe why the alert
was deferred at the time of the last scan. Verify live before reusing this reasoning:

| Alert(s) | Package | Submodule(s) | Reason last deferred | Re-check command |
|---|---|---|---|---|
| #656 #657 #658 | `file-type` (npm) | `cos-to-sql`, `fruit-counter`, `trusted-profiles/node` | v17+ is ESM-only; CJS callers need `await import()` migration | `npm show file-type@latest type` β€” if not `"module"`, CJS is back |
| #123 | `cookie` (npm) | `fotobox/frontend-app` | Fixed version blocked inside `@sveltejs/kit@2.x` | `npm show @sveltejs/kit@latest dependencies \| grep cookie` |
| #1192 | `accelerate` (pip) | `serverless-fleets/tutorials/inferencing/src` | No upstream patched version | `pip index versions accelerate \| head -1` then compare to alert's `patched_version` |

If live re-check shows the blocker is gone, treat the alert as **actionable
now** and apply the fix following Steps 3–5.

### Other previously noted upgrade considerations

- **`golang.org/x/net` β‰₯ v0.55.0**: requires Go 1.26+. The `go.mod` `go`
directive will be auto-bumped by `go get`. Safe, but changes the minimum
toolchain version recorded in `go.mod` β€” verify CI uses a matching version.

- **`mongo-driver` v1 series**: officially deprecated in favour of v2.
Bumping within v1.x is safe. Migration to v2 is a separate breaking effort
and should not be done as part of a routine dependency update.

---

## Step 3 β€” Apply fixes

### npm
```bash
cd <submodule-dir>
npm update # resolves transitive bumps within declared ranges
npm audit # confirm remaining issues
npm ci # validate lock file installs cleanly
```

Never use `npm audit fix --force` without reviewing what it would change β€”
it can downgrade to breaking versions.

### Go
```bash
cd <submodule-dir>
go get golang.org/x/net@latest
go get go.mongodb.org/mongo-driver@latest # example; target the specific module
go mod tidy
go build ./... # confirm the module still compiles
```

### Python / pip
```bash
cd <submodule-dir>
pip-audit -r requirements.txt # show vulnerabilities
pip install --upgrade <package>
pip freeze > requirements.txt
pip-audit -r requirements.txt # confirm resolved
```

---

## Step 4 β€” Verify

### Native (preferred locally β€” no container needed)
```bash
npm ci # npm: clean install from updated lock file
go build ./... # Go: confirm compilation
go vet ./...
python -m py_compile <main>.py # Python: syntax check
```

### Container-based (matches CI)
Each submodule has a `build` and/or `verify` shell script that runs
`docker build --platform linux/amd64 .`.

**On Apple Silicon (arm64):** QEMU x86_64 emulation via Podman is slow and
can SIGSEGV during `npm install` inside the container. Prefer native `npm ci`
locally; let CI handle the full container build.

```bash
podman machine start # start Podman VM first
bash <submodule-dir>/verify # run one at a time β€” parallel runs crash under QEMU
```

CI workflow: `.github/workflows/dependabot-build-verify.yml` β€” runs on every
PR that touches a submodule with a `build`/`verify` script on native x86_64.

---

## Step 5 β€” Commit and PR hygiene

Stage **only** dependency files. The `verify`/`build` scripts get
permission-bit changes from `chmod +x` β€” do not commit those:

```bash
# Restore permission-only changes to scripts
git checkout -- $(git diff --name-only | grep -E '/(verify|build)$')
```

Commit message format:
```
fix(deps): remediate Dependabot security alerts

npm updates
-----------
<submodule>:
- <package> <old> -> <new> (GHSA: <summary>, Severity #alert-number)

Go module updates
-----------------
<submodule>:
- <module> <old> -> <new> (Severity #alert-number)

Deferred / no fix available
----------------------------
- <package> (#alert): <reason>
```

PR description must include: summary table, deferred section with reasons,
and a testing section noting what was validated locally vs. what CI covers.

---

## Reporting format (inline with code change)

```
## Dependency vulnerability check β€” <submodule-dir>

Ecosystem: <Go|npm|pip>

| Package | Current | Fix | Severity | Action |
|---------|---------|-----|----------|--------|
| example | 1.2.3 | 1.2.4 | High | Updated in this PR |
| other | 2.0.0 | 3.0.0 | Medium | Major bump β€” needs approval |

Remaining open alerts: <n>
```

If no vulnerabilities found: `βœ… No known vulnerabilities in <submodule-dir>.`

---

## Project map β€” all submodules with dependency files

| Submodule | Ecosystem |
|---|---|
| `app-n-event-notification` | Go |
| `auth-oidc-proxy/auth` | npm |
| `auth-oidc/node` | npm |
| `cloudant-change-listener/job` | npm |
| `cos-to-sql` | npm |
| `fotobox/download-app` | Go |
| `fotobox/frontend-app` | npm |
| `fotobox/upload-function` | pip |
| `fruit-counter` | npm |
| `gallery/app`, `gallery/job`, `gallery/function` | npm |
| `github-webhook` | Go |
| `grpc` | Go |
| `helloworld-samples/app-python` | pip |
| `helloworld-samples/*-nodejs*` | npm |
| `helloworld-samples/*-python*` | pip |
| `kafka` | Go |
| `kafka-observer` | Go |
| `llm-translator-app` | npm + pip |
| `logging/go-*` | Go |
| `logging/node-*` | npm |
| `logging/python-*` | pip |
| `metrics-collector` | Go |
| `metrics-examples/go` | Go |
| `metrics-examples/node` | npm |
| `metrics-examples/python` | pip |
| `private-path-to-vpc-vsi/ce-app` | Go |
| `remote-bob/apiserver`, `remote-bob/job-agent` | Go |
| `satellite-connector-to-vpc-vsi/ce-app` | Go |
| `serverless-fleets/tutorials/inferencing/src` | pip |
| `sessions` | Go |
| `thumbnail/eventer`, `thumbnail/v1`, `thumbnail/v2` | Go |
| `trusted-profiles/go` | Go |
| `trusted-profiles/node` | npm |
| `trusted-profiles/python` | pip |
2 changes: 1 addition & 1 deletion app-n-event-notification/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ require (
github.com/leodido/go-urn v1.2.3 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/oklog/ulid v1.3.1 // indirect
go.mongodb.org/mongo-driver v1.11.3 // indirect
go.mongodb.org/mongo-driver v1.17.10 // indirect
golang.org/x/crypto v0.52.0 // indirect
golang.org/x/sys v0.45.0 // indirect
golang.org/x/text v0.37.0 // indirect
Expand Down
9 changes: 4 additions & 5 deletions app-n-event-notification/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,8 @@ github.com/go-playground/validator/v10 v10.13.0 h1:cFRQdfaSMCOSfGCCLB20MHvuoHb/s
github.com/go-playground/validator/v10 v10.13.0/go.mod h1:dwu7+CG8/CtBiJFZDz4e+5Upb6OLw04gtBYw0mcG/z4=
github.com/golang/snappy v0.0.1/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q=
github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/uuid v1.1.1 h1:Gkbcsh/GbpXz7lPftLA3P6TYMwjCLYm83jiFQZF/3gY=
github.com/google/uuid v1.1.1/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/mux v1.8.0 h1:i40aqfkR1h2SlN9hojwV5ZA91wcXFOvkdNIeFDP5koI=
Expand Down Expand Up @@ -79,15 +79,14 @@ github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.2 h1:+h33VjcLVPDHtOdpUCuF+7gSuG3yGIftsP1YvFihtJ8=
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/tidwall/pretty v1.0.0 h1:HsD+QiTn7sK6flMKIvNmpqz1qrpP3Ps6jOKIKMooyg4=
github.com/tidwall/pretty v1.0.0/go.mod h1:XNkn88O1ChpSDQmQeStsy+sBenx6DDtFZJxhVysOjyk=
github.com/xdg-go/pbkdf2 v1.0.0/go.mod h1:jrpuAogTd400dnrH08LKmI/xc1MbPOebTwRqcT5RDeI=
github.com/xdg-go/scram v1.1.1/go.mod h1:RaEWvsqvNKKvBPvcKeFjrG2cJqOkHTiyTpzz23ni57g=
github.com/xdg-go/stringprep v1.0.3/go.mod h1:W3f5j4i+9rC0kuIEJL0ky1VpHXQU3ocBgklLGvcBnW8=
github.com/youmark/pkcs8 v0.0.0-20181117223130-1be2e3e5546d/go.mod h1:rHwXgn7JulP+udvsHwJoVG1YGAP6VLg4y9I5dyZdqmA=
go.mongodb.org/mongo-driver v1.10.0/go.mod h1:wsihk0Kdgv8Kqu1Anit4sfK+22vSFbUrAVEYRhCXrA8=
go.mongodb.org/mongo-driver v1.11.3 h1:Ql6K6qYHEzB6xvu4+AU0BoRoqf9vFPcc4o7MUIdPW8Y=
go.mongodb.org/mongo-driver v1.11.3/go.mod h1:PTSz5yu21bkT/wXpkS7WR5f0ddqw5quethTUn9WM+2g=
go.mongodb.org/mongo-driver v1.17.10 h1:kdAgQvu8TROXZpSkJQd5wzfaNCCrMbpZyKFtQ6qkPCE=
go.mongodb.org/mongo-driver v1.17.10/go.mod h1:LlOhpH5NUEfhxcAwG0UEkMqwYcc4JU18gtCdGudk/tQ=
golang.org/x/crypto v0.0.0-20220622213112-05595931fe9d/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
golang.org/x/crypto v0.52.0 h1:RMs7fP2rXdep0CftQlK8Uf+kibLm7qkCcradZWYz988=
golang.org/x/crypto v0.52.0/go.mod h1:1QgfPxDqh0T2M/elOJtp9RvuR95kVjir0e6/BvEmGbc=
Expand Down
Loading
Loading