fix: enable CI checks to run from devcontainer by vdstrizhkova · Pull Request #1136 · microsoft/code-with-engineering-playbook

vdstrizhkova · 2026-07-21T07:15:42Z

What are you trying to address

This PR makes all CI checks runnable from the devcontainer, so contributors can validate their changes locally before pushing.

Changes:

.devcontainer/devcontainer.json — Adds the docker-in-docker devcontainer feature, required to run the MegaLinter Docker image from inside the devcontainer. Also migrates settings to the current customizations.vscode schema and adds the ms-azuretools.vscode-docker extension.
.devcontainer/devcontainer-lock.json — New lock file pinning the docker-in-docker feature to v2.17.0 for reproducible builds.
.devcontainer/Dockerfile — Updates the base image to mcr.microsoft.com/devcontainers/python:1-${VARIANT} and removes a stale Yarn apt repository that caused apt-get update failures.
.devcontainer/run-megalinter.sh — New helper script that mirrors the MegaLinter CI step locally. Running bash .devcontainer/run-megalinter.sh scans only changed files (matching VALIDATE_ALL_CODEBASE: false in CI); VALIDATE_ALL_CODEBASE=true bash .devcontainer/run-megalinter.sh scans everything.
requirements-docs.txt — Pins Pygments>=2,<2.20 to avoid an AttributeError introduced in Pygments 2.20.0, which caused the mkdocs build to crash when pymdownx.highlight passes filename=None to HtmlFormatter.
lychee.toml — Accepts HTTP 403 and 429 responses in addition to 2xx, preventing false-positive link-check failures on sites (including GitHub) that block automated crawlers. Also removes an invalid header table entry and excludes additional bot-blocking domains.

Checklist

Changes follow the repo structure and land in the appropriate folder and section
No confidential information
No duplicated content
Labeled appropriately
This PR was reviewed by at least one subject matter expert
No lint check errors related to your changes

Note: You may see link check errors on pages you have not touched. This is normal, and due to either broken links or sites that reject link checker bots. The reviewer will help you get to a green state on these.

- Add Docker-in-Docker feature and run-megalinter.sh script so MegaLinter can be executed locally via .devcontainer/run-megalinter.sh - Update devcontainer base image to devcontainers/python:1 and remove stale Yarn apt repository that causes apt-get update failures - Pin Pygments<2.20 in requirements-docs.txt to fix mkdocs build: Pygments 2.20.0 calls html.escape() on a filename option that pymdownx can pass as None, causing AttributeError during documentation build - Accept HTTP 403/429 in lychee.toml to prevent false-positive link failures from GitHub rate-limiting and bot-protection in CI; also exclude known bot-blocking domains

Shiran Rubin (shiranr) · 2026-08-07T10:49:03Z

+# 403 (Forbidden) and 429 (Too Many Requests) are accepted because they indicate
+# the target page exists but the host blocks or throttles automated clients in
+# CI. Treating them as failures produces false positives for live links.
+accept = ["200..=299", "403", "429"]


Globally accepting 403 is risky — 403 often indicates a genuinely removed/protected resource, not just bot-blocking. Since the PR already adds per-URL excludes for the known bot-blocked domains, prefer keeping accept narrow (e.g. ["200..=299", "429"]) and continuing to handle exceptions via exclude. Otherwise real breakage on any 403-returning host will silently pass CI.

Shiran Rubin (shiranr) · 2026-08-07T10:49:54Z


 # Custom request headers
-header = ["name=value", "other=value"]
+# header = { name = "value", other = "value" }


Do we really need this if it's commented out?

Shiran Rubin (shiranr) · 2026-08-07T10:50:35Z

+  -w /tmp/lint \
+  -e VALIDATE_ALL_CODEBASE="${VALIDATE_ALL}" \
+  -e DEFAULT_BRANCH=main \
+  oxsecurity/megalinter:v8.1.0


v8.1.0 is hardcoded here but .github/workflows/mega-linter.yml has its own pinned version. If they drift, the local run stops mirroring CI, defeating the purpose. Consider sourcing the version from a shared file, or at least adding a comment reminding maintainers to bump both together.

Shiran Rubin (shiranr) · 2026-08-07T10:50:55Z

+set -euo pipefail
+
+REPO_ROOT="$(git rev-parse --show-toplevel)"
+VALIDATE_ALL="${VALIDATE_ALL_CODEBASE:-false}"


MegaLinter's VALIDATE_ALL_CODEBASE=false mode typically needs DEFAULT_WORKSPACE=/tmp/lint set (and a mounted .git) to diff against DEFAULT_BRANCH=main. Worth testing that bash .devcontainer/run-megalinter.sh (no env override) actually detects only changed files rather than silently linting everything.

Shiran Rubin (shiranr) · 2026-08-07T11:04:30Z

+# Reports are written to ./megalinter-reports/ in the repo root.
+set -euo pipefail
+
+REPO_ROOT="$(git rev-parse --show-toplevel)"


git rev-parse --show-toplevel will error out if the script is run outside a git worktree — fine for the documented use case, but consider a friendlier error message.

Shiran Rubin (shiranr) · 2026-08-07T11:05:01Z


 	// Use 'postCreateCommand' to run commands after the container is created.
-	"postCreateCommand": "pip3 install -r requirements-docs.txt",
+	"postCreateCommand": "pip3 install -r requirements-docs.txt && chmod +x .devcontainer/run-megalinter.sh",


Redundant chmod +x in postCreateCommand — the file is already committed with mode 100755. The chmod is harmless but unnecessary.

Shiran Rubin (shiranr) · 2026-08-07T11:06:04Z

 mkdocs==1.3.*
 mkdocs-material==8.2.*
 pymdown-extensions>=10.0
+Pygments>=2,<2.20


Pygments>=2,<2.20 — reasonable stopgap. The root cause is that mkdocs==1.3.* and mkdocs-material==8.2.* are years old (current is 1.6+/9.x); this cap is a symptom. Fine to unblock CI, but worth filing a follow-up issue to modernize the docs toolchain.

Shiran Rubin (shiranr) · 2026-08-07T11:06:15Z

@@ -1,6 +1,7 @@
 mkdocs==1.3.*


I think we should consider updating the dependencies in this file first in a separate PR

Shiran Rubin (shiranr) · 2026-08-07T11:06:39Z

+FROM mcr.microsoft.com/devcontainers/python:1-${VARIANT}
+
+# Remove stale Yarn apt repository that has an expired GPG key and causes apt-get update failures
+RUN rm -f /etc/apt/sources.list.d/yarn.list /etc/apt/sources.list.d/yarn.list.save


Yarn cleanup runs rm -f before any apt-get update. If another expired repo shows up later, contributors will hit the same class of failure. Consider running apt-get update in the same layer to fail fast at build time rather than at postCreateCommand.

Copilot

Pull request overview

This PR aims to make the repository’s CI checks (notably MegaLinter and link checking) runnable from inside the devcontainer so contributors can validate changes locally before pushing.

Changes:

Adds Docker-in-Docker support to the devcontainer (plus lockfile pinning) so Docker-based CI tooling (MegaLinter) can run inside the container.
Introduces a local helper script to run MegaLinter similarly to CI.
Adjusts docs/tooling dependencies and link-check configuration to reduce CI false-failures (Pygments pin; lychee acceptance/exclusions).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`requirements-docs.txt`	Pins Pygments to avoid a documented mkdocs build crash.
`lychee.toml`	Broadens accepted HTTP statuses and expands exclusions to reduce link-check false positives in CI.
`.devcontainer/run-megalinter.sh`	Adds a local MegaLinter runner script intended to mirror the CI configuration.
`.devcontainer/Dockerfile`	Switches to the newer devcontainers Python base image and removes a stale Yarn apt repo entry.
`.devcontainer/devcontainer.json`	Adds docker-in-docker feature, migrates to `customizations.vscode`, and wires post-create steps for local tooling.
`.devcontainer/devcontainer-lock.json`	Pins the docker-in-docker feature version for reproducible devcontainer builds.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+docker run --rm \
+  -v "${REPO_ROOT}:/tmp/lint" \
+  -w /tmp/lint \
+  -e VALIDATE_ALL_CODEBASE="${VALIDATE_ALL}" \
+  -e DEFAULT_BRANCH=main \
+  oxsecurity/megalinter:v8.1.0


@@ -96,6 +100,15 @@ exclude = [
  "^https://github.com/marketplace/?$",
  # Other:
  "^https://opensource.org/license/",  # works locally but fails with 403 on Github CI.


vdstrizhkova requested review from Tess Ferrandez (TessFerrandez), N Youens (nyouens) and Shiran Rubin (shiranr) as code owners July 21, 2026 07:15

Shiran Rubin (shiranr) reviewed Aug 7, 2026

View reviewed changes

Shiran Rubin (shiranr) requested a lite review from Copilot August 7, 2026 11:07

Copilot started reviewing on behalf of Shiran Rubin (shiranr) August 7, 2026 11:09 View session

Copilot AI reviewed Aug 7, 2026

View reviewed changes

Navigation Menu

Uh oh!

fix: enable CI checks to run from devcontainer - #1136

fix: enable CI checks to run from devcontainer#1136
vdstrizhkova wants to merge 1 commit into
microsoft:mainfrom
vdstrizhkova:feat/devcontainer-docker-megalinter

vdstrizhkova commented Jul 21, 2026

Uh oh!

Shiran Rubin (shiranr) Aug 7, 2026

Uh oh!

Shiran Rubin (shiranr) Aug 7, 2026

Uh oh!

Shiran Rubin (shiranr) Aug 7, 2026

Uh oh!

Shiran Rubin (shiranr) Aug 7, 2026

Uh oh!

Shiran Rubin (shiranr) Aug 7, 2026

Uh oh!

Shiran Rubin (shiranr) Aug 7, 2026

Uh oh!

Shiran Rubin (shiranr) Aug 7, 2026

Uh oh!

Shiran Rubin (shiranr) Aug 7, 2026

Uh oh!

Shiran Rubin (shiranr) Aug 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

vdstrizhkova commented Jul 21, 2026

What are you trying to address

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants