Skip to content

ci: compile linked bigquery-storage in pnpm-lockfile-check - #9614

Draft
ohmayr wants to merge 1 commit into
mainfrom
ci/compile-linked-workspace-packages
Draft

ohmayr wants to merge 1 commit into
mainfrom
ci/compile-linked-workspace-packages

Conversation

@ohmayr

@ohmayr ohmayr commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Context & Problem

The "Link In-Repo Workspace Packages" step that #9569 added to pnpm-lockfile-check links matching in-repo packages after the frozen install, then restores the committed lockfile, so turbo doesn't compile them. The step compiles gaxios and teeny-request itself. The Arrow PRs for @google-cloud/bigquery (#9596–#9605) add a devDependency on @google-cloud/bigquery-storage, which handwritten/bigquery-storage satisfies, so the step links it to source without a build/ directory and requiring it fails with MODULE_NOT_FOUND in every unit and Bun job.

Summary of Changes

  • .github/actions/pnpm-lockfile-check/action.yaml: compile handwritten/bigquery-storage in the link step, alongside gaxios and teeny-request.

Verification

Ran the step's commands locally after pnpm install --frozen-lockfile --ignore-scripts:

@ohmayr
ohmayr requested a review from a team as a code owner October 13, 2026 11:23
@github-actions
github-actions Bot requested a review from shivanee-p October 13, 2026 11:23

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request replaces hardcoded package compilation in the lockfile check action with a dynamic script that resolves and compiles all in-repo linked workspace packages, including deep links. It introduces a Node.js helper script to list these directories by parsing workspace projects and scanning node_modules symlinks, accompanied by comprehensive unit tests. Review feedback recommends improving the robustness of the script by adding error handling and directory checks when reading the .pnpm store, and validating the JSON structure parsed from standard input to prevent runtime crashes.

Comment on lines +77 to +83
for (const name of fs.existsSync(store) ? fs.readdirSync(store) : []) {
// `.pnpm/node_modules` hoists every package for pnpm's own resolution;
// nothing in the repository loads from it.
if (name !== 'node_modules') {
nodeModulesDirs.push(path.join(store, name, 'node_modules'));
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using fs.existsSync alone does not guarantee that the path is a directory. If .pnpm exists but is a file or cannot be read, fs.readdirSync will throw an unhandled exception. Verifying that it is a directory using fs.statSync().isDirectory() and wrapping the operation in a try...catch block makes the file system operations more robust.

  let storeEntries = [];
  try {
    if (fs.statSync(store).isDirectory()) {
      storeEntries = fs.readdirSync(store);
    }
  } catch {
    // Ignore if .pnpm directory does not exist or cannot be read.
  }
  for (const name of storeEntries) {
    // `.pnpm/node_modules` hoists every package for pnpm's own resolution;
    // nothing in the repository loads from it.
    if (name !== 'node_modules') {
      nodeModulesDirs.push(path.join(store, name, 'node_modules'));
    }
  }

Comment on lines +110 to +115
process.stdin.on('end', () => {
const projectDirs = JSON.parse(input).map(project => project.path);
for (const dir of linkedWorkspaceDirs(process.cwd(), projectDirs)) {
console.log(dir);
}
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

When parsing JSON from external processes or APIs, it is important to validate that the input is not empty and that the parsed object has the expected structure (e.g., an array of objects with a path property) to prevent runtime crashes. This adheres to the general rule of validating JSON API responses before accessing their keys.

  process.stdin.on('end', () => {
    let projectDirs = [];
    try {
      if (input.trim()) {
        const parsed = JSON.parse(input);
        if (Array.isArray(parsed)) {
          projectDirs = parsed
            .filter(project => project && typeof project === 'object' && typeof project.path === 'string')
            .map(project => project.path);
        }
      }
    } catch (err) {
      console.error('Failed to parse workspace projects JSON:', err);
      process.exit(1);
    }
    for (const dir of linkedWorkspaceDirs(process.cwd(), projectDirs)) {
      console.log(dir);
    }
  });
References
  1. When parsing and iterating over JSON API responses, validate that the root structure is of the expected type (e.g., a list) and that individual elements are mappings before accessing their keys to prevent runtime crashes on unexpected or error responses.

@ohmayr
ohmayr marked this pull request as draft October 13, 2026 11:27
The "Link In-Repo Workspace Packages" step links matching in-repo
packages after the frozen install, then restores the committed
lockfile, so turbo doesn't compile them. The step compiles gaxios and
teeny-request itself.

A devDependency of @google-cloud/bigquery on
@google-cloud/bigquery-storage is satisfied by
handwritten/bigquery-storage, so the step links it, but nothing
compiles it. Requiring it then fails with MODULE_NOT_FOUND because
build/ doesn't exist. Compile it alongside gaxios and teeny-request.
@ohmayr ohmayr changed the title ci: compile every in-repo package the link step links ci: compile linked bigquery-storage in pnpm-lockfile-check Oct 13, 2026
@ohmayr
ohmayr force-pushed the ci/compile-linked-workspace-packages branch from 73edb7d to 9bfb931 Compare October 13, 2026 11:34

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant