[showcase] Improve Python bytecode selection with static analysis - #4
[showcase] Improve Python bytecode selection with static analysis#4ivanmilevtues wants to merge 1 commit into
Conversation
CodeBoarding reviewStatus: 3 changed components See the full change in CodeBoarding. graph LR
n_CLI_Core_Orchestration["CLI Core #38; Orchestration"]
n_Python_Serverless_Runtime["Python Serverless Runtime"]
n_Python_Package_Distribution["Python Package Distribution"]
n_Local_Development_Execution["Local Development #38; Execution"]
n_Rust_Serverless_Runtime["Rust Serverless Runtime"]
n_Build_Utilities_Resource_Management["Build Utilities #38; Resource Management"]
n_Infrastructure_Proxy_Services["Infrastructure #38; Proxy Services"]
n_Build_Tooling_Binary_Distribution["Build Tooling #38; Binary Distribution"]
n_CLI_Core_Orchestration -- "delegates local server lifecycle management" --> n_Local_Development_Execution
n_CLI_Core_Orchestration -- "queries framework metadata for project initiali…" --> n_Infrastructure_Proxy_Services
n_Python_Serverless_Runtime -- "defines deployment artifacts via Lambda abstrac…" --> n_Build_Utilities_Resource_Management
n_Python_Package_Distribution -- "provides runtime dependencies and wheels" --> n_Python_Serverless_Runtime
n_Local_Development_Execution -- "utilizes IPC proxy for cross-language request r…" --> n_Infrastructure_Proxy_Services
n_Rust_Serverless_Runtime -- "standardizes build outputs and error reporting" --> n_Build_Utilities_Resource_Management
n_Build_Tooling_Binary_Distribution -- "provides optimized binaries for distribution" --> n_CLI_Core_Orchestration
n_CLI_Core_Orchestration -- "resolves project configuration and file maps" --> n_Build_Utilities_Resource_Management
classDef added fill:#1f883d,stroke:#0b5d23,color:#ffffff;
classDef modified fill:#bf8700,stroke:#7d4e00,color:#ffffff;
classDef deleted fill:#cf222e,stroke:#82071e,color:#ffffff,stroke-dasharray:5 3;
class n_CLI_Core_Orchestration modified;
linkStyle 0,6 stroke:#7d4e00,stroke-width:2px;
linkStyle 7 stroke:#82071e,stroke-width:2px,stroke-dasharray:5 3;
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 990c52132b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const rel = relative(root, p); | ||
| if (rel && !rel.startsWith('..') && !isAbsolute(rel)) { | ||
| keys.add(rel.split(sep).join('/')); | ||
| break; |
There was a problem hiding this comment.
When an application shadows an installed module at the same relative path (for example, both roots contain pkg/__init__.py), this conversion collapses the app and site-packages paths into the same key. Both bytecode candidates are consequently marked imported by annotateBytecodeItems(), even though the closure resolved only the app copy, so during an overflowing fill the unused vendor bytecode can consume the imported tier and displace bytecode for modules that actually run at startup. Keep the root identity in the key or match candidates by sourceAbsPath.
Useful? React with 👍 / 👎.
| } else if ((await statKind(join(root, rel), cache)) === 'dir') { | ||
| target = join(root, rel); | ||
| targetKind = 'dir'; // PEP 420 namespace package | ||
| } |
There was a problem hiding this comment.
When an earlier search root contains only a namespace-package directory but a later root contains a regular package of the same name, this branch accepts the namespace directory immediately. CPython continues searching and gives the later regular package precedence; for import foo or from foo import bar, the closure therefore omits the actual package initializer and its dependencies, causing their bytecode to fall out of the imported tier when capacity overflows. Namespace portions need to be accumulated while searching for a regular module/package rather than treated as the first-root winner.
Useful? React with 👍 / 👎.
| try { | ||
| return await Promise.race([promise, timeout]); | ||
| } finally { | ||
| clearTimeout(timer); |
There was a problem hiding this comment.
When import analysis exceeds 30 seconds because the graph is large or filesystem operations are slow, Promise.race() only stops awaiting the result; collectImportClosure() continues reading and parsing every subsequent frontier in the background. That work can keep consuming CPU and I/O, compete with the remaining bundle steps, and keep the builder process alive past the advertised timeout. Pass a deadline or AbortSignal into the closure traversal and stop scheduling batches once the timeout fires.
Useful? React with 👍 / 👎.
1 participant
Showcase context
Exact patch reproduction of vercel/vercel#17333.
This architecture-heavy change adds a Rust-backed static import graph and uses compile-time value density to choose Python bytecode under Lambda size constraints. It spans Python analysis, runtime packing, tracing, fallbacks, tests, and package changesets.
The synthetic base contains the CodeBoarding workflow with this upstream patch removed; this head reapplies the exact upstream diff.