Repository navigation
flow: FLOW_INPUT_VARIANT, a stage reads the previous stage's result from another variant - #4812
Conversation
Before, every stage after synthesis globbed $(RESULTS_DIR)/memories for *.lef and *.lib, the klayout tech took $(wildcard memories/*.lef), and macro placement, regfile dissolve and synthesis read regfiles.txt, inline.txt and regfile_cores.txt only if they existed. In a tree that kept files from an earlier run, a .lib or .lef no longer generated was read with the current ones, and a missing list was skipped. Now each step reads the <m>.lib/<m>.lef of every module blackboxes.txt lists, and the lists themselves without an existence check: a file no list names is not read, and a missing list stops the step at the read. The readers move from synth_preamble.tcl to util.tcl so OpenROAD and yosys share them, and generate_klayout_tech.py takes --blackboxes. On a clean tree the files read, and the results, are the same. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
…rom another variant Before, a variant that wanted another variant's synthesis or placement had to copy that variant's results into its own directory first. Now FLOW_INPUT_VARIANT names the variant a stage reads the previous stage's result from: INPUT_RESULTS_DIR. The six places a stage reads its input say so explicitly, in its script (load_stage_input) and in its Makefile prerequisite; every other read stays in RESULTS_DIR. It defaults to FLOW_VARIANT, which leaves every path unchanged. When it names another variant, load_stage_input stops if the input is in both variants: one invocation that ran two stages, or a copy left from an earlier run, where reading the other variant's copy would mix results. It also stops with AUTO_MEMORIES=1: the memories are read from the variant that synthesised them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
There was a problem hiding this comment.
Code Review
This pull request introduces the FLOW_INPUT_VARIANT mechanism, allowing a flow run to read inputs from a shared upstream variant directory while writing outputs to its own variant directory. This is supported by helper functions in Tcl and updated Makefile variables. However, the feedback highlights a critical issue in flow/util/utils.mk where passing only INPUT_*_DIR to genMetrics.py in the metadata-generate and update_metadata_autotuner targets will result in incorrect or incomplete metadata, as metrics for stages run in the current variant will be read from the upstream variant instead.
d426f98 to
371ea26
Compare
🔍 QoR checkMetrics reflect the PR merge build — i.e. what will land on the target branch. Commit 62 design(s) checked — 0 with regression(s), 0 without a comparable baseline. |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
variables.json is unused and The-OpenROAD-Project#4805 deletes it; editing it here only causes conflicts. variables.yaml carries the change. Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
…IANT density_fill.tcl reads 5_route.odb with load_stage_input, from INPUT_RESULTS_DIR, but its USE_FILL=0 branch copied the pair from RESULTS_DIR. In a variant forked at final, RESULTS_DIR holds no 5_route, and 6_1_fill stopped at the copy: cp: cannot stat '.../<fork>/5_route.sdc': No such file or directory The branch copies from INPUT_RESULTS_DIR, as global_place_skip_io.tcl does for GPL_PLACE_IOS=1. With FLOW_INPUT_VARIANT unset the two directories are the same and nothing changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Øyvind Harboe <oyvind.harboe@zylin.com>
Draft: waits for #4751 and #4810. This branch merges both so it can be built and tested; review only the last commit, 371ea26. It is rebased onto master once they merge.
Before: a variant that wanted another variant's synthesis or placement had to copy that variant's results into its own directory first.
Now:
FLOW_INPUT_VARIANTnames the variant a stage reads the previous stage's result from. One synthesis, floorplan or CTS is shared by many experiments, each forked off it with nothing copied:(
do-routerunning only route is #4811.)This redoes #4475, which was closed after review: it touched 13 files to resolve every read against a search path (this variant first, then the other), and every reader had to remember to use the helper. #4751 and #4810 remove what made that necessary:
.odbwith its.sdc, so a stage's input is one stem read in one place,load_design, andfind_sdc_file's ranking across two directories is gone.memories/holds.What is left is one explicit read per stage, and no search.
How
variables.mk:FLOW_INPUT_VARIANT ?= $(FLOW_VARIANT)andINPUT_RESULTS_DIR. With the default every path is what it was.load.tcl:load_stage_inputreads fromINPUT_RESULTS_DIR,load_designfromRESULTS_DIR; both areload_design_from <dir> <file>.load_stage_input(floorplan.tcl1_synth,global_place_skip_io.tcl2_floorplan,cts.tcl3_place,global_route.tcl4_cts,detail_route.tcl5_1_grt,density_fill.tcl5_route), and their Makefile prerequisites name$(INPUT_RESULTS_DIR). Every other read, a stage's own earlier steps among them, stays inRESULTS_DIR.variables.yaml/.json,FlowVariables.md: the variable, its stages and its rules.Fails loudly
When
FLOW_INPUT_VARIANTnames another variant,load_stage_inputstops:make FLOW_INPUT_VARIANT=base do-grt do-routesay, would read route's input,5_1_grt.odb, frombase, not the one grt just wrote; a copy left in the fork from an earlier run is the same ambiguity. The fork reads another variant for its first stage only, and the stages after it withFLOW_INPUT_VARIANTunset.Known limit
After a fork, a file target such as
make FLOW_VARIANT=high_effort routefollows its prerequisites back through the fork's own directory and reruns the flow from synthesis there: correct, and the cost of the whole flow.do-targets run on the files in the tree and do not. #4811 makesdo-routerun only route, as the otherdo-<stage>targets do; until then, a fork at grt runsdo-5_2_route do-5_3_fillcell do-5_routeafterdo-grt.make metadatain a fork reads the fork's ownLOG_DIRandREPORTS_DIR: it reports the stages the fork ran, and the shared stages readN/A/ERR, as for any stage that did not run; their metrics are in the input variant's metadata.copy_endpoint_slackreads the input step's endpoint-slack report fromREPORTS_DIRand skips it when it is not there, so a fork whose first step only copies its input (GPL_PLACE_IOS=1at place) has no endpoint-slack report for that step. That existence check predates this pull request.Testing
tclfmt/tclint0.7.0 clean.asap7/gcdwith ORFS's make on this branch:basethroughcts, then a fork:do-5_1_grtwithFLOW_INPUT_VARIANT=base, thendo-5_2_route do-5_3_fillcell do-5_route do-6_1_fillin the fork. Againstbaserun straight on, the.odband.sdcof5_1_grt,5_2_route,5_3_fillcell,5_routeand6_1_fillare byte-identical. The fork's directory holds only those; it readbase/4_cts.odbonce.do-5_1_grt do-5_2_routein one fork invocation, withbasealready holding5_1_grt.odb: route stops at5_1_grt.odb is in both FLOW_VARIANT=span and FLOW_INPUT_VARIANT=base. .... Without the check it would readbase's global route.4_cts.odbin the fork:do-5_1_grtstops at the same error for4_cts.odb.grt routefile targets in one fork invocation beforebasehas grt: make stops atNo rule to make target '.../base/5_1_grt.odb'.AUTO_MEMORIES=1in a fork: stops atAUTO_MEMORIES=1 reads the memories synthesised in this variant, ....No CI design sets
FLOW_INPUT_VARIANT. With the default, every CI run takesload_stage_inputwithINPUT_RESULTS_DIRequal toRESULTS_DIR; a fork exercised in CI would need a step in the CI pipeline, which lives outside this repository.🤖 Generated with Claude Code