Generate env files from EJSON with local overrides - #580
Generate env files from EJSON with local overrides#580kieran-osgood-shopify wants to merge 3 commits into
Conversation
a2ee887 to
a02421c
Compare
1aa2ce8 to
25fe0a4
Compare
131eb0a to
eba4266
Compare
25fe0a4 to
acf432e
Compare
eba4266 to
e94d61b
Compare
9cddd46 to
16e70ad
Compare
6c4b15a to
6d1b182
Compare
16e70ad to
70b94be
Compare
6d1b182 to
f69378a
Compare
9bf0306 to
d4951e0
Compare
508a790 to
ab0ce29
Compare
d4951e0 to
0041792
Compare
ab0ce29 to
4f132b4
Compare
0041792 to
191de7a
Compare
6ffb33e to
df7a799
Compare
329ae4f to
f0bbb0f
Compare
df7a799 to
81de4d9
Compare
854b848 to
9039d2b
Compare
7238234 to
111f3ea
Compare
9039d2b to
0cfc1a6
Compare
ee190fc to
4d3d211
Compare
0cfc1a6 to
cfd107d
Compare
4d3d211 to
816fba7
Compare
setup_storefront_env took the first match for a duplicated key while run_maestro took the last. Last-wins is the predictable rule: in a file kept by hand, the last uncommented assignment is the active one. .env.local now overrides .env per key. Nothing writes to .env.local, so it survives every sync and gives a developer a place to point the sample apps at their own store. Write decisions about .env deliberately ignore the overlay. Resolved values carry overrides, so writing them back would make an override permanent. While .env.local exists, .env is left alone and the reason is reported. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
cfd107d to
e017ab2
Compare
scripts/generate_env_files decrypts config/secrets/*.ejson into .env and e2e/.env, so setup_storefront_env becomes read-only towards .env. A developer who keeps their own store in .env.local now keeps it: nothing overwrites the file that a sync used to rewrite. The two optional prompt flags go away with the writing. CI callers already supply their values through the process environment, so they only needed the flag to suppress a prompt that no longer exists. scripts/migrate_env_to_local preserves a hand-written .env as .env.local on the first `dev up` after this change. STOREFRONT_MERCHANT_IDENTIFIER leaves the shared storefront config: it belongs beside each sample app's own Apple Pay entitlement. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Assisted-By: devx/252dfd24-6c25-4bb4-8463-27702ec564eb
e017ab2 to
1c51847
Compare
| "environment": { | ||
| "STOREFRONT_DOMAIN": "EJ[1:Vsi9n4WYYvGd935C37cvRzygtUXcpBHu1CkCTFFUqzQ=:MDf5DnXyI+E2j9n1zeMMNprubZ0QpUSS:Ratyqcyc9xd5Ha14St4V0YFjKbt8DhbLK5cOC5GSbGdFyBnHuvtm5mdomVFHqCetPuulRsk=]", | ||
| "STOREFRONT_ACCESS_TOKEN": "EJ[1:Vsi9n4WYYvGd935C37cvRzygtUXcpBHu1CkCTFFUqzQ=:31/QaDjKTxu+CgWP1+Pps3/TSphwS16+:oWNA0Zg7iehiUz/kzXPI2sgBV4us6UNEtj2akQTeb3+vF0XRVYSjJfUIa6wqL2Sf]", | ||
| "STOREFRONT_DOMAIN": "EJ[1:l7BO8tiknXaR3V2OCydmWpvD6Nsd2/6ZKXWEdXagawo=:afgDjqCHl4uwBnLq0ghL3BAuVs4D2gd+:oyP/5BKL3/TM64sPlmdo5q78zCgKcxzYCqBZWNbCuDjGYPA4vdVfNKVe95dgGhyNYt8=]", |
There was a problem hiding this comment.
This sets the domain for sample app to shared staff store instead of the e2e test store
Install this buildOpen Tophat, select your target device, then click Install. Links open on the Mac running Tophat.
Checkout Kit E2E results
|
ee691bc to
aa596d1
Compare
|
|
||
| echo "migrate_env_to_local: copied .env to .env.local, because .env is now generated." | ||
| echo "migrate_env_to_local: .env.local wins over .env, so your values keep working." | ||
| echo "migrate_env_to_local: delete .env.local to follow the shared config instead." |
There was a problem hiding this comment.
We don't mean physically delete the file here do we?
If we delete the file and we want to update to the shared config, we could find ourselves in an odd scenario where we have the shared config in .env but the samples are using a stale config.
This helped me visualize:
before
.env token A
.env.local token A
sample files token A
delete .env.local
team updates shared token to B
run dev up
migration copies old .env token A into new .env.local
generator updates .env to token B
sample setup prefers .env.local token A
after
.env token B
.env.local token A
sample files token A <--- stale
Maybe this might be more clear:
“Remove entries from .env.local, but keep file, to fall back to shared config.”
There was a problem hiding this comment.
Great catch - I naively implemented it as "if you dont have a .env.local yet we'll just copy your .env over there"
I later forgot about that with this delete guidance
I think the correct fix here is to instead check for a marker in the .env to indicate whether its generated or not
e.g. I can check that this string " # Generated by scripts/generate_env_files from config/secrets/demo.ejson." is present in the file as that is setup in each generation
| # Nonempty, not plain: a blank line in .env means ejson has no value yet, so the | ||
| # process environment must still be able to supply one. Neither chain ends in a | ||
| # default, so a key that is genuinely unset still resolves to blank. | ||
| CUSTOMER_ACCOUNT_API_CLIENT_ID_VALUE="$(root_or_source_nonempty_value CUSTOMER_ACCOUNT_API_CLIENT_ID \ |
There was a problem hiding this comment.
I could be missing something but say I previously configured a Customer Account client ID, so generated sample config contains A. Then I clear CUSTOMER_ACCOUNT_API_CLIENT_ID in .env.local and rerun dev up because I want it gone. Since this only accepts a non-empty root value, won’t it fall back to A from generated config and write A back out again? Seems like I can no longer clear it. Same for shop ID. Or is there a work-around for this scenario.

Stack Context
This PR is two phased with #582 to achieve: Local and E2E runs will now always run the same configs - no drift.
#580 generate
.envfrom .ejson#582 generate
e2e/.envfrom ejson, use thee2e/.envin Bitrise, meaning we can delete all the secrets in the bitrise UI as they're sourced from ejson now, and developers can have the exact config of bitrise locallyWhy?
Relying on sharing 1password files around was slow and prone to drift
We had no solution for running the same config as CI locally
The encrypted files provide one reviewable source for shared configuration
.env.localkeeps personal overrides persistent acrossdev upruns.The generator remains separate from
setup_storefront_env, so public contributors do not need EJSON access and their hand-written.envremains untouched.What?
.envande2e/.envare now generated from committed encrypted EJSON files duringdev upscripts/setup_storefront_envread-only toward.env..env.localas an override on top of the.env(last key wins).envby migrating it to.env.local**For outside contributors the workflow remains the same, copy
.env.exampleto.env, fill it in, and run the samplesAlso noticed there was some
--skip-optional-promptsthat are stale/were removed recently so did some clean upHow to test
run:
dev upYou should now see the following changes:
your existing
.envwill be copied to.env.local.envhas been generated from the storedconfig/secrets/demo.ejsone2e/.envgenerated fromconfig/secrets/e2e.ejsonCompare the
.envand.env.local- consider whether you need the .local overrides and remove unnecessary overrides to receive same config as the rest of the team