Repository navigation
Conversation
|
Live provider verification for this change. Android Appium was repeated on
Commands used (scripts below contain the production client/interactor/acquisition/presentation calls): printf 'no\n' | avdmanager create avd -n tmp-issue-3529 -k 'system-images;android-35;google_apis;arm64-v8a' -d pixel_7
emulator -avd tmp-issue-3529 -port 5562 -no-snapshot -no-window
pnpm --dir /private/tmp/issue-3529-appium add appium appium-xcuitest-driver appium-uiautomator2-driver
# From /private/tmp/issue-3529-appium:
node node_modules/appium/build/lib/main.js --port 4739 --address 127.0.0.1 --log /private/tmp/issue-3529-appium-server.log
# From the issue worktree:
node --experimental-strip-types /private/tmp/issue-3529-live.ts ios
node --experimental-strip-types /private/tmp/issue-3529-live-final.ts android
set -a && . /Users/michal/Developer/agent-device/.env && set +a
node --experimental-strip-types /private/tmp/issue-3529-limrun.tsCleanup: all Appium sessions deleted, the late-completing first WDA allocation explicitly deleted, WDA terminated on the assigned iPhone, private Appium server stopped, private ADB server on port 5039 stopped, private AVD shut down and deleted, every private Limrun instance deleted. The T3 Safari preparation session was closed. Appium reproduction script (run from the checkout; use node --experimental-strip-types --input-type=module - ios/android with this code on stdin)import {parseWebDriverSourceFacts} from './packages/provider-webdriver/src/webdriver-source.ts';
import fs from 'node:fs/promises';
import { WebDriverClient } from './packages/provider-webdriver/src/webdriver-client.ts';
import { createWebDriverInteractor } from './packages/provider-webdriver/src/webdriver-interactor.ts';
import { createCloudWebDriverCapabilities } from './packages/provider-webdriver/src/capabilities.ts';
import { presentIosSnapshotAcquisition } from './packages/capture-kit/src/snapshot/ios-snapshot-runtime.ts';
const platform = process.argv[2] === 'ios' ? 'ios' : 'android';
const client = new WebDriverClient({ clientVersion: 'issue-3529', endpoint: 'http://127.0.0.1:4739', requestPolicy: { timeoutMs: 30_000, sessionCreateTimeoutMs: 360_000 } });
const caps = platform === 'ios' ? { platformName: 'iOS', 'appium:automationName': 'XCUITest', 'appium:udid': '1C2C9FD4-B720-4160-BAEB-7D4724D7B5F3', 'appium:bundleId': 'com.apple.mobilesafari', 'appium:noReset': true, 'appium:wdaLocalPort': 8199, 'appium:mjpegServerPort': 9199, 'appium:derivedDataPath': '/private/tmp/issue-3529-wda' } : { platformName: 'Android', 'appium:automationName': 'UiAutomator2', 'appium:udid': 'emulator-5562', 'appium:appPackage': 'com.android.settings', 'appium:appActivity': '.Settings', 'appium:noReset': true, 'appium:adbPort': 5039, 'appium:systemPort': 8299, 'appium:adbExecTimeout': 60000 };
let created=false;
try {
const session=await client.createSession(caps);
created=true;
await client.activateApp(platform === 'ios' ? 'com.apple.mobilesafari' : 'com.android.settings');
await new Promise(r=>setTimeout(r,1500));
const interactor = createWebDriverInteractor({ client, backend: platform === 'ios' ? 'xctest' : 'android', capabilities: createCloudWebDriverCapabilities({ provider: 'test', platform }), targetId: platform === 'ios' ? caps['appium:udid'] : 'emulator-5562' });
const report = async (phase: string) => {
const raw = await interactor.snapshot();
const result = 'stage' in raw ? presentIosSnapshotAcquisition(raw) : raw;
await fs.writeFile(`/private/tmp/issue-3529-${platform}-${phase}.json`, JSON.stringify(result));
console.log(JSON.stringify({ platform, phase, producer: result.producer, keyboard: result.keyboard, nodes: result.nodes?.length, fields: result.nodes?.filter(n => /textfield|edittext|searchfield/i.test(n.type ?? '')).map(n=>({type:n.type,label:n.label,identifier:n.identifier,rect:n.rect})) }));
return result;
};
let before = await report('before');
const search = before.nodes?.find(n => /search/i.test(n.label ?? n.identifier ?? '') && n.rect && n.rect.width > 0 && n.rect.height > 0);
const field = before.nodes?.find(n => /textfield|edittext|searchfield/i.test(n.type ?? '') && n.rect && n.rect.width > 0);
const target = field ?? search;
if (!target?.rect) throw new Error('No text/search target available');
await interactor.tap(target.rect.x+target.rect.width/2, target.rect.y+target.rect.height/2);
await new Promise(r=>setTimeout(r,1000));
let up = await report('keyboard-up');
const nextField=up.nodes?.find(n=>/edittext/i.test(n.type??'') && n.rect);
if(platform==='android' && nextField?.rect) { await interactor.tap(nextField.rect.x+nextField.rect.width/2,nextField.rect.y+nextField.rect.height/2); await new Promise(r=>setTimeout(r,1000)); up=await report('focused-keyboard'); await client.screenshot('/private/tmp/issue-3529-android-keyboard.png'); await fetch(`http://127.0.0.1:4739/session/${session.sessionId}/appium/settings`, {method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify({settings:{enableMultiWindows:true}})}); await fs.writeFile('/private/tmp/issue-3529-android-multiwindow.xml',await client.source()); console.log(JSON.stringify({platform,unboundedVisibility:await client.isKeyboardShown(15_000)})); }
if (platform === 'android') {
const tapNamed = async (label) => {
const facts=parseWebDriverSourceFacts(await client.source(),'android');
const node=facts.nodes.find(n=>n.label===label && n.rect);
if(!node?.rect) return false;
await interactor.tap(node.rect.x+node.rect.width/2,node.rect.y+node.rect.height/2);
await new Promise(r=>setTimeout(r,1000));return true;
};
if(await tapNamed('Open features menu')) {
if(await tapNamed('Floating')) {
await report('floating-keyboard');
await client.screenshot('/private/tmp/issue-3529-android-floating.png');
} else console.log('Floating option was not available');
}
}
await client.hideKeyboard().catch(()=>undefined);
await report('after');
} finally { if (created) await client.deleteSession(); }Limrun reproduction script (same stdin setup; credentials stay in the host environment)import fs from 'node:fs/promises';
import Limrun from './node_modules/@limrun/api/index.mjs';
import { createInstanceClient } from './node_modules/@limrun/api/ios-client.mjs';
import { captureLimrunIosSnapshot } from './packages/provider-limrun/src/ios-snapshot-adapter.ts';
import { presentIosSnapshotAcquisition } from './packages/capture-kit/src/snapshot/ios-snapshot-runtime.ts';
const api = new Limrun({ apiKey: process.env.LIMRUN_API_KEY });
let instance, client;
try {
instance = await api.iosInstances.create({ wait: true, metadata: { displayName:'issue-3529 keyboard carriage', labels:{issue:'3529'} }, spec:{} });
console.log(JSON.stringify({allocated:instance.metadata.id}));
client = await createInstanceClient({apiUrl:instance.status.apiUrl,token:instance.status.token,logLevel:'warn'});
await client.launchApp('com.apple.Preferences');
await new Promise(r=>setTimeout(r,1500));
const report = async (phase) => {
const result = presentIosSnapshotAcquisition(await captureLimrunIosSnapshot({client,instanceId:instance.metadata.id}));
console.log(JSON.stringify({phase,producer:result.producer,keyboard:result.keyboard,nodes:result.nodes.length, targets:result.nodes.filter(n=>/search|textfield/i.test((n.type??'')+' '+(n.label??''))).map(n=>({type:n.type,label:n.label,rect:n.rect}))}));
return result;
};
const before = await report('before');
const field = before.nodes.find(n=>/textfield|searchfield/i.test(n.type??'') && n.rect && n.rect.width>0);
if (field) {await client.tap(field.rect.x+field.rect.width/2,field.rect.y+field.rect.height/2);await new Promise(r=>setTimeout(r,1500)); await report('keyboard-up'); const shot=await client.screenshot(); await fs.writeFile('/private/tmp/issue-3529-limrun-keyboard.jpg',Buffer.from(shot.base64,'base64')); }
else console.log('No text field was available for keyboard-up check');
} catch(error) { console.log(JSON.stringify({error: error instanceof Error ? error.message : String(error)}));process.exitCode=1; }
finally { client?.disconnect(); if(instance){await api.iosInstances.delete(instance.metadata.id); console.log('Deleted private Limrun instance');} } |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed across 23 files
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
|
Round 2 live verification at
Commands: pnpm build
pnpm build:android
AGENT_DEVICE_STATE_DIR=/private/tmp/3593-state pnpm clean:daemon
mkdir -p /private/tmp/3593-avds /private/tmp/3593-android-home
ANDROID_AVD_HOME=/private/tmp/3593-avds ANDROID_USER_HOME=/private/tmp/3593-android-home avdmanager create avd -n review-3593 -k 'system-images;android-35;google_apis;arm64-v8a' -d pixel_7 --path /private/tmp/3593-avds/review-3593.avd < /dev/null
ANDROID_AVD_HOME=/private/tmp/3593-avds ANDROID_USER_HOME=/private/tmp/3593-android-home emulator -avd review-3593 -port 5562 -no-snapshot -no-window
APPIUM_HOME=/private/tmp/issue-3529-appium node /private/tmp/issue-3529-appium/node_modules/appium/build/lib/main.js --port 4739 --address 127.0.0.1 --log /private/tmp/3593-appium-server.log
AGENT_DEVICE_STATE_DIR=/private/tmp/3593-state node --experimental-strip-types /private/tmp/3593-live.ts ios
AGENT_DEVICE_STATE_DIR=/private/tmp/3593-state node --experimental-strip-types /private/tmp/3593-live.ts android
set -a && . /Users/michal/Developer/agent-device/.env && set +a
AGENT_DEVICE_STATE_DIR=/private/tmp/3593-state node --experimental-strip-types /private/tmp/3593-limrun.tsBoth Appium sessions returned HTTP 200 on DELETE. WDA was terminated only on the assigned simulator. Private server stopped, ADB port 5039 stopped, AVD shut down and deleted; device panels closed. No shared agent-device session was opened. Exact Appium reproduction scriptimport {parseWebDriverSourceFacts} from '/Users/michal/.t3/worktrees/agent-device/t3-f028aae4/packages/provider-webdriver/src/webdriver-source.ts';
import fs from 'node:fs/promises';
import { WebDriverClient } from '/Users/michal/.t3/worktrees/agent-device/t3-f028aae4/packages/provider-webdriver/src/webdriver-client.ts';
import { createWebDriverInteractor } from '/Users/michal/.t3/worktrees/agent-device/t3-f028aae4/packages/provider-webdriver/src/webdriver-interactor.ts';
import { createCloudWebDriverCapabilities } from '/Users/michal/.t3/worktrees/agent-device/t3-f028aae4/packages/provider-webdriver/src/capabilities.ts';
import { presentIosSnapshotAcquisition } from '/Users/michal/.t3/worktrees/agent-device/t3-f028aae4/packages/capture-kit/src/snapshot/ios-snapshot-runtime.ts';
const platform = process.argv[2] === 'ios' ? 'ios' : 'android';
const client = new WebDriverClient({ clientVersion: '3593-r2', endpoint: 'http://127.0.0.1:4739', requestPolicy: { timeoutMs: 30_000, sessionCreateTimeoutMs: 360_000 } });
const caps = platform === 'ios' ? { platformName: 'iOS', 'appium:automationName': 'XCUITest', 'appium:udid': '1C2C9FD4-B720-4160-BAEB-7D4724D7B5F3', 'appium:bundleId': 'com.apple.mobilesafari', 'appium:noReset': true, 'appium:wdaLocalPort': 8199, 'appium:mjpegServerPort': 9199, 'appium:derivedDataPath': '/private/tmp/3593-wda' } : { platformName: 'Android', 'appium:automationName': 'UiAutomator2', 'appium:udid': 'emulator-5562', 'appium:appPackage': 'com.android.settings', 'appium:appActivity': '.Settings', 'appium:noReset': true, 'appium:adbPort': 5039, 'appium:systemPort': 8299, 'appium:adbExecTimeout': 60000 };
let created=false;
try {
const session=await client.createSession(caps);
created=true;
await client.activateApp(platform === 'ios' ? 'com.apple.mobilesafari' : 'com.android.settings');
await new Promise(r=>setTimeout(r,1500));
const interactor = createWebDriverInteractor({ client, backend: platform === 'ios' ? 'xctest' : 'android', capabilities: createCloudWebDriverCapabilities({ provider: 'test', platform }), targetId: platform === 'ios' ? caps['appium:udid'] : 'emulator-5562' });
const report = async (phase: string) => {
const raw = await interactor.snapshot();
const result = 'stage' in raw ? presentIosSnapshotAcquisition(raw) : raw;
await fs.writeFile(`/private/tmp/3593-${platform}-${phase}.json`, JSON.stringify(result));
console.log(JSON.stringify({ platform, phase, producer: result.producer, keyboard: result.keyboard, nodes: result.nodes?.length, fields: result.nodes?.filter(n => /textfield|edittext|searchfield/i.test(n.type ?? '')).map(n=>({type:n.type,label:n.label,identifier:n.identifier,rect:n.rect})) }));
return result;
};
let before = await report('before');
const search = before.nodes?.find(n => /search/i.test(n.label ?? n.identifier ?? '') && n.rect && n.rect.width > 0 && n.rect.height > 0);
const field = before.nodes?.find(n => /textfield|edittext|searchfield/i.test(n.type ?? '') && n.rect && n.rect.width > 0);
const target = field ?? search;
if (!target?.rect) throw new Error('No text/search target available');
await interactor.tap(target.rect.x+target.rect.width/2, target.rect.y+target.rect.height/2);
await new Promise(r=>setTimeout(r,1000));
let up = await report('keyboard-up');
const nextField=up.nodes?.find(n=>/edittext/i.test(n.type??'') && n.rect);
if(platform==='android' && nextField?.rect) { await interactor.tap(nextField.rect.x+nextField.rect.width/2,nextField.rect.y+nextField.rect.height/2); await new Promise(r=>setTimeout(r,1000)); up=await report('focused-keyboard'); await client.screenshot('/private/tmp/3593-android-keyboard.png'); await fetch(`http://127.0.0.1:4739/session/${session.sessionId}/appium/settings`, {method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify({settings:{enableMultiWindows:true}})}); await fs.writeFile('/private/tmp/3593-android-multiwindow.xml',await client.source()); console.log(JSON.stringify({platform,unboundedVisibility:await client.isKeyboardShown(15_000)})); }
if (platform === 'android') {
const tapNamed = async (label) => {
const facts=parseWebDriverSourceFacts(await client.source(),'android');
const node=facts.nodes.find(n=>n.label===label && n.rect);
if(!node?.rect) return false;
await interactor.tap(node.rect.x+node.rect.width/2,node.rect.y+node.rect.height/2);
await new Promise(r=>setTimeout(r,1000));return true;
};
if(await tapNamed('Open features menu')) {
if(await tapNamed('Floating')) {
await report('floating-keyboard');
await client.screenshot('/private/tmp/3593-android-floating.png');
} else console.log('Floating option was not available');
}
}
await client.hideKeyboard().catch(()=>undefined);
await report('after');
} finally { if (created) await client.deleteSession(); }Exact Limrun reproduction scriptimport fs from 'node:fs/promises';
import Limrun from '/Users/michal/.t3/worktrees/agent-device/t3-f028aae4/node_modules/@limrun/api/index.mjs';
import { createInstanceClient } from '/Users/michal/.t3/worktrees/agent-device/t3-f028aae4/node_modules/@limrun/api/ios-client.mjs';
import { captureLimrunIosSnapshot } from '/Users/michal/.t3/worktrees/agent-device/t3-f028aae4/packages/provider-limrun/src/ios-snapshot-adapter.ts';
import { presentIosSnapshotAcquisition } from '/Users/michal/.t3/worktrees/agent-device/t3-f028aae4/packages/capture-kit/src/snapshot/ios-snapshot-runtime.ts';
const api = new Limrun({ apiKey: process.env.LIMRUN_API_KEY });
let instance, client;
try {
instance = await api.iosInstances.create({ wait: true, metadata: { displayName:'issue-3529 keyboard carriage', labels:{issue:'3529'} }, spec:{} });
console.log(JSON.stringify({allocated:instance.metadata.id}));
client = await createInstanceClient({apiUrl:instance.status.apiUrl,token:instance.status.token,logLevel:'warn'});
await client.launchApp('com.apple.Preferences');
await new Promise(r=>setTimeout(r,1500));
const report = async (phase) => {
const result = presentIosSnapshotAcquisition(await captureLimrunIosSnapshot({client,instanceId:instance.metadata.id}));
console.log(JSON.stringify({phase,producer:result.producer,keyboard:result.keyboard,nodes:result.nodes.length, targets:result.nodes.filter(n=>/search|textfield/i.test((n.type??'')+' '+(n.label??''))).map(n=>({type:n.type,label:n.label,rect:n.rect}))}));
return result;
};
const before = await report('before');
const field = before.nodes.find(n=>/textfield|searchfield/i.test(n.type??'') && n.rect && n.rect.width>0);
if (field) {await client.tap(field.rect.x+field.rect.width/2,field.rect.y+field.rect.height/2);await new Promise(r=>setTimeout(r,1500)); await report('keyboard-up'); const shot=await client.screenshot(); await fs.writeFile('/private/tmp/3593-limrun-keyboard.jpg',Buffer.from(shot.base64,'base64')); }
else console.log('No text field was available for keyboard-up check');
} catch(error) { console.log(JSON.stringify({error: error instanceof Error ? error.message : String(error)}));process.exitCode=1; }
finally { client?.disconnect(); if(instance){await api.iosInstances.delete(instance.metadata.id); console.log('Deleted private Limrun instance');} } |
5d537cb to
0b5e645
Compare
|
Size Report
Startup median (7 runs, lower is better):
|
|
Android Appium taps near a docked keyboard stop being refused with this change, so I can't recommend merging commit 0b5e645 yet. Android Appium acquisition calls observeProviderKeyboard with The change also alters tap decisions for existing producers, and no run shows that behavior. That covers Android Appium, and the new ownership rule in keyboard-occlusion.ts:54 for every iOS producer, the Apple runner included. The live runs used direct interactor and presenter scripts at a commit I cannot find in the repository (c13634284), so the "policy sources identical" claim is unverified. They show snapshot facts only, and none reached the daemon tap guard. Please run through the daemon CLI on this head, or on a commit in the repository. (a) Android Appium with a docked Gboard: Would it be simpler to keep the probe additive? Not blocking, take or leave: tap-keyboard-occlusion.ts:147 now treats app nodes under Checks are still running on this commit (Smoke Tests, Analyze, Compatibility & Provenance, Typecheck & Package), and none has failed yet. The diff touches typecheck and package budgets and the provider-scenario routes ( Two earlier inline threads still apply: the viewport:null thread and the marker ancestry thread, and the probe-before-tree thread holds only where acquisition has a viewport, not on Android. The lower-priority threads are addressed at this head and can be resolved: tap-keyboard-occlusion.test.ts:240 now compares against the tree-only fact, cli-help-workflows.ts:55 says "dispatched tap point", snapshots.md states the presence and absence rule, snapshot-unchanged.test.ts:85 checks both directions, record.ts:93-96 drops the trust order, and interactor-types.ts:269 covers every variant. |
This branch has not been deployed
1 participant
Summary
Carry Appium iOS/Android and Limrun keyboard observations through acquisition and projected snapshots.
snapshot --jsondistinguishes measuredvisible, policyderived(docked-tree),visibility-only, and unknownunmeasurablefacts. For example,{kind: "visibility-only", visible: true}confirms presence without claiming a frame.Derived bands require usable docking, orientation and continuous-column evidence. Missing keyboard nodes never prove absence. Android Appium supplies no display measurement on this path, so app-content bounds cannot establish a derived band. Resolvable docked tree evidence outweighs a disagreeing visibility probe. Assistant markers own only themselves; app controls beneath them remain subject to tap occlusion after projection.
Updates help/docs and fixes all nine review findings (25 files).
Closes #3529.
Validation
Commit
0b5e645372957901aac7aa1c8820ef8d21085e08.5d537cb06)pnpm check:affected --run: 1,139 files / 9,540 tests passed. CI eager-closure/test-size gates: 845 tests passed. Layering passed after commit. Ownership and unchanged-comparison mutants fail.Live on pre-rebase
c13634284(provider acquisition and keyboard-policy sources are identical at this head): assigned iOS Appium, private Android Appium AVD (including floating Gboard), and Limrun passed; commands and outputs: #3593 (comment). Safari's hideKeyboard retained its visible keyboard. Private resources cleaned up. CI: 17 passed after bounded monitoring; two smoke checks, compatibility/provenance, and typecheck/package remain pending. Cubic passed; zero unresolved threads.