[Android] Dinamically set test cases dir under app's specific storage

[IvanBM18]

Bug: b/545195031

Since Android API level 30(and all apps targeting android 11+), apps have scoped storage access, and we can only give it external storage permissions at runtime by a user facing dialog. To avoid this, we now copy the testcases to the app's external app-specific storage , which is always readable by our app's package.

We didn't noticed this issues because all test cases execute using am start which almost the 99% of the time returns no error codes, hence we though that the test case executed but that is not the case.

Example for chrome: Previously, if we tried to execute our test case, the app would tell us it can't find the file(because of the read external storage permissions):

But after this changes the html renders successfully!

Learn more:

• https://developer.android.com/about/versions/11/privacy/storage
• https://developer.android.com/training/data-storage#scoped-storage

Alternative

• The other alternative is to serve the testcases html/js/css files trough a simple localhost server trough adb reverse tcp:8000 tcp:8000, this is what we usually do in regular chrome tests, clusterfuzz already support's this, and this is controlled by the fuzzer, not by the job, if the fuzzer testcases have the 'http' word in the filename then clusterfuzz will serve them trough http, but i think nevertheless we need to make this change to keep our options open for any case we don't want to serve them trough http.

Changes

• Always pushes files to this directory /sdcard/Android/data/{PKG_NAME}/files/ which is always reable by the given app.
• Dynamically calculate the Test case directory based off the app's pkg
• no longer allows the job to override DEVICE_TESTCASES_DIR
• Now at fuzz session setup we clean from /sdcard/Android/data/{PKG_NAME}/files/*.

[IvanBM18]

Maybe i should always grab the default value here, this way jobs that already define a custom test cases dir path continue to work

[IvanBM18]

wait, but if jobs define a custom device testcase dir they will still continue to fail silently(i.e. the file is not readable by the app)

[IvanBM18]

Removed the ability to override the DEVICE_TESTCASES_DIR

[letitz]

LGTM % small changes. Thanks for sending this and sorry for the wait!

[letitz]

Meant to approve.

[decoNR]

Although I do not have much context on this, the changes are specific to Android and do not affect other parts. Additionally, as noted in the description, they have been tested. LGTM.

[AashutoshMurthy]

Issues when PKG_NAME is unset & directory cleanup

1. Malformed path (/sdcard/Android/data//files) when PKG_NAME is not set

When PKG_NAME is not set (e.g., non-APK jobs, or before an APK is identified), get_package_name() returns None and the fallback '' produces /sdcard/Android/data//files. This causes two problems:

• Host string replacement mismatch (FileNotFoundError):
  In testcase_manager.py:464 (convert_dependency_url_to_local_path), ClusterFuzz maps device paths back to the host using literal substring matching:

  remote_dir = android.app.get_testcases_directory()  # '/sdcard/Android/data//files'
  local_path = local_path.replace(remote_dir, local_inputs_dir)

  If a browser or crash logger normalizes the path in logs to a single slash (file:///sdcard/Android/data/files/test.html), "data//files" fails to match "data/files". replace() does nothing, leaving a device path on the host bot and causing a FileNotFoundError.
• Breaks Android Scoped Storage permissions:
  On device, the double slash resolves to a rogue folder named files directly under /sdcard/Android/data/ (where Android expects <package_name> directories). Under Android 11+ Scoped Storage, the FUSE daemon restricts non-root apps to their own /sdcard/Android/data/<package_name> directory. Because no app owns files, non-root apps cannot access it (EACCES), breaking the exact permission model this PR was created to support.

2. Incomplete cleanup with rm -rf /*

In device.py:clear_testcase_directory(), changing adb.remove_directory(DIR, recreate=True) to rm -rf {app.get_testcases_directory()}/* preserves the root directory and relies on shell wildcard expansion. Because * in POSIX shell does not match leading dots, any hidden files/directories (dotfiles) created during runs will be silently ignored, and orphaned folders (like /sdcard/Android/data//files) will persist across runs.

Suggested Fix

1. Fall back to a valid default directory when package_name is missing:

def get_testcases_directory():
  """Returns the testcases directory."""
  package_name = get_package_name()
  if not package_name:
    return '/sdcard/fuzzer-testcases'
  return f'/sdcard/Android/data/{package_name}/files'

2. For cleanup in clear_testcase_directory(), delete contents including hidden files (e.g., find {dir} -mindepth 1 -delete) so dotfiles are not left behind.

[IvanBM18]

Hi @AashutoshMurthy

Thanks for looking into this, regarding the issues you found...

• Sure makes sense to add the device test cases android fuzzing with no pacakges nor apks(assumed all android fuzzing used one of this at least) im sending a PR for this today.
  • I'm also adding quite a ton of debug level logs in this PR([Android] Adding debug logs #5461) which should better help the dev track when one of their android job fails for any reason not tracked previously.
  • FWIW when the PKG_NAME is missing in the job definition CF infeers it from your apk(see https://github.com/google/clusterfuzz/blob/master/src/clusterfuzz/_internal/platforms/android/app.py#L66C5-L66C21) so the only possible reason for the app package to be missing is if you dont have an apk in a android job.
• For the second issue that's a nice catch! Thanks for the suggestion!