Clear ARM64 instruction cache on mprotect exec

[doanbaotrung]

Flush data cache and invalidate instruction cache using __builtin___clear_cache when sys_mprotect marks memory with LINUX_PROT_EXEC.

This prevents Apple Silicon CPUs from executing stale cached instructions when dynamic JIT compilers like Android ART generate and execute new ARM64 machine code.

---

Summary by cubic

When sys_mprotect marks memory executable, it now flushes the data cache and invalidates the instruction cache (via __builtin___clear_cache). This prevents Apple Silicon from executing stale cached instructions, so JIT-generated code like Android ART's runs correctly.

^{Written for commit fbd0003. Summary will update on new commits.}

[cubic-dev-ai]

2 issues found across 1 file

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/syscall/mem.c">

<violation number="1" location="src/syscall/mem.c:4543">
P1: The icache flush is added only in the high-VA branch (`addr >= g->guest_size`), but ordinary guest memory (text/heap/mmap below `guest_size`, which is the 64 GiB to 1 TiB primary window where ART JIT buffers live) is handled by the second `mprot_off` branch below, which never calls `__builtin___clear_cache`. For any mprotect(PROT_EXEC) at a normal guest VA, `sys_mprotect` returns without invalidating the instruction cache, so the stale-instruction bug the PR targets remains unfixed in the common case.</violation>

<violation number="2" location="src/syscall/mem.c:4544">
P2: In the high-VA branch the range passed to `__builtin___clear_cache` is computed as `g->host_base + addr`, but pages with `addr >= guest_size` are not backed at `host_base + addr`. Per the comment at mem.c:1439-1441 and `host_ptr_for_gpa` (mem.c:796), high-VA pages live in named-mapping/overflow-segment host buffers, so this flushes the wrong (primary-window) memory and leaves the actual instruction cache untouched. Resolve the host range with `host_ptr_for_gpa(g, addr)` / `host_ptr_for_gpa(g, mprot_end)` instead.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P1: The icache flush is added only in the high-VA branch (addr >= g->guest_size), but ordinary guest memory (text/heap/mmap below guest_size, which is the 64 GiB to 1 TiB primary window where ART JIT buffers live) is handled by the second mprot_off branch below, which never calls __builtin___clear_cache. For any mprotect(PROT_EXEC) at a normal guest VA, sys_mprotect returns without invalidating the instruction cache, so the stale-instruction bug the PR targets remains unfixed in the common case.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/syscall/mem.c, line 4543:

<comment>The icache flush is added only in the high-VA branch (`addr >= g->guest_size`), but ordinary guest memory (text/heap/mmap below `guest_size`, which is the 64 GiB to 1 TiB primary window where ART JIT buffers live) is handled by the second `mprot_off` branch below, which never calls `__builtin___clear_cache`. For any mprotect(PROT_EXEC) at a normal guest VA, `sys_mprotect` returns without invalidating the instruction cache, so the stale-instruction bug the PR targets remains unfixed in the common case.</comment>

<file context>
@@ -4540,6 +4540,11 @@ int64_t sys_mprotect(guest_t *g, uint64_t addr, uint64_t length, int prot)
                     return -LINUX_ENOMEM;
             }
             guest_region_set_prot(g, addr, mprot_end, prot);
+            if (prot & LINUX_PROT_EXEC) {
+                char *host_start = (char *) g->host_base + addr;
+                char *host_end = (char *) g->host_base + mprot_end;
</file context>

[cubic-dev-ai]

P2: In the high-VA branch the range passed to __builtin___clear_cache is computed as g->host_base + addr, but pages with addr >= guest_size are not backed at host_base + addr. Per the comment at mem.c:1439-1441 and host_ptr_for_gpa (mem.c:796), high-VA pages live in named-mapping/overflow-segment host buffers, so this flushes the wrong (primary-window) memory and leaves the actual instruction cache untouched. Resolve the host range with host_ptr_for_gpa(g, addr) / host_ptr_for_gpa(g, mprot_end) instead.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At src/syscall/mem.c, line 4544:

<comment>In the high-VA branch the range passed to `__builtin___clear_cache` is computed as `g->host_base + addr`, but pages with `addr >= guest_size` are not backed at `host_base + addr`. Per the comment at mem.c:1439-1441 and `host_ptr_for_gpa` (mem.c:796), high-VA pages live in named-mapping/overflow-segment host buffers, so this flushes the wrong (primary-window) memory and leaves the actual instruction cache untouched. Resolve the host range with `host_ptr_for_gpa(g, addr)` / `host_ptr_for_gpa(g, mprot_end)` instead.</comment>

<file context>
@@ -4540,6 +4540,11 @@ int64_t sys_mprotect(guest_t *g, uint64_t addr, uint64_t length, int prot)
             }
             guest_region_set_prot(g, addr, mprot_end, prot);
+            if (prot & LINUX_PROT_EXEC) {
+                char *host_start = (char *) g->host_base + addr;
+                char *host_end = (char *) g->host_base + mprot_end;
+                __builtin___clear_cache(host_start, host_end);
</file context>

[jserv]

Provide an MRE (Minimal reproducible example) demonstrating an mprotect execution issue caused by self-modifying code.