From 877bafa3fce5372c371b4b89d818f67fe2cc0260 Mon Sep 17 00:00:00 2001 From: "Werner, Stefan" Date: Mon, 21 Sep 2026 16:49:51 +0200 Subject: [PATCH] Fix out-of-bounds write in point query instance stack The point query variant of instance_id_stack::push() guarded the instance stack capacity only with an assert, which is compiled out in release builds. It then unconditionally wrote the instance ID and both 4x4 transform matrices at context->instStackSize and always returned true. Since rtcPointQuery() recurses into nested instances and all call sites gate recursion on the return value of push(), the missing runtime check meant a scene nested deeper than RTC_MAX_INSTANCE_LEVEL_COUNT drove instStackSize past the fixed-size arrays of the caller provided RTCPointQueryContext, writing instance transform data out of bounds. With the default RTC_MAX_INSTANCE_LEVEL_COUNT of 1, an instance of an instance is enough to corrupt memory past the context; deeper nesting scales the write distance arbitrarily. Add the same runtime capacity check that the intersect/occluded variant of push() already performs: return false before any write when the stack is full. This makes point queries silently ignore instances nested deeper than RTC_MAX_INSTANCE_LEVEL_COUNT, which is the behaviour already documented for instancing and already implemented for ray queries. The existing call sites handle a false return by skipping the instance without a matching pop(), so the stack stays balanced. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- kernels/common/instance_stack.h | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/kernels/common/instance_stack.h b/kernels/common/instance_stack.h index 32b57b48a3..f4cf821c05 100644 --- a/kernels/common/instance_stack.h +++ b/kernels/common/instance_stack.h @@ -84,7 +84,14 @@ RTC_FORCEINLINE bool push(RTCPointQueryContext* context, { assert(context); const size_t stackSize = context->instStackSize; - assert(stackSize < RTC_MAX_INSTANCE_LEVEL_COUNT); + + /* We assert here because instances are silently dropped when the stack is full. + This might be quite hard to find in production. */ + const bool spaceAvailable = stackSize < RTC_MAX_INSTANCE_LEVEL_COUNT; + assert(spaceAvailable); + if (unlikely(!spaceAvailable)) + return false; + context->instID[stackSize] = instanceId; #if defined(RTC_GEOMETRY_INSTANCE_ARRAY) context->instPrimID[stackSize] = instancePrimId;