Repository navigation
Copy transient HTTP and handshake buffers before queuing TCP writes - #510
Conversation
mathieucarbou
left a comment
There was a problem hiding this comment.
@labodj : we won't accept an AI-generated PR fixing a maybe issue (because it does not take into account how lwip / async_tcp works behind).
Please get rid of all this AI crap, and work on a real reproductible test case like we have in our examples that can prove the issue is real and reproductible with an ESP32.
|
@willmmiles : I will let you verify - this relates to how lwip works so you'll be better at answering this PR ;-) Thanks! |
|
Thanks for pointing out the lwIP behavior. |
Arduino Core 2 is not supported anymore and same goes for ESP8266 (see: #511 (comment)) |
|
@labodj fyi @willmmiles did some versification and the issue is valid also on esp32 - we will fix it through another pr. Thanks for letting us know about it! |
ebe25a1 to
37bba2a
Compare
|
Conclusion: |
|
@copilot resolve the merge conflicts in this pull request |
be29fe7 to
7a71f47
Compare
ESP8266's ESPAsyncTCP defaults to borrowing the buffers passed to
add()andwrite(). Several response paths pass storage that is cleared, reused or destroyed before TCP has finished reading it: assembled HTTP headers, basic and buffered response bodies, and the local strings used for WebSocket and SSE handshakes. A delayed TCP read can therefore access freed memory or transmit overwritten data.Request
ASYNC_WRITE_FLAG_COPYexplicitly at those six call sites. TCP then owns the queued bytes when the response clears its header, releases a body buffer or returns from a handshake callback. Transports that already copy by default retain that behavior.The regression test executes the actual production methods with a TCP peer that delays reading until after the temporary buffer or response has been destroyed. It covers basic headers, basic bodies, abstract-response headers, abstract-response bodies, WebSocket handshakes and SSE handshakes. All six scenarios report heap-use-after-free under AddressSanitizer against unchanged
mainwith borrowing enabled; all six pass with this fix. The same scenarios also pass when the transport copies by default.Native tests pass with GCC and Clang under AddressSanitizer and UBSan. Run them with:
The included workflow runs both compilers. These tests exercise buffer ownership deterministically; they do not emulate lwIP or prove retransmission behavior on every target.
On borrowing transports, these writes now allocate a TCP-owned copy of the queued bytes.
Build verification
Local builds passed for:
d1_miniPerfTestsesp32devPerfTestsesp32devPerfTestsesp32-c3-devkitm-1PerfTestsrpipicowPerfTestsrpipico2wPerfTestsgeneric-bk7231n-qfn32-tuyaAsyncResponseStreamgeneric-rtl8710bn-2mb-788kAsyncResponseStreamESP32 Arduino 3.3.11 was built from the official git tag with the matching IDF
b774170flibraries. LibreTiny was built in an isolated Python 3.12 / PlatformIO 6.1.19 environment. These are compilation checks; no boards were flashed.