Skip to content

Split large write buffers into chunks libuv accepts - #793

Open
jensbjorgensen wants to merge 1 commit into
MagicStack:masterfrom
jensbjorgensen:stream_write_nbufs
Open

jensbjorgensen wants to merge 1 commit into
MagicStack:masterfrom
jensbjorgensen:stream_write_nbufs

Conversation

@jensbjorgensen

Copy link
Copy Markdown
Contributor

Summary

libuv 1.52.0 added uv__check_before_write() (libuv/libuv#5014), which makes
uv_write() and uv_try_write() return UV_EINVAL when a request carries
more than 1024*1024 buffers. uvloop 0.23.0 picked that up with the libuv
1.52.1 upgrade (#753).

UVStream appends every transport.write() to self._buffer while libuv
still has pending data for the stream, and _exec_write() then hands the
whole list to libuv as a single request. Any code that issues more than 1M
write() calls in one loop iteration, or one writelines() with more than
1M items, now gets the transport closed with OSError(22, 'Invalid argument')
instead of the data being sent. uvloop 0.22.1 (libuv 1.48.0) and vanilla
asyncio, which coalesces into a bytearray, both handle this fine.

We hit this in production on a server that streams a large snapshot to a
newly connected client as a few million small write() calls from one
callback; every snapshot died with EINVAL and the client reconnect loop
pegged a CPU.

Fix

_exec_write() now hands the buffered data to libuv in chunks of at most
1024*1024 buffers, one uv_write() request per chunk, and leaves the last
chunk to the existing code path. Writes under the limit are unaffected.

Reproducer

import asyncio
import uvloop


async def main():
    async def handler(reader, writer):
        while await reader.read(1 << 20):
            pass

    server = await asyncio.start_server(handler, '127.0.0.1', 0)
    reader, writer = await asyncio.open_connection(
        *server.sockets[0].getsockname())

    writer.writelines([b'x'] * (1024 * 1024 + 1))
    await writer.drain()
    print('ok')


uvloop.run(main())

On uvloop 0.23.0 the transport is closed with OSError(22, 'Invalid argument') and drain() raises ConnectionResetError: Connection lost.
With this change it prints ok. Same result with PYTHONASYNCIODEBUG=1.

Changes

- uvloop/handles/stream.pyx, stream.pxd: add __MAX_WRITE_BUFS and
  UVStream._exec_write_chunks(), called from _exec_write() when
  len(self._buffer) exceeds the limit.
- tests/test_tcp.py: test_many_queued_writes covers both the empty-queue
  path (writelines() of 1M+1 buffers on a fresh connection) and the
  queued path (1M+1 write() calls while libuv has data pending). It fails
  on master and passes with this change, on normal and UVLOOP_DEBUG builds.

Tested on Linux with 3.14 against libuv 1.53.0.

libuv >= 1.52 rejects uv_write()/uv_try_write() requests carrying more
than 1024*1024 buffers with UV_EINVAL.  A transport that keeps receiving
write() calls while libuv still has pending data accumulates every
buffer in UVStream._buffer and flushed them in a single request, so a
burst of more than 1M small writes in one loop iteration killed the
connection with OSError(22, 'Invalid argument').

Hand the buffered writes to libuv in chunks of at most 1024*1024
buffers, one uv_write() request each.

@Vizonex Vizonex left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Workflows seem to be failing for unrelated reasons but I like where this PR is going.

@jensbjorgensen

Copy link
Copy Markdown
Contributor Author

yes I would be so grateful if we could get this through. it's an unfortunate case but it worked before, would work with the vanilla loop etc and more importantly for me it occasionally happens

@Vizonex

Vizonex commented Nov 18, 2026

Copy link
Copy Markdown

yes I would be so grateful if we could get this through. it's an unfortunate case but it worked before, would work with the vanilla loop etc and more importantly for me it occasionally happens

@jensbjorgensen agreed. Will just be patient for right now. I've been helping review these to ease everyone's burdens.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants