Skip to content

zephyr-cp: don't drop console input past 128 unread bytes - #11549

Open
dhalbert wants to merge 1 commit into
adafruit:mainfrom
dhalbert:zephyr-cp-repl-long-lines
Open

dhalbert wants to merge 1 commit into
adafruit:mainfrom
dhalbert:zephyr-cp-repl-long-lines

Conversation

@dhalbert

@dhalbert dhalbert commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator

Zephyr ports were truncating long REPL lines.

Change by Claude, directed by @dhalbert.

The problem

serial_cb() in common-hal/busio/UART.c emptied the Zephyr driver's receive buffer on every callback, and discarded bytes once the 128-byte RX queue was full. Any input more than 128 bytes ahead of the reader was lost:

  • A single REPL line of 128 bytes or more lost its tail, including the newline, so the REPL waited for the rest of the line.
  • Several short lines sent at once also lost everything past 128 unread bytes.

The fix

  • When the queue is full, serial_cb() stops reading and calls uart_irq_rx_disable(). The bytes stay in the driver, and the Zephyr CDC ACM driver stops accepting packets, so the host waits.
  • read() and clear_rx_buffer() turn RX back on with a new resume_rx().

As on TinyUSB ports, a Ctrl-C queued behind a full, unread backlog is not seen until something reads the input.

busio.UART shares this code but was not tested.

Testing

On a Feather nRF52840:

  • REPL lines of 64 to 2000 characters, each sent in one write. Before, every line of 128 bytes or more stalled. Now all complete.
  • 30 short REPL lines (510 bytes) sent in one write: all run.
  • sys.stdin.read() of 1000 and 5000 bytes sent at once: data intact.
  • Ctrl-C still interrupts while True: pass.

serial_cb() drained the driver unconditionally and discarded bytes once
the 128-byte msgq was full, so REPL lines of 128 bytes or more lost
their tail and newline and the REPL waited forever.

Pause RX with uart_irq_rx_disable() instead, leaving the bytes in the
driver so USB CDC applies flow control, and resume it when read() or
clear_rx_buffer() makes room.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dhalbert

dhalbert commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator Author

Zephyr doesn't have the equivalent of tud_cdc_set_wanted_char(), which we use to check for Ctrl-C as each USB packet arrives. On zephyr-cp, Ctrl-C is only checked when serial_cb() copies bytes out of the driver. So while RX is paused, a Ctrl-C already in the driver's 1024-byte ring buffer isn't seen until the backlog ahead of it is read. TinyUSB ports only miss a Ctrl-C that is still on the host.

@dhalbert
dhalbert requested a review from tannewt October 10, 2026 22:15

@mikeysklar mikeysklar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works for long USB input, but you found the marble in the oatmeal.

  • with RX paused, Ctrl-C waits behind the backlog and never arrives.

Two inline suggestions below plus a few lines outside the diff fix it.

  • UART: the sender can't wait, so pausing only hides Ctrl-C. Inline suggestions 1 and 2 skip the pause on UART.
  • USB: the host waits, but Ctrl-C sits behind the 128-byte queue. A 1 KB USB buffer fixes it, same as TinyUSB on nRF. These boards have the memory for a larger rbuffer.

Tested on 5 Zephyr boards:

Test Boards main this PR PR + suggestions
4000-byte REPL line nRF54LM20, AT-START-F435 (USB) stalls ok ok
4000-byte paste nRF54LM20, AT-START-F435 (USB) fails ok ok
Ctrl-C, 1000 bytes unread nRF54LM20, AT-START-F435 (USB) 10/10 stuck 10/10
Ctrl-C, 200-1000 bytes unread SiWx917, ESP32-S31 (UART) 10/10 stuck 10/10
Ctrl-C in time.sleep(), 200 unread all four 5/5 stuck 5/5
new CI test below (native_sim) pass fail pass

Outside the diff, so not click-to-accept:

--- a/ports/zephyr-cp/common-hal/busio/UART.h
+++ b/ports/zephyr-cp/common-hal/busio/UART.h
     bool rx_paused;     // set by irq if no space in rbuf
+    // Set when the sender waits for unread data (USB CDC), so RX can
+    // pause on a full msgq instead of dropping bytes.
+    bool flow_control;
--- a/ports/zephyr-cp/common-hal/busio/UART.c
+++ b/ports/zephyr-cp/common-hal/busio/UART.c
@@ common_hal_busio_uart_construct_from_device
     self->rx_paused = false;
+    self->flow_control = false;
--- a/ports/zephyr-cp/common-hal/usb_cdc/Serial.c
+++ b/ports/zephyr-cp/common-hal/usb_cdc/Serial.c
     common_hal_busio_uart_construct_from_device(self, uart_device, receiver_buffer_size, receiver_buffer);
+    self->flow_control = true;
--- a/ports/zephyr-cp/supervisor/usb.c
+++ b/ports/zephyr-cp/supervisor/usb.c
-        uint8_t *receiver_buffer = port_malloc(128, true);
+        uint8_t *receiver_buffer = port_malloc(1024, true);
-            common_hal_usb_cdc_serial_construct_from_device(&usb_cdc_console_obj, DEVICE_DT_GET(DT_NODELABEL(cdc_acm_console)), 128, receiver_buffer);
+            common_hal_usb_cdc_serial_construct_from_device(&usb_cdc_console_obj, DEVICE_DT_GET(DT_NODELABEL(cdc_acm_console)), 1024, receiver_buffer);
(same for the cdc_acm_data block)

CI test for ports/zephyr-cp/tests/test_basics.py, next to test_ctrl_c_interrupts_long_sleep:

@pytest.mark.circuitpy_drive({"code.py": 'print("spinning")\nwhile True:\n    pass\n'})
@pytest.mark.duration(10)
def test_ctrl_c_after_unread_input(circuitpython):
    """Ctrl+C must still interrupt when the console queue is full of unread input."""
    circuitpython.serial.wait_for("spinning")
    circuitpython.serial.write("x" * 200)
    time.sleep(0.5)
    circuitpython.serial.write("\x03")
    circuitpython.wait_until_done()
    assert "KeyboardInterrupt" in circuitpython.serial.all_output

UART consoles still drop long input (64-byte queue). I'll send a separate PR for that.

while (uart_fifo_read(dev, &c, 1) == 1) {
/* read until FIFO empty or msgq full */
while (true) {
if (k_msgq_num_free_get(&self->msgq) == 0) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only pause where the sender can wait (USB CDC). A UART sender can't, so pausing there only hides Ctrl-C.

Suggested change
if (k_msgq_num_free_get(&self->msgq) == 0) {
if (self->flow_control && k_msgq_num_free_get(&self->msgq) == 0) {

Comment on lines +77 to +78
} else {
k_msgq_put(&self->msgq, &c, K_NO_WAIT);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UART no longer pauses, so the put can fail when full. Only wake on success.

Suggested change
} else {
k_msgq_put(&self->msgq, &c, K_NO_WAIT);
} else if (k_msgq_put(&self->msgq, &c, K_NO_WAIT) == 0) {

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants