F-Stack exits with "double free detected in tcache 2"

Hi team, I'm testing on my AWS machine with an ENS NIC. I encountered an ABORT error when stopping the app, where I want to stop the busy poll by calling ff_stop_run(). Below is my environment, test app, my compile command and the execution result. I tried to stop it in another thread but failed, then I switch to test the single thread app but still see the issue. This issue seems to happen inside the ff_unload_config.

Linux ip-10-20-88-228 6.8.0-1057-aws #60~22.04.1-Ubuntu SMP Wed May 27 08:16:59 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux
F-stack git version: 
commit 2422d12eb09a9d8b33056a8834d28f42e5706380 (HEAD -> dev, origin/dev, origin/HEAD)
Author: fengbojiang <fengbojiang@tencent.com>
Date:   Thu Jun 18 14:35:14 2026 +0800

    feat(stack-coexist): drive FIONBIO/FIOASYNC ioctl on the paired host fd for dual-stack sockets

    ff_ioctl now syncs set-direction ioctls (FIONBIO/FIOASYNC) to the paired host fd after the F-Stack call (raw Linux request, mirroring ff_fcntl); query ioctls like FIONREAD are not forwarded to avoid clobbering argp. Macro-gated, macro-off byte-identical; updates zh/en spec + three-layer docs

// test.cpp
#include <ff_api.h>

#include <atomic>
#include <chrono>
#include <iostream>
#include <thread>

std::atomic<bool> running{false};
std::atomic<int> iteration{0};

int run(void* arg) {
    // if (running) {
        auto it = iteration.fetch_add(1);
        if (it % 100'000 == 0) {
            std::cout << "iteration " << it << std::endl;
        }
        if (it == 1'000'000) {
            // running = {false};
            ff_stop_run();
        }
    // }
    return 0;
}

int main(int argc, char* argv[]) {
    // std::thread test([&](){
        ff_init(argc, argv);
        // running.store(true);
        ff_run(run, nullptr);
    // });

    std::this_thread::sleep_for(std::chrono::milliseconds(500));
    std::cout << "Finished" << std::endl;

    return 0;
}

 g++ -g -O0 -fsanitize=address -std=c++20   -I/data/f-stack/lib -I/usr/local/include   test.cpp   -Wl,--whole-archive   -L/data/f-stack/lib -lfstack $(pkg-config --libs --static libdpdk)   -Wl,--no-whole-archive   -lnuma -lpthread -lrt -lm -ldl -lcrypto -export-dynamic -lz -o test

ens6: Successed to register dpdk interface
iteration 0
iteration 100000
iteration 200000
iteration 300000
iteration 400000
iteration 500000
iteration 600000
iteration 700000
iteration 800000
iteration 900000
iteration 1000000
=================================================================
==896655==ERROR: AddressSanitizer: attempting double-free on 0x502000002030 in thread T0:
    #0 0x7ad290eb4537 in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:127
    #1 0x640a06fcb81e in ff_unload_config (/home/ubuntu/playground/test+0x72881e)
    #2 0x640a06fd1e71 in ff_dpdk_run (/home/ubuntu/playground/test+0x72ee71)
    #3 0x640a06ca05e0 in main /home/ubuntu/playground/test.cpp:30
    #4 0x7ad290029d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #5 0x7ad290029e3f in __libc_start_main_impl ../csu/libc-start.c:392
    #6 0x640a06ca0364 in _start (/home/ubuntu/playground/test+0x3fd364)

0x502000002030 is located 0 bytes inside of 8-byte region [0x502000002030,0x502000002038)
freed by thread T0 here:
    #0 0x7ad290eb4537 in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:127
    #1 0x640a06fcb81e in ff_unload_config (/home/ubuntu/playground/test+0x72881e)

previously allocated by thread T0 here:
    #0 0x7ad290e5b9a7 in __interceptor_strdup ../../../../src/libsanitizer/asan/asan_interceptors.cpp:454
    #1 0x640a06fc8dc0 in dpdk_args_setup.constprop.0 (/home/ubuntu/playground/test+0x725dc0)
    #2 0x640a06fcc332 in ff_load_config (/home/ubuntu/playground/test+0x729332)

SUMMARY: AddressSanitizer: double-free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:127 in __interceptor_free
==896655==ABORTING

Update: I recompiled the DPDK and F-Stack library in the master branch(), seems okay now, but ASAN reports some memory leaks, maybe fixed in a middle commit, I'll close it for now:

commit 0b8ed6d8bd8bdb089df1626e9ffb6ce2cab9b2a0 (HEAD -> master, origin/master)
Author: fengbojiang <fengbojiang@tencent.com>
Date:   Mon May 25 15:25:55 2026 +0800

    Fix an issue of LD_PRELOAD.

    When there are no requests, add a lock window to prevent applications such as Nginx from being starved
    to death when they attempt to initialize and acquire a lock when idle_sleep is set to 0.

ena_rx_queue_release(): Rx queue 0:0 released
ena_tx_queue_release(): Tx queue 0:0 released
Finished

=================================================================
==1131517==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 74800 byte(s) in 14 object(s) allocated from:
    #0 0x7b5fcc4b4887 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x57e2ff605041 in malloc (/home/ubuntu/playground/test+0x347041)

Direct leak of 279 byte(s) in 28 object(s) allocated from:
    #0 0x7b5fcc45b9a7 in __interceptor_strdup ../../../../src/libsanitizer/asan/asan_interceptors.cpp:454
    #1 0x57e2ff85525b in zone_alloc_sysctl (/home/ubuntu/playground/test+0x59725b)

Direct leak of 208 byte(s) in 23 object(s) allocated from:
    #0 0x7b5fcc45b9a7 in __interceptor_strdup ../../../../src/libsanitizer/asan/asan_interceptors.cpp:454
    #1 0x57e2ff85a35b in uma_startup3 (/home/ubuntu/playground/test+0x59c35b)

Direct leak of 17 byte(s) in 1 object(s) allocated from:
    #0 0x7b5fcc45b9a7 in __interceptor_strdup ../../../../src/libsanitizer/asan/asan_interceptors.cpp:454
    #1 0x57e2ff86f0fd in dpdk_args_setup.constprop.0 (/home/ubuntu/playground/test+0x5b10fd)
    #2 0x57e2ff871c05 in ff_load_config (/home/ubuntu/playground/test+0x5b3c05)

Indirect leak of 21576 byte(s) in 3 object(s) allocated from:
    #0 0x7b5fcc4b4887 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:145
    #1 0x57e2ff605041 in malloc (/home/ubuntu/playground/test+0x347041)

SUMMARY: AddressSanitizer: 96880 byte(s) leaked in 69 allocation(s).

I would like to reopen this issue because I found similar double free issue on v2.0.

v2.0: double free in `ff_unload_config()` when `ff_run()` returns (rte_eal_init aliases dpdk_argv entries)

This is still reproducible on v2.0 (6a520b3) and current dev (813b820); follow-up to #1089, which was closed because 1.25/master doesn't have ff_unload_config() yet.

Symptom

Any application whose ff_run() returns (after ff_stop_run()) aborts in the cleanup at the end of ff_dpdk_run():

free(): double free detected in tcache 2

or, depending on heap state, later inside rte_eal_cleanup() (malloc_consolidate(): invalid chunk size in rte_intr_instance_free / pci_cleanup).

Reproduce

Stock config.ini (only [port0] addresses changed), F-Stack v2.0, bundled DPDK 24.11.8, AWS ENA + igb_uio, Ubuntu 22.04, gcc 12:

#include <stdio.h>
#include "ff_api.h"
static int n;
static int loop(void *arg) { (void)arg; if (++n == 100000) ff_stop_run(); return 0; }
int main(int argc, char *argv[]) {
    if (ff_init(argc, argv) < 0) return 1;
    ff_run(loop, NULL);
    printf("ff_run returned after %d iterations\n", n);
    return 0;
}

gcc -O0 -g -fsanitize=address repro.c -o repro -I$FF_PATH/lib \
    -Wl,--whole-archive,$FF_PATH/lib/libfstack.a,--no-whole-archive \
    $(pkg-config --static --libs libdpdk) -lrt -lm -ldl -lcrypto -lz -pthread -lnuma
sudo ./repro --conf config.ini

ASan (libfstack built with DEBUG="-O0 -g"):

==39989==ERROR: AddressSanitizer: attempting double-free on 0x502000001db0 in thread T0:
    #0 in __interceptor_free
    #1 in ff_unload_config lib/ff_config.c:1880
    #2 in ff_dpdk_run lib/ff_dpdk_if.c:4166
    #3 in ff_run lib/ff_init.c:63
    #4 in main repro.c:7
freed by thread T0 here:
    #0 in __interceptor_free
    #1 in ff_unload_config lib/ff_config.c:1880
    #2 in ff_dpdk_run lib/ff_dpdk_if.c:4166
previously allocated by thread T0 here:
    #0 in __interceptor_strdup
    #1 in dpdk_args_setup lib/ff_config.c:1197      <- dpdk_argv[0] = strdup("f-stack")
    #2 in ff_load_config lib/ff_config.c:1735
    #3 in ff_init lib/ff_init.c:41

Root cause

ff_init() passes the global dpdk_argv[] straight to rte_eal_init(). DPDK's argument parser overwrites one slot with the program name at the end of parsing (lib/eal/linux/eal.c, eal_parse_args()):

if (optind >= 0)
    argv[optind-1] = prgname;

All F-Stack EAL arguments are options, so optind == argc and dpdk_argv[argc-1] now points to the same string as dpdk_argv[0]. With the default config, gdb after rte_eal_init() shows:

dpdk_argc = 4
dpdk_argv[0] = 0x555557737650 "f-stack"
dpdk_argv[1] = 0x5555577450d0 "-c1"
dpdk_argv[2] = 0x555557742780 "-n4"
dpdk_argv[3] = 0x555557737650 "f-stack"     <- was "--proc-type=auto"

ff_unload_config() (new in 2.0) frees every dpdk_argv[i], so "f-stack" is freed twice. The original dpdk_argv[argc-1] string leaks, which is the 17-byte dpdk_args_setup leak reported in #1089 on master. Any config hits this, because -c/-n are always generated.

Fix

Give EAL a scratch copy of the pointer array, so dpdk_argv[] keeps the pointers ff_unload_config() owns. I submitted a PR for review, pls let me know if my understanding is wrong or not.

#1098 has merged， thanks very much.

Navigation Menu

Repository navigation

F-Stack exits with "double free detected in tcache 2" #1089

v2.0: double free in `ff_unload_config()` when `ff_run()` returns (rte_eal_init aliases dpdk_argv entries)

Symptom

Reproduce

Root cause

Fix

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

F-Stack exits with "double free detected in tcache 2" #1089

Description

Activity

Frodocz commented on Jul 23, 2026

Frodocz commented on Oct 26, 2026

v2.0: double free in ff_unload_config() when ff_run() returns (rte_eal_init aliases dpdk_argv entries)

Symptom

Reproduce

Root cause

Fix

jfb8856606 commented on Oct 27, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

v2.0: double free in `ff_unload_config()` when `ff_run()` returns (rte_eal_init aliases dpdk_argv entries)