`malloc_normal.current` / `malloc_huge.current` drifts when a thread rarely allocates and frees other threads' memory periodically

[molnsona]

A thread that rarely allocates and often frees memory allocated by other threads subtracts the freed bytes only from its own (theap) stats.

Allocation statistics are kept per thread and are merged into the shared stats only when that thread collects (mi_collect()) or exits. A thread merges its stats only after every generic_collect allocations, never on a free. A thread that does not allocate enough doesn’t get its stats merged. mi_stats_get() therefore over-counts by everything that thread has freed since its last collect, and becomes correct again only when the thread collects or exits.

We first noticed this issue in v3.4.3. v3.5.2 with the new per-page statistics partially fixes it. A thread that has never allocated has no theap, so its frees go directly into the shared stats.

We are running services with dedicated logging threads. Worker threads write events into a queue, the logging thread reads the events, deallocates some data attached to the events and flushes the log. We compute "currently allocated bytes" of the process as mi_stats_t.malloc_normal.current + mi_stats_t.malloc_huge.current

With the logger enabled the value drifts upward. We tried comparing this with tcmalloc. With the logger disabled it behaves as expected and matches tcmalloc's generic.current_allocated_bytes on the same workload.

The logging thread generally doesn’t allocate, so its stats are never collected. Lowering generic_collect doesn’t help. This was tested on v3.4.3 but is reproducible on v3.5.2 as well.

Logger on (green: mimalloc, yellow: tcmalloc, y-axis in GB):

Logger off (green: mimalloc, yellow: tcmalloc, y-axis in GB):

Reproduction:

// cc -O2 -pthread repro.c -lmimalloc
// (default build, MI_STATS=1)
#include <mimalloc.h>
#include <mimalloc-stats.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdatomic.h>
#include <unistd.h>

mi_stats_t collect_stats() {
  mi_stats_t s;
  mi_stats_init(&s);
  mi_stats_get(&s);
  return s;
}

size_t collect_rss() {
  size_t rss = 0;
  mi_process_info(NULL, NULL, NULL, &rss, NULL, NULL, NULL, NULL);
  return rss;
}

#define BLOCK 4096
#define BATCH 1024
#define MiB 1048576.0
static void** mailbox;
static atomic_bool full, stop;
static atomic_llong live;

static void* producer(void* arg) {
  while (!stop) {
    void** b = malloc(BATCH * sizeof(void*));
    for (int i = 0; i < BATCH; i++) {
      b[i] = malloc(BLOCK);
      memset(b[i], 1, 64);
      live += mi_usable_size(b[i]);
    }
    while (full && !stop);
    mailbox = b; full = true;
  }
  return NULL;
}

static void* consumer(void* arg) {
  // remove this line to fix stats
  mi_free(mi_malloc(64));

  while (!stop) {
    while (!full) { if (stop) return NULL; }
    void** b = mailbox; full = false;
    for (int i = 0; i < BATCH; i++) {
      live -= mi_usable_size(b[i]);
      free(b[i]);
    }
    free(b);
  }

  return NULL;
}

int main(void) {
  pthread_t p, c;
  pthread_create(&p, NULL, producer, NULL);
  pthread_create(&c, NULL, consumer, NULL);

  for (int t = 1; t <= 4; t++) {
    sleep(1);

    mi_stats_t s = collect_stats();
    int64_t allocated = s.malloc_normal.current + s.malloc_huge.current;

    size_t rss = collect_rss();

    printf("t=%ds  stats = %10.1f MiB   actually live = %5.1f MiB   rss = %5.1f MiB\n",
           t, allocated / MiB, live / MiB, rss / MiB);
  }

  stop = true;
  pthread_join(p, NULL);
  pthread_join(c, NULL);
  
  mi_stats_t s = collect_stats();
  int64_t allocated = s.malloc_normal.current + s.malloc_huge.current;

  size_t rss = collect_rss();

  printf("after both threads exited: %.1f MiB (live %.1f MiB, rss %.1f MiB)\n",
         allocated / MiB, live / MiB, rss / MiB);

  return 0;
}

Example output:

t=1s  stats =    11898.4 MiB   actually live =  10.4 MiB   rss =  15.6 MiB
t=2s  stats =    24119.4 MiB   actually live =  11.2 MiB   rss =  15.6 MiB
t=3s  stats =    39981.5 MiB   actually live =  10.8 MiB   rss =  15.6 MiB
t=4s  stats =    54445.6 MiB   actually live =  10.8 MiB   rss =  15.6 MiB
after both threads exited: 8.1 MiB (live 8.0 MiB, rss 15.6 MiB)

Removing mi_free(mi_malloc(64)) causes the thread to write directly into the shared atomic stats, so the frees become visible immediately.

Environment

• mimalloc v3.5.2 (636510a), default CMake Release build (MI_STATS=1, MI_PROFILE=1, MI_DEFAULT_ALLOW_THP=2), malloc/free overridden by linking -lmimalloc.
• Debian 12 (bookworm), kernel 6.1.0-41-amd64, x86-64; gcc 12.2.0, glibc 2.36.