Jemalloc 5.4.0

Source: github.com
119 points by gkfasdfasdf 4 hours ago on hackernews | 31 comments

This release contains over 160 commits, focusing on the technical debts
cleaning including refactorings, bug fixes, test coverage improvement, and
option cleanups. The release also includes portability improvements per
upstream issues report.

New features:

  • Add EXTENT_ALLOC_FLAG_PINNED so custom extent-allocation hooks can
    mark non-reclaimable mappings, such as HugeTLB pages, for preferential
    reuse outside the decay and purge pipeline. Add the mallctl interfaces
    stats.pinned, stats.arenas.<i>.pinned,
    stats.arenas.<i>.extents.<j>.npinned,
    stats.arenas.<i>.extents.<j>.pinned_bytes, and
    stats.arenas.<i>.mutexes.extents_pinned.{counter} to report
    pinned-memory usage and mutex statistics. (@binliu19: be2de8c)
  • Allow resuming per-CPU arena selection via thread.arena.
    (@Algunenano: 68c35f6)
  • Better align the contents of human-readable and JSON malloc statistics.
    (@spredolac: 68fe1be, 30b1a41, 04aad97)
  • Replace the runtime experimental_infallible_new option with the
    compile-time --enable-cxx-infallible-new option, enabling
    compiler-level optimizations and optimization in move constructors,
    and fix the new(std::nothrow) contract (@spredolac: 160ab9d, fe33667)

Incompatible changes:

  • Adapt tcache fill and retention targets per bin to demand observed
    between GC events, replacing the fixed refill/flush policy. Remove
    seven legacy non-experimental controls: lg_tcache_nslots_mul,
    tcache_nslots_small_min, tcache_nslots_small_max,
    tcache_nslots_large, tcache_gc_delay_bytes,
    lg_tcache_flush_small_div, and lg_tcache_flush_large_div.
    Corresponding malloc_conf settings are silently ignored, matching
    opt.* mallctls return ENOENT, and tcache_ncached_max remains
    supported. (@spredolac: d13fe91)

Bug fixes:

  • Preserve errno across free, free_sized, and free_aligned_sized,
    and across process_madvise-based page purging. (@spredolac: 3a77966,
    86f0582)
  • Fix numeric overflow checks in size classes. (@spredolac: 6b24522)
  • Accept NULL in free_sized() and free_aligned_sized() (C23
    correctness). (@bigbruno: 7ce8b91)
  • Fix TSD lifecycle edge cases by 1) initializing thread-cache bins before
    marking the cache enabled, preventing reentrant bootstrap allocations
    from using uninitialized state, and 2) avoiding TSD recreation for late
    deallocations after thread teardown on generic-TSD platforms.
    (@fzakaria: 54f22c8, @spredolac: fb5499a)
  • Use O_CLOEXEC when opening the THP sysfs file in init_thp_state.
    (@ibookstein: d070554)
  • Fix duplicate opt.stats_print and opt.stats_print_opts fields in
    malloc_stats_print output. (@spredolac: dfe3a2e)
  • Fix a potential deadlock during arena_reset. (@guangli-dai: 6957341)
  • Fix a prof-sampling / guard-page interaction bug in the SAN. (@gctony:
    e36a0fa)

Optimizations and refactors:

  • Modularize jemalloc's front end by extracting arena management,
    initialization, fork orchestration, and allocation dispatch from
    jemalloc.c; untangle tcache/arena ownership; and consolidate the
    internal header graph to eliminate circular dependencies. (@spredolac:
    ba1e2fe, 8874597, 9d75722, de9ad14, ...)
  • Simplify ctl dispatch by refactoring arena helpers, internalizing an
    implementation-only control, replacing control-flow macros with typed
    helpers, and organizing ctl.c by subsystem. (@spredolac: 9e14346,
    901365d, 4e903a0, 5bc8d6e)
  • Cap the base-block growth heuristic to avoid virtual-memory exhaustion
    under rare racy conditions. (@guangli-dai: 2f4db8c)
  • Move background-thread lifecycle and state operations into the
    background-thread module, clarifying ownership independently of PAC/HPA
    callers. (@guangli-dai: f1f0792)
  • Simplify the page-allocation boundary by replacing PAI vtable dispatch
    with direct PAC/HPA calls, removing the obsolete pai_t/pai.h
    abstraction, and moving deferred-work and decay orchestration out of the
    arena. (@guangli-dai: 1dfa6f7, 8edd101, d410f43)
  • Refactor statistics collection and rendering into separate
    gather/emission stages and descriptor-driven tables. (@spredolac:
    184f304, 80c8fcb)
  • Introduce an OS abstraction layer and move platform-dependent
    file/process I/O, time, synchronization, CPU, virtual-memory, atfork,
    error-handling, profiling, thread-yield, and configuration-access
    operations out of allocator core code. (@guangli-dai: c4158ac,
    c8e2e01, ...)

Portability improvements:

  • Replace the std::__throw_bad_alloc call with standard C++ (#2900).
    (@lexprfuncall: 1a15fe3)
  • Make arena_s use a flexible array member (bin_t all_bins[]) for C99
    or newer. (@grueninger: 300b58b)
  • Fix rdtscp detection with --with-lg-vaddr. (@xinydev: e8a0d2b)
  • Fix malloc_getcpu on macOS to read the current CPU number correctly.
    (@Algunenano: 5aabbc8)
  • Use CLOCK_MONOTONIC for background-thread sleep to prevent
    clock-rollback stalls, and detect monotonic-condvar support at configure
    time. (@antonio2368: ebacec3, 8361239)
  • Fix compilation warnings on macOS. (@gctony: abb0a8a)
  • Fix thread-exit TSD cleanup on MinGW builds. (@gctony: 1e92317)
  • Fix GCC 16 build warnings by removing -Wpedantic violations in macro
    and function syntax and explicitly NUL-terminating profiling thread-name
    copies to resolve -Wstringop-truncation. (@grueninger: 5acdcee,
    c22d929, @jasonangelov: 6a245e0)
  • Parse PID-namespace symlinks without glibc-dependent strtok/atol,
    and return identifiers as uint64_t. (@guangli-dai: 278d90a)