You are scanning a massive codebase or a server directory with millions of files. You run a standard ripgrep command. Instead of a list of matches, the terminal spits out a single, unhelpful line: Segmentation fault (core dumped). Or maybe it just exits silently with status code 139.
You check your command. It's a basic search. You run it on another machine, and it works fine. The difference is simple. The machine where it crashed is running a statically compiled ripgrep binary built against musl libc. The working machine is running a standard dynamically linked glibc binary.
This issue hits developers who deploy Rust tools to minimal environments like Alpine Docker containers or bare-metal scratch environments. The static binary seems like the perfect solution until a deep directory structure triggers a crash.
Let's look at what happens when ripgrep crashes. If you run the binary inside a debugger, the crash point looks something like this:
Program received signal SIGSEGV, Segmentation fault.
0x0000555555628a40 in core::fmt::write ()
(gdb) bt
#0 0x0000555555628a40 in core::fmt::write ()
#1 0x0000555555589124 in std::io::Write::write_fmt ()
#2 0x000055555559f130 in ignore::WalkState::clone ()
#3 0x000055555559f450 in ignore::dir::read_dir ()
...
#120 0x000055555559f450 in ignore::dir::read_dir ()
#121 0x00005555555a1200 in ignore::WalkParallel::run ()The backtrace shows deep recursion inside the ignore crate, which ripgrep uses to traverse filesystems in parallel. The stack frame keeps growing until it hits a wall.
If you run ulimit -a, you might see that your stack size limit is set to 8192 KB (8MB). That should be plenty of space for a few hundred directory levels. The crash happens anyway.
The issue lies in how musl libc handles thread stack sizes.
When you compile a Rust binary for the x86_64-unknown-linux-musl target, the resulting binary uses musl's thread implementation. Glibc allocates a generous default stack size for new threads, usually 8MB. Musl takes a different path. It prioritizes low memory usage and predictable resource footprints.
Historically, musl set the default stack size for threads created via pthread_create to 80KB or 128KB. While newer musl versions have increased this limit in certain configurations, statically compiled Rust binaries targeting musl often inherit these tight constraints.
Rust's standard library spawns threads using the platform's default settings. When ripgrep initializes its parallel directory walker, it spawns worker threads using the standard library. On a musl system, these worker threads get the default musl stack size.
An 80KB stack is tiny. A single stack frame in Rust can consume a few hundred bytes, especially if it contains local variables, buffers, or structs from the ignore or regex crates. If your directory structure is deep, or if the search logic requires deep recursion, the stack pointer moves past the allocated boundary. It hits the guard page, and the kernel terminates the process.
You can simulate this environment to see the crash yourself. Let's create a deeply nested directory tree.
mkdir -p test_deep
cd test_deep
for i in {1..500}; do
mkdir subdir
cd subdir
done
echo "target_string" > target.txtIf you run a glibc-linked rg "target_string" from the root of test_deep, it traverses all 500 levels, finds the file, and exits successfully.
If you run a statically compiled musl version of rg on the same directory, it will crash before it gets halfway down the tree. The stack runs out of room long before the directory traversal finishes.
You might wonder why this doesn't happen with the dynamically linked version of ripgrep on the same system.
When you run a dynamically linked binary on a glibc-based system (like Ubuntu or Debian), the binary uses glibc's dynamic loader and glibc's pthread implementation. Glibc's defaults apply.
When you run a static musl binary, the entire libc runtime is packed inside the executable. The binary does not look at the host system's libc. It uses the compiled-in musl code, which uses the tiny stack allocation strategy. Even if you run the static binary on a glibc system, it still crashes because the thread creation logic is baked into the static binary itself.
You have several options to prevent these segfaults, depending on how you build and run your tools.
The quickest fix is to tell the Rust runtime to allocate larger stacks for its threads. You can do this without recompiling the binary by setting the RUST_MIN_STACK environment variable.
export RUST_MIN_STACK=8388608
rg "target_string"This environment variable forces Rust's standard library to request a specific stack size when spawning new threads. Setting it to 8388608 (8MB) matches the glibc default.
This variable only affects threads spawned by the Rust standard library (std::thread::spawn). If a third-party library bypasses the standard library and calls pthread_create directly through raw system bindings, it might ignore this variable. Fortunately, ripgrep relies on standard Rust threading primitives, so this fix works.
If you are distributing the static binary and want to make sure users do not have to set environment variables, you can configure the linker to set a larger default stack size.
You can pass custom flags to the linker when building your Rust project. For musl targets, you can use the -Wl,-z,stack-size flag.
Add the following configuration to your .cargo/config.toml file:
[target.x86_64-unknown-linux-musl]
rustflags = ["-C", "link-arg=-Wl,-z,stack-size=2097152"]This configuration tells the linker to set the default stack size for the main thread to 2MB (2097152 bytes).
Keep in mind that this flag primarily affects the main thread. Spawning new threads might still fall back to the libc default unless the thread spawning code explicitly requests a size or honors RUST_MIN_STACK.
If you do not absolutely need a static binary, switch to a dynamically linked glibc binary.
Statically compiled musl binaries are excellent for containerized applications where you want to run a scratch image. But if you are running on a standard Linux distribution like Ubuntu, CentOS, or Arch, the dynamic glibc binary is more resilient. It handles deep recursion better because of the larger default stack limits.
If you are building Docker containers, you can switch from an Alpine-based image to a slim Debian-based image. This allows you to use the glibc binary and avoids the stack size issue.
If you are developing Rust tools that might be compiled with musl, you should design your code to avoid deep recursion.
Instead of recursive directory traversal, use an iterative approach with a work queue. The ignore crate handles this internally, but its parallel implementation still relies on recursive patterns in some of its state management.
If you must use recursion, keep your stack frames small. Avoid allocating large arrays or structs on the stack. Instead, allocate them on the heap using Box, Vec, or Arc.
Consider this example of stack-heavy code:
fn process_directory(path: &Path) {
let mut buffer = [0u8; 16384]; // 16KB on the stack
// Read and process files...
for entry in read_dir(path) {
process_directory(&entry.path()); // Recursive call
}
}On a musl thread with an 80KB stack, this function will overflow the stack after only five levels of nesting.
You can refactor it to use heap allocation:
fn process_directory(path: &Path) {
let mut buffer = vec![0u8; 16384]; // Allocated on the heap
// Read and process files...
for entry in read_dir(path) {
process_directory(&entry.path());
}
}Now, each stack frame only holds the pointer to the vector, which is 24 bytes on x86_64. This allows the recursion to go much deeper before running out of stack space.
Static compilation is highly valued in the Rust ecosystem. It makes deployment simple. You copy a single file to a server, and it runs.
But static binaries hide the differences between C runtimes. When you compile statically, you choose musl, and you accept its design decisions. Musl's focus on predictability and low resource usage makes sense for embedded systems, but it can cause unexpected failures in desktop or server applications that handle unpredictable workloads.
Knowing the limits of your target runtime helps you debug these issues quickly. The next time a static binary crashes without a trace, check the stack size.
