Fast Codebase Inspection and Structural Analysis with Rust Glancer

Opening a new Rust codebase can be a slow, heavy experience. If the project is large, your editor might stall for minutes while the language server indexes the code. It compiles build scripts, downloads dependencies, and drives up CPU temperatures. Running local documentation tools can take just as long because they require a full compilation pass.

Sometimes you do not need a full compiler. You do not need type checking, borrow checking, or macro expansion. You just want to see the layout of the code. You want to know what structs exist, what functions they expose, and how the modules are organized.

We can build a lightweight tool to do this. Let's call it Glancer. By avoiding compilation and focusing purely on syntax tree traversal, we can map out a massive Rust project in milliseconds.

Why Grep and LSPs Fall Short

Text search tools are fast, but they lack syntax awareness. If you search for a struct name, you get every file where that struct is instantiated, imported, or mentioned in a comment. Writing regex patterns to find only the definitions is fragile and breaks on multi-line formatting.

Language servers are syntax-aware, but they are designed for deep, stateful analysis. They build index databases, track type information across crates, and monitor file changes in real time. This is perfect when you are writing code, but it is overkill when you are just exploring.

An AST-based structural analyzer sits in the middle. It parses the source code into an Abstract Syntax Tree, extracts the definitions, and discards the rest. It is fast because it treats each file as an independent unit of text. This makes it an ideal foundation for building custom developer tools that need to run instantly.

The Core Architecture

To build Glancer, we need three core components:

1. A fast directory walker that respects project configurations.
2. A syntax parser that can turn Rust source code into a structured tree.
3. A parallel execution queue to process files concurrently.

We will use the ignore crate for directory walking. It handles .gitignore rules automatically (though you might want to watch out for issues like ripgrep musl segfaults on large searches), so we do not waste time parsing build artifacts in the target directory. For parsing, we will use the syn crate. For parallel processing, we will use rayon.

Reading the Filesystem

First, we need to find all the Rust files in the project. We want to skip hidden files, ignored directories, and non-Rust files. The ignore crate makes this straightforward.

Here is how we set up the file walker:

use ignore::WalkBuilder;
use std::path::PathBuf;
 
fn find_rust_files(root: &str) -> Vec<PathBuf> {
    let mut files = Vec::new();
    let walker = WalkBuilder::new(root)
        .standard_filters(true)
        .build();
 
    for entry in walker {
        if let Ok(entry) = entry {
            let path = entry.path();
            if path.is_file() && path.extension().map_or(false, |ext| ext == "rs") {
                files.push(path.to_path_buf());
            }
        }
    }
    files
}

This function returns a list of paths to Rust files. It runs quickly because it skips directories like target and .git by default.

Parsing Rust Syntax

Once we have the files, we need to extract their structural elements. We do not need to parse function bodies or variable assignments. We only care about item declarations: structs, enums, traits, and impl blocks.

The syn crate provides a visitor pattern through the syn::visit::Visit trait. We can define a struct that implements this trait and records the items we care about.

use syn::visit::Visit;
use syn::{ItemStruct, ItemImpl, ItemTrait, ItemFn};
 
struct StructuralVisitor {
    structs: Vec<String>,
    impls: Vec<String>,
    traits: Vec<String>,
    functions: Vec<String>,
}
 
impl<'ast> Visit<'ast> for StructuralVisitor {
    fn visit_item_struct(&mut self, node: &'ast ItemStruct) {
        let name = node.ident.to_string();
        let visibility = format!("{:?}", node.vis);
        self.structs.push(format!("struct {} (vis: {})", name, visibility));
        syn::visit::visit_item_struct(self, node);
    }
 
    fn visit_item_impl(&mut self, node: &'ast ItemImpl) {
        let self_ty = &node.self_ty;
        let self_ty_str = quote::quote!(#self_ty).to_string();
        
        if let Some((_, ref trait_path, _)) = node.trait_ {
            let trait_name = quote::quote!(#trait_path).to_string();
            self.impls.push(format!("impl {} for {}", trait_name, self_ty_str));
        } else {
            self.impls.push(format!("impl {}", self_ty_str));
        }
        syn::visit::visit_item_impl(self, node);
    }
 
    fn visit_item_trait(&mut self, node: &'ast ItemTrait) {
        let name = node.ident.to_string();
        self.traits.push(format!("trait {}", name));
        syn::visit::visit_item_trait(self, node);
    }
 
    fn visit_item_fn(&mut self, node: &'ast ItemFn) {
        let name = node.sig.ident.to_string();
        self.functions.push(format!("fn {}()", name));
        syn::visit::visit_item_fn(self, node);
    }
}

By overriding these methods, we collect structural metadata while ignoring the logic inside the functions. This keeps memory usage low and processing fast. This approach helps maintain small functions and readable code by making the structure obvious.

Parallelizing the Parse Pipeline

Parsing files is a CPU-bound task. Since each file can be parsed independently, we can use rayon to distribute the work across all available CPU cores.

We will read the contents of each file, parse it into an AST, and run our visitor.

use rayon::prelude::*;
use std::fs;
 
struct FileSummary {
    path: PathBuf,
    structs: Vec<String>,
    impls: Vec<String>,
    traits: Vec<String>,
    functions: Vec<String>,
}
 
fn analyze_files(files: &[PathBuf]) -> Vec<FileSummary> {
    files
        .par_iter()
        .filter_map(|path| {
            let content = fs::read_to_string(path).ok()?;
            let syntax_tree = syn::parse_file(&content).ok()?;
            
            let mut visitor = StructuralVisitor {
                structs: Vec::new(),
                impls: Vec::new(),
                traits: Vec::new(),
                functions: Vec::new(),
            };
            
            visitor.visit_file(&syntax_tree);
            
            Some(FileSummary {
                path: path.clone(),
                structs: visitor.structs,
                impls: visitor.impls,
                traits: visitor.traits,
                functions: visitor.functions,
            })
        })
        .collect()
}

Using par_iter instead of a standard loop can reduce the execution time by a factor of four or five on modern multi-core processors.

Optimizing Syn Features

The syn crate is large. By default, it compiles with features that we do not need for structural analysis. To keep our build times fast and our binary small, we should configure the features in our Cargo.toml file.

We only need the parsing and printing features. We do not need the full Rust evaluation suite.

[dependencies]
syn = { version = "2.0", features = ["clone-impls", "extra-traits", "parsing", "printing", "visit"] }
quote = "1.0"
ignore = "0.4"
rayon = "1.8"

By disabling the full feature of syn, we reduce the compile time of our tool. The parser runs faster because it does not have to build complex structures for every statement inside function bodies.

Handling Macro Expansion and Conditional Compilation

A pure AST parser has limitations. It does not run the compiler's preprocessor. This means two things:

1. It cannot expand macros.
2. It cannot resolve conditional compilation attributes like #[cfg(target_os = "windows")].

If a struct is defined inside a macro invocation, our visitor will see the macro call, but not the struct definition. For example, if a library uses a macro to generate error types, Glancer will only see the macro name.

But we can handle this by tracking macro calls. We can add a visitor method for macros:

impl<'ast> Visit<'ast> for StructuralVisitor {
    fn visit_macro(&mut self, node: &'ast syn::Macro) {
        let path = quote::quote!(#node.path).to_string();
        self.functions.push(format!("macro! {}", path));
        syn::visit::visit_macro(self, node);
    }
}

This gives us a hint that code generation is happening at that location, even if we do not expand the macro. For a fast inspection tool, this is a reasonable trade-off. If we wanted full macro expansion, we would need to invoke the compiler, which defeats the purpose of a fast, compiler-free inspection.

Formatting the Output for Humans

Once we have the structural summaries, we need to present them in a readable format. A flat list of structs and functions can be overwhelming. We should group them by file and module path. Improving readability is the main goal of structural analysis.

We can print the output as a tree structure in the terminal:

src/main.rs
├── struct Config
├── impl Config
│   └── fn load()
└── fn main()

src/parser.rs
├── trait Parse
├── struct Parser
└── impl Parse for Parser

This layout improves readability when scanning a new project. While variable naming best practices keep code clean, you first have to find where it lives.

Performance Comparisons

How fast is this approach? On a medium-sized project with 50,000 lines of Rust code, Glancer can walk the directory, parse all files, and print the structural tree in less than 50 milliseconds.

On the same codebase, running cargo doc can take 10 to 15 seconds, depending on whether the dependencies are cached. Starting a language server and waiting for it to index can take 5 to 10 seconds.

This speed difference changes how you explore code. Instead of waiting for your editor to load, you can run a quick query in your terminal, find the struct you need, and open the specific file.

Alternative Parsers: Tree-Sitter

We used the syn crate because it is the standard tool for parsing Rust code within the Rust ecosystem. But it has one drawback: it expects valid Rust code. If a file has a syntax error, syn::parse_file will fail and return an error.

If you want to inspect codebases that are in a transient, broken state, you might consider tree-sitter-rust. Tree-sitter is an incremental parsing library. It is designed to handle invalid syntax gracefully. If there is an error on line 50, it can still parse the rest of the file.

But Tree-sitter requires C bindings and is more complex to set up in a pure Rust project. For most use cases, syn is fast enough and easier to maintain.

Integrating with Terminal Workflows

Because Glancer outputs structured text, you can integrate it with other command-line tools. For example, you can pipe the output into fzf to create an interactive symbol finder:

glancer | fzf -preview 'git grep -n {2}'

This command lets you search through all structs and functions in your project and jump directly to their definitions. Many developer tools focus on deep analysis, but simple utilities that integrate with unix pipelines often provide the best workflow improvements.

By keeping the tool simple, fast, and focused on a single task, we can make codebase exploration much smoother. You do not always need a full compiler to understand your code. Sometimes, a quick glance is all you need.