Introduction
When it comes to scripting and system-level programming, Bash and Rust serve very different purposes. Bash is beloved for quick automation and command chaining, while Rust is renowned for speed and safety. This post explores a practical comparison between the two: how efficiently can they match lines between two large files? We’ll walk through a test that measures execution speed and effectiveness in both environments.
Why It Matters
File comparison is a common task in system administration, data processing, and software development. The speed and accuracy of such operations can significantly impact overall system performance, especially at scale. This comparison reveals not just how much faster Rust can be, but also when it’s worth investing in a compiled language over a simple script.
Background
For a long time—probably since 2013 when I started coding Linux Kodachi—I mainly relied on Bash scripts for the backend and Gambas for the frontend. However, as I began working on version 9 of the UI, I thought: why not give some new back end languages a try?
I experimented with several options, but I was genuinely amazed by how fast Rust performed. This led me to make a pivotal decision: I’m shifting the backend of Linux Kodachi from Bash scripting to Rust. You can already see some of the components and how their speed compares to the Bash versions in the v9 behind-the-scenes progress section.
To help others experience this improvement, I’ve included a testing script you can try yourself. Rust with native APIs can save you a lot of processing time and system resources. However, it’s important to note: Rust isn’t easy to learn. It demands time and dedication—but the payoff in performance is worth it.
Key Features
- Automatically generates two large text files with controlled differences.
- Implements line-by-line comparison using both Bash and Rust.
- Times each implementation down to the millisecond.
- Reports number of matching lines and relative performance gain.
- Performs Rust source code compilation and execution within the script.
How It Works
The script initiates by validating the presence of the Rust compiler. It then generates two test files:
file1.txt
: Contains 100,000 lines in the formatLine X
.file2.txt
: Mirrorsfile1.txt
except every 10th line is replaced withChanged Line X
.
Bash Implementation:
- Opens both files using file descriptors.
- Reads and compares each line pair.
- Counts matching lines.
- Measures total execution time using
date +%s%3N
to get milliseconds.
Rust Implementation:
- Generates a Rust source file with buffered I/O for optimal performance.
- Compiles the source code using
rustc
. - Opens and reads both files with error handling.
- Compares each line using iterators and counts matches.
- Measures elapsed time using Rust’s
std::time::Instant
.
Performance Analysis:
After both implementations run, the script:
- Extracts Rust’s execution time.
- Computes and displays the speedup factor (
Bash time / Rust time
).
Quick Start
To run the comparison:
- Ensure you have Rust installed (
rustc
must be available in your PATH). - Save the script as
rust-speed.sh
. - Make it executable:
chmod +x rust-speed.sh
- Run the script:
./rust-speed.sh
The output will display:
- Matched lines by Bash
- Bash execution time
- Rust matched lines and execution time
- Relative speed improvement
Benefits
- Hands-on Benchmarking: Helps you understand performance differences in real terms.
- Automatic Test Case Generation: No need to prepare your own test data.
- Cross-Language Insight: Highlights the strengths and tradeoffs between scripting and compiled languages.
- Reproducible & Extensible: Easily modify file sizes or logic for broader comparisons.
- Kodachi Optimization Example: See real-world usage in Kodachi’s v9 backend.
Conclusion
This performance comparison is a great example of choosing the right tool for the job. Bash excels in quick, simple tasks and scripting workflows. But when performance and scalability are crucial, Rust’s compiled efficiency becomes a game changer. Whether you’re optimizing a live system like Kodachi or building a new backend from scratch, investing in Rust can pay off significantly.
That said, the transition hasn’t been easy for me. Rewriting years of Bash scripts in Rust is no small feat—it takes time, patience, and a willingness to embrace a steep learning curve. But the speed improvements and resource savings make it all worth the effort. If you’re thinking about modernizing your stack, Rust is a powerful path forward.
#!/bin/bash
# Rust vs Bash Performance Comparison Script
# ==========================================
#
# Author: Warith Al Maawali
# Copyright (c) 2025 Warith Al Maawali
# License: See /home/kodachi/LICENSE
#
# Version: 1.0.0
# Last updated: 2025-03-23
#
# Description:
# This script compares the performance of Bash and Rust for a simple file comparison task.
# It generates two large text files with slight differences, then measures how long it takes
# for both Bash and Rust implementations to compare them line by line.
#
# Usage:
# ./rust-speed.sh
#
# Output Information:
# - Number of matched lines found by each implementation
# - Execution time in milliseconds for each implementation
#
# Dependencies:
# - rustc: Rust compiler for compiling the Rust implementation
# - bash: For running the script and the Bash implementation
# - date: For timing the Bash implementation
#
# Return Values:
# The script returns 0 on success, 1 if Rust compilation fails
#
# Examples:
# ./rust-speed.sh # Run the performance comparison test
#
# Links:
# - Website: https://www.digi77.com
# - GitHub: https://github.com/WMAL
# - Discord: https://discord.gg/KEFErEx
# - LinkedIn: https://www.linkedin.com/in/warith1977
# - X (Twitter): https://x.com/warith2020
# Check for required dependencies
if ! command -v rustc >/dev/null 2>&1; then
echo "Error: rustc is required but not installed."
exit 1
fi
# Create a directory for test files if it doesn't exist
TEST_DIR="test_files"
mkdir -p "$TEST_DIR"
# Generate two test files for comparison
# file1.txt contains 100,000 lines in the format "Line X"
# file2.txt is identical to file1.txt except every 10th line is changed to "Changed Line X"
echo "Generating test files..."
>"$TEST_DIR/file1.txt"
>"$TEST_DIR/file2.txt"
for i in $(seq 1 100000); do
echo "Line $i" >>"$TEST_DIR/file1.txt"
if ((i % 10 == 0)); then
# Every 10th line in file2.txt is different
echo "Changed Line $i" >>"$TEST_DIR/file2.txt"
else
# All other lines are identical
echo "Line $i" >>"$TEST_DIR/file2.txt"
fi
done
# -------------------
# Bash Comparison
# -------------------
echo "Running Bash file comparison..."
# Record start time in milliseconds
start_bash=$(date +%s%3N)
# Initialize counter for matching lines
matched=0
# Open both files for reading using file descriptors
exec 3<"$TEST_DIR/file1.txt"
exec 4<"$TEST_DIR/file2.txt"
# Read both files line by line and compare
while true; do
# Read a line from each file
read -r line1 <&3 || break # Break loop if end of file1
read -r line2 <&4 || break # Break loop if end of file2
# Compare lines and increment counter if they match
if [ "$line1" = "$line2" ]; then
((matched++))
fi
done
# Close file descriptors
exec 3<&-
exec 4<&-
# Record end time and calculate elapsed time in milliseconds
end_bash=$(date +%s%3N)
elapsed_bash=$((end_bash - start_bash))
# Display results for Bash implementation
echo "Bash matched lines: $matched"
echo "Bash execution time: ${elapsed_bash} ms"
# -------------------
# Rust Comparison
# -------------------
echo "Creating Rust comparison program..."
# Create a Rust source file with equivalent functionality
cat <<'EOF' >"$TEST_DIR/rust_compare.rs"
use std::fs::File;
use std::io::{BufRead, BufReader};
use std::time::Instant;
use std::path::Path;
fn main() {
// Define file paths
let file1_path = Path::new("test_files/file1.txt");
let file2_path = Path::new("test_files/file2.txt");
// Start timing
let start = Instant::now();
// Open both files with proper error handling
let file1 = match File::open(&file1_path) {
Ok(file) => file,
Err(e) => {
eprintln!("Failed to open {}: {}", file1_path.display(), e);
std::process::exit(1);
}
};
let file2 = match File::open(&file2_path) {
Ok(file) => file,
Err(e) => {
eprintln!("Failed to open {}: {}", file2_path.display(), e);
std::process::exit(1);
}
};
// Create buffered readers for efficient line-by-line reading
let reader1 = BufReader::new(file1);
let reader2 = BufReader::new(file2);
// Initialize counter for matching lines
let mut matched = 0;
// Zip both readers together to process lines in parallel
for (line1_result, line2_result) in reader1.lines().zip(reader2.lines()) {
// Safely unwrap lines with error handling
let line1 = match line1_result {
Ok(line) => line,
Err(e) => {
eprintln!("Error reading from file1: {}", e);
continue;
}
};
let line2 = match line2_result {
Ok(line) => line,
Err(e) => {
eprintln!("Error reading from file2: {}", e);
continue;
}
};
// Compare lines and increment counter if they match
if line1 == line2 {
matched += 1;
}
}
// Calculate elapsed time
let duration = start.elapsed();
// Display results
println!("Rust matched lines: {}", matched);
println!("Rust execution time: {} ms", duration.as_millis());
}
EOF
# Compile the Rust program
echo "Compiling Rust program..."
rustc "$TEST_DIR/rust_compare.rs" -o "$TEST_DIR/rust_compare"
# Check if compilation was successful
if [[ $? -ne 0 ]]; then
echo "Rust compilation failed. Is rustc installed correctly?"
exit 1
fi
# Run the compiled Rust program
echo "Running Rust file comparison..."
"$TEST_DIR/rust_compare"
# Calculate and display performance difference
if [[ $elapsed_bash -gt 0 ]]; then
# Get Rust execution time from the output
rust_time=$(grep -o "[0-9]* ms" <<<"$("$TEST_DIR/rust_compare" | grep "execution time")" | grep -o "[0-9]*")
if [[ -n "$rust_time" ]]; then
speedup=$(echo "scale=2; $elapsed_bash / $rust_time" | bc)
echo -e "\nPerformance Comparison:"
echo "Rust is approximately ${speedup}x faster than Bash for this task."
fi
fi
# Cleanup
echo -e "\nCleaning up temporary files..."
# Uncomment the line below to enable cleanup
# rm -rf "$TEST_DIR"
echo "Test completed successfully."
exit 0
Bash