Rust for Pythonistas

Lesson 22 of 26

Threads without a GIL

In CPython, the GIL (global interpreter lock) lets only one thread run Python code at a time. Threads still help while waiting on the network or disk, but CPU-heavy work doesn't get faster. And the GIL doesn't protect your data: count += 1 is several steps, and another thread can run in between, so you still need locks. Nothing checks that you used them.

Rust has no GIL. Threads run truly in parallel, on all cores. In exchange, the compiler checks every piece of data that more than one thread can reach. Two marker traits decide it: Send ("may move to another thread") and Sync ("may be used from several threads at once"). You rarely write them, but you see them in errors.

For reading code, four tools cover most of it:

  • thread::spawn(move || ...) starts a thread; .join() waits for it.
  • Arc<Mutex<T>> is data that many threads share and change: the thread-safe Rc<RefCell<T>> from lesson 21.
  • mpsc::channel() is a queue.Queue between threads.
  • thread::scope is like with ThreadPoolExecutor(): all its threads finish at the end.

Start threads and collect their results

Python's ThreadPoolExecutor reuses a few threads; thread::spawn starts one operating system thread per call. A Rust thread can return a value, and join() hands it back. (For many small jobs, real code often uses the rayon crate instead. When you read .par_iter(), it means "this loop runs on all cores".)

Python
from concurrent.futures import ThreadPoolExecutor


def square(n):
    return n * n


with ThreadPoolExecutor() as pool:
    results = list(pool.map(square, [1, 2, 3, 4]))
print(results)
Rust
use std::thread;

fn square(n: i32) -> i32 {
    n * n
}

fn main() {
    let mut handles = Vec::new();
    for n in [1, 2, 3, 4] {
        handles.push(thread::spawn(move || square(n)));
    }
    let results: Vec<i32> = handles.into_iter().map(|h| h.join().unwrap()).collect();
    println!("{:?}", results);
}
Output
[1, 4, 9, 16]

A shared counter: Arc<Mutex<T>>

Both programs start 4 threads that each add 1 a thousand times, under a lock. The difference is where the lock lives. In Python the lock sits next to the data, and you have to remember to use it. In Rust the data sits inside the Mutex, and lock() is the only way to reach it.

Python
import threading

count = 0
lock = threading.Lock()


def work():
    global count
    for _ in range(1000):
        with lock:
            count += 1


threads = [threading.Thread(target=work) for _ in range(4)]
for t in threads:
    t.start()
for t in threads:
    t.join()
print(count)
Rust
use std::sync::{Arc, Mutex};
use std::thread;

fn main() {
    let count = Arc::new(Mutex::new(0));
    let mut handles = Vec::new();
    for _ in 0..4 {
        let count = Arc::clone(&count);
        handles.push(thread::spawn(move || {
            for _ in 0..1000 {
                *count.lock().unwrap() += 1;
            }
        }));
    }
    for h in handles {
        h.join().unwrap();
    }
    println!("{}", count.lock().unwrap());
}
Output
4000

Rc can't cross threads: a compile error

In lesson 21, Rc<RefCell<T>> was the way to share data you can change. Try that with threads and Rust refuses to compile, because Rc is not Send: its count isn't safe to change from two threads at once. Python lets every object cross threads and leaves the rest to the GIL and to you. The fix is the thread-safe pair: Arc instead of Rc, Mutex instead of RefCell.

Python
import threading

names = []


def add(name):
    names.append(name)


threads = [threading.Thread(target=add, args=(n,)) for n in ["ada", "bob", "cy"]]
for t in threads:
    t.start()
for t in threads:
    t.join()
print(", ".join(sorted(names)))
Rust
use std::cell::RefCell;
use std::rc::Rc;
use std::thread;

fn main() {
    let names = Rc::new(RefCell::new(Vec::new()));
    let mut handles = Vec::new();
    for name in ["ada", "bob", "cy"] {
        let names = Rc::clone(&names);
        handles.push(thread::spawn(move || names.borrow_mut().push(name)));
    }
    for h in handles {
        h.join().unwrap();
    }
    let mut all = names.borrow().clone();
    all.sort();
    println!("{}", all.join(", "));
}
Output
ada, bob, cy

Scoped threads can borrow

thread::spawn needs move because the thread might outlive everything around it. thread::scope promises that it won't: every thread started inside the scope is finished when the scope ends, just like the threads of a with ThreadPoolExecutor() block. Because of that promise, scoped threads may borrow local data, with no Arc and no copy.

Python
from concurrent.futures import ThreadPoolExecutor

words = ["apple", "kiwi", "banana"]
with ThreadPoolExecutor() as pool:
    lengths = list(pool.map(len, words))
print(lengths)
print(", ".join(words))
Rust
use std::thread;

fn main() {
    let words = vec!["apple", "kiwi", "banana"];
    let lengths: Vec<usize> = thread::scope(|s| {
        let handles: Vec<_> = words.iter().map(|w| s.spawn(move || w.len())).collect();
        handles.into_iter().map(|h| h.join().unwrap()).collect()
    });
    println!("{:?}", lengths);
    println!("{}", words.join(", "));
}
Output
[5, 4, 6]
apple, kiwi, banana

Channels: queue.Queue between threads

Instead of sharing data, threads can send it to each other. A channel has a sending end and a receiving end. When a value is sent, the receiver becomes its owner, so the two threads never touch the same value. There is no need for a None at the end: when the sender is dropped, the channel closes.

Python
import queue
import threading

q = queue.Queue()


def produce():
    for n in range(1, 4):
        q.put(f"job {n}")
    q.put(None)


threading.Thread(target=produce).start()
while (item := q.get()) is not None:
    print(f"got {item}")
print("done")
Rust
use std::sync::mpsc;
use std::thread;

fn main() {
    let (tx, rx) = mpsc::channel();
    thread::spawn(move || {
        for n in 1..=3 {
            tx.send(format!("job {n}")).unwrap();
        }
    });
    for item in rx {
        println!("got {item}");
    }
    println!("done");
}
Output
got job 1
got job 2
got job 3
done

Where the Python analogy breaks

Cheat card

PythonRust
threading.Thread(target=f).start()let h = thread::spawn(move || f());
t.join()h.join().unwrap() // also gives the return value
lock = threading.Lock() with lock: count += 1let count = Mutex::new(0); *count.lock().unwrap() += 1;
# the same object in many threadslet shared = Arc::clone(&data); // Arc<Mutex<T>>
with ThreadPoolExecutor() as pool:thread::scope(|s| { s.spawn(...); });
queue.Queue()let (tx, rx) = mpsc::channel();
q.put(x)tx.send(x).unwrap();
q.get()rx.recv().unwrap() // or: for x in rx { }
# GIL: one thread runs Python at a time// threads run in parallel; data races do not compile

← Shared ownership: Rc, RefCell and Weak The crates you'll meet everywhere →