Closures in Rust

Today, I want to note down my thoughts on closures. Closures are important in Rust, because they are extensively used in iterator adapters paramount in development highly performant programs. However, to my point of view this topic is not well-covered in The Book. This may be a reason why it is considered among the most difficult parts of the language. In this post, I will try to shed more light on it, hopefully making it more clear to Rust learners. Note that I am still a novice to the language, and my understanding may not be fully correct.

Closure Definition

According to The Book, closure is an anonymous function that can capture its environment. There are two things that have to be highlighted in this definition:

• A closure can be imagined like a function;
• Contrary to a function, it can capture its environment (capturing the environment means that in a closure you can use the variables defined outside the closure body but accessible in its scope).

The Rust Reference explains the closure concepts and their connection with the Rust language components:

A closure expression produces a closure value with a unique, anonymous type that cannot be written out.

Let’s consider a simple example of a closure:

fn main() {
    let count = 0;
    let print_count_closure = || println!("Count value: {}", count);
}

In this example, the closure expression || println!("Count value: {}", count) produces a closure value stored in the variable print_count_closure that has a unique, anonymous type. This closure captures the variable count. What we usually understand under the term closure is what is stored in the print_count_closure variable in the example.

As we have already mentioned, closures are similar to function, therefore in order to use them we need to call them. In the simplest case (when closure does not depend on parameters), we just add parenthesis after the closure variable in order to execute it:

fn main() {
    let count = 0;
    let print_count_closure = || println!("Count value: {}", count);
    print_count_closure();
}

It is possible to pass arguments to a closure. In order to do this, just add arguments names inside the bars ||. For instance, in the example below the closure accepts two arguments name and age. Note, we are not obliged to specify the types of the arguments because Rust try to deduce them considering the types of the values passed as parameters to the closure. However, if a closure is called with parameters of different types, a compile-time error will be generated (see the commented call of the closure).

fn main() {
    let intro = String::from("Participant age");
    let print_user_age = |name, age| println!("{}\n\t{}: {}", intro, name, age);

    for (name, age) in [
        (String::from("Alice"), 5),
        (String::from("Bob"), 7),
        (String::from("Mallory"), 20),
    ]
    .iter()
    {
        print_user_age(name, age);
    }

    //print_user_age(String::from("Eve"), "infinity"); //error: mismatched types
}

Variable Capture Modes

The main difference between closures and functions is that the formers can capture variables, meaning they can use in their computation the values of the variables accessible in the same scope. Contrary to functions that gets their input from arguments, which access mode is defined explicitly (immutable borrow, mutable borrow, or move), closures capture variables implicitly. In order to explain the connection between capture modes and their influence on closed over variables, I have drawn the following diagram (Figure 1) (I think this is the main contribution of the article). Let’s consider its upper part.

Capture by Immutable Borrow

Let’s consider the following code:

fn main() {
    let immut_val = String::from("immut");
    let fn_closure = || {
        println!("Len: {}", immut_val.len());
    };
}

Here, we create the variable immut_val of type String containing the value "immut". Then, we declare a closure that prints the length of this string. In order to understand how to capture immut_val variable, the Rust compiler performs the analysis of the closure body. As we can see, in the closure body we call len() method on the captured variable. If we check the signature of this method fn len(&self) -> usize, we can see that this method does not modify the value (it borrows immutably the variable). So as the value is not modified, the fn_closure closure captures the immut_val variable by immutable borrow. This means that we can have other immutable references to the variable simultaneously with the closure, e.g.:

fn main() {
    let immut_val = String::from("immut");
    let fn_closure = || {
        println!("Len: {}", immut_val.len());
    };

    println!("Value: {}", immut_val); //ok
    fn_closure();                     //ok

    // cannot borrow mutably because it is already borrowed immutably
    // immut_val.push_str("-push");   
    // fn_closure();
}

Capture by Mutable Borrow

In the following example, we create the mutable mut_val variable of type String with the value "mut". Then, we create a closure that appends to our string the "-new" substring.

fn main() {
    let mut mut_val = String::from("mut");
    let mut fnmut_closure = || {
        mut_val.push_str("-new");
    };
}

You can note several differences from the previous example. The first one is clear: in Rust if you change the value of a variable, you need to declare it as mutable. However, the second is not that obvious: you have to declare a closure variable as mutable if the closure modifies the environment.

If we consider the body of the closure, we can see that the push_str() method is called over the captured mut_val variable. This method has the signature fn push_str(&mut self, string: &str), the &mut self part means that it is used to modify the value (the variable is mutably borrowed by this method). So as the value is modified, the closure captures the variable by mutable borrow (sometimes, mutable borrow is also called as unique mutable borrow). Therefore, we cannot have simultaneously other references to the variable:

fn main() {
    let mut mut_val = String::from("mut");
    let mut fnmut_closure = || {
        mut_val.push_str("-new");
    };

    // cannot borrow immutable because the variable is borrowed as mutable
    // println!("{}", mut_val);
    
    // cannot borrow mutably second time
    // mut_val.push_str("another_string");
    
    fnmut_closure();

    // ok because closure is already dropped 
    println!("{}", mut_val); 
}

Capture by Move

In the following example, a String variable called mov_val is created. Then, in the closure we use a hack to move a value: we assign the value of the variable mov_val to a new variable moved_value.

fn main() {
    let mov_val = String::from("value");
    let fnonce_closure = || {
        let moved_value = mov_val;
    };
}

We know, in Rust assigning a non-copy type variable to a new variable moves the value the new variable (changes the owner of the value). Therefore, in the closure body we move the ownership of the closed over variable. Thus, the closure captures the variable by move. So as the variable is moved, we cannot use it anywhere else:

fn main() {
    let mov_val = String::from("value");
    let fnonce_closure = || {
        let moved_value = mov_val;
    };
    
    fnonce_closure(); // ok
    
    // cannot print it because it is captured in the closure
    // borrow of moved value
    // println!("{}", mov_val);
    // cannot call closure the second time
    // fnonce_closure();
}

Capture by Unique Immutable Borrow

There is one particular case not considered on the diagram, also usually not mentioned in books and articles: capture by unique immutable borrow. I discovered this case when I was reading the Rust Reference chapter on closures. It arises when you capture an immutable reference to a mutable variable and use it to modify the referenced value. For instance, let’s consider the following example:

fn main() {
    let mut s = String::from("hello");
    let x = &mut s;
    
    let mut mut_closure = || {
        (*x).push_str(" world");
    };
}

Here, the closure captures the immutable variable x, a reference to a mutable String variable. The closure does not modify the reference value, thus the closure should capture x by immutable borrow. Therefore, we should be able to have other references to this variable simultaneously. However, this is not true as, e.g., in the following example that compiles with the error:

fn main() {
    let mut s = String::from("hello");
    let x = &mut s;
    
    let mut mut_closure = || {
        (*x).push_str(" world");
    };

    // cannot borrow `x` as immutable because previous closure requires unique access
    println!("{:?}", x); // error happens here
    mut_closure();
}

The reason is that in this case the variable is captured by unique immutable borrow. As far as I understand, this is quite a rare case, and the compiler is good at saying what is the problem, therefore this case is usually omitted (the diagram also does not pick out it separately). However, for a complete cover of the topic, I discuss it in this section.

Joint Capturing

Currently, any variable named in a closure will be fully captured. This creates some inconveniences if you capture variables of complex types, e.g., structs. Let’s consider the following artificial example:

struct Person {
    first_name: String,
    last_name: String,
}

fn main() {
    let mut alice_wonderland = Person {
        first_name: String::from("Alice"),
        last_name: String::from("Wonder"),
    };
    let print_first_name = || println!("First name: {}", alice_wonderland.first_name);
    alice_wonderland.last_name.push_str("land");
    print_first_name();
}

In this example, we create a struct called Person that consists of two fields: first_name and second_name. Then, in the main function, we create an instance of this struct, alice_wonderland. We define a closure that prints the first name, and then we modify the value stored in the last_name field of the instance. Although the closure and modification statement deal with different fields of the Person instance, this code generates the error:

error[E0502]: cannot borrow `alice_wonderland.last_name` as mutable because it is also borrowed as immutable
  --> src/bin/disjoin_fields.rs:12:5
   |
11 |     let print_first_name = || println!("First name: {}", alice_wonderland.first_name);
   |                            --                            ---------------- first borrow occurs due to use of `alice_wonderland` in closure
   |                            |
   |                            immutable borrow occurs here
12 |     alice_wonderland.last_name.push_str("land");
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ mutable borrow occurs here
13 |     print_first_name();
   |     ---------------- immutable borrow later used here

error: aborting due to previous error

This error arises because the closure, despite dealing with only one field of the struct, borrows immutably the whole struct instance, thus, you cannot modify the other field of the structure simultaneously.

This creates inconsistencies with current borrow and move rules. There is an RFC proposal on capturing disjoint fields separately. Hopefully, soon it will be implemented in Rust (most probably, in a new edition of the language).

Calling Closures

If you use closures only by calling them directly (by adding parenthesis with the argument values), this does not give you a lot of benefits. That is similar to simple inlining of the closure body instead of the corresponding calls.

The true power of closures is revealed when you pass closures as arguments to other functions (usually called as higher-order functions), where they are called. For instance, closures can be used as callbacks or as processors of elements in iterator adapters. In Rust, in order to define a function we need either to know the types of function arguments or specify their trait bounds.

As we know, a closure variable has a unique, anonymous type that cannot be written out. To exemplify this for instance, let’s consider the following example assigning the fake type to our closure variable:

fn main() {
    let count = 0;
    let print_count_closure: i32 = || println!("Count value: {}", count);
    print_count_closure();
}

If we try to build this program, the compiler will generate the following error:

error[E0308]: mismatched types
 --> src/bin/simple_closure_call.rs:3:36
  |
3 |     let print_count_closure: i32 = || println!("Count value: {}", count);
  |                              ---   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected `i32`, found closure
  |                              |
  |                              expected due to this
  |
  = note: expected type `i32`
          found closure `[closure@src/bin/simple_closure_call.rs:3:36: 3:73]`

As you can see, the compiler expects a strange type [closure@src/bin/simple_closure_call.rs:3:36: 3:73] for our print_count_closure variable. However, even if you add this type to the example the compiler will still complain.

Similarly, we cannot specify the type of closure argument in a function definition. Thus, the only way to define that a function should accept a closure as an argument is through trait bounds. Rust provides three different traits Fn, FnMut, and FnOnce that can be used as trait bounds for closure arguments. Each closure implements one of these three traits, and what trait is automatically implemented depends on how the closure captures its environment.

Our diagram in Figure 1 explains the connection between capture modes and what traits are automatically implemented for a closure. By words, this can be expressed with the following set of rules:

• Non-capturing closures (closures that do not capture variables from its environment) or closures that capture variables only by immutable borrow, implements automatically the Fn trait.
• If all variables are captured by mutable and immutable borrows, then the closure implements automatically the FnMut trait.
• If at least one variable is captured by move, then the closure implements automatically the FnOnce trait.

Moreover, on the diagram you can see the column (:) sign between the closure traits. These columns show the supertrait relation between the corresponding traits. For instance, FnOnce is a supertrait for FnMut. In its turn, FnMut can be considered as a supertrait for the Fn trait. This means that if you have a function that impose FnOnce trait bound on a parameter, you can pass to it FnMut and Fn closures as well. Indeed, if your function expects a closure that can be called only once (FnOnce trait bound), then it is clear that you can pass there a closure that does not modify the environment (Fn) or that modifies the environment (FnMut) because they will be called only once.

Let’s consider the following example. Let’s define the functions call_fn, call_fn_mut and call_fn_once that accepts one generic parameter with a trait bound that allows the function to accept Fn, FnMut and FnOnce closures correspondingly and call them. Note that the call_fn_mut impose the closure argument to be mutable.

fn call_fn<F>(f: F)
where
    F: Fn(),
{
    f();
}

fn call_fn_mut<F>(mut f: F)
where
    F: FnMut(),
{
    f();
}

fn call_fn_once<F>(f: F)
where
    F: FnOnce(),
{
    f();
}
...

Now, let’s try to use the closures defined in our previous section as an argument to these functions:

...
fn main() {
    let immut_val = String::from("immut");
    let fn_closure = || {
        println!("Len: {}", immut_val.len());
    };

    let mut mut_val = String::from("mut");
    let mut fnmut_closure = || {
        mut_val.push_str("-new");
    };

    let value = String::from("value");
    let fnonce_closure = || {
        let moved_value = value;
    };

    call_fn(fn_closure);
    call_fn_mut(fn_closure);
    call_fn_once(fn_closure);

    // call_fn(fnmut_closure); //error: fnmut_closure implements `FnMut`, not `Fn`
    call_fn_mut(fnmut_closure);
    call_fn_once(fnmut_closure);

    // call_fn(fnonce_closure); //error: fnonce_closure implements `FnOnce`, not `Fn`
    // call_fn_mut(fnonce_closure); //error: fnonce_closure implements `FnOnce`, not `FnMut`
    call_fn_once(fnonce_closure);
}

As you can see, the closure fn_closure, which automatically implements the Fn trait because it only borrows variables immutably, can be passed as an argument to any of these three functions. This means that if the closure implements Fn trait it automatically implements FnMut and FnOnce traits also.

From the error explanation, we understand that the fnmut_closure closure does not implement the Fn trait. Similarly, the fnonce_closure closure does not implement the Fn and FnMut traits. Thus, they cannot be passed as parameters to the corresponding functions.

References to Closures

It should be highlighted that:

• If type F implements the Fn trait then &F also implements the Fn trait.
• If type F implements the FnMut trait then &mut F also implements the FnMut trait.

This means that in our example instead of passing to higher order functions closures implementing the Fn and FnMut traits, we could pass the references to them:

...
fn main() {
    let immut_val = String::from("immut");
    let fn_closure = || {
        println!("Len: {}", immut_val.len());
    };

    let mut mut_val = String::from("mut");
    let mut fnmut_closure = || {
        mut_val.push_str("-new");
    };
    call_fn(&fn_closure);
    call_fn_mut(&fn_closure);
    call_fn_once(&fn_closure);

    // call_fn(&mut fnmut_closure); //error fnmut_closure implements `FnMut`, not `Fn`
    call_fn_mut(&mut fnmut_closure);
    call_fn_once(&mut fnmut_closure);
}

Move Keyword

Let’s consider the following example. Here, we define a function called get_print_closure that returns a closure that prints the value of the name variable defined in the same function.

fn main() {
    let closure = get_print_closure();
    closure();
}

fn get_print_closure() -> impl Fn() {
    let name = String::from("User");
    || { //error
        println!("Hello {}!", name);
    }
}

If we analyze the body of the closure, we understand that the value of the name variable is captured by immutable reference. After the function returns with closure as a returned value, the variable name gets out of the scope and should be dropped. However, in this case we will get an incorrect memory state, a dangling pointer. Obviously, this code leads to an error in Rust:

error[E0373]: closure may outlive the current function, but it borrows `name`, which is owned by the current function
 --> src/bin/move_keyword.rs:8:5
  |
8 |     || {
  |     ^^ may outlive borrowed value `name`
9 |         println!("Hello {}!", name);
  |                               ---- `name` is borrowed here
  |
note: closure is returned here
 --> src/bin/move_keyword.rs:6:27
  |
6 | fn get_print_closure() -> impl Fn() {
  |                          

Luckily, the error tells us exactly how to correct the issue: we just need to add the move keyword before the closure’s first bar:

fn get_print_closure() -> impl Fn() {
    let name = String::from("User");
    move || {
        println!("Hello {}!", name);
    }
}

What happens if we add this magic keyword? In this case, the name variable will be moved inside the closure (the closure will becode the owner of the name variable) and no error will be generated.

Beside being used in expressions where a closure is returned, the move keyword is often used when we create a new thread and execute a closure in it. For instance, the following program will not compile unless we uncomment the move keyword:

use std::thread;

fn main() {
    let name = String:: from("Alice");
    let print_closure = /*move*/ || println!("Name: {}", name);
    let handler = thread::spawn(print_closure);
    handler.join().expect("Error happened!");
}

Conclusion

This is all for now. Hopefully, this was a useful reading for you. As I am new to Rust, it took me a lot of time to understand this topic and write this article, however I do not regret so as I get to this level of understanding. If you find any mistakes or have suggestions what subtopics I should extend, I will be grateful if you point me to them, so I can improve the article.