HSPACE Rust Lectures #10 — Macros

If you've been burned by C macros, the word alone makes you flinch. Rust macros are a different animal: not text substitution but syntax extension over token trees. This session is about that difference.

Macro basics

Anything with a ! — like println! — is a macro. Why can't it be a function?

println!("{}, {}", a, b);
println!("{} {} {}", x, y, z);

Because the arity varies and the format string is checked at compile time. Rust functions have no varargs, so this must be a macro — and because it is one, println!("{}", ) is a compile error. What blew up at runtime in C's printf is caught before the program runs.

You define macros with macro_rules!:

macro_rules! my_vec {
    () => {
        Vec::new()
    };
    ($($x:expr),*) => {
        {
            let mut v = Vec::new();
            $( v.push($x); )*
            v
        }
    };
}

It resembles match, except what's matched isn't values but tokens.

Macro expansion

Macros expand early in compilation, and the result must be syntactically valid Rust — you can't emit arbitrary text the way C macros can.

assert_eq!(gcd(6, 10), 2);

// expands to roughly
match (&gcd(6, 10), &2) {
    (left_val, right_val) => {
        if !(*left_val == *right_val) {
            panic!("assertion failed: `(left == right)` ...");
        }
    }
}

Matching through & matters: nothing is moved, so the operands remain usable after assert_eq!.

To see expansions directly, use cargo expand — or the Playground's Expand macros, mentioned back in the intro. When a macro misbehaves, reading the expansion is the fastest route.

Unintended consequences

I liked that the lecture called these out explicitly. Macros have traps.

1. Arguments can be evaluated more than once

macro_rules! max {
    ($a:expr, $b:expr) => {
        if $a > $b { $a } else { $b }
    };
}

max!(expensive_call(), 0)   // expensive_call() may run twice

The classic C macro problem, unchanged. The fix is to bind once:

macro_rules! max {
    ($a:expr, $b:expr) => {{
        let a = $a;
        let b = $b;
        if a > b { a } else { b }
    }};
}

2. Operator precedence

Something captured as $a:expr is a single expression node, so you don't need the defensive parentheses C macros require. max!(1 + 2, 3) cannot re-associate into if 1 + 2 > 3, because you're handling a parsed expression, not tokens. A clear win over C.

3. Blocks and semicolons

That's why macro bodies often use {{ }}: the outer braces delimit the macro body, and the inner ones are the block expression actually generated.

Repetition

The real power is in repetition syntax:

$( ... )*      // zero or more
$( ... )+      // one or more
$( ... )?      // zero or one
$( ... ),*     // zero or more, comma-separated

The macro from session one's example is a good illustration:

macro_rules! impl_from_num_for_json {
    ( $( $t:ident )* ) => {
        $(
            impl From<$t> for Json {
                fn from(arg: $t) -> Json {
                    Json::Number(arg as f64)
                }
            }
        )*
    };
}

impl_from_num_for_json!(u8 i8 u16 i16 u32 i32 u64 i64 usize isize f32 f64);

One line generates impl blocks for twelve types. By hand that's twelve copy-pastes, and easy to forget when adding a type. Eliminating boilerplate is the most honest use of macros.

Fragment types

In $name:kind, the kind is the fragment type and it decides what gets captured.

Fragment	Matches	Example
expr	expression	2 + 2, f(x)
ident	identifier	foo, u32
ty	type	Vec<String>, i32
pat	pattern	Some(x), _
literal	literal	42, "hi"
block	block	{ ... }
stmt	statement	let x = 1
item	item	fn f() {}, struct S;
path	path	std::io::Read
tt	one token tree	anything

tt is the most flexible and carries the least structure, which is why recursive macros lean on it.

Choosing the wrong fragment produces odd errors. Something captured as $t:expr is already an expression node and can't be matched into further — which is exactly when you switch to tt and recurse.

Recursion

A macro can call itself.

macro_rules! count {
    () => { 0 };
    ($head:tt $($tail:tt)*) => { 1 + count!($($tail)*) };
}

The default recursion limit is 128 levels; raise it with #![recursion_limit = "256"].

Building the json! macro

The centerpiece of the session: a macro that lets you write JSON literals directly in Rust.

#[derive(Clone, PartialEq, Debug)]
enum Json {
    Null,
    Boolean(bool),
    Number(f64),
    String(String),
    Array(Vec<Json>),
    Object(Box<HashMap<String, Json>>),
}

#[macro_export]
macro_rules! json {
    (null) => {
        $crate::Json::Null
    };
    ([ $( $element:tt ),* ]) => {
        $crate::Json::Array(vec![ $( json!($element) ),* ])
    };
    ({ $( $key:tt : $value:tt ),* }) => {
        {
            let mut fields = $crate::macros::Box::new($crate::macros::HashMap::new());
            $(
                fields.insert($crate::macros::ToString::to_string($key), json!($value));
            )*
            $crate::Json::Object(fields)
        }
    };
    ($other:tt) => {
        $crate::Json::from($other)
    };
}

Which lets you write:

let value = json!({
    "math_works": (4 - 2 == 2),
    "en": HELLO,
    HELLO: "bonjour!"
});

The key is that arrays and objects are handled recursively — each element goes through json! again, so nesting can be arbitrarily deep. And the final rule ($other:tt) => { Json::from($other) } converts numbers, strings and booleans through From — the very From impls that the repetition macro above generated.

Why $element:tt and not $element:expr? Because as an expr, an array literal like [1, 2] freezes into one expression node you can't match into. Capturing it as tt keeps it open for recursion.

Scoping and hygiene

Hygiene is where Rust macros diverge most sharply from C.

macro_rules! using_a {
    ($e:expr) => {
        {
            let a = 42;
            $e
        }
    };
}

let a = 10;
let x = using_a!(a + 1);   // 11, not 43!

The a inside the macro and the caller's a are different variables, because the compiler tracks a syntax context per identifier. No more naming temporaries __tmp_xyz_123.

Hygiene applies only to local variable names, though. Paths — types, functions, macro names — resolve in the caller's context, which breaks when the macro is used outside its home crate:

macro_rules! bad_json {
    (null) => { Json::Null };   // requires Json to be imported in user code
}

The fix is $crate:

macro_rules! json {
    (null) => { $crate::Json::Null };   // always this crate's Json
}

$crate expands to "the crate where this macro was defined." That's also why the json! example spells out $crate::macros::Box and $crate::macros::ToString, with the macros module re-exporting them:

pub use std::boxed::Box;
pub use std::collections::HashMap;
pub use std::string::ToString;

It looked excessive to me at first — until you consider that a user might have defined their own type named Box.

Importing and exporting macros

#[macro_export] places the macro at the crate root.

#[macro_export]
macro_rules! json { /* ... */ }

Users then use my_crate::json;. Worth remembering that macros don't follow the module hierarchy the way other items do.

Within the same crate, the definition must appear before its uses — unlike functions, macros are order-sensitive. Hence the convention of putting mod macros; at the very top of lib.rs.

Macros with traits

Generating trait implementations in bulk is a pattern the standard library uses too — impl_from_num_for_json! above being the example.

But the lecture drew a line: use derive where derive works, and generics where generics work. Macros are for the cases neither covers, i.e. when you want to change the syntax itself. More macros means worse error messages, weaker IDE support, and readers who have to imagine the expansion.

Avoiding syntax errors during matching

Macro rules are tried top to bottom, and a failed match falls through to the next rule. The problem is a rule that matches but expands to something invalid — there's no fallthrough then, just an error.

So rule order matters: specific patterns first, general ones last. That's why json! lists (null), ([...]) and ({...}) before ($other:tt). Put $other:tt on top and it swallows everything.

Beyond macro_rules!

Some things declarative macros can't do. For those there are procedural macros: you write a function in a separate crate that takes a TokenStream and returns a TokenStream, so analysis and generation can be arbitrary Rust.

• Derive macros — #[derive(Serialize)], as in serde
• Attribute macros — #[tokio::main], #[test], transforming an item
• Function-like macros — called like sqlx::query!, processed procedurally

The syn and quote crates make parsing and code generation much easier. And thiserror's #[derive(Error)] from the error-handling session is a procedural macro too.

Assignments

All three build new syntax with macro_rules!.

1. hashmap! — the HashMap counterpart of vec!.

#[macro_export]
macro_rules! hashmap {
    () => {
        ::std::collections::HashMap::new()
    };
    ( $( $key:expr => $value:expr ),* $(,)? ) => {
        {
            let mut map = ::std::collections::HashMap::new();
            $(
                map.insert($key, $value);
            )+
            map
        }
    };
}

let map = hashmap!(1 => "one", 2 => "two");
let map = hashmap!(1 => "one", 2 => "two",);   // trailing comma OK
let empty: HashMap<u32, u32> = hashmap!();

The trick is $(,)?, which optionally allows a trailing comma. Without it hashmap!(1 => "one",) doesn't compile. The real vec! uses the same technique.

What I liked about this exercise is that it ships cases that must fail under src/invalid/ — no-comma.rs, double-commas.rs, only-arrow.rs, missing-argument.rs. It makes the point that designing a macro means deciding what to reject as much as what to accept.

2. for_2d! — a doubly-nested for loop as one construct.

macro_rules! for_2d {
    ($x_name:ident <$x_type:ty> in $x_expr:expr,
     $y_name:ident <$y_type:ty> in $y_expr:expr,
     $block:block) => {
        for $x_name in $x_expr {
            let $x_name: $x_type = $x_name;
            for $y_name in $y_expr {
                let $y_name: $y_type = $y_name;
                $block
            }
        }
    };
}

for_2d!(row <i32> in 1..5, col <i32> in 2..7, {
    (Coordinate { x: col, y: row }).show()
});

let values = [1, 3, 5];
for_2d!(x <u16> in values, y <u16> in values, {
    (Coordinate { x: x.into(), y: y.into() }).show()
});

Four fragment types — ident, ty, expr, block — in one rule. And <i32> in that position isn't Rust syntax at all, which is precisely the point: a macro can define its own syntax.

let $x_name: $x_type = $x_name; is a nice touch — rebinding the loop variable under the same name to pin its type, i.e. shadowing from an earlier session, used inside a macro.

3. graph! — adjacency-list syntax.

macro_rules! graph {
    ( $($from:literal -> ( $( $to:literal),* );)* ) => {
        {
            let mut vec = Vec::new();
            $( $(vec.push(($from, $to));)* )*
            vec
        }
    }
}

let my_graph = graph!(
    1 -> (2, 3, 4, 5);
    2 -> (1, 3);
    3 -> (2);
    4 -> ();
    5 -> (1, 2, 3);
);

Nested repetition is the lesson. The outer $( ... ;)* takes each line, the inner $( $to:literal ),* takes that line's destinations, and the expansion unrolls both levels with $( $(...)* )*. An empty inner list like 4 -> () works fine.

-> and ; mean other things in Rust, but in a macro pattern they're just separator tokens — a clear reminder that macros operate on token trees, not meaning.

Takeaways

• Rust macros work on token trees, not text, so precedence bugs don't exist
• fragment types decide what gets captured; recurse with tt
• repetition ($(...)*) and nested repetition kill boilerplate
• hygiene keeps local names from colliding, but paths need $crate
• rules match in order, so put specific patterns first
• when macro_rules! isn't enough, reach for procedural macros

Next: async programming, part 1 — futures, async/await, and how they differ from threads.