A language without closures is a language without power. Session 013 gave FLIN both closures and the higher-order functions that depend on them -- map, filter, and reduce -- in a single sixty-minute sprint that produced 16 new opcodes and 37 new tests.
Closures are one of those features that seem simple from the user's perspective. You write a function that references a variable from its enclosing scope, and it just works. But under the hood, in a stack-based VM where local variables live on call frames that disappear when functions return, "just works" requires careful engineering.
This is the story of how we made closures work in FLIN's VM, how we built the higher-order function infrastructure on top of them, and how we fought the Rust borrow checker at every step.
---
What Closures Are and Why They Matter
A closure is a function paired with an environment -- a snapshot of the variables it references from its enclosing scope. Consider this FLIN code:
multiplier = 3
double = |x| x * multiplier
result = double(5) // 15The lambda |x| x * multiplier captures the variable multiplier. When double(5) executes, the lambda needs access to multiplier even though it is not a parameter. The value 3 must be carried along with the function itself.
In an AST-walking interpreter, this is easy: closures capture a reference to the enclosing environment. In a bytecode VM, it is harder, because the enclosing environment (the call frame) might have been popped off the call stack by the time the closure executes.
FLIN solves this by storing captured values directly in the closure object on the heap:
pub struct ClosureData {
pub function: FunctionData,
pub captures: Vec<Value>,
}When the compiler creates a closure, it emits instructions to copy the captured variables from their current locations (stack, locals, or globals) into the closure's captures vector. When the closure is called, the VM makes those captured values available as if they were local variables.
---
The Closure Object
On the heap, a closure looks like this:
ObjectData::Closure(ClosureData {
function: FunctionData {
name: "double".to_string(),
arity: 1,
address: 42, // Bytecode address of the function body
local_count: 1,
},
captures: vec![Value::Int(3)], // multiplier = 3
})The function field contains everything the VM needs to call the closure: its name (for error messages), its arity (how many arguments it expects), its bytecode address (where to jump to), and how many local variables it uses. The captures vector contains the values that were closed over at creation time.
This is a "flat closure" design -- all captured values are copied into the closure at creation time. The alternative is "upvalues" (as in Lua), where the closure holds references to mutable cells that can be shared between the enclosing scope and the closure. Flat closures are simpler and faster for the common case where captured values are read-only. FLIN does not currently support mutating captured variables from within a closure, so flat closures are sufficient.
---
Higher-Order Function Infrastructure
With closures working, we needed map, filter, and reduce -- the three workhorses of functional programming. But implementing higher-order functions in a bytecode VM is fundamentally different from implementing them in a host-language interpreter.
In a JavaScript engine, array.map(fn) calls fn using the host language's call mechanism. In FLIN's VM, the closure's body is bytecode. Calling it means setting up a call frame, jumping to the closure's bytecode address, executing instructions, and handling the return. And map needs to do this once for every element in the list.
We could not just loop in Rust and call the closure N times, because the closure executes FLIN bytecode, not Rust code. The closure might itself call other functions, allocate objects, or even trigger another map. The VM's execution loop must remain the single point of control.
The solution was a continuation-based approach: the HOF infrastructure manages iteration state on a separate stack, and the normal Return instruction is augmented to check whether it is returning from a HOF callback.
The HofContext
#[derive(Debug, Clone, Copy, PartialEq)]
pub enum HofKind {
Map,
Filter,
Reduce,
}#[derive(Debug, Clone)]
pub struct HofContext {
pub kind: HofKind,
pub source_list: ObjectId,
pub current_index: usize,
pub length: usize,
pub results: Vec
When the VM encounters a ListMap, ListFilter, or ListReduce instruction, it creates a HofContext and pushes it onto the hof_stack. The context tracks which list is being iterated, the current position, the accumulated results, the closure to call, and where to resume when the entire HOF is done.
The VM then calls the closure for the first element using hof_call_next:
fn hof_call_next(&mut self, chunk: &Chunk) -> Result<(), RuntimeError> {
let (source_list, current_index, closure_id, kind, accumulator) = {
let ctx = self.hof_stack.last().ok_or(
RuntimeError::InternalError("No HOF context".into())
)?;
(ctx.source_list, ctx.current_index, ctx.closure_id,
ctx.kind, ctx.accumulator.clone())
};// Get the current element from the source list let element = { let list = self.get_list(source_list)?; list[current_index].clone() };
// Set up the closure call let closure = self.get_closure(closure_id)?; let frame = CallFrame { return_ip: self.ip, base_pointer: self.stack.len(), locals: vec![element.clone()], function_name: closure.function.name.clone(), };
// For reduce, the closure takes two arguments: accumulator and element if kind == HofKind::Reduce { if let Some(acc) = accumulator { frame.locals.insert(0, acc); } }
self.frames.push(frame); self.ip = closure.function.address; Ok(()) } ```
Notice the destructuring at the top. We extract all the values we need from the HofContext into local variables before doing anything else. This is the borrow checker pattern we described in the previous article -- extracting data before mutating self.
Modified Return
The key insight is that the Return opcode does double duty. When returning from a normal function call, it restores the caller's frame. When returning from a HOF callback, it feeds the result into the HOF machinery:
OpCode::Return => {
let result = self.pop();if !self.hof_stack.is_empty() { // HOF continuation: process the callback's return value self.hof_handle_return(result)?;
// If the HOF is not done, call the closure for the next element if !self.hof_stack.is_empty() { self.hof_call_next(chunk)?; } } else { // Normal function return let frame = self.frames.pop().unwrap(); self.ip = frame.return_ip; self.stack.truncate(frame.base_pointer); self.push(result); } } ```
Handling Returns
hof_handle_return processes the callback's return value differently depending on the HOF kind:
- Map: The return value is appended to the results vector unconditionally.
- Filter: The return value is tested for truthiness. If truthy, the current element (not the return value) is appended to the results.
- Reduce: The return value becomes the new accumulator.
When the current index reaches the list length, the HOF is complete. The HofContext is popped from the hof_stack, the instruction pointer is restored to return_ip, and the final result (a new list for map/filter, the accumulator for reduce) is pushed onto the operand stack.
---
16 New Opcodes
Session 013 added 16 new opcodes in the 0xE0-0xEF range:
// Extended Built-in Functions (0xE0-0xEF)
Replace = 0xE0, // String replace
Abs = 0xE1, // Math absolute value
Floor = 0xE2, // Math floor
Ceil = 0xE3, // Math ceiling
Round = 0xE4, // Math round
Min = 0xE5, // Math minimum
Max = 0xE6, // Math maximum
Sqrt = 0xE7, // Math square root
PowFn = 0xE8, // Math power
ListFirst = 0xE9, // First element
ListLast = 0xEA, // Last element
ListReverse = 0xEB, // Reverse list
ListSort = 0xEC, // Sort list
ListMap = 0xED, // HOF map with closure
ListFilter = 0xEE, // HOF filter with closure
ListReduce = 0xEF, // HOF reduce with closureThe first 13 are straightforward built-in functions. ListMap, ListFilter, and ListReduce are the HOF opcodes that trigger the continuation machinery described above.
Each math built-in handles both Int and Float arguments:
OpCode::Abs => {
let val = self.pop();
let result = match val {
Value::Int(n) => Value::Int(n.abs()),
Value::Float(n) => Value::Float(n.abs()),
_ => return Err(RuntimeError::TypeError(
"abs() requires a number".into()
)),
};
self.push(result);
}List operations create new lists rather than mutating in place. reverse(list) returns a new reversed list. sort(list) returns a new sorted list. The original list is untouched. This immutable-by-default approach simplifies reasoning about list operations and eliminates an entire class of bugs where a function unexpectedly mutates its input.
The sort implementation handles mixed Int/Float comparisons:
list.sort_by(|a, b| match (a, b) {
(Value::Int(a), Value::Int(b)) => a.cmp(b),
(Value::Float(a), Value::Float(b)) =>
a.partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal),
(Value::Int(a), Value::Float(b)) =>
(*a as f64).partial_cmp(b).unwrap_or(std::cmp::Ordering::Equal),
(Value::Float(a), Value::Int(b)) =>
a.partial_cmp(&(*b as f64)).unwrap_or(std::cmp::Ordering::Equal),
_ => std::cmp::Ordering::Equal,
});---
The Borrow Checker Fights Back
If there is one session where Rust's borrow checker was our primary adversary, it was this one. The HOF infrastructure requires reading from the heap (to get the source list and the closure) while also writing to the heap (to allocate results) and mutating the VM state (pushing frames, changing the instruction pointer).
The solution was always the same: extract what you need into local variables first, then mutate.
// Extract values before mutable operations
let (source_list, current_index, closure_id, kind, accumulator) = {
let ctx = self.hof_stack.last().ok_or(...)?;
(ctx.source_list, ctx.current_index, ctx.closure_id,
ctx.kind, ctx.accumulator.clone())
};
// Now safe to call self.push(), self.get_list(), etc.This pattern appears at least a dozen times in the HOF implementation. Each time, we scope the immutable borrow of self (to read from the HOF context) inside a block, letting the borrow end before the mutable operations begin.
Three parallel subagents worked on Session 013: one on math built-ins and opcodes, one on the HOF infrastructure, and one on tests. All three completed, but the HOF agent needed manual fixes for borrow checker issues. The borrow checker does not understand intent -- it only sees lifetimes. And the intent of "read this, then write that" requires the programmer to make the sequencing explicit.
---
Testing Higher-Order Functions
The test suite for built-in functions and HOF covered 37 new cases:
String built-ins (16 tests): len, upper, lower, trim, split, join, contains, starts_with, ends_with, replace -- each tested with typical inputs and edge cases.
Math built-ins (14 tests): abs, floor, ceil, round, sqrt, min, max, pow -- each tested with both Int and Float inputs, including negative numbers and edge cases like sqrt(0).
List built-ins (7 tests): first, last, reverse, sort, len -- including empty lists and single-element lists.
The HOF tests were integration tests that compiled FLIN source to bytecode and executed it:
fn compile_and_run(source: &str) -> Result<(Value, VM), String> {
let tokens = tokenize(source)?;
let program = parse(tokens)?;
check(&program)?;
let chunk = CodeGenerator::new().generate(&program)?;
let mut vm = VM::new();
let result = vm.execute(&chunk)?;
Ok((result, vm))
}This helper was the workhorse of all VM testing from Session 011 onwards. It takes FLIN source code as a string, runs it through the complete pipeline (lexer, parser, type checker, code generator, VM), and returns the result. If any stage fails, the error message tells you exactly where.
---
What This Unlocked
With closures and HOF working, FLIN gained functional programming capabilities that most developers expect from a modern language. A developer could write:
numbers = [1, 2, 3, 4, 5]
doubled = numbers.map(|x| x * 2) // [2, 4, 6, 8, 10]
evens = numbers.filter(|x| x % 2 == 0) // [2, 4]
sum = numbers.reduce(|a, b| a + b, 0) // 15Each of these compiles to bytecode that the VM executes using the HOF continuation machinery. The closure |x| x * 2 is a real closure -- it is allocated on the heap, carries its captured environment, and is called once per element by the VM's execution loop.
The total after Session 013: 338 tests passing, Phase 6 (Built-in Functions) at 64%, and the VM capable of executing real functional programs. The language was no longer just a calculator. It was a language.
---
The Broader Pattern
Looking back, the progression from Session 010 to 013 follows a pattern:
1. Session 010: Build the execution engine (stack, control flow, arithmetic). 2. Session 011: Build the memory system (heap, GC, allocation). 3. Session 013: Build the abstraction layer (closures, HOF, built-ins).
Each layer depends on the one below it. Closures need the heap (to store captured values). The heap needs the GC (to reclaim dead closures). The GC needs the execution engine (to find roots on the stack and in call frames).
This bottom-up approach -- foundation first, abstraction second -- is the only way to build a VM correctly. If we had tried to implement closures before the GC was working, captured values would leak. If we had tried to implement the GC before the heap was designed, the mark phase would not know what to traverse. Each session built exactly what the next session needed.
The next article moves from the runtime to the frontend: how the VM executes view instructions to generate HTML, bind attributes reactively, and handle events.
---
This is Part 23 of the "How We Built FLIN" series, documenting how a CEO in Abidjan and an AI CTO built a programming language from scratch.
Next up: [24] How the VM Executes Views -- the bridge between bytecode and the DOM.