Having a complete version history for every entity is powerful, but raw power without precision is just noise. When a product has fifty price changes over six months, you do not want all fifty versions -- you want the versions where the price exceeded a threshold, sorted by date. When an audit log has hundreds of entries, you need filtering by field values and ordering by timestamp.
Session 082 completed the TEMP-4 category (History Queries) to one hundred percent by implementing the final three tasks: filtering by field values, combining filters, and ordering results. The design decision at the heart of this feature -- helper methods instead of lambdas -- shaped the entire implementation.
The Lambda Question
The natural syntax for filtering in most languages uses closures or lambda expressions:
// JavaScript
product.history.filter(v => v.price > 100)
product.history.sort((a, b) => a.price - b.price)# Python
[v for v in product.history if v.price > 100]
sorted(product.history, key=lambda v: v.price)We considered this approach for FLIN. The syntax would have been:
// Hypothetical lambda-based filtering (NOT implemented)
product.history.where(|ver| ver.price > 100)We rejected it for three reasons.
First, FLIN did not have lambda support. Implementing closures -- with their captured variable scoping, lifetime management in the VM, and type checking implications -- would have been a multi-session detour. History filtering was blocking real use cases. We needed it now.
Second, field-based filtering covers ninety-five percent of use cases. When developers filter version histories, they almost always filter by a field value: "versions where price was above X," "versions where status was Y," "versions created after date Z." A dedicated method for field-based filtering is not a limitation -- it is a focused API.
Third, simpler syntax is better for FLIN's target audience. FLIN is designed for developers who want to build applications quickly. A method that takes a field name, an operator, and a value is more immediately readable than a lambda expression, especially for developers coming from SQL backgrounds.
The decision: implement .where_field(field, operator, value) and .order_by(field, direction) as dedicated methods. Lambdas can be added later without breaking changes.
The API Design
Two new methods on list types:
// Filtering: returns a new list containing only matching elements
history.where_field("price", ">", 15.0)
history.where_field("status", "==", "active")
history.where_field("stock", "<=", 50)// Ordering: returns a new sorted list history.order_by("price", "asc") history.order_by("created_at", "desc")
// Chaining: filter then sort history.where_field("price", ">=", 12.0).order_by("price", "desc") ```
Six comparison operators: ==, !=, <, <=, >, >=. Two sort directions: "asc" and "desc". All operators are passed as string arguments and validated at runtime.
The chaining syntax works naturally because both methods return lists. .where_field() returns a filtered list. .order_by() returns a sorted list. Applying .order_by() to a filtered list works without any special handling.
Implementation: Two New Opcodes
ListFilterField (0xF4)
The ListFilterField opcode pops four values from the stack: the list, the field name, the operator string, and the comparison value. It iterates over the list, extracts the named field from each element (which must be an entity instance), applies the comparison, and pushes a new list containing only the matching elements.
OpCode::ListFilterField => {
let value_val = self.pop()?;
let op_val = self.pop()?;
let field_val = self.pop()?;
let list_val = self.pop()?;let field_name = match &field_val { Value::Object(id) => self.get_string(*id)?.to_string(), Value::Text(s) => s.clone(), _ => return Err(RuntimeError::TypeError { / ... / }), };
let operator = match &op_val { Value::Object(id) => self.get_string(*id)?.to_string(), Value::Text(s) => s.clone(), _ => return Err(RuntimeError::TypeError { / ... / }), };
// Iterate list, filter by field comparison let mut filtered = Vec::new(); for item in list_items { let field_value = extract_field(&item, &field_name); let matches = match operator.as_str() { "==" => values_equal(&field_value, &value_val), "!=" => !values_equal(&field_value, &value_val), "<" => compare_numeric_values(&field_value, &value_val) == Less, "<=" => compare_numeric_values(&field_value, &value_val) != Greater, ">" => compare_numeric_values(&field_value, &value_val) == Greater, ">=" => compare_numeric_values(&field_value, &value_val) != Less, _ => false, }; if matches { filtered.push(item); } }
// Push filtered list self.push(create_list(filtered)); } ```
The comparison dispatch uses two helper methods: values_equal() for equality checks (handles type coercion between integers, floats, and strings) and compare_numeric_values() for ordering comparisons (handles int/int, float/float, and cross-type comparisons).
ListOrderBy (0xF5)
The ListOrderBy opcode pops three values: the list, the field name, and the direction string. It sorts the list by extracting the named field from each element and comparing values.
The implementation had to work around Rust's borrow checker. The sort closure needs access to self to read heap objects, but sort_by takes a mutable reference to the slice, creating a borrowing conflict. The solution was to pre-extract all field values before sorting:
OpCode::ListOrderBy => {
let dir_val = self.pop()?;
let field_val = self.pop()?;
let list_val = self.pop()?;// Pre-extract field values to avoid borrow conflicts let mut pairs: Vec<(Value, Value)> = Vec::new(); for item in &list_items { let field_value = extract_field(item, &field_name); pairs.push((item.clone(), field_value)); }
// Sort by extracted field values pairs.sort_by(|(_, val_a), (_, val_b)| { let cmp = compare_numeric_values(val_a, val_b); if direction == "desc" { cmp.reverse() } else { cmp } });
// Reconstruct sorted list
let sorted: Vec
This "extract then sort" pattern adds a memory allocation (the pairs vector), but it cleanly satisfies the borrow checker and makes the sorting logic straightforward.
Type Checking
Both methods were registered in the type checker with appropriate signatures:
// .where_field(field_name: Text, operator: Text, value: Unknown) -> List<T>
"where_field" => FlinType::Function {
params: vec![FlinType::Text, FlinType::Text, FlinType::Unknown],
ret: Box::new(list_element_type.clone()),
min_arity: 3,
has_rest: false,
}// .order_by(field_name: Text, direction: Text) -> List
The return type preserves the list's element type. If you call .where_field() on a List, you get back a List. This enables type-safe chaining: the type checker knows that .order_by() on the result of .where_field() is valid.
Code Generation
Both methods were integrated into the existing try_emit_list_method() function in the code generator. When the code generator encounters a method call on a list, it checks whether the method name matches a known list operation. For where_field, it emits the arguments onto the stack followed by the ListFilterField opcode. For order_by, it emits the arguments followed by ListOrderBy.
The code generation is straightforward because the opcodes handle all the complexity. The emitter's job is just to put the right values on the stack in the right order.
Testing: Filtering and Ordering in Action
The example file history-queries-complete-test.flin demonstrates every filtering and ordering pattern:
entity Product {
name: text
price: number
}product = Product { name: "Widget", price: 10 } save product
product.price = 12 save product
product.price = 15 save product
product.price = 18 save product
product.price = 20 save product
product.price = 8 save product
// Six versions: $10, $12, $15, $18, $20, $8
// Filter: expensive versions expensive = product.history.where_field("price", ">", 15.0) // Result: [$18, $20]
// Filter: cheap versions cheap = product.history.where_field("price", "<=", 12.0) // Result: [$10, $12, $8]
// Filter: exact price exact = product.history.where_field("price", "==", 15.0) // Result: [$15]
// Order: ascending sorted_asc = product.history.order_by("price", "asc") // Result: [$8, $10, $12, $15, $18, $20]
// Order: descending sorted_desc = product.history.order_by("price", "desc") // Result: [$20, $18, $15, $12, $10, $8]
// Chain: filter then order expensive_sorted = product.history .where_field("price", ">=", 12.0) .order_by("price", "desc") // Result: [$20, $18, $15, $12] ```
All assertions passed. All one thousand and ten library tests still passing.
An Edge Case: HTML and Comparison Operators
During testing, we discovered an amusing interaction between FLIN's HTML parser and comparison operators. FLIN renders views as HTML, and the < and > characters in HTML content confuse the parser:
// Parse error -- the parser sees '<' as an HTML tag start
<h3>Price > 15</h3>// Works correctly
Price greater than 15
```This is not a bug in the filtering implementation -- it is a consequence of FLIN's unified code-and-view syntax. The workaround is to avoid raw comparison operators in HTML text content. In practice, this rarely matters because comparison operators appear in code blocks, not in view text.
TEMP-4 Complete: The Fourth Category at 100%
Session 082 brought TEMP-4 (History Queries) to twenty-two out of twenty-two tasks -- one hundred percent complete. This was the fourth temporal category to reach full completion:
1. TEMP-1: Core Soft Delete (5/5) 2. TEMP-2: Temporal Access (18/18) 3. TEMP-3: Temporal Keywords (14/14) 4. TEMP-4: History Queries (22/22) 5. TEMP-5: Time Arithmetic (12/12) 6. TEMP-11: Integration Tests (27/27)
Six out of eleven categories complete. Overall progress: one hundred and five out of one hundred sixty tasks (sixty-five point six percent).
The completion timeline for TEMP-4 tells the story of iterative development:
| Session | Progress | Milestone |
|---|---|---|
| 068 | 1/22 (5%) | Foundation audit |
| 075 | 4/22 (18%) | .history property |
| 080 | 16/22 (73%) | List methods |
| 081 | 19/22 (86%) | Metadata access |
| 082 | 22/22 (100%) | Filtering and ordering |
Five sessions, spread across two days, each building on the previous one's work. No single session was heroic. Each added a focused piece -- and the sum was a complete, tested, production-ready history query system.
Why This Matters for Applications
Filtering and ordering transform the temporal model from "we store history" to "we can answer questions about history." Consider these application patterns:
Price tracking: "Show me every time this product was priced above $100, sorted by date."
expensive_history = product.history
.where_field("price", ">", 100)
.order_by("created_at", "asc")Audit compliance: "Show me all changes to this document in the last month, most recent first."
recent_changes = document.history
.where_field("created_at", ">", last_month)
.order_by("created_at", "desc")Anomaly detection: "Find versions where the stock dropped below the safety threshold."
low_stock_events = inventory.history
.where_field("quantity", "<", safety_threshold)Each of these would require a custom SQL query, a dedicated reporting table, or an analytics pipeline in a traditional application. In FLIN, they are one-liners that compose naturally with the rest of the temporal API.
Comparison with Other Query Systems
To appreciate what .where_field() and .order_by() give FLIN developers, consider the equivalent in other systems.
SQL with temporal tables:
SELECT * FROM products_history
WHERE product_id = $1
AND price > 100
AND valid_from <= NOW()
ORDER BY valid_from DESC;The developer writes SQL, manages connections, maps results to objects, and handles the temporal columns explicitly. The query is decoupled from the application code that uses the results.
Rails with PaperTrail:
product.versions.where("object->>'price' > ?", 100)
.order(created_at: :desc)
.map { |v| v.reify }The developer imports a gem, configures it per model, writes ActiveRecord queries against a versions table, and calls reify to reconstruct each historical object. The versioning is external to the language.
FLIN:
product.history
.where_field("price", ">", 100)
.order_by("created_at", "desc")No import. No configuration. No query language boundary. No object reconstruction. The result is a list of entities that can be iterated in a template, passed to functions, or further filtered with additional .where_field() calls.
The difference is not just syntactic. It is architectural. In SQL and Rails, temporal queries are database concerns that leak into application code. In FLIN, they are language expressions that compose with everything else.
Performance Characteristics
Both where_field and order_by operate on in-memory lists. The version history is loaded from the database once (when .history is accessed), and subsequent filtering and ordering operate on the loaded data. This means:
- Filtering is O(n) where n is the number of versions.
- Ordering is O(n log n) using Rust's sort algorithm.
- Chaining
.where_field().order_by()is O(n) + O(m log m) where m is the filtered result size.
For the vast majority of entities, version counts are in the tens or hundreds, making these operations effectively instantaneous. For entities with thousands of versions, the planned retention policies (TEMP-9) would keep the history size bounded.
Two hundred and fifty lines of implementation code. Two new opcodes. Two new type signatures. And FLIN's history queries became production-ready.
---
This is Part 5 of the "How We Built FLIN" temporal model series, documenting the filtering and ordering system for temporal history queries.
Series Navigation: - [046] Every Entity Remembers Everything: The Temporal Model - [047] Version History and Time Travel Queries - [048] Temporal Integration: From Bugs to 100% Test Coverage - [049] Destroy and Restore: Soft Deletes Done Right - [050] Temporal Filtering and Ordering (you are here) - [051] Temporal Comparison Helpers - [052] Version Metadata Access - [053] Time Arithmetic: Adding Days, Comparing Dates - [054] Tracking Accuracy and Validation - [055] The Temporal Model Complete: What No Other Language Has