derive-deftly 1.0 is released.
derive-deftly is a template-based derive-macro facility for Rust. It has been a great success. Your codebase may benefit from it too!
Rust programmers will appreciate its power, flexibility, and consistency, compared to
macro_rules; and its convenience and simplicity, compared to proc macros.
Programmers coming to Rust from scripting languages will appreciate derive-deftly s convenient automatic code generation, which works as a kind of compile-time introspection.
Rust s two main macro systems
I m often a fan of metaprogramming, including macros. They can help
remove duplication and flab, which are often the enemy of correctness.
Rust has two macro systems. derive-deftly offers much of the power of the more advanced (proc_macros), while beating the simpler one (macro_rules) at its own game for ease of use.
(Side note: Rust has at least three other ways to do metaprogramming: generics;
build.rs; and, multiple module inclusion via
#[path=]. These are beyond the scope of this blog post.)
macro_rules!
macro_rules! aka pattern macros , declarative macros , or sometimes macros by example are the simpler kind of Rust macro.
They involve writing a sort-of-BNF pattern-matcher, and a template which is then expanded with substitutions from the actual input. If your macro wants to accept comma-separated lists, or other simple kinds of input, this is OK. But often we want to emulate a
#[derive(...)] macro: e.g., to define code based on a struct, handling each field. Doing that with macro_rules is very awkward:
macro_rules! s pattern language doesn t have a cooked way to match a data structure, so you have to hand-write a matcher for Rust syntax, in each macro. Writing such a matcher is very hard in the general case, because
macro_rules lacks features for matching important parts of Rust syntax (notably, generics). (If you
really need to, there s a
horrible technique as a workaround.)
And, the invocation syntax for the macro is awkward: you must enclose the whole of the struct in
my_macro! . This makes it hard to apply more than one macro to the same struct, and produces rightward drift.
Enclosing the struct this way means the macro must reproduce its input - so it can have bugs where it mangles the input, perhaps subtly. This also means the reader cannot be sure precisely whether the macro modifies the struct itself. In Rust, the types and data structures are often the key places to go to understand a program, so this is a significant downside.
macro_rules also has various other weird deficiencies too specific to list here.
Overall, compared to (say) the C preprocessor, it s great, but programmers used to the power of Lisp macros, or (say) metaprogramming in Tcl, will quickly become frustrated.
proc macros
Rust s
second macro system is much more advanced. It is a fully general system for processing and rewriting code. The macro s implementation is Rust code, which takes the macro s input as arguments, in the form of
Rust tokens, and returns Rust tokens to be inserted into the actual program.
This approach is more similar to Common Lisp s macros than to most other programming languages macros systems. It is extremely powerful, and is used to implement many very widely used and powerful facilities. In particular, proc macros can be applied to data structures with
#[derive(...)]. The macro receives the data structure, in the form of Rust tokens, and returns the code for the new implementations, functions etc.
This is used very heavily in the standard library for basic features like
#[derive(Debug)] and
Clone, and for important libraries like
serde and
strum.
But, it is a complete pain in the backside to
write and
maintain a proc_macro.
The Rust types and functions you deal with in your macro are very low level. You must manually handle every possible case, with runtime conditions and pattern-matching. Error handling and recovery is so nontrivial there are macro-writing
libraries and even
more macros to help. Unlike a Lisp codewalker, a Rust proc macro must deal with Rust s highly complex syntax. You will probably end up dealing with
syn, which is a complete Rust parsing library, separate from the compiler; syn is capable and comprehensive, but a proc macro must still contain a lot of often-intricate code.
There are build/execution environment problems. The proc_macro code can t live with your application; you have to put the proc macros in a separate cargo package, complicating your build arrangements. The proc macro package environment is weird: you can t test it separately, without
jumping through hoops. Debugging can be awkward. Proper tests can only realistically be done with the help of complex
additional tools, and will involve a pinned version of Nightly Rust.
derive-deftly to the rescue
derive-deftly lets you use a write a
#[derive(...)] macro, driven by a data structure, without wading into any of that stuff.
Your macro definition is a template in a simple syntax, with predefined
$-substitutions for the various parts of the input data structure.
Example
Here s a
real-world example from a personal project:
define_derive_deftly!
export UpdateWorkerReport:
impl $ttype
pub fn update_worker_report(&self, wr: &mut WorkerReport)
$(
$ when fmeta(worker_report)
wr.$fname = Some(self.$fname.clone()).into();
)
#[derive(Debug, Deftly, Clone)]
...
#[derive_deftly(UiMap, UpdateWorkerReport)]
pub struct JobRow
...
#[deftly(worker_report)]
pub status: JobStatus,
pub processing: NoneIsEmpty<ProcessingInfo>,
#[deftly(worker_report)]
pub info: String,
pub duplicate_of: Option<JobId>,
This is a nice example, also, of how using a macro can avoid bugs. Implementing this update by hand without a macro would involve a lot of cut-and-paste. When doing that cut-and-paste it can be very easy to accidentally write bugs where you forget to update some parts of each of the copies:
pub fn update_worker_report(&self, wr: &mut WorkerReport)
wr.status = Some(self.status.clone()).into();
wr.info = Some(self.status.clone()).into();
Spot the mistake? We copy
status to
info. Bugs like this are extremely common, and not always found by the type system. derive-deftly can make it much easier to make them impossible.
Special-purpose derive macros are now worthwhile!
Because of the difficult and cumbersome nature of proc macros, very few projects have site-specific, special-purpose
#[derive(...)] macros.
The
Arti codebase has no bespoke proc macros, across its 240kloc and 86 crates. (We did
fork one upstream proc macro package to add a feature we needed.) I have only
one bespoke, case-specific, proc macro amongst all of my personal Rust projects; it predates derive-deftly.
Since we have started using derive-deftly in Arti, it has become an important tool in our toolbox. We have 37 bespoke derive macros, done with derive-deftly. Of these, 9 are exported for use by downstream crates. (For comparison there are 176 macro_rules macros.)
In my most recent
personal Rust project, I have 22 bespoke derive macros, done with with derive-deftly, and 19 macro_rules macros.
derive-deftly macros are easy and straightforward enough that they can be used as readily as macro_rules macros. Indeed, they are often
clearer than a macro_rules macro.
Stability without stagnation
derive-deftly is already highly capable, and can solve many advanced problems.
It is mature software, well tested, with excellent documentation, comprising both comprehensive
reference material and the
walkthrough-structured user guide.
But declaring it 1.0 doesn t mean that it won t improve further.
Our
ticket tracker has a laundry list of possible features. We ll sometimes be cautious about committing to these, so we ve added a
beta feature flag, for opting in to less-stable features, so that we can prototype things without painting ourselves into a corner. And, we intend to further develop the Guide.
comments
For my PhD, my colleagues/collaborators and I built a distributed
stream-processing system using Haskell.
There are several other Haskell stream-processing systems. How do they
compare?
First, let's briefly discuss and define streaming in this context.
Structure and Interpretation of Computer Programs introduces Streams
as an analogue of lists, to support delayed evaluation. In brief, the
inductive list type (a list is either an empty list or a head element
pre-pended to another list) is replaced with a structure with a head
element and a promise which, when evaluated, will generate the tail
(which in turn may have a head element and a promise to generate another
tail, culminating in the equivalent of an empty list.) Later on SICP
also covers lazy evaluation.
However, the streaming we're talking about originates in the relational
community, rather than the functional one, and is subtly different. It's
about building a pipeline of processing that receives and emits data but
doesn't need to (indeed, cannot) reference the whole stream (which may
be infinite) at once.
Haskell streaming systems
Now let's go over some Haskell streaming systems.
conduit (2011-)
Conduit is the oldest of the ones I am reviewing here, but I doubt it's
the first in the Haskell ecosystem. If I've made any obvious omissions,
please let me know!
Conduit provides a new set of types to model streaming data, and a completely
new set of functions which are analogues of standard Prelude functions, e.g.
As of today, 
I can t remember exactly the joke I was making at the time in my
The 
TL;DR
It's difficult to find the right Debian image. We have thousands of
ISO files and cloud images and we support multiple CPU architectures
and several download methods. The directory structure of our main image server
is like a maze, and our web pages for downloading are also confusing.
Most important facts from this blog post
Stable Wireshark PPA:
Like each month, have a look at the work funded by 
Last November, the DebConf25 Team
![[DebConf25 Logo Contest Winner]](https://bits.debian.org/images/juliana-dc25-logo1-vertical.png){kind=logo}
Juliana also shared with us a bit of her motivation, creative process and
inspiration when designing her logo:
If you re automating CI builds, there s even a GitHub Action to make your workflows faster essentially doing what 
Warning: