Russ Allbery: Review: And Shall Machines Surrender
Series: | Machine Mandate #1 |
Publisher: | Prime Books |
Copyright: | 2019 |
ISBN: | 1-60701-533-1 |
Format: | Kindle |
Pages: | 86 |
Series: | Machine Mandate #1 |
Publisher: | Prime Books |
Copyright: | 2019 |
ISBN: | 1-60701-533-1 |
Format: | Kindle |
Pages: | 86 |
Series: | Dead Djinn Universe #1 |
Publisher: | Tordotcom |
Copyright: | 2021 |
ISBN: | 1-250-26767-6 |
Format: | Kindle |
Pages: | 391 |
Publisher: | Tor |
Copyright: | 2021 |
ISBN: | 1-250-78907-9 |
Format: | Kindle |
Pages: | 371 |
Publisher: | New American Library |
Copyright: | 2016 |
ISBN: | 0-698-18327-4 |
Format: | Kindle |
Pages: | 572 |
A Wizard of Earthsea (1971) Ursula K. Le Guin How did it come to be that Harry Potter is the publishing sensation of the century, yet Ursula K. Le Guin's Earthsea is only a popular cult novel? Indeed, the comparisons and unintentional intertextuality with Harry Potter are entirely unavoidable when reading this book, and, in almost every respect, Ursula K. Le Guin's universe comes out the victor. In particular, the wizarding world that Le Guin portrays feels a lot more generous and humble than the class-ridden world of Hogwarts School of Witchcraft and Wizardry. Just to take one example from many, in Earthsea, magic turns out to be nurtured in a bottom-up manner within small village communities, in almost complete contrast to J. K. Rowling's concept of benevolent government departments and NGOs-like institutions, which now seems a far too New Labour for me. Indeed, imagine an entire world imbued with the kindly benevolence of Dumbledore, and you've got some of the moral palette of Earthsea. The gently moralising tone that runs through A Wizard of Earthsea may put some people off:
Vetch had been three years at the School and soon would be made Sorcerer; he thought no more of performing the lesser arts of magic than a bird thinks of flying. Yet a greater, unlearned skill he possessed, which was the art of kindness.Still, these parables aimed directly at the reader are fairly rare, and, for me, remain on the right side of being mawkish or hectoring. I'm thus looking forward to reading the next two books in the series soon.
Blood Meridian (1985) Cormac McCarthy Blood Meridian follows a band of American bounty hunters who are roaming the Mexican-American borderlands in the late 1840s. Far from being remotely swashbuckling, though, the group are collecting scalps for money and killing anyone who crosses their path. It is the most unsparing treatment of American genocide and moral depravity I have ever come across, an anti-Western that flouts every convention of the genre. Blood Meridian thus has a family resemblance to that other great anti-Western, Once Upon a Time in the West: after making a number of gun-toting films that venerate the American West (ie. his Dollars Trilogy), Sergio Leone turned his cynical eye to the western. Yet my previous paragraph actually euphemises just how violent Blood Meridian is. Indeed, I would need to be a much better writer (indeed, perhaps McCarthy himself) to adequately 0utline the tone of this book. In a certain sense, it's less than you read this book in a conventional sense, but rather that you are forced to witness successive chapters of grotesque violence... all occurring for no obvious reason. It is often said that books 'subvert' a genre and, indeed, I implied as such above. But the term subvert implies a kind of Puck-like mischievousness, or brings to mind court jesters licensed to poke fun at the courtiers. By contrast, however, Blood Meridian isn't funny in the slightest. There isn't animal cruelty per se, but rather wanton negligence of another kind entirely. In fact, recalling a particular passage involving an injured horse makes me feel physically ill. McCarthy's prose is at once both baroque in its language and thrifty in its presentation. As Philip Connors wrote back in 2007, McCarthy has spent forty years writing as if he were trying to expand the Old Testament, and learning that McCarthy grew up around the Church therefore came as no real surprise. As an example of his textual frugality, I often looked for greater precision in the text, finding myself asking whether who a particular 'he' is, or to which side of a fight some two men belonged to. Yet we must always remember that there is no precision to found in a gunfight, so this infidelity is turned into a virtue. It's not that these are fair fights anyway, or even 'murder': Blood Meridian is just slaughter; pure butchery. Murder is a gross understatement for what this book is, and at many points we are grateful that McCarthy spares us precision. At others, however, we can be thankful for his exactitude. There is no ambiguity regarding the morality of the puppy-drowning Judge, for example: a Colonel Kurtz who has been given free license over the entire American south. There is, thank God, no danger of Hollywood mythologising him into a badass hero. Indeed, we must all be thankful that it is impossible to film this ultra-violent book... Indeed, the broader idea of 'adapting' anything to this world is, beyond sick. An absolutely brutal read; I cannot recommend it highly enough.
Bodies of Light (2014) Sarah Moss Bodies of Light is a 2014 book by Glasgow-born Sarah Moss on the stirrings of women's suffrage within an arty clique in nineteenth-century England. Set in the intellectually smoggy cities of Manchester and London, this poignant book follows the studiously intelligent Alethia 'Ally' Moberly who is struggling to gain the acceptance of herself, her mother and the General Medical Council. You can read my full review from July.
House of Leaves (2000) Mark Z. Danielewski House of Leaves is a remarkably difficult book to explain. Although the plot refers to a fictional documentary about a family whose house is somehow larger on the inside than the outside, this quotidian horror premise doesn't explain the complex meta-commentary that Danielewski adds on top. For instance, the book contains a large number of pseudo-academic footnotes (many of which contain footnotes themselves), with references to scholarly papers, books, films and other articles. Most of these references are obviously fictional, but it's the kind of book where the joke is that some of them are not. The format, structure and typography of the book is highly unconventional too, with extremely unusual page layouts and styles. It's the sort of book and idea that should be a tired gimmick but somehow isn't. This is particularly so when you realise it seems specifically designed to create a fandom around it and to manufacturer its own 'cult' status, something that should be extremely tedious. But not only does this not happen, House of Leaves seems to have survived through two exhausting decades of found footage: The Blair Witch Project and Paranormal Activity are, to an admittedly lesser degree, doing much of the same thing as House of Leaves. House of Leaves might have its origins in Nabokov's Pale Fire or even Derrida's Glas, but it seems to have more in common with the claustrophobic horror of Cube (1997). And like all of these works, House of Leaves book has an extremely strange effect on the reader or viewer, something quite unlike reading a conventional book. It wasn't so much what I got out of the book itself, but how it added a glow to everything else I read, watched or saw at the time. An experience.
Milkman (2018) Anna Burns This quietly dazzling novel from Irish author Anna Burns is full of intellectual whimsy and oddball incident. Incongruously set in 1970s Belfast during The Irish Troubles, Milkman's 18-year-old narrator (known only as middle sister ), is the kind of dreamer who walks down the street with a Victorian-era novel in her hand. It's usually an error for a book that specifically mention other books, if only because inviting comparisons to great novels is grossly ill-advised. But it is a credit to Burns' writing that the references here actually add to the text and don't feel like they are a kind of literary paint by numbers. Our humble narrator has a boyfriend of sorts, but the figure who looms the largest in her life is a creepy milkman an older, married man who's deeply integrated in the paramilitary tribalism. And when gossip about the narrator and the milkman surfaces, the milkman beings to invade her life to a suffocating degree. Yet this milkman is not even a milkman at all. Indeed, it's precisely this kind of oblique irony that runs through this daring but darkly compelling book.
The First Fifteen Lives of Harry August (2014) Claire North Harry August is born, lives a relatively unremarkable life and finally dies a relatively unremarkable death. Not worth writing a novel about, I suppose. But then Harry finds himself born again in the very same circumstances, and as he grows from infancy into childhood again, he starts to remember his previous lives. This loop naturally drives Harry insane at first, but after finding that suicide doesn't stop the quasi-reincarnation, he becomes somewhat acclimatised to his fate. He prospers much better at school the next time around and is ultimately able to make better decisions about his life, especially when he just happens to know how to stay out of trouble during the Second World War. Yet what caught my attention in this 'soft' sci-fi book was not necessarily the book's core idea but rather the way its connotations were so intelligently thought through. Just like in a musical theme and varations, the success of any concept-driven book is far more a product of how the implications of the key idea are played out than how clever the central idea was to begin with. Otherwise, you just have another neat Borges short story: satisfying, to be sure, but in a narrower way. From her relatively simple premise, for example, North has divined that if there was a community of people who could remember their past lives, this would actually allow messages and knowledge to be passed backwards and forwards in time. Ah, of course! Indeed, this very mechanism drives the plot: news comes back from the future that the progress of history is being interfered with, and, because of this, the end of the world is slowly coming. Through the lives that follow, Harry sets out to find out who is passing on technology before its time, and work out how to stop them. With its gently-moralising romp through the salient historical touchpoints of the twentieth century, I sometimes got a whiff of Forrest Gump. But it must be stressed that this book is far less certain of its 'right-on' liberal credentials than Robert Zemeckis' badly-aged film. And whilst we're on the topic of other media, if you liked the underlying conceit behind Stuart Turton's The Seven Deaths of Evelyn Hardcastle yet didn't enjoy the 'variations' of that particular tale, then I'd definitely give The First Fifteen Lives a try. At the very least, 15 is bigger than 7. More seriously, though, The First Fifteen Lives appears to reflect anxieties about technology, particularly around modern technological accelerationism. At no point does it seriously suggest that if we could somehow possess the technology from a decade in the future then our lives would be improved in any meaningful way. Indeed, precisely the opposite is invariably implied. To me, at least, homo sapiens often seems to be merely marking time until we can blow each other up and destroying the climate whilst sleepwalking into some crisis that might precipitate a thermonuclear genocide sometimes seems to be built into our DNA. In an era of cli-fi fiction and our non-fiction newspaper headlines, to label North's insight as 'prescience' might perhaps be overstating it, but perhaps that is the point: this destructive and negative streak is universal to all periods of our violent, insecure species.
The Goldfinch (2013) Donna Tartt After Breaking Bad, the second biggest runaway success of 2014 was probably Donna Tartt's doorstop of a novel, The Goldfinch. Yet upon its release and popular reception, it got a significant number of bad reviews in the literary press with, of course, an equal number of predictable think pieces claiming this was sour grapes on the part of the cognoscenti. Ah, to be in 2014 again, when our arguments were so much more trivial. For the uninitiated, The Goldfinch is a sprawling bildungsroman that centres on Theo Decker, a 13-year-old whose world is turned upside down when a terrorist bomb goes off whilst visiting the Metropolitan Museum of Art, killing his mother among other bystanders. Perhaps more importantly, he makes off with a painting in order to fulfil a promise to a dying old man: Carel Fabritius' 1654 masterpiece The Goldfinch. For the next 14 years (and almost 800 pages), the painting becomes the only connection to his lost mother as he's flung, almost entirely rudderless, around the Western world, encountering an array of eccentric characters. Whatever the critics claimed, Tartt's near-perfect evocation of scenes, from the everyday to the unimaginable, is difficult to summarise. I wouldn't label it 'cinematic' due to her evocation of the interiority of the characters. Take, for example: Even the suggestion that my father had close friends conveyed a misunderstanding of his personality that I didn't know how to respond it's precisely this kind of relatable inner subjectivity that cannot be easily conveyed by film, likely is one of the main reasons why the 2019 film adaptation was such a damp squib. Tartt's writing is definitely not 'impressionistic' either: there are many near-perfect evocations of scenes, even ones we hope we cannot recognise from real life. In particular, some of the drug-taking scenes feel so credibly authentic that I sometimes worried about the author herself. Almost eight months on from first reading this novel, what I remember most was what a joy this was to read. I do worry that it won't stand up to a more critical re-reading (the character named Xandra even sounds like the pharmaceuticals she is taking), but I think I'll always treasure the first days I spent with this often-beautiful novel.
Beyond Black (2005) Hilary Mantel Published about five years before the hyperfamous Wolf Hall (2004), Hilary Mantel's Beyond Black is a deeply disturbing book about spiritualism and the nature of Hell, somewhat incongruously set in modern-day England. Alison Harte is a middle-aged physic medium who works in the various towns of the London orbital motorway. She is accompanied by her stuffy assistant, Colette, and her spirit guide, Morris, who is invisible to everyone but Alison. However, this is no gentle and musk-smelling world of the clairvoyant and mystic, for Alison is plagued by spirits from her past who infiltrate her physical world, becoming stronger and nastier every day. Alison's smiling and rotund persona thus conceals a truly desperate woman: she knows beyond doubt the terrors of the next life, yet must studiously conceal them from her credulous clients. Beyond Black would be worth reading for its dark atmosphere alone, but it offers much more than a chilling and creepy tale. Indeed, it is extraordinarily observant as well as unsettlingly funny about a particular tranche of British middle-class life. Still, the book's unnerving nature that sticks in the mind, and reading it noticeably changed my mood for days afterwards, and not necessarily for the best.
The Wall (2019) John Lanchester The Wall tells the story of a young man called Kavanagh, one of the thousands of Defenders standing guard around a solid fortress that envelopes the British Isles. A national service of sorts, it is Kavanagh's job to stop the so-called Others getting in. Lanchester is frank about what his wall provides to those who stand guard: the Defenders of the Wall are conscripted for two years on the Wall, with no exceptions, giving everyone in society a life plan and a story. But whilst The Wall is ostensibly about a physical wall, it works even better as a story about the walls in our mind. In fact, the book blends together of some of the most important issues of our time: climate change, increasing isolation, Brexit and other widening societal divisions. If you liked P. D. James' The Children of Men you'll undoubtedly recognise much of the same intellectual atmosphere, although the sterility of John Lanchester's dystopia is definitely figurative and textual rather than literal. Despite the final chapters perhaps not living up to the world-building of the opening, The Wall features a taut and engrossing narrative, and it undoubtedly warrants even the most cursory glance at its symbolism. I've yet to read something by Lanchester I haven't enjoyed (even his short essay on cheating in sports, for example) and will be definitely reading more from him in 2022.
The Only Story (2018) Julian Barnes The Only Story is the story of Paul, a 19-year-old boy who falls in love with 42-year-old Susan, a married woman with two daughters who are about Paul's age. The book begins with how Paul meets Susan in happy (albeit complicated) circumstances, but as the story unfolds, the novel becomes significantly more tragic and moving. Whilst the story begins from the first-person perspective, midway through the book it shifts into the second person, and, later, into the third as well. Both of these narrative changes suggested to me an attempt on the part of Paul the narrator (if not Barnes himself), to distance himself emotionally from the events taking place. This effect is a lot more subtle than it sounds, however: far more prominent and devastating is the underlying and deeply moving story about the relationship ends up. Throughout this touching book, Barnes uses his mastery of language and observation to avoid the saccharine and the maudlin, and ends up with a heart-wrenching and emotive narrative. Without a doubt, this is the saddest book I read this year.
Series: | Fractured Fables #1 |
Publisher: | Tordotcom |
Copyright: | 2021 |
ISBN: | 1-250-76536-6 |
Format: | Kindle |
Pages: | 121 |
"You know it wasn't originally a spinning wheel in the story?" I offer, because alcohol transforms me into a chatty Wikipedia page.A Spindle Splintered is told from Zinnia's first-person perspective, and Zinnia is great. My favorite thing about Harrow's writing is the fierce and complex emotions of her characters. The overall tone is lighter than The Once and Future Witches or The Ten Thousand Doors of January, but Harrow doesn't shy away from showing the reader Zinnia's internal thought process about her illness (and her eye-rolling bemusement at some of the earlier emotional stages she went through).
Dying girl rule #3 is no romance, because my entire life is one long trolley problem and I don't want to put any more bodies on the tracks. (I've spent enough time in therapy to know that this isn't "a healthy attitude towards attachment," but I personally feel that accepting my own imminent mortality is enough work without also having a healthy attitude about it.)There's a content warning for parents here, since Harrow spends some time on the reaction of Zinnia's parents and the complicated dance between hope, despair, smothering, and freedom that she and they had to go through. There were no easy answers and all balances were fragile, but Zinnia always finds her feet. For me, Harrow's character writing is like emotional martial arts: rolling with punches, taking falls, clear-eyed about the setbacks, but always finding a new point of stability to punch back at the world. Zinnia adds just enough teenage irreverence and impatience to blunt the hardest emotional hits. I really enjoy reading it. The one caution I will make about that part of the story is that the focus is on Zinnia's projected lifespan and not on her illness specifically. Harrow uses it as setup to dig into how she and her parents would react to that knowledge (and I thought those parts were done well), but it's told from the perspective of "what would you do if you knew your date of death," not from the perspective of someone living with a disability. It is to some extent disability as plot device, and like the fairy tale that it's based on, it's deeply invested in the "find a cure" approach to the problem. I'm not disabled and am not the person to ask about how well a story handles disability, but I suspect this one may leave something to be desired. I thought the opening of this story is great. Zinnia is a great first-person protagonist and the opening few chapters are overflowing with snark and acerbic commentary. Dumping Zinnia into another world but having text messaging still work is genius, and I kind of wish Harrow had made that even more central to the book. The rest of the story was good but not as good, and the ending was somewhat predictable and a bit of a deus ex machina. But the characters carried it throughout, and I will happily read more of this. Recommended, with the caveat about disability and the content warning for parents. Followed by A Mirror Mended, which I have already pre-ordered. Rating: 8 out of 10
Series: | Eternal Sky #2 |
Publisher: | Tor |
Copyright: | March 2013 |
ISBN: | 0-7653-2755-4 |
Format: | Hardcover |
Pages: | 333 |
Series: | Raybearer #1 |
Publisher: | Amulet Books |
Copyright: | 2020 |
ISBN: | 1-68335-719-1 |
Format: | Kindle |
Pages: | 308 |
For the kid scanning fairy tales for a hero with a face like theirs. And for the girls whose stories we compressed into pities and wonders, triumphs and cautions, without asking, even once, for their names.I think she was successful on both parts of that promise, and it makes for some great reading. Recommended. Followed by Redemptor. Rating: 8 out of 10
mbsync
).
But I also evaluated OfflineIMAP which was resurrected from the Python
2 apocalypse, and because I had used it before, for a long time.
Read on for the details.
$ notmuch count --exclude=false
372758
$ du -sh --exclude xapian Maildir
13G Maildir
The baseline we are comparing against is SMD (syncmaildir) which
performs the sync in about 7-8 seconds locally (3.5 seconds for each
push/pull command) and about 10-12 seconds remotely.
Anything close to that or better is good enough. I do not have recent
numbers for a SMD full sync baseline, but the setup
documentation
mentions 20 minutes for a full sync. That was a few years ago, and the
spool has obviously grown since then, so that is not a reliable
baseline.
A baseline for a full sync might be also set with rsync, which copies
files at nearly 40MB/s, or 317Mb/s!
anarcat@angela:tmp(main)$ time rsync -a --info=progress2 --exclude xapian shell.anarc.at:Maildir/ Maildir/
12,647,814,731 100% 37.85MB/s 0:05:18 (xfr#394981, to-chk=0/395815)
72.38user 106.10system 5:19.59elapsed 55%CPU (0avgtext+0avgdata 15988maxresident)k
8816inputs+26305112outputs (0major+50953minor)pagefaults 0swaps
That is 5 minutes to transfer the entire spool. Incremental syncs are
obviously pretty fast too:
anarcat@angela:tmp(main)$ time rsync -a --info=progress2 --exclude xapian shell.anarc.at:Maildir/ Maildir/
0 0% 0.00kB/s 0:00:00 (xfr#0, to-chk=0/395815)
1.42user 0.81system 0:03.31elapsed 67%CPU (0avgtext+0avgdata 14100maxresident)k
120inputs+0outputs (3major+12709minor)pagefaults 0swaps
As an extra curiosity, here's the performance with tar
, pretty
similar with rsync
, minus incremental which I cannot be bothered to
figure out right now:
anarcat@angela:tmp(main)$ time ssh shell.anarc.at tar --exclude xapian -cf - Maildir/ pv -s 13G tar xf -
56.68user 58.86system 5:17.08elapsed 36%CPU (0avgtext+0avgdata 8764maxresident)k
0inputs+0outputs (0major+7266minor)pagefaults 0swaps
12,1GiO 0:05:17 [39,0MiB/s] [===================================================================> ] 92%
Interesting that rsync
manages to almost beat a plain tar
on file
transfer, I'm actually surprised by how well it performs here,
considering there are many little files to transfer.
(But then again, this maybe is exactly where rsync
shines: while
tar
needs to glue all those little files together, rsync
can just
directly talk to the other side and tell it to do live
changes. Something to look at in another article maybe?)
Since both ends are NVMe drives, those should easily saturate a
gigabit link. And in fact, a backup of the server mail spool achieves
much faster transfer rate on disks:
anarcat@marcos:~$ tar fc - Maildir pv -s 13G > Maildir.tar
15,0GiO 0:01:57 [ 131MiB/s] [===================================] 115%
That's 131Mibyyte per second, vastly faster than the gigabit
link. The client has similar performance:
anarcat@angela:~(main)$ tar fc - Maildir pv -s 17G > Maildir.tar
16,2GiO 0:02:22 [ 116MiB/s] [==================================] 95%
So those disks should be able to saturate a gigabit link, and they are
not the bottleneck on fast links. Which begs the question of what is
blocking performance of a similar transfer over the gigabit link, but
that's another question altogether, because no sync program ever
reaches the above performance anyways.
Finally, note that when I migrated to SMD, I wrote a small
performance
comparison that
could be interesting here. It show SMD to be faster than OfflineIMAP,
but not as much as we see here. In fact, it looks like OfflineIMAP
slowed down significantly since then (May 2018), but this could be due
to my larger mail spool as well.
mbsync
) project is written in C and supports
syncing Maildir and IMAP folders, with possibly multiple replicas. I
haven't tested this but I suspect it might be possible to sync between
two IMAP servers as well. It supports partial mirorrs, message flags,
full folder support, and "trash" functionality.
.mbsyncrc
configuration file:
SyncState *
Sync New ReNew Flags
IMAPAccount anarcat
Host imap.anarc.at
User anarcat
PassCmd "pass imap.anarc.at"
SSLType IMAPS
CertificateFile /etc/ssl/certs/ca-certificates.crt
IMAPStore anarcat-remote
Account anarcat
MaildirStore anarcat-local
# Maildir/top/sub/sub
#SubFolders Verbatim
# Maildir/.top.sub.sub
SubFolders Maildir++
# Maildir/top/.sub/.sub
# SubFolders legacy
# The trailing "/" is important
#Path ~/Maildir-mbsync/
Inbox ~/Maildir-mbsync/
Channel anarcat
# AKA Far, convert when all clients are 1.4+
Master :anarcat-remote:
# AKA Near
Slave :anarcat-local:
# Exclude everything under the internal [Gmail] folder, except the interesting folders
#Patterns * ![Gmail]* "[Gmail]/Sent Mail" "[Gmail]/Starred" "[Gmail]/All Mail"
# Or include everything
Patterns *
# Automatically create missing mailboxes, both locally and on the server
#Create Both
Create slave
# Sync the movement of messages between folders and deletions, add after making sure the sync works
#Expunge Both
Long gone are the days where I would spend a long time reading a
manual page to figure out the meaning of every option. If that's your
thing, you might like this one. But I'm more of a "EXAMPLES
section" kind of person now, and I somehow couldn't find a sample file
on the website. I started from the Arch wiki one but it's
actually not great because it's made for Gmail (which is not a usual
Dovecot server). So a sample config file in the manpage would be a
great addition. Thankfully, the Debian packages ships one in
/usr/share/doc/isync/examples/mbsyncrc.sample
but I only found that
after I wrote my configuration. It was still useful and I recommend
people take a look if they want to understand the syntax.
Also, that syntax is a little overly complicated. For example, Far
needs colons, like:
Far :anarcat-remote:
Why? That seems just too complicated. I also found that sections are
not clearly identified: IMAPAccount
and Channel
mark section
beginnings, for example, which is not at all obvious until you learn
about mbsync
's internals. There are also weird ordering issues: the
SyncState
option needs to be before IMAPAccount
, presumably
because it's global.
Using a more standard format like .INI or TOML could improve that
situation.
rsync
.
The incremental runs are roughly 2 seconds, which is even more
impressive, as that's actually faster than rsync
:
===> multitime results
1: mbsync -a
Mean Std.Dev. Min Median Max
real 2.015 0.052 1.930 2.029 2.105
user 0.660 0.040 0.592 0.661 0.722
sys 0.338 0.033 0.268 0.341 0.387
Those tests were performed with isync 1.3.0-2.2 on Debian
bullseye. Tests with a newer isync release originally failed because
of a corrupted message that triggered bug 999804 (see
below). Running 1.4.3 under valgrind works around the bug, but adds a
50% performance cost, the full sync running in 1h35m.
Once the upstream patch is applied, performance with 1.4.3 is fairly
similar, considering that the new sync included the register
folder
with 4000 messages:
120.74user 213.19system 59:47.69elapsed 9%CPU (0avgtext+0avgdata 105420maxresident)k
29128inputs+28284376outputs (0major+45711minor)pagefaults 0swaps
That is ~13GB in ~60 minutes, which gives us 28.3Mbps. Incrementals
are also pretty similar to 1.3.x, again considering the double-connect
cost:
===> multitime results
1: mbsync -a
Mean Std.Dev. Min Median Max
real 2.500 0.087 2.340 2.491 2.629
user 0.718 0.037 0.679 0.711 0.793
sys 0.322 0.024 0.284 0.320 0.365
Those tests were all done on a Gigabit link, but what happens on a
slower link? My server uplink is slow: 25 Mbps down, 6 Mbps up. There
mbsync
is worse than the SMD baseline:
===> multitime results
1: mbsync -a
Mean Std.Dev. Min Median Max
real 31.531 0.724 30.764 31.271 33.100
user 1.858 0.125 1.721 1.818 2.131
sys 0.610 0.063 0.506 0.600 0.695
That's 30 seconds for a sync, which is an order of magnitude slower
than SMD.
mbsync
UI is kind of neat:
anarcat@angela:~(main)$ mbsync -a
Notice: Master/Slave are deprecated; use Far/Near instead.
C: 1/2 B: 204/205 F: +0/0 *0/0 #0/0 N: +1/200 *0/0 #0/0
(Note that nice switch away from slavery-related terms too.)
The display is minimal, and yet informative. It's not obvious what
does mean at first glance, but the manpage is useful at least for
clarifying that:
This represents the cumulative progress over channels, boxes, and messages affected on the far and near side, respectively. The message counts represent added messages, messages with updated flags, and trashed messages, respectively. No attempt is made to calculate the totals in advance, so they grow over time as more information is gathered. (Emphasis mine).In other words:
C 2/2
: channels done/total (2 done out of 2)B 204/205
: mailboxes done/total (204 out of 205)F
: changes on the far sideN: +10/200 *0/0 #0/0
: changes on the "near" side:
+10/200
: 10 out of 200 messages downloaded*0/0
: no flag changed#0/0
: no message deletedS
to "mark spam", which basically assigns the tag spam
to the
message and removes a bunch of others. Then I have a
notmuch-purge script which moves that message to the spam folder,
for training purposes. It basically does this:
notmuch search --output=files --format=text0 "$search_spam" \
xargs -r -0 mv -t "$HOME/Maildir/$ PREFIX junk/cur/"
This method, which worked fine in SMD (and also OfflineIMAP) created
this error on sync:
Maildir error: duplicate UID 37578.
And indeed, there are now two messages with that UID in the mailbox:
anarcat@angela:~(main)$ find Maildir/.junk/ -name '*U=37578*'
Maildir/.junk/cur/1637427889.134334_2.angela,U=37578:2,S
Maildir/.junk/cur/1637348602.2492889_221804.angela,U=37578:2,S
This is actually a known limitation or, as mbsync(1) calls it, a
"RECOMMENDATION":
When using the more efficient default UID mapping scheme, it is important that the MUA renames files when moving them between Maildir fold ers. Mutt always does that, while mu4e needs to be configured to do it:So it seems I would need to fix my script. It's unclear how the paths should be renamed, which is unfortunate, because I would need to change my script to adapt to(setq mu4e-change-filenames-when-moving t)
mbsync
, but I can't tell how just from
reading the above.
(A manual fix is actually to rename the file to remove the U=
field:
mbsync
will generate a new one and then sync correctly.)
Fortunately, someone else already fixed that issue: afew, a
notmuch tagging script (much puns, such hurt), has a move mode
that can rename files correctly, specifically designed to deal
with mbsync
. I had already been told about afew, but it's one more
reason to standardize my notmuch hooks on that project, it looks like.
Update: I have tried to use afew and found it has significant
performance issues. It also has a completely different paradigm
to what I am used to: it assumes all incoming mail has a new
and
lays its own tags on top of that (inbox
, sent
, etc). It can only
move files from one folder at a time (see this bug) which
breaks my spam training workflow. In general, I sync my tags into
folders (e.g. ham
, spam
, sent
) and message flags (e.g. inbox
is F
, unread
is "not S
", etc), and afew is not well suited for
this (although there are hacks that try to fix this). I have
worked hard to make my tagging scripts idempotent, and it's something
afew doesn't currently have. Still, it would be better to have
that code in Python than bash, so maybe I should consider my options
here.
mbsync
is really fast, but the downside of that is that
it's written in C, and with that comes a whole set of security
issues. The Debian security tracker has only three CVEs on
isync, but the above issues show there could be many more.
Reading the source code certainly did not make me very comfortable
with trusting it with untrusted data. I considered sandboxing it with
systemd (below) but having systemd run as a --user
process makes
that difficult. I also considered using an apparmor profile but
that is not trivial because we need to allow SSH and only some parts
of it...
Thankfully, upstream has been diligent at addressing the issues I have
found. They provided a patch within a few days which did fix the sync
issues.
Update: upstream actually took the issue very seriously. They not only
got CVE-2021-44143 assigned for my bug report, they also
audited the code and found several more issues collectively identified
as CVE-2021-3657, which actually also affect 1.3 (ie. Debian
11/bullseye/stable). Somehow my corpus doesn't trigger that issue, but
it was still considered serious enough to warrant a CVE. So one the
one hand: excellent response from upstream; but on the other hand: how
many more of those could there be in there?
mbsync
as a
systemd service. It suggests using the --verbose
(-V
) flag which
is a little intense here, as it outputs 1444 lines of messages.
I have used the following .service
file:
[Unit]
Description=Mailbox synchronization service
ConditionHost=!marcos
Wants=network-online.target
After=network-online.target
Before=notmuch-new.service
[Service]
Type=oneshot
ExecStart=/usr/bin/mbsync -a
Nice=10
IOSchedulingClass=idle
NoNewPrivileges=true
[Install]
WantedBy=default.target
And the following .timer
:
[Unit]
Description=Mailbox synchronization timer
ConditionHost=!marcos
[Timer]
OnBootSec=2m
OnUnitActiveSec=5m
Unit=mbsync.service
[Install]
WantedBy=timers.target
Note that we trigger notmuch
through systemd, with the Before
and
also by adding mbsync.service
to the notmuch-new.service
file:
[Unit]
Description=notmuch new
After=mbsync.service
[Service]
Type=oneshot
Nice=10
ExecStart=/usr/bin/notmuch new
[Install]
WantedBy=mbsync.service
An improvement over polling repeatedly with a .timer
would be to
wake up only on IMAP notify, but neither imapnotify nor
goimapnotify seem to be packaged in Debian. It would also not
cover for the "sent folder" use case, where we need to wake up on
local changes.
IMAPStore remote
Tunnel "ssh -q host.remote.com /usr/sbin/imapd"
Add BatchMode
, restrict to IdentitiesOnly
, provide a password-less
key just for this, add compression (-C
), find the Dovecot imap
binary, and you get this:
IMAPAccount anarcat-tunnel
Tunnel "ssh -o BatchMode=yes -o IdentitiesOnly=yes -i ~/.ssh/id_ed25519_mbsync -o HostKeyAlias=shell.anarc.at -C anarcat@imap.anarc.at /usr/lib/dovecot/imap"
And it actually seems to work:
$ mbsync -a
Notice: Master/Slave are deprecated; use Far/Near instead.
C: 0/2 B: 0/1 F: +0/0 *0/0 #0/0 N: +0/0 *0/0 #0/0imap(anarcat): Error: net_connect_unix(/run/dovecot/stats-writer) failed: Permission denied
C: 2/2 B: 205/205 F: +0/0 *0/0 #0/0 N: +1/1 *3/3 #0/0imap(anarcat)<1611280><90uUOuyElmEQlhgAFjQyWQ>: Info: Logged out in=10808 out=15396642 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=1 body_bytes=8087
It's a bit noisy, however. dovecot/imap
doesn't have a "usage" to
speak of, but even the source code doesn't hint at a way to disable
that Error
message, so that's unfortunate. That socket is owned by
root:dovecot
so presumably Dovecot runs the imap
process as
$user:dovecot
, which we can't do here. Oh well?
Interestingly, the SSH setup is not faster than IMAP.
With IMAP:
===> multitime results
1: mbsync -a
Mean Std.Dev. Min Median Max
real 2.367 0.065 2.220 2.376 2.458
user 0.793 0.047 0.731 0.776 0.871
sys 0.426 0.040 0.364 0.434 0.476
With SSH:
===> multitime results
1: mbsync -a
Mean Std.Dev. Min Median Max
real 2.515 0.088 2.274 2.532 2.594
user 0.753 0.043 0.645 0.766 0.804
sys 0.328 0.045 0.212 0.340 0.393
Basically: 200ms slower. Tolerable.
mbsync
on my first workstation. The
work on the second one was more streamlined, especially since the
corruption on mailboxes was fixed:
dpkg -i isync_1.4.3-1.1~_amd64.deb
rsync -a --info=progress2 angela:Maildir/ Maildir-mbsync/
find Maildir-mbsync/ -type f -name '*.angela,*' -print0 rename -0 's/\.angela,/\.curie,/'
rm -rf Maildir-mbsync/.notmuch/xapian/
systemctl --user --now disable smd-pull.service smd-pull.timer smd-push.service smd-push.timer notmuch-new.service notmuch-new.timer
smd-pull --show-tags ; smd-push --show-tags ; notmuch new ; notmuch-sync-flagged -v
notmuch dump pv > notmuch.dump
cp -al Maildir Maildir-bak
ssh-keygen -t ed25519 -f .ssh/id_ed25519_mbsync
cat .ssh/id_ed25519_mbsync.pub
.ssh/authorized_keys
on the server, like this:
command="/usr/lib/dovecot/imap",restrict ssh-ed25519 AAAAC...mv Maildir Maildir-smd
mv Maildir-mbsync Maildir
notmuch new
pv notmuch.dump notmuch restore
smd-*
services:
systemctl --user enable mbsync.timer notmuch-new.service
systemctl --user start mbsync.timer
rm ~/.config/systemd/user/smd*
systemctl daemon-reload[...]
Warning: cannot apply tags to missing message: CAN6gO7_QgCaiDFvpG3AXHi6fW12qaN286+2a7ERQ2CQtzjSEPw@mail.gmail.com
Warning: cannot apply tags to missing message: CAPTU9Wmp0yAmaxO+qo8CegzRQZhCP853TWQ_Ne-YF94MDUZ+Dw@mail.gmail.com
Warning: cannot apply tags to missing message: F5086003-2917-4659-B7D2-66C62FCD4128@gmail.com
[...]
Warning: cannot apply tags to missing message: mailman.2.1316793601.53477.sage-members@mailman.sage.org
Warning: cannot apply tags to missing message: mailman.7.1317646801.26891.outages-discussion@outages.org
Warning: cannot apply tags to missing message: notmuch-sha1-000458df6e48d4857187a000d643ac971deeef47
Warning: cannot apply tags to missing message: notmuch-sha1-0079d8e0c3340e6f88c66f4c49fca758ea71d06d
Warning: cannot apply tags to missing message: notmuch-sha1-0194baa4cfb6d39bc9e4d8c049adaccaa777467d
Warning: cannot apply tags to missing message: notmuch-sha1-02aede494fc3f9e9f060cfd7c044d6d724ad287c
Warning: cannot apply tags to missing message: notmuch-sha1-06606c625d3b3445420e737afd9a245ae66e5562
Warning: cannot apply tags to missing message: notmuch-sha1-0747b020f7551415b9bf5059c58e0a637ba53b13
[...]
As detailed in the crash report,
all of those were actually innocuous and could be ignored.
Also note that we completely trash the notmuch
database because it's
actually faster to reindex from scratch than let notmuch
slowly
figure out that all mails are new and all the old mails are
gone. The fresh indexing took:
nov 19 15:08:54 angela notmuch[2521117]: Processed 384679 total files in 23m 41s (270 files/sec.).
nov 19 15:08:54 angela notmuch[2521117]: Added 372610 new messages to the database.
While a reindexing on top of an existing database was going twice as
slow, at about 120 files/sec.
SyncState *
Sync All
# IMAP side, AKA "Far"
IMAPAccount anarcat-imap
Host imap.anarc.at
User anarcat
PassCmd "pass imap.anarc.at"
SSLType IMAPS
CertificateFile /etc/ssl/certs/ca-certificates.crt
IMAPAccount anarcat-tunnel
Tunnel "ssh -o BatchMode=yes -o IdentitiesOnly=yes -i ~/.ssh/id_ed25519_mbsync -o HostKeyAlias=shell.anarc.at -C anarcat@imap.anarc.at /usr/lib/dovecot/imap"
IMAPStore anarcat-remote
Account anarcat-tunnel
# Maildir side, AKA "Near"
MaildirStore anarcat-local
# Maildir/top/sub/sub
#SubFolders Verbatim
# Maildir/.top.sub.sub
SubFolders Maildir++
# Maildir/top/.sub/.sub
# SubFolders legacy
# The trailing "/" is important
#Path ~/Maildir-mbsync/
Inbox ~/Maildir/
# what binds Maildir and IMAP
Channel anarcat
Far :anarcat-remote:
Near :anarcat-local:
# Exclude everything under the internal [Gmail] folder, except the interesting folders
#Patterns * ![Gmail]* "[Gmail]/Sent Mail" "[Gmail]/Starred" "[Gmail]/All Mail"
# Or include everything
#Patterns *
Patterns * !register !.register
# Automatically create missing mailboxes, both locally and on the server
Create Both
#Create Near
# Sync the movement of messages between folders and deletions, add after making sure the sync works
Expunge Both
# Propagate mailbox deletion
Remove both
IMAPAccount anarcat-register-imap
Host imap.anarc.at
User register
PassCmd "pass imap.anarc.at-register"
SSLType IMAPS
CertificateFile /etc/ssl/certs/ca-certificates.crt
IMAPAccount anarcat-register-tunnel
Tunnel "ssh -o BatchMode=yes -o IdentitiesOnly=yes -i ~/.ssh/id_ed25519_mbsync -o HostKeyAlias=shell.anarc.at -C register@imap.anarc.at /usr/lib/dovecot/imap"
IMAPStore anarcat-register-remote
Account anarcat-register-tunnel
MaildirStore anarcat-register-local
SubFolders Maildir++
Inbox ~/Maildir/.register/
Channel anarcat-register
Far :anarcat-register-remote:
Near :anarcat-register-local:
Create Both
Expunge Both
Remove both
Note that it may be out of sync with my live (and private)
configuration file, as I do not publish my "dotfiles" repository
publicly for security reasons.
notmuch
, and
would sometimes crash mysteriously. It's been a while, so my memory is
hazy on that.
It also kind of died in a fire when Python 2 stop being
maintained. The main author moved on to a different project,
imapfw which could serve as a framework to build IMAP clients,
but never seemed to implement all of the OfflineIMAP features and
certainly not configuration file compatibility. Thankfully, a new team
of volunteers ported OfflineIMAP to Python 3 and we can now test that
new version to see if it is an improvement over mbsync
.
Copy message from RemoteAnarcat:junk:
ERROR: Copying message 30624 [acc: Anarcat]
decoding with 'X-EUC-TW' codec failed (AttributeError: 'memoryview' object has no attribute 'decode')
Thread 'Copy message from RemoteAnarcat:junk' terminated with exception:
Traceback (most recent call last):
File "/usr/share/offlineimap3/offlineimap/imaputil.py", line 406, in utf7m_decode
for c in binary.decode():
AttributeError: 'memoryview' object has no attribute 'decode'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/share/offlineimap3/offlineimap/threadutil.py", line 146, in run
Thread.run(self)
File "/usr/lib/python3.9/threading.py", line 892, in run
self._target(*self._args, **self._kwargs)
File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 802, in copymessageto
message = self.getmessage(uid)
File "/usr/share/offlineimap3/offlineimap/folder/IMAP.py", line 342, in getmessage
data = self._fetch_from_imap(str(uid), self.retrycount)
File "/usr/share/offlineimap3/offlineimap/folder/IMAP.py", line 908, in _fetch_from_imap
ndata1 = self.parser['8bit-RFC'].parsebytes(data[0][1])
File "/usr/lib/python3.9/email/parser.py", line 123, in parsebytes
return self.parser.parsestr(text, headersonly)
File "/usr/lib/python3.9/email/parser.py", line 67, in parsestr
return self.parse(StringIO(text), headersonly=headersonly)
File "/usr/lib/python3.9/email/parser.py", line 56, in parse
feedparser.feed(data)
File "/usr/lib/python3.9/email/feedparser.py", line 176, in feed
self._call_parse()
File "/usr/lib/python3.9/email/feedparser.py", line 180, in _call_parse
self._parse()
File "/usr/lib/python3.9/email/feedparser.py", line 385, in _parsegen
for retval in self._parsegen():
File "/usr/lib/python3.9/email/feedparser.py", line 298, in _parsegen
for retval in self._parsegen():
File "/usr/lib/python3.9/email/feedparser.py", line 385, in _parsegen
for retval in self._parsegen():
File "/usr/lib/python3.9/email/feedparser.py", line 256, in _parsegen
if self._cur.get_content_type() == 'message/delivery-status':
File "/usr/lib/python3.9/email/message.py", line 578, in get_content_type
value = self.get('content-type', missing)
File "/usr/lib/python3.9/email/message.py", line 471, in get
return self.policy.header_fetch_parse(k, v)
File "/usr/lib/python3.9/email/policy.py", line 163, in header_fetch_parse
return self.header_factory(name, value)
File "/usr/lib/python3.9/email/headerregistry.py", line 601, in __call__
return self[name](name, value)
File "/usr/lib/python3.9/email/headerregistry.py", line 196, in __new__
cls.parse(value, kwds)
File "/usr/lib/python3.9/email/headerregistry.py", line 445, in parse
kwds['parse_tree'] = parse_tree = cls.value_parser(value)
File "/usr/lib/python3.9/email/_header_value_parser.py", line 2675, in parse_content_type_header
ctype.append(parse_mime_parameters(value[1:]))
File "/usr/lib/python3.9/email/_header_value_parser.py", line 2569, in parse_mime_parameters
token, value = get_parameter(value)
File "/usr/lib/python3.9/email/_header_value_parser.py", line 2492, in get_parameter
token, value = get_value(value)
File "/usr/lib/python3.9/email/_header_value_parser.py", line 2403, in get_value
token, value = get_quoted_string(value)
File "/usr/lib/python3.9/email/_header_value_parser.py", line 1294, in get_quoted_string
token, value = get_bare_quoted_string(value)
File "/usr/lib/python3.9/email/_header_value_parser.py", line 1223, in get_bare_quoted_string
token, value = get_encoded_word(value)
File "/usr/lib/python3.9/email/_header_value_parser.py", line 1064, in get_encoded_word
text, charset, lang, defects = _ew.decode('=?' + tok + '?=')
File "/usr/lib/python3.9/email/_encoded_words.py", line 181, in decode
string = bstring.decode(charset)
AttributeError: decoding with 'X-EUC-TW' codec failed (AttributeError: 'memoryview' object has no attribute 'decode')
Last 1 debug messages logged for Copy message from RemoteAnarcat:junk prior to exception:
thread: Register new thread 'Copy message from RemoteAnarcat:junk' (account 'Anarcat')
ERROR: Exceptions occurred during the run!
ERROR: Copying message 30624 [acc: Anarcat]
decoding with 'X-EUC-TW' codec failed (AttributeError: 'memoryview' object has no attribute 'decode')
Traceback:
File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 802, in copymessageto
message = self.getmessage(uid)
File "/usr/share/offlineimap3/offlineimap/folder/IMAP.py", line 342, in getmessage
data = self._fetch_from_imap(str(uid), self.retrycount)
File "/usr/share/offlineimap3/offlineimap/folder/IMAP.py", line 908, in _fetch_from_imap
ndata1 = self.parser['8bit-RFC'].parsebytes(data[0][1])
File "/usr/lib/python3.9/email/parser.py", line 123, in parsebytes
return self.parser.parsestr(text, headersonly)
File "/usr/lib/python3.9/email/parser.py", line 67, in parsestr
return self.parse(StringIO(text), headersonly=headersonly)
File "/usr/lib/python3.9/email/parser.py", line 56, in parse
feedparser.feed(data)
File "/usr/lib/python3.9/email/feedparser.py", line 176, in feed
self._call_parse()
File "/usr/lib/python3.9/email/feedparser.py", line 180, in _call_parse
self._parse()
File "/usr/lib/python3.9/email/feedparser.py", line 385, in _parsegen
for retval in self._parsegen():
File "/usr/lib/python3.9/email/feedparser.py", line 298, in _parsegen
for retval in self._parsegen():
File "/usr/lib/python3.9/email/feedparser.py", line 385, in _parsegen
for retval in self._parsegen():
File "/usr/lib/python3.9/email/feedparser.py", line 256, in _parsegen
if self._cur.get_content_type() == 'message/delivery-status':
File "/usr/lib/python3.9/email/message.py", line 578, in get_content_type
value = self.get('content-type', missing)
File "/usr/lib/python3.9/email/message.py", line 471, in get
return self.policy.header_fetch_parse(k, v)
File "/usr/lib/python3.9/email/policy.py", line 163, in header_fetch_parse
return self.header_factory(name, value)
File "/usr/lib/python3.9/email/headerregistry.py", line 601, in __call__
return self[name](name, value)
File "/usr/lib/python3.9/email/headerregistry.py", line 196, in __new__
cls.parse(value, kwds)
File "/usr/lib/python3.9/email/headerregistry.py", line 445, in parse
kwds['parse_tree'] = parse_tree = cls.value_parser(value)
File "/usr/lib/python3.9/email/_header_value_parser.py", line 2675, in parse_content_type_header
ctype.append(parse_mime_parameters(value[1:]))
File "/usr/lib/python3.9/email/_header_value_parser.py", line 2569, in parse_mime_parameters
token, value = get_parameter(value)
File "/usr/lib/python3.9/email/_header_value_parser.py", line 2492, in get_parameter
token, value = get_value(value)
File "/usr/lib/python3.9/email/_header_value_parser.py", line 2403, in get_value
token, value = get_quoted_string(value)
File "/usr/lib/python3.9/email/_header_value_parser.py", line 1294, in get_quoted_string
token, value = get_bare_quoted_string(value)
File "/usr/lib/python3.9/email/_header_value_parser.py", line 1223, in get_bare_quoted_string
token, value = get_encoded_word(value)
File "/usr/lib/python3.9/email/_header_value_parser.py", line 1064, in get_encoded_word
text, charset, lang, defects = _ew.decode('=?' + tok + '?=')
File "/usr/lib/python3.9/email/_encoded_words.py", line 181, in decode
string = bstring.decode(charset)
Folder junk [acc: Anarcat]:
Copy message UID 30626 (29008/49310) RemoteAnarcat:junk -> LocalAnarcat:junk
Command exited with non-zero status 100
5252.91user 535.86system 3:21:00elapsed 47%CPU (0avgtext+0avgdata 846304maxresident)k
96344inputs+26563792outputs (1189major+2155815minor)pagefaults 0swaps
That only transferred about 8GB of mail, which gives us a transfer
rate of 5.3Mbit/s, more than 5 times slower than mbsync
. This bug is
possibly limited to the bullseye
version of offlineimap3
(the
lovely 0.0~git20210225.1e7ef9e+dfsg-4
), while the current sid
version (the equally gorgeous 0.0~git20211018.e64c254+dfsg-1
) seems
unaffected.
*** Finished account 'Anarcat' in 511:12
ERROR: Exceptions occurred during the run!
ERROR: Exception parsing message with ID (<20190619152034.BFB8810E07A@marcos.anarc.at>) from imaplib (response type: bytes).
AttributeError: decoding with 'X-EUC-TW' codec failed (AttributeError: 'memoryview' object has no attribute 'decode')
Traceback:
File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 810, in copymessageto
message = self.getmessage(uid)
File "/usr/share/offlineimap3/offlineimap/folder/IMAP.py", line 343, in getmessage
data = self._fetch_from_imap(str(uid), self.retrycount)
File "/usr/share/offlineimap3/offlineimap/folder/IMAP.py", line 910, in _fetch_from_imap
raise OfflineImapError(
ERROR: Exception parsing message with ID (<40A270DB.9090609@alternatives.ca>) from imaplib (response type: bytes).
AttributeError: decoding with 'x-mac-roman' codec failed (AttributeError: 'memoryview' object has no attribute 'decode')
Traceback:
File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 810, in copymessageto
message = self.getmessage(uid)
File "/usr/share/offlineimap3/offlineimap/folder/IMAP.py", line 343, in getmessage
data = self._fetch_from_imap(str(uid), self.retrycount)
File "/usr/share/offlineimap3/offlineimap/folder/IMAP.py", line 910, in _fetch_from_imap
raise OfflineImapError(
ERROR: IMAP server 'RemoteAnarcat' does not have a message with UID '32686'
Traceback:
File "/usr/share/offlineimap3/offlineimap/folder/Base.py", line 810, in copymessageto
message = self.getmessage(uid)
File "/usr/share/offlineimap3/offlineimap/folder/IMAP.py", line 343, in getmessage
data = self._fetch_from_imap(str(uid), self.retrycount)
File "/usr/share/offlineimap3/offlineimap/folder/IMAP.py", line 889, in _fetch_from_imap
raise OfflineImapError(reason, severity)
Command exited with non-zero status 1
8273.52user 983.80system 8:31:12elapsed 30%CPU (0avgtext+0avgdata 841936maxresident)k
56376inputs+43247608outputs (811major+4972914minor)pagefaults 0swaps
"offlineimap -o " took 8 hours 31 mins 15 secs
This is 8h31m for transferring 12G, which is around 3.1Mbit/s. That is
nine times slower than mbsync
, almost an order of magnitude!
Now that we have a full sync, we can test incremental
synchronization. That is also much slower:
===> multitime results
1: sh -c "offlineimap -o true"
Mean Std.Dev. Min Median Max
real 24.639 0.513 23.946 24.526 25.708
user 23.912 0.473 23.404 23.795 24.947
sys 1.743 0.105 1.607 1.729 2.002
That is also an order of magnitude slower than mbsync
, and
significantly slower than what you'd expect from a sync process. ~30
seconds is long enough to make me impatient and distracted; 3 seconds,
less so: I can wait and see the results almost immediately.
mbsync
over a slow
link, but I Haven't tested that theory.
The OfflineIMAP mail spool is missing quite a few messages as well:
anarcat@angela:~(main)$ find Maildir-offlineimap -type f -type f -a \! -name '.*' wc -l
381463
anarcat@angela:~(main)$ find Maildir -type f -type f -a \! -name '.*' wc -l
385247
... although that's probably all either new messages or the
register
folder, so OfflineIMAP might actually be in a better
position there. But digging in more, it seems like the actual
per-folder diff is fairly similar to mbsync
: a few messages missing
here and there. Considering OfflineIMAP's instability and poor
performance, I have not looked any deeper in those discrepancies.
doveadm
and offlineimap
, but requires running an IMAP server
locally, Perlrsmtp
which is a nice name
for rsendmail
. not evaluated because it seems awfully complex to
setup, Haskellmbsync
to sync my mail. I'm a little disappointed by
the synchronisation times over the slow link, but I guess that's on
par for the course if we use IMAP. We are bound by the network speed
much more than with custom protocols. I'm also worried about the C
implementation and the crashes I have witnessed, but I am encouraged
by the fast upstream response.
Time will tell if I will stick with that setup. I'm certainly curious
about the promises of interimap and mail-sync, but I have ran out of
time on this project.
Series: | The Scholomance #2 |
Publisher: | Del Rey |
Copyright: | 2021 |
ISBN: | 0-593-12887-7 |
Format: | Kindle |
Pages: | 388 |
[She] came round to me and put her arm around my waist and said under her breath, "Hey, she can be taught," with a tease in her voice that wobbled a little, and when I looked at her, her eyes were bright and wet, and I put my arm around her shoulders and hugged her.You'll know it when you get there. The Last Graduate also gives the characters other than El and Orion more room, which is part of how it handles the chosen one trope. It's been obvious since early in the first book that Orion is a sort of chosen one, and it becomes obvious to the reader that El may be as well. But Novik doesn't let the plot focus only on them; instead, she uses that trope to look at how alliances and collective action happen, and how no one can carry the weight by themselves. As El learns more and gains power, she also becomes less central to the plot resolution and has to learn how to be less self-reliant. This is not a book where one character is trained to save the world. It's a book where she manages to enlist the support of a kick-ass project manager and becomes part of a team. Middle books of a trilogy are notoriously challenging. Often they're travel books: the first book sets up a problem, the second book moves the characters both physically and emotionally into a position to solve the problem, and the third book is the payoff. Travel books often sag. They can feel obligatory but somewhat boring, like a chore on the way to the third-book climax. The Last Graduate is not a travel book; it is, instead, a pivot book, which is my favorite form of trilogy. It's a book that rewrites the problem the first book set up, both resolving it and expanding the scope beyond what the reader had expected. This is immensely satisfying when done well, and Novik does it extremely well. This is not a flawless book. There are some pacing hiccups, there is a romance angle that didn't work for me (although it does arrive at some character insights that I thought were spot on), and although I think Novik is doing something interesting with the trope, there is a lot of chosen one power escalation happening here. It's not the sort of book that I can claim is perfectly written. Instead, it's the sort of book that uses some of my favorite plot elements and emotional beats in such an effective way and with such a memorable character that I do not have it in me to care about any of the flaws. Your mileage may therefore vary, but I would be happy to read books like this until the end of time. As mentioned above, The Last Graduate ends on another cliffhanger. This time I was worried that Novik might have ended the series there, since there's enough of an internal climax that I could imagine some literary fiction (which often seems allergic to endings) would have stopped here. Thankfully, Novik's web site says this is not the case. The next year is going to be a difficult wait. The third book of this series is going to be incredibly difficult to write, and I hope Novik is up to the challenge she's made for herself. But she handled the transition between the first and second book so well, and this book is so good that I have a lot of hope. If the third book is half as good as I'm hoping, this is going to be one of my favorite fantasy series of all time. Followed by an as-yet-untitled third book. Rating: 10 out of 10
Series: | Seraphina #2 |
Publisher: | Ember |
Copyright: | 2015 |
ISBN: | 0-375-89659-7 |
Format: | Kindle |
Pages: | 458 |
tools.deps.alpha
, a library for dependency graph
resolution and classpath building, and the CLI tool clj
, for REPL
interaction. If time permitted, I was also to improve the quality of
both new and existing Clojure packages, and the overall Debian Clojure
packaging process. My mentor was Louis-Louis-Philippe V ronneau, and my
co-mentor was Utkarsh Gupta.clojure-debian-helper
). The second reason for which we only
currently have a suboptimal Clojure experience in Debian, and probably
the root of the previous one, is that many core build tools and
libraries for the language have not simply been packaged yet. My project
aimed to attack that seemingly root cause.As I said, another reason for me choosing this project is my own
experience as the Co-founder and Leader of, probably, the first Free
Software Community experience in my hometown of San Juan, Argentina.
That interest in Free Software evolved in a first PhD attempt in what is
now known as the field of Peer Production. A subject that has lived
within me as a research interest during my day job at a University.Being a Clojure fan, it felt only logical combining all those interests
somehow. And this project seemed like the ideal combination.clojure-cli
.clojure-cli
.clojure
to use clojure-cli
.clojure-cli
scripts.Series: | Between Earth and Sky #1 |
Publisher: | Saga Press |
Copyright: | October 2020 |
ISBN: | 1-5344-3769-X |
Format: | Kindle |
Pages: | 454 |
$_vbe_prompt_compact
set to 1 when we want a compact prompt. We use the
following function to define the prompt appearance:
_vbe_prompt () local retval=$? # When compact, just time + prompt sign if (( $_vbe_prompt_compact )); then # Current time (with timezone for remote hosts) _vbe_prompt_segment cyan default "%D %H:%M$ SSH_TTY+ %Z " # Hostname for remote hosts [[ $SSH_TTY ]] && \ _vbe_prompt_segment black magenta "%B%M%b" # Status of the last command if (( $retval )); then _vbe_prompt_segment red default $ PRCH[reta] else _vbe_prompt_segment green cyan $ PRCH[ok] fi # End of prompt _vbe_prompt_end return fi # Regular prompt with many information # [ ] setopt prompt_subst PS1='$(_vbe_prompt) '
Update (2021.05) The following part has been rewritten to be more robust. The code is stolen from Powerlevel10k s issue #888. See the comments for more details.
_vbe-zle-line-init() [[ $CONTEXT == start ]] return 0 # Start regular line editor (( $+zle_bracketed_paste )) && print -r -n - $zle_bracketed_paste[1] zle .recursive-edit local -i ret=$? (( $+zle_bracketed_paste )) && print -r -n - $zle_bracketed_paste[2] # If we received EOT, we exit the shell if [[ $ret == 0 && $KEYS == $'\4' ]]; then _vbe_prompt_compact=1 zle .reset-prompt exit fi # Line edition is over. Shorten the current prompt. _vbe_prompt_compact=1 zle .reset-prompt unset _vbe_prompt_compact if (( ret )); then # Ctrl-C zle .send-break else # Enter zle .accept-line fi return ret zle -N zle-line-init _vbe-zle-line-init
bind-key -T copy-mode M-w \ send -X copy-pipe-and-cancel "sed 's/ .* /%/g' xclip -i -selection clipboard" \;\ display-message "Selection saved to clipboard!"
14:21 % ssh eizo.luffy.cx Linux eizo 4.19.0-16-amd64 #1 SMP Debian 4.19.181-1 (2021-03-19) x86_64 Last login: Fri Apr 23 14:20:39 2021 from 2a01:cb00:3f:b02:9db6:efa4:d85:7f9f 14:21 CEST % uname -a Linux eizo 4.19.0-16-amd64 #1 SMP Debian 4.19.181-1 (2021-03-19) x86_64 GNU/Linux 14:21 CEST % Connection to eizo.luffy.cx closed. 14:22 % git status On branch article/zsh-transient Untracked files: (use "git add <file>..." to include in what will be committed) ../../media/images/zsh-compact-prompt@2x.jpg nothing added to commit but untracked files present (use "git add" to track)
Next.