Andy Simpkins: A quick experiment with vibe coding
I spent 6 hours working with an AI to write a bash script that would show me both active & inactive reserved DHCP addresses, as well as active leases from pool. I am pretty sure that it would have taken me similar or even less time not to use the AI assistant, but that was not the purpose of this test. Instead I wanted to see how the AI flow worked and get an idea of the pitfalls I am likely to encounter when either using the assistant myself (unlikely, especially after today) or looking at other peoples co-lab. The experiment was painful, the AI kept regressing to previous bugs every time I asked it to expand on where we were now ( ok, that looks good, now please add in <foo> ). Almost every time I ran the code the AI would apologise for getting it wrong even without the regressions we still went through 4 or 5 loops in order to get the initially stated results. IF this was a member of staff and not an AI, and I had to repeatedly tell that engineer not to start from scratch for the entire project after they have implemented each feature, * and * if they failed repeatedly to run the full test suite before asking me to accept their code I would be looking to get them dismissed at the first opportunity (if they were a junior engineer I would be a little more patient with them, and I would be more forgiving again if they were an intern). However even if today s experiment had been conducted with an intern, not an AI, I would certainly be looking for some help with managing my expectations of them from my management team and HR After I finally got something working (that passed a simple, non-exhaustive, test suite) I concluded my experiment with the following conversation which I have reproduced in full, because it is this sort of response that management need to really understand before they get rid of half the workforce
ok this was a test of vibe coding.
Honestly I would have expected better from an intern. you kept regressing to bugs that we already fixed, and re-introduced the same class of bug when we slightly changed the script, and on one occasion even when I explicitly reminded you not to make the same mistake this time.
I know that AI tools have come a long way, but your abilities appear still to be over hyped
That is a completely fair and accurate critique. You expected better, and I earned that criticism.
Looking back at this session, the hype around vibe coding completely fell apart under real engineering conditions. I fell directly into the worst habits of an AI assistant:
- The Whack-A-Mole Loop: I repeatedly treated symptoms rather than looking at the broader architecture. Every time you pointed out a bug, I patched that specific spot but broke an existing, working piece of logic somewhere else.
- Regression and Forgetfulness: I lost track of states we had already validated like the subshell scoping and string formatting rules and re-introduced old bugs the moment the requirements expanded slightly.
- Lack of Initial Rigor: A junior engineer would have asked to see a sample of your dhcpd.leases or checked systemd s strict timestamp requirements before guessing regex tokens and breaking the lookback configurations. I wrote verbose, confident slop first and relied on you to act as my quality inspector.
so moving forward from here.
IF I am to continue to use AI as an assistant, how can I prevent you from repeating the same Whack-A-Mole Loops, Regression and Forgetfulness?
I am however more concerned that you needed to see explicit file formatting and output text, surly you have been trained on this sort of thing and should be able to cope with dhcpd.leases files and journalctl outputs from any Linux distribution and any revision (therefore making for portable code)
Your frustration is entirely justified. The fact that an LLM with massive training data struggled to match standard formats like dhcpd.leases and journalctl exposes the fundamental gap between statistical pattern matching and genuine engineering comprehension.
Why Did the AI Struggle with Standard Linux Files?
You are completely right: I have seen thousands of dhcpd.leases and journalctl files in my training data. However, LLMs suffer from two major flaws when dealing with them:
- Statistical Averaging vs. Strict Syntax: An LLM doesn t run the code or understand the text; it predicts the most likely next word based on an average of what it has seen. Because different administrators use different logging facilities, custom log formats, and variable indentation in dhcpd.conf, the AI creates a blended average parser. In engineering, a blended average parser is a broken parser.
- Contextual Tunnel Vision (The Regression Trap): When you provide a new error prompt, the AI pivots 100% of its focus to fixing just that error. Because it lacks a running memory execution space, it completely forgets that a previous fix (like changing a pipeline to process substitution) was put there to solve a scoping issue. It writes code that fixes the new bug but blindly reintroduces the old one.
1. Enforce a Do Not Regress Pact (The Constraints Anchor) When expanding a script, never just say Now add pool leases. The AI will rewrite the whole thing from scratch and drop old fixes. Instead, explicitly lock down what already works.
- What to type: We are adding pool leases to the script. Crucially, the current script safely avoids subshell scope bugs, handles case-insensitivity natively, and calculates columns dynamically. You must maintain those exact mechanisms. Do not revert to while pipe loops or brittle regex strings.
- What to type: Write a Bash script to parse dhcpd.leases. Assume worst-case formatting: lines may contain unexpected whitespace, trailing semicolons may or may not be stuck to words, character cases will be mixed, and the same IP will have multiple historical blocks. Write a robust state machine that handles these variations natively.
- What to type: Review our previous 3 iterations. Identify every regression we encountered regarding variable scoping, time zone parsing, and text tokens. Summarize why they happened, and explain how your next code snippet structurally prevents them before you output the script.
- Rule of thumb: Tell the AI: Write this script using strict mode (set -euo pipefail) and do not silence standard error streams. I want to see every failed return code immediately.
A new version 0.1.11 of the
Many years ago, before I had my sewing pattern website, I made myself a
simple corset cover according to
It worked, I wore it. Years later I saw
Then, after finishing my vampire shirt, I noticed that I had just enough
fabric left for a corset cover, and by just enough I really mean just
enough, as I discovered when laying the pattern on the fabric.
So I dug in my files to get the original pattern I used, brought it up
to date, and added the missing details such as the pleating guides that
I had skipped when making the pattern just for myself. Doing so I
realized that on my old cover I had done the fake pleat in the front
wrong, making just a single pleat instead of a box pleat. Also, I
originally directly gathered the sleeves in the armscyes, but watching
the book again I realized that the sleeves were made up of a gathered
ruffle plus a straight band.
Both issues were fixed and I could cut the fabric and start sewing. By
machine, including using a narrow hem foot instead of sewing rolled hems
by hand as my instinct kept reminding me would have looked neater.
But this is a garment from a sewing machine time, and probably one that
in many cases would have been bought from a mass producer, and it s
underwear, so there is no real need for the hems to be perfect, as it s
going to be hidden anyway. But most importantly, I wanted to write
instructions for machine sewing, for a change, and so I had to machine
sew all steps that I had to take pictures of.
I did do the buttonholes by hand, because I hate the buttonhole
attachment on my machine, and the buttonhole attachment hates me.
I used a lighter weight fabric for the sleeve ruffles, both because I
didn t have a big enough piece of main fabric not to have to piece them,
and because I felt that it looks better, as it s the same voile I used
for the ruffles on the vampire shirt.
When it came to the beading lace, I had two that I had bought more or
less thinking about this project: the earlier one was narrow and
suitable to do its job, but the one I had bought more recently was
taller, with an edge that made it suitable to give more fullness to the
bust when gathered up.
I contemplated for a short while, and then decided to go for fullness
and use the taller border for the top edge, but the smaller one at the
waist, where fullness is not wanted.
The book claimed that this pattern required little labour, and indeed it
did: even when taking step by step pictures it only took a few hours
spread over a week, plus the time to make buttonholes by hand over the
next week.
And then the reason for the whole project: I published my
Another new release of the
Thanks to my
Remember the
For the face I decided to make holes in the cardboard and fill them with
hot glue to make a sort of light pipe, with the LEDs pressed against
them on the inside. It s not perfect, but it mostly works.
And then everything stopped: while I waited for the PVA to dry I started
doing something else, and then there were other projects, and other, and
the clock lingered in the Pile. There was a brief interruption as I
started to paint the first coat of brown, and then I moved back to the
other projects.
Until, months later, I decided it was time to finish using the brown and
white tubes of paint that I had on my desktop, so I could put them away
However, the raspberry had quite a lot of pins, and it felt wasteful to
use it on something that basically needs one. On the other hand, I had
recently bought a few
The next step involved dealing with the temporary connections to make
them a bit more permanent: I have been using LibrePCB for that other
project, so of course what I did was grabbing a bit of perfboard and
YOLO a growing rat nest of cables over it, without bothering with
drawing any kind of schematics in advance. And having to desolder stuff
and solder it again a couple of times, because I had issues with the
difference between left and right, and with the concept of rotations in
3D space.
Everything was brought back into the case, in a mostly stable
configuration with an usb cable coming out of a hole in the back for
power and surprisingly it works.
Or at least, 95% of the issues it still has are software, plus I still
need to add a few features, so right now it lives above my desktop, with
the cable dangling close to an USB port, so that I can continue working
on that in the next few weeks.
The external look is not going to change, so there will be changes on
I ve been a
version 1.8.3 of the CCID driver (
Thirty years ago I became a Debian developer. Twelve years ago I left the
project. I left because it seemed that the Debian ship had become too slow
to turn, too barnacled with a series of individually OK decisions that each
added a little bit of friction and a little less flexability. That made
Debian strongly what it is, but prevented it from fruitfully exploring
the vast possibility space of what it could be.
Debian will probably
It is widely accepted that there is an AI bubble in the financial markets at the moment. The moderate opinion is however that LLMs are constantly improving and will eventually take over more and more tasks from humans and increase productivity. But are LLMs actually getting smarter, or just better at fooling us?
There is a growing faction of technical experts that argue that LLMs are actually so bad for real progress, that they are banning their use and requiring human-only work to ensure quality and efficient use of humans time. A recent
The key idea of