Ryan Malloy
235ffc955c
The talk exists as audio now: a synthesised narration, four and a half minutes, on a /talk/ page with the full transcript underneath. The transcript is the authoritative version, and the audio is preload="none" so the 2.5 MB file is only fetched if somebody presses play. A second player sits on the homepage, the way spicebook surfaces its pitch. The badge follows the homestar.ink pattern: a small webring-style button in the footer that opens a short modal about the studio. It is a Starlight footer override, so it appears site-wide rather than only on the landing page, and it closes on Escape or backdrop click with focus returned to the badge. Talk script and narration text are kept in drafts/ for reuse.
6.2 KiB
Give an Agent a Hypervisor
Lightning talk. Five minutes as written without slide 6, just under six with it. Slide 6 is marked optional for exactly that reason: drop it if the timer is strict, keep it if you have the room. Timings below are cumulative and assume you skip it. Slide cues in brackets. Everything here happened; nothing is dramatized.
[0:00] Slide 1: KolibriOS, with a game of Tetris running
I want to start with this picture, because it is the whole talk in one frame.
That is a virtual machine. The operating system is KolibriOS, written almost entirely in assembly, and it boots to a full graphical desktop in about six seconds. The game of Tetris in the middle was opened, and played, by a language model. It scored eleven. Not good. Not zero.
There is no automation script behind that. The model called tools, looked at what came back, and decided what to do next.
[0:30] Slide 2: QMP: JSON over a unix socket
This is easier than it sounds, because QEMU already speaks a machine protocol. QMP is JSON over a unix socket, and it has been sitting there for years waiting for something to talk to it. So the project is an MCP server that maps it onto tools an agent can call. Thirty three of them.
One detail matters more than the rest. Screenshots come from the framebuffer, and keys go in as scancodes, below the operating system. Nothing is installed in the guest. So this works at a BIOS menu, at a bootloader, and on an OS from 1998 that has never heard of you.
[1:10] Slide 3: look, act, look
Now the part I actually want to talk about.
My first attempt to drive that desktop was a tidy sequence of keystrokes. Open the menu, arrow up to Game Center, press Enter. I sent it blind, the way you would write a shell script.
The screenshot afterwards showed a dialog box. The highlight had not been where I assumed. The item I actually landed on was Shutdown, and the dialog was asking if I would like to reboot, with Enter as the default.
I was one keystroke from turning off the machine I was trying to drive.
What saved it was not cleverness. It was taking a screenshot before pressing the next key. Look, act, look. Every time I skipped the second look, I got surprised.
[1:55] Slide 4: fifteen rounds, then two
It got worse before it got better. KolibriOS ignores the device that lets you click an exact pixel, so the cursor had to be driven like a real PS/2 mouse. Relative motion has no idea where it currently is, and the guest applies its own acceleration, so a hundred pixels is not a hundred pixels.
The technique that worked is the one you would use in the dark. Shove the cursor into a corner until it stops, because now you know where it is. Move in small steps. Screenshot and check before you click.
That took fifteen rounds of trial and error. So I put it in the tool. The rematch, cold boot to Tetris running, took two calls.
Do the fiddly thing once, by hand, badly. Then put the lesson in the tool so nobody does it by hand again.
[2:40] Slide 5: ignore_errors=True
Then I had the code reviewed properly, in the Margaret Hamilton style. Thirty two findings. Here is the worst one, because it is embarrassing and because I think it is common.
The tool that destroys a sandbox tells QEMU to quit, waits ten seconds for the process to die, then deletes the disk.
Read that again. It waits ten seconds, and then it deletes the disk. It never looks at whether the process died. If the VM was wedged, the delete happened anyway, out from under a machine that still had the file open.
And the delete itself was written like this.
shutil.rmtree(target, ignore_errors=True)
That flag turns "I could not delete this" into silence. The function then
returned destroyed: True. Unconditionally. It reported success without ever
asking.
Both problems are one if statement away from correct. That is why it is worth
showing you. It is not exotic. It is the ordinary shape of a bug in code that
waits for something.
[optional, +45s] Slide 6: two things the gates caught
Once I started checking properly, the checking kept paying.
An acceptance test against real QEMU found a bug in my own fix. A hundred and twenty five unit tests had missed it, because every one of them mocked the function involved.
And the audit before publishing found my source distribution had swallowed the entire documentation site, node modules and all. Seven thousand files, seventy one megabytes, for a package with thirty source files. PyPI versions are immutable, so that would have been permanent.
Neither was found by writing code. Both came from checking what I had actually produced, rather than what I meant to produce.
[4:00] Slide 7: the same picture again
So you can give a language model a hypervisor, and it will boot an operating system out of the Internet Archive and play Tetris in it. That part took an afternoon and it is genuinely fun.
The part worth keeping is smaller. Both halves of this ran on one rule. The model has to look at the screen before it presses the next key. The code has to check the process died before it deletes the disk.
Verify the thing you waited for. That is the talk.
It is on PyPI as mcqemu. Docs at mcqemu.warehack.ing.
Notes for delivery
- The Shutdown dialog is the laugh. Do not rush it, and do not explain it afterwards.
- Slide 5 is the load-bearing slide. Never cut it.
- Slide 6 is the pressure valve. Drop it entirely for a hard five minutes, or keep only the PyPI story, which is the most concrete of the two.
- Have the Tetris screenshot up during questions. People ask about it.
- Spare thirty seconds if you need it: archive.org was serving 500s from a dead storage node that day, and the fix was asking its metadata API which mirror was healthy. Cut this first.
Alternative angle, if the room is more ops than dev
Same demo, different spine: disposable VMs as agent sandboxes. A copy-on-write overlay means a fresh clone of a twenty gigabyte base image costs kilobytes and about a second. Outbound networking is off by default, because QEMU's user-mode networking maps the host loopback to 10.0.2.2, and "it is only on localhost" is exactly the assumption that breaks. Same closing rule, applied to the destroy path.