The Repository key named a repo that was never created and 404'd for everyone,
which PyPI would have frozen into the version permanently. It now points at
warehack.ing/mcbebop on Gitea, which is public and anonymously reachable, with
Homepage on the docs site and an Issues link. All three verified with an
unauthenticated request.
Audited the package against the two-stage procedure before a first PyPI
publish. The sdist and wheel were already tight, but three things needed
fixing and the controls needed to become real rather than documented.
The simulator volunteered high="PI04" as its product serial, which is the
real Bebop 2 serial prefix. Nothing unique to one aircraft, but a realistic
prefix invites being quoted into a bug report as a specimen, so it now reads
"N0TAREAL" / "0000000SIM" with a comment saying why it is nonsense on
purpose. The low half and the 500.0 no-fix GPS sentinel were already fake.
Hardened [tool.uv.build-backend] source-exclude well past the directories
that exist today: captures at any depth, log dumps, recorded media by
extension, caches, and anything credential-shaped. .gitignore governs git
and source-exclude governs the sdist; a capture can sit in one and not the
other, which is how this kind of data reaches an immutable index. Verified
the broad patterns do not over-reach: arsdk-xml/ with PROVENANCE.md and
tools/logs.py both still ship.
Added LICENSE (MIT) and LICENSE-arsdk-xml (Parrot SA's BSD-3-Clause), and
corrected the declared licence to "MIT AND BSD-3-Clause". The vendored XML
ships in both artifacts because nothing here decodes a command without it,
so MIT alone understated what is in the box. Both texts now appear in the
artifacts and in the metadata.
test_packaging.py grows privacy guards that fail on a serial prefix, a P7
CPU id, any MAC, a high-precision coordinate, an absolute home path, or any
private address other than the drone's own documented 192.168.42.0/24. Each
pattern was checked against the real identifiers to confirm it bites, since
a guard that passes on an empty tree proves nothing.
Example address in test_arsdk_session.py moved to RFC 5737 space.
504 tests pass, ruff clean.
The simulator can now stream video, so a client's whole decode path can be
developed without an aircraft: an RFC 6184 packetiser for Annex-B H.264, a
capture/replay format for real drone RTP, and both wired into the sim behind
VideoEnable.
Proved by decoding rather than by assertion: ffmpeg reads the simulated stream
at 856x480 and 30 fps, including a join two seconds in at a random mid-GOP
offset, which is the case a goggle viewer actually faces.
The simulated stream is RFC-correct but shaped differently from the aircraft's,
which wraps far more NALs in STAP-A than repeating parameter sets explains. A
client tested only against the synthetic path is tested against the wrong shape;
that is what the replay path is for.
The sim takes an optional video source. Nothing flows until
ardrone3.MediaStreaming.VideoEnable arrives with 1; RTP then goes from the
sim's own 5004 to whatever arstream2_client_stream_port the controller named
in its handshake, and stops on a 0, on a link loss, or at shutdown. The
MediaStreamingState.VideoEnableChanged reply goes out either way, because
that reports the aircraft's state and not its bitrate, so a client waiting on
the confirmation must not hang for want of a file to stream.
Video stays optional: a FakeBebop() with no source gains no thread and sends
no packets.
One source object for the life of the sim, so sequence numbers and timestamps
keep advancing across both the loop point in the file and a disable/enable
cycle. A decoder handed a timestamp that went backwards treats the stream as
corrupt and stays that way.
The test that proves any of this works is the ffmpeg decode: frames come out
at 856x480, including when ffmpeg joins a stream that has already been running
for two seconds, which is the case a goggle viewer actually faces. The clip is
generated at setup and never committed.
Also fixed stop() on a sim that was never started: the sockets bind in
__post_init__, so it had ports to release, and joining an unstarted thread
raised and left them held.
The packetiser is RFC 6184: single-NAL packets, FU-A fragmentation, and
STAP-A aggregation for parameter sets and SEI. Pure stdlib, because the
simulator ships in the package and cannot drag a media library behind it.
A VCL NAL is never aggregated and interleaved mode is absent; the docstring
says so rather than leaving it to be discovered.
Parameter sets repeat rather than appearing once at the head of the stream,
which is what the aircraft does and the only reason a viewer joining a flight
already in progress can recover. A start offset, random with a controllable
seed, exists to hand a decoder a stream that begins mid-GOP on purpose.
Capture and replay is the other half, and the two are different instruments.
The packetised path paces at a nominal frame rate, which proves a decoder and
a renderer work. A replay reproduces the recorded inter-packet gaps packet for
packet, which is what a latency or jitter measurement needs. The aircraft's
packet-type mix cannot be derived from first principles: the measurement we
have read only the outer header byte of each packet, so what its 740 FU-A
packets carried is still unknown, and only a real capture settles it.
The loop point is the one gap a file cannot describe. It is estimated from the
capture's mean frame period rather than its median packet gap, which was the
first thing tried and is wrong: with a dozen packets to a frame most gaps are
intra-frame, so the stream looped a frame early every time.
The drone sends BatteryStateChanged from two places with different values: the
true figure on buffer 126, corroborated by its own log and tracking a smooth
discharge, and a constant 0 on buffer 127 that appears in no log line. Keeping
the newest frame meant a healthy aircraft read as flat and preflight refused.
Events now carry their buffer and the store keeps the last value per source, so
a disagreement is visible instead of silently resolved. get_state reports the
sources whenever they differ, rather than handing over a winner.
preflight breaks the tie on physical grounds, not preference: an aircraft that
is powered and holding a link is not at 0%, so a zero from a live drone is not
credible while a non-zero one is. It says the number was contested either way,
and a genuine zero from every source still reports zero and still blocks.
The CKCM log format turned out to be documented in a file deleted from
Parrot's ulog repository in 2017, so it is cited rather than claimed as
reverse engineering, with four points the capture settled that the source
leaves open or states wrongly.
The COMMANDS tag is deliberately NOT resolved to protocol commands: measured,
only 4 of 11 symbols match the XML by name, covering 3.5% of the tag, and a
resolver would read as authoritative while guessing, with its silence on the
rest reading as 'not a command'.
Tested on the aircraft. A second ARSDK handshake is accepted and telemetry is
redirected to it; the first session's frames stop while it still reports
connected = True. So the claim inherited from pyparrot's error text, which had
reached our error messages, tool descriptions, simulator behaviour and a test
name, was wrong in the most misleading direction: a refusal would be loud, and
this is silent.
The simulator now models the takeover by default; refusal stays available
because a client must handle a non-zero status anyway.
read_log fetches the live log over FTP and filters by tag, severity and
message before it returns anything, because half an hour of uptime is
five thousand entries. It excludes shp_usbmode by default, a five-second
USB poll that can be a third of the log, and says in the result that it
did so. Only the current file is reachable: the archives the drone
rotates into are a previous owner's sessions and carry network and
location data, which is a decision for a person, not a tool.
bebop://commands is the resource worth having, because it needs no
drone: the whole catalogue is readable with the aircraft switched off.
The ones that do need a link return an explanation rather than raising,
since a resource that errors looks broken while one that explains itself
is empty for a reason. bebop://state also reports telemetry age, because
the aircraft accepts a second controller and silently redirects
telemetry to it, so a connected flag proves nothing.
The log says nothing resolvable about ARSDK, measured rather than
assumed: no command ids appear anywhere in it, and the COMMANDS function
names match arsdk-xml for 4 of 11 symbols, 3.5% of the tag. So nothing
here resolves them, and read_log says where to look instead.
_host moves into tools/_common as drone_host now that three tool modules
need it, and head/tail in the shell allow-list gains the reason they
must take a file argument: ld.so.preload makes SIGPIPE fatal, so a pipe
into head writes a crash report to the aircraft's flash.
The format is Parrot's own, from ulogcat/libulogcat_ckcm.c, which was
deleted from Parrot-Developers/ulog in 2017; the module docstring cites
it and records what the 560 KB capture settles that the source leaves
open, notably that this firmware's stamps are monotonic microseconds
since boot with no wall clock anywhere.
Robustness is the point as much as the layout. The log is live and grew
from 445 KB to 560 KB inside a minute, so a fetch landing mid-record is
the normal case and ends the parse quietly. The markers are not escaped
either, so the lengths are what the parser trusts.
Parses the capture to the byte: 5520 entries, no resyncs, no residue.
15 commands go on the unacknowledged buffer, so acked=false is their normal
outcome rather than a failure. Reporting it bare made a working camera move
look broken; the note now says to confirm with get_state instead.
list_dir and fetch take a name and resolve it themselves; the tools layer
resolved it first and handed over the object, so every call failed with
'No FTP area named Area(...)'. Found by calling the tool for real.
Acknowledge on data type alone, never on buffer id. A live Bebop 2 sends
DATA_WITH_ACK on buffer 126 and plain DATA on 127, the opposite of what the
buffer names imply, so requiring both to agree meant nothing was ever
acknowledged. The drone resent its state instead of continuing: 35 telemetry
keys where there should be 192, and a connect that took 4 s instead of 1.7.
Record argument-less events. Eight events carry no arguments and their
arrival is the whole message, including AllStatesChanged and
AllSettingsChanged, which mark the end of a state dump. Keying only by
argument decoded them to an empty dict and lost them.
Wait for those terminators instead of a quiet period. The drone streams
attitude at about 5 Hz throughout, so the link is never quiet and the wait
always ran to its timeout with a partial burst.
Also: preflight no longer reports ready when a blocking check has no data.
It answered 'ready' on an aircraft it knew almost nothing about, which is
worse than refusing to answer.
connect/disconnect/status, list_commands/command_info, the generic
send_command, get_state/watch_state/preflight_check, camera snapshot and
record, FTP listing and fetch, the read-only shell, and arm/disarm.
The safety gate is tested exhaustively rather than by sample: every motion
and envelope command is asserted to refuse while unarmed, and Landing and
Emergency are asserted to work without arming, because refusing to bring
down an airborne aircraft is the more dangerous answer.
preflight_check reads the per-element sensor keys, so a single failing
self-test is visible instead of being hidden by whichever arrived last.
17 events are MAP_ITEM or LIST_ITEM: they arrive once per element, all
carrying the same argument names, so a flat store kept only whichever landed
last. SensorsStatesListChanged reports six sensors that way, and the
simulator's deliberate magnetometer fault was invisible through state() as a
result, which is exactly the fact preflight_check needs to see.
Entries are now keyed Command[element]_arg. Parrot marks these events but
never names the key; it is the first argument by convention.
76 tests. Implements the protocol directly rather than through pyparrot,
whose receive thread prints to stdout and would corrupt JSON-RPC.
Two corrections to the observed notes, both adopted: the emergency buffer
carries DATA_WITH_ACK with unlimited retries in libARController rather than
the low-latency type pyparrot sends, and fire-and-forget is the wrong
property for the command that cuts the motors; and the handshake reply must
be read until its NUL terminator rather than from a single recv, because it
has grown across firmware versions.
decode_event raises on an id triple the XML does not carry, and the XML
is missing at least FlatTrim, so this happens on a real aircraft. The
receive loop already counted and carried on; now something checks it.
protocol/codec.py landed with decode_event returning ids and a values
dict, so the session looks the command name up in the index rather than
expecting it in the result. Verified end to end against that stream's
codec in a stitched tree: the sim encodes events through it, the session
decodes through it, and the test that skips until it exists now passes.
The argument name cannot be recovered by splitting a telemetry key,
because an argument can contain an underscore, so the expectation
predicate matches on the suffix instead.
The transport is threaded and the session puts an async face on it, so
pings and acks are answered whether or not anyone is awaiting a
coroutine. Telemetry is stored per key with a timestamp, because a drone
that has stopped reporting otherwise reads identically to one repeating
itself.
Encoding goes through protocol/codec lazily, so this lands without
waiting for that stream; the sim carries a small encoder of its own for
the events it sends, which also keeps it from agreeing with the client
about a shared mistake.
Ported from bebop-2's sim.py, retargeted at the vendored XML and with
the identity burst now sent per controller attach rather than once per
process.
116 tests. Video is the port of what was proven against the aircraft: no
RTSP anywhere, SDP describing our own port, ffmpeg bound before the stream
is enabled, -c copy for recording.
Changes the stream made and justified: the video session takes a sender
callback rather than owning a connection, so media/ imports nothing from
arsdk or protocol; blocking subprocess waits moved off the event loop;
exposure is restored even when entering the session fails.
FTP exposes media, flightplans and logs. Port 51 is the firmware-write
channel and is deliberately unreachable. The shell is an allow-list of
command names with shell metacharacters refused outright, because the
drone's telnet login is an unauthenticated root shell.
Ports bebop-2's working video.py, which was verified against the live
aircraft. Firmware 4.7.1 serves no RTSP, so there is no URL and no RTSP
path here: we describe our own receiving port in an SDP and let ffmpeg
bind it before VideoEnable goes out, because RTP is connectionless and
packets that arrive before the sink is listening are gone.
Three changes on top of the port. StreamSession no longer owns the drone
link, since the tools layer holds a long-lived session; it takes an async
sender callable instead, which also keeps media/ loadable while arsdk/
and protocol/ are still being written. The context manager is async for
the same reason, with the blocking subprocess waits moved off the event
loop. And downscale() shrinks a frame before it reaches a model, because
a full 856x480 is most of a context window spent on pixels nobody asked
for.
FTP exposes media (21), flightplans (61) and logs (21, scoped to the
Debug tree). Port 51 is deliberately absent: it serves /update as root,
it is how firmware is pushed, it has no read use case, and the drone's
Wi-Fi is open. Fetches stream to capture_dir under a size cap so a 1080p
recording cannot be pulled into a tool result.
The shell is an allow-list of eleven read-only command names rather than
a deny-list, because the login is `exec /bin/sh -l` with no password and
deny-lists on shells leak. Arguments carrying shell metacharacters are
refused before the socket opens. Output is bracketed between two echoed
nonce markers rather than trimmed by prompt pattern, since telnetd's pty
echoes our input with the prompt glued to the front and sends all of it
before anything runs.
90 tests. Verified against real data from the aircraft: VideoEnable encodes
to the bytes we actually sent, enums encode as 4-byte indices (pyparrot read
one byte and misaligned everything after), and events decode to the same
<Command>_<arg> keys our captures use.
Two counts in the brief were wrong and the stream corrected them with tests:
31 commands state neither result nor triggered (not 33), and 39 give no
answer on Bebop 2 support (not 32).
uv_build packages only src/mcbebop, so arsdk-xml/ at the repo root was
absent from the wheel and every install would have failed to find the
command definitions. Caught during the protocol build.
Parse Parrot's common.xml and ardrone3.xml into CommandSpec objects rather
than transcribing 264 commands by hand. Direction is derived from
comment result=/triggered= with a class-name-suffix fallback for the 31
commands that state neither, giving 101 to-drone and 163 from-drone.
ardrone3.Piloting.FlatTrim is injected: the firmware accepts it but neither
the vendored snapshot nor upstream defines it.
codec encodes the <BBH header (the command id is 16 bits, not 8) and reads
enums as i32 and strings as NUL-terminated, both of which pyparrot gets
wrong. safety assigns the arming tier by class, with Mavlink and Calibration
counted as motion because they move the aircraft. expectations parses the
undocumented #p-c-m grammar including | alternatives and this.<arg> echoes.
90 tests, counts asserted so a bad parse fails loudly.
protocol/types.py and arsdk/types.py are the interfaces the parallel streams
build against: command/arg/enum/expectation specs and the safety tiers on one
side, frame encoding and the buffer conventions on the other.
Logging goes to stderr throughout, since stdout carries JSON-RPC. That is also
why this package will speak ARSDK itself rather than through pyparrot, which
prints from its receive thread.
Bebop-era snapshot of common.xml and ardrone3.xml (BSD-3-Clause, Parrot SA),
with pyparrot's <myclass> rename reverted to Parrot's <class>. Provenance and
the comparison against upstream master are in arsdk-xml/PROVENANCE.md.
Ryan Malloy
rsp2k