Compare commits
10
Commits
bdfb2a9cc5
...
7cd7a4c87e
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
7cd7a4c87e | ||
|
|
c5e84c47ae | ||
|
|
e9b0c264c5 | ||
|
|
ae2cf54e68 | ||
|
|
0912689301 | ||
|
|
907f876773 | ||
|
|
8611d6112d | ||
|
|
41a6cb107f | ||
|
|
ad81cc6e38 | ||
|
|
38322b6885 |
@@ -0,0 +1,32 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Ryan Malloy
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
|
||||
---
|
||||
|
||||
This package also distributes a vendored third-party component under a
|
||||
different licence. See LICENSE-arsdk-xml for the full text.
|
||||
|
||||
src/mcbebop/arsdk-xml/common.xml
|
||||
src/mcbebop/arsdk-xml/ardrone3.xml
|
||||
Copyright (C) 2014 Parrot SA, BSD-3-Clause.
|
||||
|
||||
The combined distribution is therefore "MIT AND BSD-3-Clause".
|
||||
@@ -0,0 +1,37 @@
|
||||
The files src/mcbebop/arsdk-xml/common.xml and src/mcbebop/arsdk-xml/ardrone3.xml
|
||||
are Parrot SA's own ARSDK protocol definitions, redistributed unmodified except
|
||||
as recorded in src/mcbebop/arsdk-xml/PROVENANCE.md. Upstream:
|
||||
https://github.com/Parrot-Developers/arsdk-xml
|
||||
|
||||
The licence text below is reproduced from the header of those files.
|
||||
|
||||
----------------------------------------------------------------------
|
||||
|
||||
Copyright (C) 2014 Parrot SA
|
||||
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions
|
||||
are met:
|
||||
* Redistributions of source code must retain the above copyright
|
||||
notice, this list of conditions and the following disclaimer.
|
||||
* Redistributions in binary form must reproduce the above copyright
|
||||
notice, this list of conditions and the following disclaimer in
|
||||
the documentation and/or other materials provided with the
|
||||
distribution.
|
||||
* Neither the name of Parrot nor the names
|
||||
of its contributors may be used to endorse or promote products
|
||||
derived from this software without specific prior written
|
||||
permission.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
|
||||
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
|
||||
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
|
||||
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
|
||||
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
|
||||
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING,
|
||||
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
|
||||
OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
|
||||
AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
|
||||
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
|
||||
OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
|
||||
SUCH DAMAGE.
|
||||
@@ -30,6 +30,53 @@ land an airborne aircraft is the more dangerous answer.
|
||||
`connect(target="sim")` runs everything against a protocol-accurate simulator,
|
||||
which is where anything involving motion should be rehearsed.
|
||||
|
||||
## The simulator streams video
|
||||
|
||||
The simulator can push RTP/H.264 exactly as the aircraft does, so a viewer's
|
||||
whole video path can be developed and measured without a drone. It answers the
|
||||
handshake with `arstream2_server_stream_port: 5004`, sends nothing until
|
||||
`ardrone3.MediaStreaming.VideoEnable` arrives with 1, then streams from its own
|
||||
5004 to whatever `arstream2_client_stream_port` the client named, and stops on a
|
||||
0, on a link loss, or at shutdown.
|
||||
|
||||
```bash
|
||||
python -m mcbebop.sim --video clip.h264 # steady 30 fps
|
||||
python -m mcbebop.sim --video clip.h264 --start-offset random --seed 7
|
||||
python -m mcbebop.sim --video flight.rtpcap # a real capture, replayed
|
||||
```
|
||||
|
||||
`MCBEBOP_SIM_VIDEO_SOURCE=clip.h264` does the same for `connect(target="sim")`.
|
||||
|
||||
Two kinds of source, and they are **different instruments**:
|
||||
|
||||
| Source | Pacing | Use it for |
|
||||
|---|---|---|
|
||||
| `.h264` Annex-B elementary stream | packetised here, steady frame rate | does the decoder work, does the renderer work |
|
||||
| `.rtpcap` capture off the aircraft | the recorded inter-packet gaps, packet for packet | latency and jitter, bursts, loss behaviour |
|
||||
|
||||
Make the first from any video, at the resolution the aircraft streams:
|
||||
|
||||
```bash
|
||||
ffmpeg -i anything.mp4 -t 10 -vf scale=856:480 -r 30 \
|
||||
-c:v libx264 -preset ultrafast -pix_fmt yuv420p -g 30 -f h264 clip.h264
|
||||
```
|
||||
|
||||
`-f h264` already writes Annex-B, so no bitstream filter is wanted;
|
||||
`h264_mp4toannexb` converts the other direction and ffmpeg rejects it here.
|
||||
|
||||
Make the second from a real drone. Start the recorder first, because RTP is
|
||||
connectionless and anything sent before the bind is gone, then enable video
|
||||
from a session that holds the ARSDK link:
|
||||
|
||||
```bash
|
||||
python -m mcbebop.media.capture flight.rtpcap --seconds 30 # binds 55004
|
||||
```
|
||||
|
||||
`--start-offset random` is worth knowing about. It begins mid-GOP, which is
|
||||
what a viewer switched on while the drone is already flying is handed, and
|
||||
`--seed` makes a failure repeatable. Parameter sets repeat about once a second
|
||||
on the packetised path, which is what lets a late joiner recover at all.
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
@@ -37,7 +84,12 @@ uvx mcbebop
|
||||
claude mcp add mcbebop -- uvx mcbebop
|
||||
```
|
||||
|
||||
## Credits
|
||||
## Licence
|
||||
|
||||
`arsdk-xml/` is Parrot SA's own protocol definition, BSD-3-Clause. See
|
||||
`arsdk-xml/PROVENANCE.md`.
|
||||
This package is MIT (`LICENSE`), and it vendors one third-party component:
|
||||
`src/mcbebop/arsdk-xml/` is Parrot SA's own protocol definition, BSD-3-Clause
|
||||
(`LICENSE-arsdk-xml`). It ships in both the sdist and the wheel because nothing
|
||||
here can decode a single command without it. What that snapshot is and how it
|
||||
differs from upstream is recorded in `arsdk-xml/PROVENANCE.md`.
|
||||
|
||||
So the distribution as a whole is `MIT AND BSD-3-Clause`.
|
||||
|
||||
+80
-4
@@ -4,7 +4,11 @@ version = "2026.10.02"
|
||||
description = "MCP server for the Parrot Bebop 2 drone: telemetry, camera, files, and every ARSDK command"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.12"
|
||||
license = "MIT"
|
||||
# Our own code is MIT. The vendored arsdk-xml/ is Parrot SA's, BSD-3-Clause,
|
||||
# and it ships in both artifacts because the package cannot decode a single
|
||||
# command without it, so the distribution as a whole is both.
|
||||
license = "MIT AND BSD-3-Clause"
|
||||
license-files = ["LICENSE", "LICENSE-arsdk-xml"]
|
||||
authors = [{name = "Ryan Malloy", email = "ryan@supported.systems"}]
|
||||
keywords = ["mcp", "drone", "parrot", "bebop", "arsdk", "fastmcp"]
|
||||
classifiers = [
|
||||
@@ -24,8 +28,9 @@ dependencies = [
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
Homepage = "https://warehack.ing"
|
||||
Repository = "https://git.supported.systems/rsp2k/mcbebop"
|
||||
Homepage = "https://bebop-2.warehack.ing"
|
||||
Repository = "https://git.supported.systems/warehack.ing/mcbebop"
|
||||
Issues = "https://git.supported.systems/warehack.ing/mcbebop/issues"
|
||||
|
||||
[project.scripts]
|
||||
mcbebop = "mcbebop.server:main"
|
||||
@@ -35,7 +40,78 @@ requires = ["uv_build>=0.11.3,<0.12.0"]
|
||||
build-backend = "uv_build"
|
||||
|
||||
[tool.uv.build-backend]
|
||||
source-exclude = ["CLAUDE.md", ".env", ".env.*", ".mcp.json", "tests", "captures"]
|
||||
# Err toward excluding. This repo sits next to a real aircraft, and `captures/`
|
||||
# holds video of the owner's home plus `ckcm.bin` system logs that carry the
|
||||
# serial, the CPU id and a real GPS fix together. None of it may ever reach an
|
||||
# sdist, which is immutable and mirrored within minutes of upload.
|
||||
#
|
||||
# `.gitignore` does NOT protect this: it governs git, and source-exclude
|
||||
# governs the sdist. A file can easily be in one and not the other, so the
|
||||
# patterns below are deliberately broader than the directories that exist
|
||||
# today -- a capture dropped into a new directory next year is still excluded.
|
||||
# test_packaging.py asserts both halves stay in agreement.
|
||||
source-exclude = [
|
||||
# Operator-private context and anything credential-shaped.
|
||||
"CLAUDE.md",
|
||||
".env",
|
||||
".env.*",
|
||||
".mcp.json",
|
||||
".claude",
|
||||
".claude/**",
|
||||
# Dev-only trees. Tests carry fixtures copy-pasted from a live aircraft.
|
||||
"tests",
|
||||
"tests/**",
|
||||
"docs",
|
||||
"docs/**",
|
||||
"scripts",
|
||||
"scripts/**",
|
||||
"*.ipynb",
|
||||
# Captures off the aircraft, at any depth, plus log dumps.
|
||||
"captures",
|
||||
"captures/**",
|
||||
"**/captures/**",
|
||||
"**/logs/**",
|
||||
"ckcm*",
|
||||
"**/ckcm*",
|
||||
# Recorded media and binary blobs by extension, so a stray file in a
|
||||
# directory nobody thought to list is still caught.
|
||||
"*.rtpcap",
|
||||
"*.raw",
|
||||
"*.h264",
|
||||
"*.bin",
|
||||
"*.pcap",
|
||||
"*.pcapng",
|
||||
"*.mp4",
|
||||
"*.jpg",
|
||||
"*.jpeg",
|
||||
"*.png",
|
||||
"*.wav",
|
||||
"*.ulg",
|
||||
"**/*.rtpcap",
|
||||
"**/*.raw",
|
||||
"**/*.h264",
|
||||
"**/*.bin",
|
||||
"**/*.jpg",
|
||||
"**/*.jpeg",
|
||||
"**/*.png",
|
||||
# Caches and build output. .pyc files embed the absolute build path, which
|
||||
# leaks a username and the local directory layout.
|
||||
"**/__pycache__",
|
||||
"**/__pycache__/**",
|
||||
"*.pyc",
|
||||
"**/*.pyc",
|
||||
".pytest_cache",
|
||||
".pytest_cache/**",
|
||||
".ruff_cache",
|
||||
".ruff_cache/**",
|
||||
".venv",
|
||||
".venv/**",
|
||||
"dist",
|
||||
"dist/**",
|
||||
"build",
|
||||
"build/**",
|
||||
"*.egg-info",
|
||||
]
|
||||
|
||||
[dependency-groups]
|
||||
dev = ["ruff>=0.16", "pytest>=8.0", "pytest-asyncio>=0.25"]
|
||||
|
||||
@@ -23,7 +23,7 @@ import logging
|
||||
import threading
|
||||
import time
|
||||
from collections.abc import Callable, Iterable
|
||||
from dataclasses import dataclass, field
|
||||
from dataclasses import dataclass, field, replace
|
||||
from functools import cache
|
||||
from typing import Any
|
||||
|
||||
@@ -70,15 +70,27 @@ class Timeouts:
|
||||
|
||||
@dataclass
|
||||
class Reading:
|
||||
"""One telemetry key: the value, and when the drone last said so."""
|
||||
"""One telemetry key: the value, and when the drone last said so.
|
||||
|
||||
`sources` holds the last value seen from each buffer. It exists because
|
||||
this aircraft sends `BatteryStateChanged` from two places with different
|
||||
values, so "the newest frame" and "the truth" are not the same thing, and
|
||||
a consumer needs to be able to see that rather than be handed a winner.
|
||||
"""
|
||||
|
||||
value: Any
|
||||
at: float # time.monotonic()
|
||||
count: int = 1
|
||||
sources: dict[int, Any] = field(default_factory=dict)
|
||||
|
||||
def age(self, now: float | None = None) -> float:
|
||||
return (now if now is not None else time.monotonic()) - self.at
|
||||
|
||||
@property
|
||||
def conflicting(self) -> bool:
|
||||
"""True when two buffers are currently reporting different values."""
|
||||
return len(set(map(repr, self.sources.values()))) > 1
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Waiter:
|
||||
@@ -119,7 +131,7 @@ def _name_for(ids: tuple[int, int, int]) -> str:
|
||||
return spec.name if spec is not None else f"cmd_{ids[0]}_{ids[1]}_{ids[2]}"
|
||||
|
||||
|
||||
def _as_event(decoded: Any, payload: bytes) -> Event | None:
|
||||
def _as_event(decoded: Any, payload: bytes, buffer_id: int = 0) -> Event | None:
|
||||
"""Normalise whatever the codec returned into an `Event`.
|
||||
|
||||
`decode_event` returns `(ids, values)`, which is the second branch. The
|
||||
@@ -131,7 +143,7 @@ def _as_event(decoded: Any, payload: bytes) -> Event | None:
|
||||
if decoded is None:
|
||||
return None
|
||||
if isinstance(decoded, Event):
|
||||
return decoded
|
||||
return decoded if decoded.buffer_id else replace(decoded, buffer_id=buffer_id)
|
||||
ids = getattr(decoded, "ids", None)
|
||||
name = getattr(decoded, "name", None)
|
||||
values = getattr(decoded, "values", None)
|
||||
@@ -151,6 +163,7 @@ def _as_event(decoded: Any, payload: bytes) -> Event | None:
|
||||
name=str(name) if name else _name_for(ids), # type: ignore[arg-type]
|
||||
values=dict(values or {}),
|
||||
at=getattr(decoded, "at", None) or time.monotonic(),
|
||||
buffer_id=buffer_id,
|
||||
)
|
||||
|
||||
|
||||
@@ -265,7 +278,7 @@ class DroneSession:
|
||||
if len(frame.payload) < COMMAND_HEADER.size:
|
||||
return
|
||||
try:
|
||||
event = _as_event(self._decode(frame.payload), frame.payload)
|
||||
event = _as_event(self._decode(frame.payload), frame.payload, frame.buffer_id)
|
||||
except Exception:
|
||||
# An event we cannot decode is a gap in coverage, not a reason to
|
||||
# drop the link: the next frame may be the one that matters.
|
||||
@@ -282,7 +295,9 @@ class DroneSession:
|
||||
with self._lock:
|
||||
for key, value in values.items():
|
||||
prior = self._store.get(key)
|
||||
self._store[key] = Reading(value, event.at, (prior.count + 1) if prior else 1)
|
||||
sources = dict(prior.sources) if prior else {}
|
||||
sources[event.buffer_id] = value
|
||||
self._store[key] = Reading(value, event.at, (prior.count + 1) if prior else 1, sources)
|
||||
hit = [w for w in self._waiters if event.ids in w.ids and w.predicate(event)]
|
||||
for waiter in hit:
|
||||
waiter.hit = event
|
||||
@@ -309,7 +324,16 @@ class DroneSession:
|
||||
if keys is not None:
|
||||
wanted = list(keys)
|
||||
items = {k: v for k, v in items.items() if k in wanted or any(k.startswith(w) for w in wanted)}
|
||||
return {k: {"value": v.value, "age": round(v.age(now), 3)} for k, v in sorted(items.items())}
|
||||
out: dict[str, Any] = {}
|
||||
for key, reading in sorted(items.items()):
|
||||
entry: dict[str, Any] = {"value": reading.value, "age": round(reading.age(now), 3)}
|
||||
if reading.conflicting:
|
||||
# Say so rather than hand over a winner. The newest frame is not
|
||||
# necessarily the true one when two parts of the aircraft report
|
||||
# the same thing differently.
|
||||
entry["sources"] = dict(reading.sources)
|
||||
out[key] = entry
|
||||
return out
|
||||
|
||||
def values(self, keys: Iterable[str] | None = None) -> dict[str, Any]:
|
||||
"""The same thing without the staleness wrapper, for internal checks."""
|
||||
|
||||
@@ -90,6 +90,14 @@ class Event:
|
||||
name: str # the command name, e.g. "BatteryStateChanged"
|
||||
values: dict[str, Any] # keyed `<Command>_<arg>`, as our captures are
|
||||
at: float # time.monotonic()
|
||||
buffer_id: int = 0
|
||||
"""Which buffer carried it.
|
||||
|
||||
Kept because this aircraft sends the same command from more than one place
|
||||
with different values: `BatteryStateChanged` arrives as the true figure on
|
||||
one buffer and as a constant 0 on another. Without knowing the source, a
|
||||
store that keeps the newest value silently reports whichever spoke last.
|
||||
"""
|
||||
|
||||
|
||||
class HandshakeError(RuntimeError):
|
||||
|
||||
@@ -20,6 +20,13 @@ class Settings(BaseSettings):
|
||||
capture_dir: Path = Field(
|
||||
default=Path("captures"), description="Where recordings and snapshots are written."
|
||||
)
|
||||
sim_video_source: Path | None = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"An Annex-B .h264 file or a .rtpcap capture for connect(target='sim') to stream. "
|
||||
"Unset means the simulator answers VideoEnable but sends no RTP."
|
||||
),
|
||||
)
|
||||
transport: str = Field(default="stdio", description="stdio or http.")
|
||||
host: str = Field(default="127.0.0.1", description="Bind address when transport is http.")
|
||||
port: int = Field(default=8440, description="Port when transport is http.")
|
||||
|
||||
@@ -21,9 +21,14 @@ def already_connected(target: str) -> ToolError:
|
||||
|
||||
|
||||
def handshake_refused(status: int) -> ToolError:
|
||||
# Tested 2026-10-02: a Bebop 2 does NOT refuse a second controller, it
|
||||
# accepts it and redirects telemetry to it. So a non-zero status means
|
||||
# something other than "already taken", and guessing at it would send the
|
||||
# reader looking in the wrong place.
|
||||
return ToolError(
|
||||
f"The drone refused the connection (status {status}). It serves one controller at a time, "
|
||||
"so close FreeFlight on any phone or tablet that is holding the link."
|
||||
f"The drone refused the connection (status {status}). That is unusual: this aircraft accepts "
|
||||
"a second controller rather than refusing one, so the cause is more likely the drone still "
|
||||
"booting, or a firmware that differs from 4.7.1. Wait a few seconds and try again."
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -53,6 +53,12 @@ ALLOWED = frozenset(
|
||||
"bcmwl", # Broadcom wireless tool: regulatory domain, channel, rates
|
||||
"df",
|
||||
"mount",
|
||||
# Always with a FILE argument. `/etc/ld.so.preload` on this drone
|
||||
# preloads a crash-dump library into every process and it treats
|
||||
# SIGPIPE as fatal, so `producer | head` makes the producer dump a
|
||||
# ~350 KB crash report onto the aircraft's flash. FORBIDDEN_CHARS
|
||||
# already makes a pipe unbuildable from here; the hazard is anyone who
|
||||
# relaxes that, or who runs a command on the drone by hand.
|
||||
"head",
|
||||
"tail",
|
||||
}
|
||||
|
||||
@@ -0,0 +1,378 @@
|
||||
"""Parrot's ulogcat CKCM log, as firmware 4.7.1 writes it.
|
||||
|
||||
The aircraft keeps its system log at `internal_000/Debug/current/ckcm/ckcm.bin`,
|
||||
produced by `/usr/bin/ckcm_log.sh` running `ulogcat -uk -v ckcm`. It is the
|
||||
only place several facts about the machine are stated without a shell: which
|
||||
board it is, which config it loaded, what its command handler thought it was
|
||||
doing. So it is worth reading, and reading it needs the framing.
|
||||
|
||||
CKCM is a serial-debug format from Parrot's car-kit era, and the renderer that
|
||||
writes it was deleted from `Parrot-Developers/ulog` in October 2017. The file
|
||||
`ulogcat/libulogcat_ckcm.c` at the commit before `cec8877` is the reference
|
||||
for everything below, and it says in a comment that it copies its constants
|
||||
because the real CKCM headers were never published. Independently, the
|
||||
framing was derived from a 560 KB sample off this aircraft and holds across
|
||||
all of it: 5520 entries, every header consumed to the byte, no residue.
|
||||
|
||||
frame := D5 E6 <kind:u8> <body> E5 F6
|
||||
|
||||
A log entry is TWO frames written back to back in one `write()`: a header
|
||||
frame and then the data frame carrying its message. There is no file header
|
||||
and no rotation framing; the file is the raw concatenation.
|
||||
|
||||
header (kind E7) := opt:u8 priority:u8 [colour:3 if opt&01]
|
||||
stamp:u64le tid:i32le
|
||||
[tname:pstr if opt&20] [tag:pstr if opt&40]
|
||||
|
||||
data (kind 02) := pstr
|
||||
data (kind 24) := colour:3 02 pstr
|
||||
data (other) := raw bytes, for a ulog_bin entry
|
||||
|
||||
`pstr` is one length byte then that many bytes, so a message is capped at 255
|
||||
bytes and anything longer is truncated by the drone, not by us.
|
||||
|
||||
`priority` is a single ASCII letter, and it is lossy on purpose: the renderer
|
||||
maps EMERG, ALERT and CRIT all to `C`, and NOTICE and INFO both to `I`, so a
|
||||
notice cannot be told from an info here.
|
||||
|
||||
`stamp` is microseconds. Which clock it counts from depends on the ulogger
|
||||
driver build, and this firmware's answer is legible in the data: userspace
|
||||
stamps top out around 2067 s rather than 1.7e9, and they agree to a constant
|
||||
8 ms with the uptime stamps 779 of the `ARlibs` messages carry in their own
|
||||
text. So this aircraft logs **monotonic time since boot**, and there is no
|
||||
wall clock anywhere in the file. Kernel entries carry the printk uptime, which
|
||||
is the same clock but merged in from `/proc/kmsg` and so can land a few
|
||||
milliseconds out of order against the userspace stream.
|
||||
|
||||
There is no pid field. Only `tid` is stored, and the process name is smuggled
|
||||
into `tname` as `process/thread` when the two differ, which is why a `/` in
|
||||
`source` is the only signal that an entry came from a named thread rather than
|
||||
a process main. Both strings are genuinely optional: kernel messages carry a
|
||||
tag (`KERNEL`) and no name, a few processes carry a name and no tag. `-l` is
|
||||
not passed, so tags are raw and `tag == "KERNEL"` is the only way to tell a
|
||||
kernel entry from a userspace one.
|
||||
|
||||
The three colour bytes are RGB, and the drone colours its camera and ISP
|
||||
output: 23 of the 5520 sample entries carry them, white for ulogcat's own
|
||||
banner lines and oranges for the rest. The data frame clamps each channel to a
|
||||
minimum of 1, so a header colour of `ff d0 00` arrives as `ff d0 01` there;
|
||||
the header's value is the true one and the one kept.
|
||||
|
||||
Two opt bits, `02` (PC) and `10` (thread priority), are defined but never
|
||||
written by this renderer, so no public source says what they would look like.
|
||||
A header carrying either is refused rather than guessed at.
|
||||
|
||||
Robustness is not optional here: the log is live. The sample grew from 445 KB
|
||||
to 560 KB inside a minute, so a fetch lands mid-record as a matter of course
|
||||
and a truncated tail has to end the parse quietly rather than raise. Nor are
|
||||
the markers escaped, which is why the lengths are what this parser trusts and
|
||||
the end marker is only what it checks them against.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import struct
|
||||
from collections.abc import Iterator
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
FRAME_START = b"\xd5\xe6"
|
||||
FRAME_END = b"\xe5\xf6"
|
||||
|
||||
KIND_HEADER = 0xE7 # CKCM calls this PLOG
|
||||
KIND_MESSAGE = 0x02 # RT_STR
|
||||
KIND_MESSAGE_COLOURED = 0x24 # RT_COLOR, then an RT_STR
|
||||
|
||||
#: Not a byte in the file. A ulog_bin entry's data frame carries no kind and
|
||||
#: no length, so this parser needs a name for "the frame was raw payload".
|
||||
KIND_BINARY = -1
|
||||
|
||||
FLAG_COLOUR = 0x01
|
||||
FLAG_PC = 0x02 # never written by this renderer; layout unknown
|
||||
FLAG_DATE = 0x04
|
||||
FLAG_TID = 0x08
|
||||
FLAG_THREAD_PRIORITY = 0x10 # never written by this renderer; layout unknown
|
||||
FLAG_NAME = 0x20
|
||||
FLAG_TAG = 0x40
|
||||
|
||||
#: Bits whose wire shape no public source describes. A header that sets one
|
||||
#: cannot be read, and reading the fields after it would be a guess.
|
||||
FLAGS_UNSUPPORTED = FLAG_PC | FLAG_THREAD_PRIORITY | 0x80
|
||||
|
||||
COLOUR_LEN = 3
|
||||
|
||||
#: The letters the renderer writes, spelled out. `C` also covers EMERG and
|
||||
#: ALERT, and `I` also covers NOTICE, so neither is recoverable from here.
|
||||
PRIORITY_NAMES = {
|
||||
"C": "critical",
|
||||
"E": "error",
|
||||
"W": "warning",
|
||||
"I": "info",
|
||||
"D": "debug",
|
||||
}
|
||||
|
||||
#: Syslog severity numbers, so "at least a warning" can be a comparison.
|
||||
#: Lower is more severe, and an unknown letter sorts as the least severe
|
||||
#: thing there is so that a filter never silently drops it.
|
||||
PRIORITY_RANK = {"C": 2, "E": 3, "W": 4, "I": 6, "D": 7}
|
||||
UNKNOWN_RANK = 99
|
||||
|
||||
|
||||
def rank(priority: str) -> int:
|
||||
return PRIORITY_RANK.get(priority, UNKNOWN_RANK)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class LogRecord:
|
||||
"""One log entry, the header and message frames put back together."""
|
||||
|
||||
uptime_us: int
|
||||
priority: str
|
||||
"""Syslog level as the single letter the file stores. `level` spells it."""
|
||||
tag: str
|
||||
source: str
|
||||
"""`process/thread`, or empty when the entry carries no source name."""
|
||||
tid: int
|
||||
message: str
|
||||
"""Text, or a short stand-in when the entry's payload was binary."""
|
||||
offset: int
|
||||
"""Byte offset of the entry's header frame, for pointing at a bad record."""
|
||||
colour: bytes | None = None
|
||||
"""RGB the drone asked for, when it asked. Camera and ISP code does."""
|
||||
binary: bytes | None = None
|
||||
"""Raw payload of a ulog_bin entry, which carries no text and no length."""
|
||||
|
||||
@property
|
||||
def uptime(self) -> float:
|
||||
"""Seconds since the drone booted."""
|
||||
return self.uptime_us / 1_000_000
|
||||
|
||||
@property
|
||||
def level(self) -> str:
|
||||
return PRIORITY_NAMES.get(self.priority, self.priority)
|
||||
|
||||
|
||||
@dataclass
|
||||
class ParseStats:
|
||||
"""What the parse had to put up with. Worth reporting on a live log."""
|
||||
|
||||
entries: int = 0
|
||||
resyncs: int = 0
|
||||
"""Frames that did not decode, each costing a hunt for the next marker."""
|
||||
truncated: bool = False
|
||||
"""The buffer ended mid-record, which is normal for a log still being written."""
|
||||
unpaired: int = 0
|
||||
"""Headers with no message frame after them, or the reverse."""
|
||||
tags: dict[str, int] = field(default_factory=dict)
|
||||
|
||||
|
||||
class _Truncated(Exception):
|
||||
"""The frame runs past the end of the buffer."""
|
||||
|
||||
|
||||
class _Malformed(Exception):
|
||||
"""The frame decoded to something that is not a frame."""
|
||||
|
||||
|
||||
def _pstr(data: bytes, pos: int) -> tuple[str, int]:
|
||||
if pos >= len(data):
|
||||
raise _Truncated
|
||||
length = data[pos]
|
||||
end = pos + 1 + length
|
||||
if end > len(data):
|
||||
raise _Truncated
|
||||
return data[pos + 1 : end].decode("utf-8", "replace"), end
|
||||
|
||||
|
||||
def _frame(data: bytes, start: int, *, expect_data: bool) -> tuple[int, int, int]:
|
||||
"""Locate the frame whose start marker is at `start`.
|
||||
|
||||
Returns `(kind, body_start, body_end)`, with `KIND_BINARY` for a payload
|
||||
that carries no kind byte. For every other frame the end marker is
|
||||
verified, which is the only check that the lengths were read correctly:
|
||||
get them wrong and the two bytes that should close the frame are not
|
||||
there.
|
||||
|
||||
A binary payload has no kind, no length and so nothing to check, which
|
||||
makes it indistinguishable from corruption. `expect_data` is what keeps
|
||||
that from costing anything: a data frame only ever follows a header, so
|
||||
only there is an unreadable frame read as a payload. Anywhere else it is
|
||||
treated as damage and the caller resyncs, which is what keeps a corrupt
|
||||
region from swallowing the next header along with it.
|
||||
"""
|
||||
pos = start + len(FRAME_START)
|
||||
if pos >= len(data):
|
||||
raise _Truncated
|
||||
kind = data[pos]
|
||||
body = pos + 1
|
||||
|
||||
if kind == KIND_HEADER:
|
||||
end = _header_end(data, body)
|
||||
elif kind == KIND_MESSAGE:
|
||||
_, end = _pstr(data, body)
|
||||
elif (
|
||||
kind == KIND_MESSAGE_COLOURED
|
||||
and body + COLOUR_LEN < len(data)
|
||||
and data[body + COLOUR_LEN] == KIND_MESSAGE
|
||||
):
|
||||
_, end = _pstr(data, body + COLOUR_LEN + 1)
|
||||
elif expect_data:
|
||||
return _binary_frame(data, pos)
|
||||
else:
|
||||
raise _Malformed
|
||||
|
||||
if end + len(FRAME_END) > len(data):
|
||||
raise _Truncated
|
||||
if data[end : end + len(FRAME_END)] != FRAME_END:
|
||||
raise _Malformed
|
||||
return kind, body, end
|
||||
|
||||
|
||||
def _binary_frame(data: bytes, body: int) -> tuple[int, int, int]:
|
||||
end = data.find(FRAME_END, body)
|
||||
if end < 0:
|
||||
raise _Truncated
|
||||
return KIND_BINARY, body, end
|
||||
|
||||
|
||||
def _header_end(data: bytes, body: int) -> int:
|
||||
pos = body + 2 # opt, priority
|
||||
if pos > len(data):
|
||||
raise _Truncated
|
||||
flags = data[body]
|
||||
if flags & FLAGS_UNSUPPORTED:
|
||||
raise _Malformed
|
||||
if flags & FLAG_COLOUR:
|
||||
pos += COLOUR_LEN
|
||||
pos += 12 # stamp u64, tid i32
|
||||
if pos > len(data):
|
||||
raise _Truncated
|
||||
if flags & FLAG_NAME:
|
||||
_, pos = _pstr(data, pos)
|
||||
if flags & FLAG_TAG:
|
||||
_, pos = _pstr(data, pos)
|
||||
return pos
|
||||
|
||||
|
||||
def _header(data: bytes, body: int, end: int) -> tuple[str, str, str, int, int, bytes | None]:
|
||||
flags = data[body]
|
||||
priority = chr(data[body + 1])
|
||||
pos = body + 2
|
||||
colour = None
|
||||
if flags & FLAG_COLOUR:
|
||||
colour = data[pos : pos + COLOUR_LEN]
|
||||
pos += COLOUR_LEN
|
||||
stamp_us, tid = struct.unpack_from("<Qi", data, pos)
|
||||
pos += 12
|
||||
name = tag = ""
|
||||
if flags & FLAG_NAME:
|
||||
name, pos = _pstr(data, pos)
|
||||
if flags & FLAG_TAG:
|
||||
tag, pos = _pstr(data, pos)
|
||||
if pos != end:
|
||||
# Residue means the opt bits described a shape this parser does not
|
||||
# know, so the fields it did read are not trustworthy either.
|
||||
raise _Malformed
|
||||
return priority, tag, name, tid, stamp_us, colour
|
||||
|
||||
|
||||
def _message(data: bytes, kind: int, body: int) -> str:
|
||||
pos = body + COLOUR_LEN + 1 if kind == KIND_MESSAGE_COLOURED else body
|
||||
text, _ = _pstr(data, pos)
|
||||
# Most messages end in a newline the drone's own printf put there.
|
||||
return text.rstrip("\n")
|
||||
|
||||
|
||||
def parse(data: bytes, stats: ParseStats | None = None) -> Iterator[LogRecord]:
|
||||
"""Walk a ulogcat buffer, yielding one record per log entry.
|
||||
|
||||
Never raises on bad input. A frame that does not decode costs a hunt for
|
||||
the next start marker, and a buffer that ends mid-record ends the walk;
|
||||
both are counted in `stats` if one is passed, because on a live log the
|
||||
difference between "the drone logged nothing more" and "we caught it
|
||||
mid-write" is worth telling the caller about.
|
||||
"""
|
||||
stats = stats if stats is not None else ParseStats()
|
||||
pos = data.find(FRAME_START)
|
||||
if pos < 0:
|
||||
return
|
||||
pending: tuple[int, tuple] | None = None
|
||||
|
||||
while pos >= 0:
|
||||
try:
|
||||
kind, body, end = _frame(data, pos, expect_data=pending is not None)
|
||||
except _Truncated:
|
||||
# Only the tail of a live file should land here. If a start marker
|
||||
# follows, the "truncation" was really a misread frame.
|
||||
nxt = data.find(FRAME_START, pos + len(FRAME_START))
|
||||
if nxt < 0:
|
||||
stats.truncated = True
|
||||
break
|
||||
stats.resyncs += 1
|
||||
pos = nxt
|
||||
continue
|
||||
except _Malformed:
|
||||
stats.resyncs += 1
|
||||
pos = data.find(FRAME_START, pos + len(FRAME_START))
|
||||
continue
|
||||
|
||||
if kind == KIND_HEADER:
|
||||
if pending is not None:
|
||||
stats.unpaired += 1
|
||||
try:
|
||||
pending = (pos, _header(data, body, end))
|
||||
except (_Truncated, _Malformed):
|
||||
stats.resyncs += 1
|
||||
pending = None
|
||||
elif pending is None:
|
||||
stats.unpaired += 1
|
||||
else:
|
||||
offset, (priority, tag, source, tid, stamp_us, colour) = pending
|
||||
pending = None
|
||||
binary = data[body:end] if kind == KIND_BINARY else None
|
||||
if binary is None:
|
||||
try:
|
||||
message = _message(data, kind, body)
|
||||
except (_Truncated, _Malformed): # pragma: no cover - _frame checked it
|
||||
stats.resyncs += 1
|
||||
pos = end + len(FRAME_END)
|
||||
continue
|
||||
else:
|
||||
message = f"<{len(binary)} bytes of binary log payload>"
|
||||
stats.entries += 1
|
||||
stats.tags[tag] = stats.tags.get(tag, 0) + 1
|
||||
yield LogRecord(
|
||||
uptime_us=stamp_us,
|
||||
priority=priority,
|
||||
tag=tag,
|
||||
source=source,
|
||||
tid=tid,
|
||||
message=message,
|
||||
offset=offset,
|
||||
colour=colour,
|
||||
binary=binary,
|
||||
)
|
||||
|
||||
pos = end + len(FRAME_END)
|
||||
if pos >= len(data):
|
||||
break
|
||||
if data[pos : pos + len(FRAME_START)] != FRAME_START:
|
||||
nxt = data.find(FRAME_START, pos)
|
||||
if nxt < 0:
|
||||
stats.truncated = True
|
||||
break
|
||||
stats.resyncs += 1
|
||||
pos = nxt
|
||||
|
||||
if pending is not None:
|
||||
stats.unpaired += 1
|
||||
if stats.resyncs or stats.unpaired:
|
||||
logger.debug(
|
||||
"ulog parse: %d entries, %d resyncs, %d unpaired, truncated=%s",
|
||||
stats.entries,
|
||||
stats.resyncs,
|
||||
stats.unpaired,
|
||||
stats.truncated,
|
||||
)
|
||||
@@ -0,0 +1,280 @@
|
||||
"""Record the aircraft's RTP stream, and replay it packet for packet.
|
||||
|
||||
Two halves of one idea. `record` binds the port we name in the ARSDK
|
||||
handshake and writes every datagram the drone sends us, with the time it
|
||||
arrived. `ReplaySource` feeds that file back to a client with the gaps it was
|
||||
recorded with, which is the only way to put a viewer in front of the
|
||||
aircraft's real pacing without the aircraft.
|
||||
|
||||
That distinction is the point of having two paths at all.
|
||||
`rtp.PacketisedSource` builds a stream from any video file and sends it at a
|
||||
steady frame rate, which proves a decoder and a renderer work. A replay
|
||||
reproduces bursts, reordering and jitter as they happened, which is what a
|
||||
latency measurement needs. They are different instruments and a caller should
|
||||
know which one it has.
|
||||
|
||||
The file is deliberately dull: a magic string, a version, then a record per
|
||||
packet. Nothing is parsed, nothing is dropped, and the payload bytes are kept
|
||||
whole, so a later pass can answer questions the recorder never asked. Counting
|
||||
the NAL types carried inside FU-A packets is the obvious one: the only
|
||||
measurement we have of the drone's stream read the outer header byte of each
|
||||
packet and stopped there, so what the fragments carried is still unknown.
|
||||
|
||||
Recording needs the drone. Reading, writing and replaying a file do not, which
|
||||
is what makes the format testable here.
|
||||
|
||||
python -m mcbebop.media.capture out.rtpcap --seconds 10
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import random
|
||||
import socket
|
||||
import statistics
|
||||
import struct
|
||||
import time
|
||||
from collections.abc import Iterable, Iterator
|
||||
from dataclasses import dataclass, field
|
||||
from itertools import pairwise
|
||||
from pathlib import Path
|
||||
|
||||
from mcbebop.media.rtp import CLOCK_RATE, DEFAULT_FPS, parse_packet, resolve_start_offset, rewrite_header
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
MAGIC = b"BEBOPRTP"
|
||||
VERSION = 1
|
||||
_FILE_HEADER = struct.Struct("<8sHH")
|
||||
_RECORD = struct.Struct("<dI")
|
||||
|
||||
#: The port the controller names as `arstream2_client_stream_port`.
|
||||
DEFAULT_CAPTURE_PORT = 55004
|
||||
_MAX_DATAGRAM = 65535
|
||||
_POLL = 0.25
|
||||
|
||||
|
||||
class CaptureFormatError(ValueError):
|
||||
"""The file is not a capture, or is a version we do not know."""
|
||||
|
||||
|
||||
def write_capture(path: str | Path, packets: Iterable[tuple[float, bytes]]) -> int:
|
||||
"""Write `(arrival seconds, datagram)` pairs. Returns the packet count.
|
||||
|
||||
Times are relative to the first packet, so a file is comparable with
|
||||
itself regardless of when it was taken.
|
||||
"""
|
||||
out = Path(path)
|
||||
out.parent.mkdir(parents=True, exist_ok=True)
|
||||
count = 0
|
||||
with out.open("wb") as fh:
|
||||
fh.write(_FILE_HEADER.pack(MAGIC, VERSION, 0))
|
||||
for at, data in packets:
|
||||
fh.write(_RECORD.pack(at, len(data)) + data)
|
||||
count += 1
|
||||
return count
|
||||
|
||||
|
||||
def read_capture(path: str | Path) -> Iterator[tuple[float, bytes]]:
|
||||
"""Stream a capture back as `(arrival seconds, datagram)` pairs.
|
||||
|
||||
A truncated tail stops the read rather than raising: a capture cut short
|
||||
by Ctrl-C is still worth replaying, and the alternative is losing the
|
||||
whole file to its last partial record.
|
||||
"""
|
||||
with Path(path).open("rb") as fh:
|
||||
head = fh.read(_FILE_HEADER.size)
|
||||
if len(head) < _FILE_HEADER.size:
|
||||
raise CaptureFormatError(f"{path} is too short to be a capture")
|
||||
magic, version, _flags = _FILE_HEADER.unpack(head)
|
||||
if magic != MAGIC:
|
||||
raise CaptureFormatError(f"{path} does not start with {MAGIC!r}")
|
||||
if version != VERSION:
|
||||
raise CaptureFormatError(f"{path} is capture version {version}, this reads {VERSION}")
|
||||
while True:
|
||||
raw = fh.read(_RECORD.size)
|
||||
if len(raw) < _RECORD.size:
|
||||
if raw:
|
||||
log.warning("%s ends mid-record; stopping there", path)
|
||||
return
|
||||
at, length = _RECORD.unpack(raw)
|
||||
data = fh.read(length)
|
||||
if len(data) < length:
|
||||
log.warning("%s ends mid-packet; stopping there", path)
|
||||
return
|
||||
yield at, data
|
||||
|
||||
|
||||
def record(
|
||||
path: str | Path,
|
||||
*,
|
||||
port: int = DEFAULT_CAPTURE_PORT,
|
||||
host: str = "",
|
||||
seconds: float | None = None,
|
||||
max_packets: int | None = None,
|
||||
) -> int:
|
||||
"""Bind `port` and write every datagram that arrives. Returns the count.
|
||||
|
||||
Read-only with respect to the aircraft: it sends nothing, so it cannot be
|
||||
the reason a stream stops. Something else has to hold the ARSDK session
|
||||
and send VideoEnable, and it has to do it after this is listening, because
|
||||
RTP is connectionless and whatever arrives before the bind is gone.
|
||||
"""
|
||||
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
sock.bind((host, port))
|
||||
sock.settimeout(_POLL)
|
||||
deadline = None if seconds is None else time.monotonic() + seconds
|
||||
first: float | None = None
|
||||
packets: list[tuple[float, bytes]] = []
|
||||
log.info("recording RTP on %s:%d to %s", host or "0.0.0.0", port, path)
|
||||
try:
|
||||
while deadline is None or time.monotonic() < deadline:
|
||||
try:
|
||||
data, _addr = sock.recvfrom(_MAX_DATAGRAM)
|
||||
except TimeoutError:
|
||||
continue
|
||||
now = time.monotonic()
|
||||
first = now if first is None else first
|
||||
packets.append((now - first, data))
|
||||
if max_packets is not None and len(packets) >= max_packets:
|
||||
break
|
||||
except KeyboardInterrupt:
|
||||
log.info("interrupted; writing what arrived")
|
||||
finally:
|
||||
sock.close()
|
||||
count = write_capture(path, packets)
|
||||
span = packets[-1][0] if packets else 0.0
|
||||
log.info("wrote %d packets over %.2fs to %s", count, span, path)
|
||||
return count
|
||||
|
||||
|
||||
def _median_positive(values: Iterable[int | float], fallback: float) -> float:
|
||||
positive = [v for v in values if v > 0]
|
||||
return statistics.median(positive) if positive else fallback
|
||||
|
||||
|
||||
@dataclass
|
||||
class ReplaySource:
|
||||
"""A recorded RTP stream, sent again with its original inter-packet gaps.
|
||||
|
||||
Only the SSRC and the sequence numbers are rewritten, so the client sees a
|
||||
stream from us rather than one that looks like a replay of someone else's.
|
||||
Timestamps keep their recorded deltas; what is added is a running offset,
|
||||
so a loop carries the clock forward instead of jumping back to the start
|
||||
of the capture and stalling every decoder downstream.
|
||||
|
||||
The loop point is the one gap in the file that does not exist in it: the
|
||||
interval between the last packet and a next one that was never recorded.
|
||||
It is estimated as what would carry the stream to the start of the frame
|
||||
after the last, from the capture's own mean frame period. The median gap
|
||||
is the wrong estimator here and was the first thing tried: with a dozen
|
||||
packets to a frame, most gaps are the ones *inside* a frame, so the median
|
||||
is an intra-frame gap and the stream would loop a frame early every time.
|
||||
"""
|
||||
|
||||
packets_in: tuple[tuple[float, bytes], ...]
|
||||
start_offset: float | str | None = None
|
||||
seed: int | None = None
|
||||
ssrc: int = field(default_factory=lambda: random.getrandbits(32))
|
||||
|
||||
@classmethod
|
||||
def from_path(
|
||||
cls,
|
||||
path: str | Path,
|
||||
*,
|
||||
start_offset: float | str | None = None,
|
||||
seed: int | None = None,
|
||||
) -> ReplaySource:
|
||||
packets = tuple(read_capture(path))
|
||||
if not packets:
|
||||
raise CaptureFormatError(f"{path} holds no packets")
|
||||
return cls(packets_in=packets, start_offset=start_offset, seed=seed)
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
# Both counters live on the instance, not in `packets()`, so a client
|
||||
# that disables and re-enables video sees the stream continue rather
|
||||
# than a clock that jumps backwards under an unchanged SSRC.
|
||||
counters = random.Random(self.seed)
|
||||
self._seq = counters.getrandbits(16)
|
||||
self._ticks = counters.getrandbits(32)
|
||||
# A separate generator, so the chosen offset does not move when the
|
||||
# number of counter draws above changes.
|
||||
offset = resolve_start_offset(self.start_offset, self.duration, random.Random(self.seed))
|
||||
self.start_index = next(
|
||||
(i for i, (at, _) in enumerate(self.packets_in) if at >= offset),
|
||||
0,
|
||||
)
|
||||
stamps = [parse_packet(data)[2] for _at, data in self.packets_in]
|
||||
# Ticks step only between frames, so the median of the positive steps
|
||||
# is one frame's worth however many packets a frame took.
|
||||
ticks = [(b - a) % (1 << 32) for a, b in pairwise(stamps)]
|
||||
self._loop_ticks = int(_median_positive(ticks, CLOCK_RATE / DEFAULT_FPS))
|
||||
self._loop_gap = self._gap_to_the_next_frame(stamps)
|
||||
|
||||
def _gap_to_the_next_frame(self, stamps: list[int]) -> float:
|
||||
"""How long after the last recorded packet the next frame would start."""
|
||||
starts = [at for i, (at, _) in enumerate(self.packets_in) if i == 0 or stamps[i] != stamps[i - 1]]
|
||||
if len(starts) < 2:
|
||||
return 1.0 / DEFAULT_FPS
|
||||
period = (starts[-1] - starts[0]) / (len(starts) - 1)
|
||||
return max(0.0, period - (self.duration - starts[-1]))
|
||||
|
||||
@property
|
||||
def duration(self) -> float:
|
||||
return self.packets_in[-1][0] if self.packets_in else 0.0
|
||||
|
||||
@property
|
||||
def describe(self) -> str:
|
||||
return (
|
||||
f"{len(self.packets_in)} recorded packets over {self.duration:.2f}s, "
|
||||
f"replayed from index {self.start_index}"
|
||||
)
|
||||
|
||||
def packets(self) -> Iterator[tuple[float, bytes]]:
|
||||
total = len(self.packets_in)
|
||||
index = self.start_index
|
||||
prev_at, prev_ticks = None, None
|
||||
while True:
|
||||
at, data = self.packets_in[index]
|
||||
stamp = parse_packet(data)[2]
|
||||
if prev_at is None:
|
||||
delay, step = 0.0, 0
|
||||
elif index == 0:
|
||||
# The wrap: the file's own gaps say nothing about this one.
|
||||
delay, step = self._loop_gap, self._loop_ticks
|
||||
else:
|
||||
delay, step = at - prev_at, (stamp - prev_ticks) % (1 << 32)
|
||||
self._ticks = (self._ticks + step) % (1 << 32)
|
||||
yield (
|
||||
max(0.0, delay),
|
||||
rewrite_header(data, ssrc=self.ssrc, seq=self._seq, timestamp=self._ticks),
|
||||
)
|
||||
self._seq = (self._seq + 1) & 0xFFFF
|
||||
prev_at, prev_ticks = at, stamp
|
||||
index = (index + 1) % total
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
import argparse
|
||||
import sys
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="python -m mcbebop.media.capture",
|
||||
description="Record the Bebop 2's RTP video stream for later replay.",
|
||||
)
|
||||
parser.add_argument("out", help="where to write the capture")
|
||||
parser.add_argument("--port", type=int, default=DEFAULT_CAPTURE_PORT)
|
||||
parser.add_argument("--seconds", type=float, default=None, help="stop after this long")
|
||||
parser.add_argument("--packets", type=int, default=None, help="stop after this many packets")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
# stderr, because this module is importable from the MCP server and
|
||||
# stdout there is the JSON-RPC transport.
|
||||
logging.basicConfig(level=logging.INFO, format="%(message)s", stream=sys.stderr)
|
||||
count = record(args.out, port=args.port, seconds=args.seconds, max_packets=args.packets)
|
||||
return 0 if count else 1
|
||||
|
||||
|
||||
if __name__ == "__main__": # pragma: no cover - a hand-run tool
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,397 @@
|
||||
"""RTP/H.264 packetisation, so the simulator can emit a video stream.
|
||||
|
||||
The Bebop 2 does not serve RTSP. It pushes plain RTP to the port the
|
||||
controller named in its handshake, which means a simulator can imitate the
|
||||
whole video path with a UDP socket and nothing else. This module is the
|
||||
packetiser half of that: Annex-B H.264 in, RFC 6184 datagrams out. Pure
|
||||
stdlib, because the simulator ships in the package and must not drag a media
|
||||
library in behind it.
|
||||
|
||||
What is implemented, and what is not:
|
||||
|
||||
Single-NAL packets when a NAL fits the MTU, FU-A fragmentation when it does
|
||||
not, and STAP-A aggregation for runs of parameter sets and SEI. Nothing else.
|
||||
In particular a VCL NAL is never aggregated, so an access unit that fits in
|
||||
one datagram still goes out as one single-NAL packet rather than a STAP-A.
|
||||
Interleaved mode (types 25 to 29 other than 28), MTAP and the RTCP side are
|
||||
absent; a viewer that needs RTCP from the simulator needs a capture replay.
|
||||
|
||||
The live aircraft's packet-type mix is not reproduced and cannot be from
|
||||
first principles. Six seconds off the drone carried 1461 packets whose outer
|
||||
NAL byte was STAP-A, 740 FU-A and 200 non-IDR, which is around eight STAP-A
|
||||
per frame at 30 fps and far more aggregation than parameter sets alone can
|
||||
explain. Those are *outer* header bytes: for a FU-A packet the type lives in
|
||||
the next byte and was never read, so what the carried NALs were is unknown.
|
||||
Closing that gap needs a real capture, which is what `media/capture.py` and
|
||||
`ReplaySource` exist for.
|
||||
|
||||
Repeating the parameter sets is the one behaviour here that is not optional.
|
||||
The aircraft's SPS and PPS recur throughout the stream rather than appearing
|
||||
once at the start, which is why a viewer can join a flight already in
|
||||
progress and recover in a second or two. `PacketisedSource` does the same.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import random
|
||||
import struct
|
||||
from collections.abc import Iterator, Sequence
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Protocol
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
PAYLOAD_TYPE = 96 # dynamic, bound to H264/90000 by the SDP
|
||||
CLOCK_RATE = 90_000
|
||||
DEFAULT_MTU = 1400 # payload bytes, so a 1500-byte link does not fragment
|
||||
DEFAULT_FPS = 30.0
|
||||
|
||||
# version 2, no padding, no extension, no CSRC.
|
||||
_V2 = 0x80
|
||||
_MARKER = 0x80
|
||||
_RTP_HEADER = struct.Struct(">BBHII")
|
||||
_STAP_A_LENGTH = struct.Struct(">H")
|
||||
|
||||
NAL_STAP_A = 24
|
||||
NAL_FU_A = 28
|
||||
|
||||
_VCL = frozenset(range(1, 6)) # 1 non-IDR .. 5 IDR
|
||||
_SPS, _PPS = 7, 8
|
||||
# What is worth aggregating: parameter sets, SEI and the access unit
|
||||
# delimiter. All small, all repeated, none of them a picture.
|
||||
_AGGREGATABLE = frozenset({6, _SPS, _PPS, 9})
|
||||
|
||||
_START_CODE = b"\x00\x00\x01"
|
||||
|
||||
|
||||
def nal_type(nal: bytes) -> int:
|
||||
return nal[0] & 0x1F
|
||||
|
||||
|
||||
def _nri(nal: bytes) -> int:
|
||||
return nal[0] & 0x60
|
||||
|
||||
|
||||
def iter_nals(data: bytes) -> Iterator[bytes]:
|
||||
"""Split an Annex-B elementary stream into NAL units.
|
||||
|
||||
Both start codes are accepted. The three-byte form is the one actually
|
||||
searched for, and a fourth leading zero is simply a trailing zero of the
|
||||
previous NAL, which is why trailing zeros are stripped: a decoder ignores
|
||||
them but they would otherwise inflate every NAL's length and so change
|
||||
where the MTU falls.
|
||||
"""
|
||||
start = data.find(_START_CODE)
|
||||
if start < 0:
|
||||
return
|
||||
pos = start + len(_START_CODE)
|
||||
while True:
|
||||
nxt = data.find(_START_CODE, pos)
|
||||
end = len(data) if nxt < 0 else nxt
|
||||
nal = data[pos:end].rstrip(b"\x00")
|
||||
if nal:
|
||||
yield nal
|
||||
if nxt < 0:
|
||||
return
|
||||
pos = nxt + len(_START_CODE)
|
||||
|
||||
|
||||
def _opens_picture(nal: bytes) -> bool:
|
||||
"""Is this a VCL NAL whose slice starts at macroblock zero?
|
||||
|
||||
`first_mb_in_slice` is the first ue(v) in the slice header, and ue(v) is
|
||||
zero exactly when the first bit is set. That makes the boundary between
|
||||
access units readable without a bitstream parser, and it keeps a
|
||||
multi-slice picture in one access unit instead of splitting it into one
|
||||
per slice, which would put the marker bit in the wrong places.
|
||||
"""
|
||||
return len(nal) > 1 and nal_type(nal) in _VCL and bool(nal[1] & 0x80)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AccessUnit:
|
||||
"""One decodable picture, with whatever headers precede it."""
|
||||
|
||||
nals: tuple[bytes, ...]
|
||||
|
||||
@property
|
||||
def carries_parameter_sets(self) -> bool:
|
||||
return any(nal_type(n) in (_SPS, _PPS) for n in self.nals)
|
||||
|
||||
@property
|
||||
def is_idr(self) -> bool:
|
||||
return any(nal_type(n) == 5 for n in self.nals)
|
||||
|
||||
|
||||
def access_units(nals: Sequence[bytes]) -> list[AccessUnit]:
|
||||
"""Group NALs into access units.
|
||||
|
||||
A picture-opening VCL NAL closes the previous unit, so parameter sets and
|
||||
SEI attach to the picture they precede rather than to the one before.
|
||||
"""
|
||||
units: list[AccessUnit] = []
|
||||
current: list[bytes] = []
|
||||
have_picture = False
|
||||
for nal in nals:
|
||||
if _opens_picture(nal) and have_picture:
|
||||
units.append(AccessUnit(tuple(current)))
|
||||
current, have_picture = [], False
|
||||
current.append(nal)
|
||||
have_picture = have_picture or nal_type(nal) in _VCL
|
||||
if current:
|
||||
units.append(AccessUnit(tuple(current)))
|
||||
return units
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AnnexBStream:
|
||||
"""A parsed Annex-B file, ready to be looped."""
|
||||
|
||||
units: tuple[AccessUnit, ...]
|
||||
parameter_sets: tuple[bytes, ...]
|
||||
|
||||
@classmethod
|
||||
def from_bytes(cls, data: bytes) -> AnnexBStream:
|
||||
nals = list(iter_nals(data))
|
||||
if not nals:
|
||||
raise ValueError("no NAL units found; is this an Annex-B elementary stream?")
|
||||
# Keep the last of each kind: a stream whose resolution changes
|
||||
# mid-file should be repeated with the sets that are in force, and
|
||||
# x264 writes the same pair every time anyway.
|
||||
latest: dict[int, bytes] = {}
|
||||
for nal in nals:
|
||||
kind = nal_type(nal)
|
||||
if kind in (_SPS, _PPS):
|
||||
latest[kind] = nal
|
||||
sets = tuple(latest[k] for k in (_SPS, _PPS) if k in latest)
|
||||
if not sets:
|
||||
log.warning("the stream carries no SPS/PPS, so a viewer joining late cannot start")
|
||||
return cls(units=tuple(access_units(nals)), parameter_sets=sets)
|
||||
|
||||
@classmethod
|
||||
def from_path(cls, path: str | Path) -> AnnexBStream:
|
||||
return cls.from_bytes(Path(path).read_bytes())
|
||||
|
||||
|
||||
@dataclass
|
||||
class Packetiser:
|
||||
"""Annex-B access units in, RTP datagrams out.
|
||||
|
||||
Stateful on purpose. The sequence number, the timestamp and the SSRC
|
||||
belong to a stream rather than to a frame, and a looping source must not
|
||||
reset any of them: a decoder handed a timestamp that jumps backwards
|
||||
treats the whole stream as corrupt.
|
||||
"""
|
||||
|
||||
mtu: int = DEFAULT_MTU
|
||||
payload_type: int = PAYLOAD_TYPE
|
||||
ssrc: int = field(default_factory=lambda: random.getrandbits(32))
|
||||
seq: int = field(default_factory=lambda: random.getrandbits(16))
|
||||
timestamp: int = field(default_factory=lambda: random.getrandbits(32))
|
||||
aggregate: bool = True
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
# Two bytes of FU-A header, so an MTU below three could never carry a
|
||||
# payload byte and would loop forever.
|
||||
if self.mtu < 3:
|
||||
raise ValueError(f"mtu must be at least 3 payload bytes, got {self.mtu}")
|
||||
|
||||
def packetise(self, nals: Sequence[bytes], *, advance: int = 0) -> list[bytes]:
|
||||
"""Datagrams for one access unit, then move the clock on by `advance`.
|
||||
|
||||
The marker bit lands on the last packet of the unit, which is how a
|
||||
receiver knows the picture is complete without parsing it.
|
||||
"""
|
||||
payloads: list[bytes] = []
|
||||
for group in self._groups(nals):
|
||||
if len(group) > 1:
|
||||
payloads.append(self._stap_a(group))
|
||||
elif len(group[0]) <= self.mtu:
|
||||
payloads.append(group[0])
|
||||
else:
|
||||
payloads.extend(self._fu_a(group[0]))
|
||||
|
||||
packets = [self._frame(p, marker=i == len(payloads) - 1) for i, p in enumerate(payloads)]
|
||||
self.timestamp = (self.timestamp + advance) % (1 << 32)
|
||||
return packets
|
||||
|
||||
# -- grouping --------------------------------------------------------
|
||||
def _groups(self, nals: Sequence[bytes]) -> list[list[bytes]]:
|
||||
"""Runs of NALs that travel together. Singletons unless aggregating.
|
||||
|
||||
A STAP-A of one NAL is legal but pointless, so a run of length one is
|
||||
emitted as a single-NAL packet instead.
|
||||
"""
|
||||
if not self.aggregate:
|
||||
return [[nal] for nal in nals]
|
||||
groups: list[list[bytes]] = []
|
||||
run: list[bytes] = []
|
||||
used = 1 # the STAP-A header byte
|
||||
for nal in nals:
|
||||
cost = _STAP_A_LENGTH.size + len(nal)
|
||||
if nal_type(nal) in _AGGREGATABLE and used + cost <= self.mtu:
|
||||
run.append(nal)
|
||||
used += cost
|
||||
continue
|
||||
if run:
|
||||
groups.append(run)
|
||||
run, used = [], 1
|
||||
if nal_type(nal) in _AGGREGATABLE and 1 + cost <= self.mtu:
|
||||
run, used = [nal], 1 + cost
|
||||
else:
|
||||
groups.append([nal])
|
||||
if run:
|
||||
groups.append(run)
|
||||
return groups
|
||||
|
||||
# -- packet shapes ---------------------------------------------------
|
||||
def _frame(self, payload: bytes, *, marker: bool) -> bytes:
|
||||
seq = self.seq
|
||||
self.seq = (seq + 1) & 0xFFFF
|
||||
second = (_MARKER if marker else 0) | self.payload_type
|
||||
return _RTP_HEADER.pack(_V2, second, seq, self.timestamp, self.ssrc) + payload
|
||||
|
||||
def _stap_a(self, nals: Sequence[bytes]) -> bytes:
|
||||
# The aggregate's NRI is the highest of what it carries, so dropping
|
||||
# it costs a receiver no more than dropping the most important NAL in
|
||||
# it would have.
|
||||
out = bytearray([max(_nri(n) for n in nals) | NAL_STAP_A])
|
||||
for nal in nals:
|
||||
out += _STAP_A_LENGTH.pack(len(nal)) + nal
|
||||
return bytes(out)
|
||||
|
||||
def _fu_a(self, nal: bytes) -> list[bytes]:
|
||||
indicator = _nri(nal) | NAL_FU_A
|
||||
kind = nal_type(nal)
|
||||
body = nal[1:] # the original header is rebuilt by the receiver
|
||||
budget = self.mtu - 2
|
||||
chunks = [body[i : i + budget] for i in range(0, len(body), budget)]
|
||||
out = []
|
||||
for i, chunk in enumerate(chunks):
|
||||
flags = (0x80 if i == 0 else 0) | (0x40 if i == len(chunks) - 1 else 0)
|
||||
out.append(bytes([indicator, flags | kind]) + chunk)
|
||||
return out
|
||||
|
||||
|
||||
def parse_packet(packet: bytes) -> tuple[int, int, int, int, bool, bytes]:
|
||||
"""Read an RTP packet back: payload type, seq, timestamp, ssrc, marker, payload.
|
||||
|
||||
Here rather than in the tests because the capture tools and the replay
|
||||
path need it too, and a second implementation would be a second chance to
|
||||
get the field order wrong.
|
||||
"""
|
||||
if len(packet) < _RTP_HEADER.size:
|
||||
raise ValueError(f"an RTP packet is at least {_RTP_HEADER.size} bytes, got {len(packet)}")
|
||||
first, second, seq, timestamp, ssrc = _RTP_HEADER.unpack_from(packet)
|
||||
if first >> 6 != 2:
|
||||
raise ValueError(f"not RTP version 2: first byte {first:#04x}")
|
||||
csrc = first & 0x0F
|
||||
offset = _RTP_HEADER.size + 4 * csrc
|
||||
return second & 0x7F, seq, timestamp, ssrc, bool(second & _MARKER), packet[offset:]
|
||||
|
||||
|
||||
def rewrite_header(packet: bytes, *, ssrc: int, seq: int, timestamp: int) -> bytes:
|
||||
"""Restamp a recorded packet with our SSRC, sequence number and clock.
|
||||
|
||||
The caller supplies the timestamp rather than an offset because a replay
|
||||
that loops has to carry the recorded gaps forward past the loop point
|
||||
instead of jumping back to where the capture started. `ReplaySource`
|
||||
accumulates the recorded deltas to do that.
|
||||
"""
|
||||
first, second, _seq, _timestamp, _ssrc = _RTP_HEADER.unpack_from(packet)
|
||||
head = _RTP_HEADER.pack(first, second, seq & 0xFFFF, timestamp & 0xFFFFFFFF, ssrc & 0xFFFFFFFF)
|
||||
return head + packet[_RTP_HEADER.size :]
|
||||
|
||||
|
||||
class VideoSource(Protocol):
|
||||
"""Where the simulator gets datagrams and when to send them.
|
||||
|
||||
One pair per packet: how long to wait before sending it, and the bytes.
|
||||
The generator is expected to be endless, so a viewer can be left running.
|
||||
"""
|
||||
|
||||
def packets(self) -> Iterator[tuple[float, bytes]]: ...
|
||||
|
||||
@property
|
||||
def describe(self) -> str: ...
|
||||
|
||||
|
||||
def resolve_start_offset(offset: float | str | None, duration: float, rng: random.Random) -> float:
|
||||
"""Where in the stream to begin, in seconds.
|
||||
|
||||
`"random"` exists to reproduce a viewer switched on while the drone is
|
||||
already flying, which hands a stateless decoder a stream that begins
|
||||
mid-GOP. The seed is the caller's so a failure can be run again.
|
||||
"""
|
||||
if offset is None:
|
||||
return 0.0
|
||||
if isinstance(offset, str):
|
||||
if offset != "random":
|
||||
raise ValueError(f"start_offset must be a number or 'random', got {offset!r}")
|
||||
return rng.uniform(0.0, duration) if duration > 0 else 0.0
|
||||
if offset < 0:
|
||||
raise ValueError(f"start_offset must not be negative, got {offset}")
|
||||
return offset % duration if duration > 0 else 0.0
|
||||
|
||||
|
||||
@dataclass
|
||||
class PacketisedSource:
|
||||
"""An Annex-B file, packetised and paced at a nominal frame rate.
|
||||
|
||||
Steady pacing: every frame goes out one period after the last and the
|
||||
packets within a frame go out back to back. That is a synthetic
|
||||
instrument. It is the right one for checking that a decoder and a
|
||||
renderer work, and the wrong one for measuring latency or jitter against
|
||||
the aircraft, which sends in bursts this does not imitate. Use
|
||||
`ReplaySource` over a real capture for that.
|
||||
"""
|
||||
|
||||
stream: AnnexBStream
|
||||
fps: float = DEFAULT_FPS
|
||||
mtu: int = DEFAULT_MTU
|
||||
parameter_set_period: int = 30
|
||||
start_offset: float | str | None = None
|
||||
seed: int | None = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if self.fps <= 0:
|
||||
raise ValueError(f"fps must be positive, got {self.fps}")
|
||||
if not self.stream.units:
|
||||
raise ValueError("the stream has no access units")
|
||||
self.packetiser = Packetiser(mtu=self.mtu)
|
||||
self._rng = random.Random(self.seed)
|
||||
self.start_index = round(self._offset_seconds() * self.fps) % len(self.stream.units)
|
||||
|
||||
def _offset_seconds(self) -> float:
|
||||
return resolve_start_offset(self.start_offset, len(self.stream.units) / self.fps, self._rng)
|
||||
|
||||
@property
|
||||
def describe(self) -> str:
|
||||
return (
|
||||
f"{len(self.stream.units)} access units paced at {self.fps:g} fps from index {self.start_index}"
|
||||
)
|
||||
|
||||
def packets(self) -> Iterator[tuple[float, bytes]]:
|
||||
units = self.stream.units
|
||||
ticks = round(CLOCK_RATE / self.fps)
|
||||
period = 1.0 / self.fps
|
||||
index = self.start_index
|
||||
frame = 0
|
||||
while True:
|
||||
unit = units[index]
|
||||
nals = list(unit.nals)
|
||||
# Nothing else here lets a viewer join a stream in progress: it
|
||||
# cannot decode a picture whose parameter sets it never saw.
|
||||
if (
|
||||
self.parameter_set_period > 0
|
||||
and frame % self.parameter_set_period == 0
|
||||
and not unit.carries_parameter_sets
|
||||
):
|
||||
nals = [*self.stream.parameter_sets, *nals]
|
||||
datagrams = self.packetiser.packetise(nals, advance=ticks)
|
||||
for i, datagram in enumerate(datagrams):
|
||||
yield (period if i == 0 else 0.0), datagram
|
||||
index = (index + 1) % len(units)
|
||||
frame += 1
|
||||
@@ -64,6 +64,14 @@ class SendResult(BaseModel):
|
||||
class StateValue(BaseModel):
|
||||
value: object
|
||||
age: float = Field(description="Seconds since the drone last reported this. Old values may be stale.")
|
||||
sources: dict[int, object] | None = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Present only when parts of the aircraft disagree about this value, mapping the link "
|
||||
"buffer each report arrived on to what it said. `value` is then merely the most recent "
|
||||
"of them, not a decision about which is right."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
class Check(BaseModel):
|
||||
@@ -87,6 +95,29 @@ class FileEntry(BaseModel):
|
||||
is_dir: bool = False
|
||||
|
||||
|
||||
class LogEntry(BaseModel):
|
||||
uptime: float = Field(description="Seconds since the drone booted. The log has no wall-clock time.")
|
||||
level: str = Field(description="critical, error, warning, notice, info or debug.")
|
||||
tag: str = Field(description="Which subsystem logged it, e.g. KERNEL, COMMANDS, NETWORK.")
|
||||
source: str = Field(default="", description="process/thread that logged it, where the drone says.")
|
||||
message: str
|
||||
|
||||
|
||||
class LogRead(BaseModel):
|
||||
path: str = Field(description="Where the fetched log landed on this machine.")
|
||||
size: int = Field(description="Bytes fetched. The drone is still appending to it.")
|
||||
parsed: int = Field(description="Entries found in the whole file, before any filter.")
|
||||
matched: int = Field(description="Entries the filters kept. More than `returned` means capped.")
|
||||
returned: int
|
||||
newest_first: bool
|
||||
tags: dict[str, int] = Field(
|
||||
default_factory=dict,
|
||||
description="Entry count per tag across the whole file, so you can see what else is in there.",
|
||||
)
|
||||
note: str = Field(default="", description="Anything odd about the parse, such as a truncated tail.")
|
||||
entries: list[LogEntry] = Field(default_factory=list)
|
||||
|
||||
|
||||
class ArmState(BaseModel):
|
||||
armed: bool
|
||||
reason: str = ""
|
||||
|
||||
@@ -0,0 +1,204 @@
|
||||
"""MCP resources: addressable, read-only views of the protocol and the aircraft.
|
||||
|
||||
Resources are for things a client wants to *look at* rather than *do*, and the
|
||||
split that matters here is whether the drone has to be powered on.
|
||||
|
||||
`bebop://commands` is the valuable one precisely because it does not need the
|
||||
drone. The whole command set, 264 of them, is readable with the aircraft in a
|
||||
bag: an agent can learn what the protocol offers, pick what it needs and only
|
||||
then ask someone to switch a drone on. Everything under `bebop://state`,
|
||||
`bebop://files` and `bebop://log` needs a live link, and says so in its body
|
||||
instead of raising, because a resource that errors out looks broken while one
|
||||
that explains itself is just empty for a reason.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
from typing import Any
|
||||
|
||||
from fastmcp import FastMCP
|
||||
from fastmcp.exceptions import ToolError
|
||||
|
||||
from mcbebop.config import Settings
|
||||
from mcbebop.files import ftp
|
||||
from mcbebop.protocol import xml_index
|
||||
from mcbebop.tools import logs, protocol
|
||||
from mcbebop.tools._common import app, drone_host
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
JSON = "application/json"
|
||||
|
||||
#: Enough to see what a subsystem has been up to without filling a context.
|
||||
LOG_RESOURCE_LIMIT = 80
|
||||
|
||||
|
||||
def _unavailable(reason: str, **extra: Any) -> dict:
|
||||
"""A body that explains an empty resource instead of an exception."""
|
||||
# Logged as well as returned: a client that only shows the resource as
|
||||
# empty leaves no trace of why, and this is the one place that knows.
|
||||
logger.debug("resource unavailable: %s", reason)
|
||||
return {"available": False, "reason": reason, **extra}
|
||||
|
||||
|
||||
def _session():
|
||||
"""The live session, or None. Resources degrade; they do not raise."""
|
||||
state = app()
|
||||
return state.session if state.connected else None
|
||||
|
||||
|
||||
def register(mcp: FastMCP, settings: Settings) -> None:
|
||||
@mcp.resource(
|
||||
"bebop://commands",
|
||||
name="ARSDK command catalogue",
|
||||
mime_type=JSON,
|
||||
annotations={"readOnlyHint": True},
|
||||
description=(
|
||||
"Every command and event Parrot defines for this aircraft, with its tier, direction and "
|
||||
"whether the XML says the Bebop 2 supports it. Needs no drone, so it is the place to work "
|
||||
"out what is possible before anything is powered on. Read one in full at "
|
||||
"bebop://commands/{name}."
|
||||
),
|
||||
)
|
||||
def commands() -> dict:
|
||||
specs = sorted(xml_index.all_commands(), key=lambda s: s.full_name)
|
||||
return {
|
||||
"count": len(specs),
|
||||
"tiers": "observe and config are open; envelope and motion need arm()",
|
||||
"commands": [protocol.summary(s).model_dump() for s in specs],
|
||||
}
|
||||
|
||||
@mcp.resource(
|
||||
"bebop://commands/{name}",
|
||||
name="One ARSDK command",
|
||||
mime_type=JSON,
|
||||
annotations={"readOnlyHint": True},
|
||||
description=(
|
||||
"One command in full: arguments with their types and enum values, the documentation Parrot "
|
||||
"wrote, the wire ids, which link buffer carries it and which events confirm it. The name is "
|
||||
"the dotted form, e.g. ardrone3.Piloting.TakeOff."
|
||||
),
|
||||
)
|
||||
def command(name: str) -> dict:
|
||||
spec = xml_index.get(name)
|
||||
if spec is None:
|
||||
near = [c.full_name for c in xml_index.search(name)][:8]
|
||||
return _unavailable(
|
||||
f"No command named {name!r}. Browse bebop://commands for the full list.",
|
||||
did_you_mean=near,
|
||||
)
|
||||
return protocol.detail(spec).model_dump()
|
||||
|
||||
@mcp.resource(
|
||||
"bebop://state",
|
||||
name="Telemetry snapshot",
|
||||
mime_type=JSON,
|
||||
annotations={"readOnlyHint": True},
|
||||
description=(
|
||||
"Everything the drone has reported about itself, each value with how many seconds ago it "
|
||||
"said it. Age is the point: these are the last thing it said, not a fresh reading, so a "
|
||||
"large age means the link went quiet rather than that the value is current."
|
||||
),
|
||||
)
|
||||
def state() -> dict:
|
||||
session = _session()
|
||||
if session is None:
|
||||
return _unavailable("No drone session. Call connect() first.")
|
||||
raw = session.state()
|
||||
# `connected` is not evidence of a working link. The aircraft accepts a
|
||||
# second controller and silently redirects telemetry to it, leaving
|
||||
# this session reporting connected while nothing arrives, so the age
|
||||
# of the newest value is what tells a quiet link from a live one.
|
||||
stats = session.link_stats()
|
||||
return {
|
||||
"available": True,
|
||||
"keys": len(raw),
|
||||
"last_event_age": stats.get("last_event_age"),
|
||||
"link": "a last_event_age of more than a few seconds means telemetry has stopped arriving",
|
||||
"values": {k: {"value": v["value"], "age": v["age"]} for k, v in raw.items()},
|
||||
}
|
||||
|
||||
@mcp.resource(
|
||||
"bebop://state/{key}",
|
||||
name="One telemetry value",
|
||||
mime_type=JSON,
|
||||
annotations={"readOnlyHint": True},
|
||||
description=(
|
||||
"One reported value and its age. The key is '<Command>_<arg>', such as "
|
||||
"BatteryStateChanged_percent; a bare command name matches every argument under it."
|
||||
),
|
||||
)
|
||||
def state_key(key: str) -> dict:
|
||||
session = _session()
|
||||
if session is None:
|
||||
return _unavailable("No drone session. Call connect() first.", key=key)
|
||||
raw = session.state([key])
|
||||
if not raw:
|
||||
return _unavailable(
|
||||
f"The drone has not reported anything under {key!r}. "
|
||||
"bebop://state lists every key it has sent.",
|
||||
key=key,
|
||||
)
|
||||
return {"available": True, "key": key, "values": raw}
|
||||
|
||||
@mcp.resource(
|
||||
"bebop://files/{area}",
|
||||
name="Drone storage listing",
|
||||
mime_type=JSON,
|
||||
annotations={"readOnlyHint": True},
|
||||
description=(
|
||||
"A directory listing from one of the drone's FTP areas: 'media' for what it recorded, "
|
||||
"'flightplans' for stored missions, 'logs' for its blackbox and debug tree. The "
|
||||
"firmware-update channel is deliberately not among them."
|
||||
),
|
||||
)
|
||||
async def files(area: str) -> dict:
|
||||
try:
|
||||
resolved = ftp.resolve_area(area)
|
||||
host = drone_host(settings)
|
||||
except (ftp.FtpError, ToolError) as exc:
|
||||
return _unavailable(str(exc), area=area)
|
||||
try:
|
||||
entries = await _listing(resolved.name, host)
|
||||
except (ftp.FtpError, OSError) as exc:
|
||||
return _unavailable(str(exc), area=area)
|
||||
return {
|
||||
"available": True,
|
||||
"area": resolved.name,
|
||||
"describes": resolved.describe,
|
||||
"entries": entries,
|
||||
}
|
||||
|
||||
@mcp.resource(
|
||||
"bebop://log/{tag}",
|
||||
name="System log by tag",
|
||||
mime_type=JSON,
|
||||
annotations={"readOnlyHint": True},
|
||||
description=(
|
||||
"The most recent entries the drone's system log carries under one tag, newest first. "
|
||||
"KERNEL is Linux, COMMANDS is the drone's account of the ARSDK traffic it handled, NETWORK "
|
||||
"and NETMON are the link, colibry and SETTINGS are the flight config. Use read_log for "
|
||||
"filtering by severity or message text."
|
||||
),
|
||||
)
|
||||
async def log(tag: str) -> dict:
|
||||
try:
|
||||
dest, size, records, stats = await logs.fetch_and_parse(settings)
|
||||
except (ftp.FtpError, ToolError, OSError) as exc:
|
||||
return _unavailable(str(exc), tag=tag)
|
||||
kept, matched = logs.select(records, tag=tag, limit=LOG_RESOURCE_LIMIT)
|
||||
return {
|
||||
"available": True,
|
||||
"tag": tag,
|
||||
"path": str(dest),
|
||||
"size": size,
|
||||
"parsed": stats.entries,
|
||||
"matched": matched,
|
||||
"tags": dict(sorted(stats.tags.items(), key=lambda kv: -kv[1])),
|
||||
"entries": [logs.to_entry(r).model_dump() for r in reversed(kept)],
|
||||
}
|
||||
|
||||
|
||||
async def _listing(area: str, host: str) -> list[dict]:
|
||||
entries = await asyncio.to_thread(ftp.list_dir, area, "", host=host)
|
||||
return [{"name": e.name, "size": e.size, "is_dir": e.is_dir} for e in entries]
|
||||
+269
-13
@@ -19,6 +19,14 @@ would agree with it about a shared mistake.
|
||||
One deliberate fault is baked in: the magnetometer self-test reports failure,
|
||||
because "all six sensors fine" is the one answer that never exercises the code
|
||||
that reads them.
|
||||
|
||||
Video is optional and off unless a source is handed to `FakeBebop`. With one,
|
||||
the sim imitates the aircraft's ARStream2 behaviour: nothing flows until
|
||||
`ardrone3.MediaStreaming.VideoEnable` arrives with 1, RTP then goes from the
|
||||
sim's own port 5004 to the `arstream2_client_stream_port` the controller named
|
||||
in its handshake, and it stops on a 0, on a disconnect, or at shutdown. The
|
||||
`MediaStreamingState.VideoEnableChanged` reply is sent either way, because
|
||||
that is a protocol fact rather than a property of having video to send.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -36,6 +44,7 @@ from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from mcbebop.arsdk.types import COMMAND_HEADER, FRAME_HEADER, BufferId, DataType
|
||||
from mcbebop.media.rtp import DEFAULT_FPS, DEFAULT_MTU, AnnexBStream, PacketisedSource, VideoSource
|
||||
from mcbebop.protocol.types import ArgSpec, Buffer, CommandSpec, EnumSpec
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
@@ -53,6 +62,16 @@ _C2D_BUFFERS = frozenset(
|
||||
|
||||
_ALL_STATES = (0, 4, 0)
|
||||
_ALL_SETTINGS = (0, 2, 0)
|
||||
_VIDEO_ENABLE = (1, 21, 0)
|
||||
|
||||
# What the drone's handshake reply names, and where it sends from. The client
|
||||
# half (55004/55005) is the controller's to choose and arrives in its request.
|
||||
VIDEO_SERVER_STREAM_PORT = 5004
|
||||
VIDEO_SERVER_CONTROL_PORT = 5005
|
||||
VIDEO_CLIENT_STREAM_PORT = 55004
|
||||
|
||||
#: Suffixes read as a previously recorded raw RTP stream rather than H.264.
|
||||
_REPLAY_SUFFIXES = frozenset({".rtpcap"})
|
||||
|
||||
# Sensor self-test order as the aircraft reports it, with the one that fails.
|
||||
_SENSORS = ("IMU", "barometer", "ultrasound", "GPS", "magnetometer", "vertical_camera")
|
||||
@@ -173,6 +192,29 @@ def _arg(spec: CommandSpec, name: str) -> ArgSpec:
|
||||
raise KeyError(f"{spec.full_name} has no argument {name!r}")
|
||||
|
||||
|
||||
def video_source_for(
|
||||
path: str | Path,
|
||||
*,
|
||||
fps: float = DEFAULT_FPS,
|
||||
mtu: int = DEFAULT_MTU,
|
||||
start_offset: float | str | None = None,
|
||||
seed: int | None = None,
|
||||
) -> VideoSource:
|
||||
"""Turn a file into something the sim can stream, choosing by suffix.
|
||||
|
||||
A `.rtpcap` is a capture taken off the aircraft and is replayed with its
|
||||
own inter-packet timing. Anything else is read as an Annex-B H.264
|
||||
elementary stream and packetised here at a steady `fps`. See
|
||||
`media/capture.py` for why the difference matters.
|
||||
"""
|
||||
if Path(path).suffix.lower() in _REPLAY_SUFFIXES:
|
||||
from mcbebop.media.capture import ReplaySource
|
||||
|
||||
return ReplaySource.from_path(path, start_offset=start_offset, seed=seed)
|
||||
stream = AnnexBStream.from_path(path)
|
||||
return PacketisedSource(stream, fps=fps, mtu=mtu, start_offset=start_offset, seed=seed)
|
||||
|
||||
|
||||
@dataclass
|
||||
class FakeBebop:
|
||||
"""A drone-shaped thing on a socket.
|
||||
@@ -184,13 +226,34 @@ class FakeBebop:
|
||||
host: str = "127.0.0.1"
|
||||
discovery_port: int = 0 # 0 asks the OS, which is what parallel tests want
|
||||
c2d_port: int = 0
|
||||
single_controller: bool = True
|
||||
# False by default because that is what the aircraft does: it accepts a
|
||||
# second controller and redirects telemetry to it. Set True to exercise a
|
||||
# client's handling of a refusal, which some other firmware may still give.
|
||||
single_controller: bool = False
|
||||
stream_hz: float = 5.0
|
||||
battery_start: int = 87
|
||||
|
||||
# Video is off unless a source is given, so an existing FakeBebop() is
|
||||
# byte for byte the drone it was before this existed. A path is resolved
|
||||
# here rather than at VideoEnable time, so a typo fails at construction
|
||||
# instead of silently producing a drone that never streams.
|
||||
video_source: VideoSource | str | Path | None = None
|
||||
video_fps: float = DEFAULT_FPS
|
||||
video_mtu: int = DEFAULT_MTU
|
||||
#: Seconds into the stream to start, or "random" to begin mid-GOP, which
|
||||
#: is what a viewer switched on mid-flight is handed.
|
||||
video_start_offset: float | str | None = None
|
||||
video_seed: int | None = None
|
||||
#: The drone streams from its own 5004. If that port is taken, which it is
|
||||
#: whenever a second sim is already streaming, the OS picks one instead:
|
||||
#: an RTP receiver binds rather than connects, so it does not care.
|
||||
video_source_port: int = VIDEO_SERVER_STREAM_PORT
|
||||
|
||||
received: list[Received] = field(default_factory=list)
|
||||
handshakes: list[dict[str, Any]] = field(default_factory=list)
|
||||
pongs: int = 0
|
||||
video_packets_sent: int = 0
|
||||
video_bytes_sent: int = 0
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
self.specs = load_specs()
|
||||
@@ -200,6 +263,21 @@ class FakeBebop:
|
||||
self._lock = threading.Lock()
|
||||
self._battery = self.battery_start
|
||||
|
||||
self._video = (
|
||||
video_source_for(
|
||||
self.video_source,
|
||||
fps=self.video_fps,
|
||||
mtu=self.video_mtu,
|
||||
start_offset=self.video_start_offset,
|
||||
seed=self.video_seed,
|
||||
)
|
||||
if isinstance(self.video_source, str | Path)
|
||||
else self.video_source
|
||||
)
|
||||
self._video_wanted = threading.Event()
|
||||
self._video_target: tuple[str, int] | None = None
|
||||
self._video_udp: socket.socket | None = None
|
||||
|
||||
self._udp = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||
self._udp.bind((self.host, self.c2d_port))
|
||||
self.c2d_port = self._udp.getsockname()[1]
|
||||
@@ -209,13 +287,15 @@ class FakeBebop:
|
||||
self.discovery_port = self._tcp.getsockname()[1]
|
||||
self._tcp.settimeout(0.2)
|
||||
|
||||
self._threads = [
|
||||
threading.Thread(target=target, name=f"sim-{name}", daemon=True)
|
||||
for name, target in (
|
||||
targets = [
|
||||
("discovery", self._discovery_loop),
|
||||
("commands", self._command_loop),
|
||||
("stream", self._stream_loop),
|
||||
)
|
||||
]
|
||||
if self._video is not None:
|
||||
targets.append(("video", self._video_loop))
|
||||
self._threads = [
|
||||
threading.Thread(target=target, name=f"sim-{name}", daemon=True) for name, target in targets
|
||||
]
|
||||
|
||||
# -- lifecycle -------------------------------------------------------
|
||||
@@ -234,18 +314,34 @@ class FakeBebop:
|
||||
|
||||
def stop(self) -> None:
|
||||
self._stop.set()
|
||||
self._video_wanted.clear()
|
||||
for thread in self._threads:
|
||||
# The sockets bind in __post_init__ but the threads only start in
|
||||
# start(), so a sim that was built and never run still has ports
|
||||
# to release. Joining an unstarted thread raises, which would
|
||||
# leave those ports held for the rest of the process.
|
||||
if thread.ident is not None:
|
||||
thread.join(timeout=2)
|
||||
self._udp.close()
|
||||
self._tcp.close()
|
||||
video, self._video_udp = self._video_udp, None
|
||||
if video is not None:
|
||||
video.close()
|
||||
|
||||
@property
|
||||
def occupied(self) -> bool:
|
||||
return self._d2c is not None
|
||||
|
||||
@property
|
||||
def video_streaming(self) -> bool:
|
||||
return self._video_wanted.is_set()
|
||||
|
||||
def release(self) -> None:
|
||||
"""Forget the current controller, as a real drone does on link loss."""
|
||||
self._d2c = None
|
||||
# The aircraft's stream dies with the controlling link, which is the
|
||||
# behaviour a viewer has to survive.
|
||||
self._video_wanted.clear()
|
||||
|
||||
def wait_for_controller(self, timeout: float = 5.0) -> bool:
|
||||
deadline = time.monotonic() + timeout
|
||||
@@ -295,19 +391,30 @@ class FakeBebop:
|
||||
continue
|
||||
self.handshakes.append(request)
|
||||
if self.single_controller and self.occupied:
|
||||
# What the aircraft does to a second controller, and the
|
||||
# only way to test that path without two phones.
|
||||
# NOT what a real Bebop 2 does. Tested on the aircraft
|
||||
# 2026-10-02: it accepts a second controller and silently
|
||||
# redirects telemetry to it, leaving the first starved but
|
||||
# still believing it is connected. This refusal path is kept
|
||||
# because a client must handle a non-zero status anyway, but
|
||||
# the default models the takeover. See docs network notes.
|
||||
conn.sendall(json.dumps({"status": 1}).encode() + b"\x00")
|
||||
continue
|
||||
self._d2c = (addr[0], int(request["d2c_port"]))
|
||||
# The controller names where video should go; the drone does
|
||||
# not choose it. A controller that names nothing gets the
|
||||
# usual port, which is what libARController would have sent.
|
||||
self._video_target = (
|
||||
addr[0],
|
||||
int(request.get("arstream2_client_stream_port", VIDEO_CLIENT_STREAM_PORT)),
|
||||
)
|
||||
reply = {
|
||||
"status": 0,
|
||||
"c2d_port": self.c2d_port,
|
||||
"arstream_fragment_size": 65000,
|
||||
"arstream_fragment_maximum_number": 128,
|
||||
"arstream_max_ack_interval": -1,
|
||||
"arstream2_server_stream_port": 5004,
|
||||
"arstream2_server_control_port": 5005,
|
||||
"arstream2_server_stream_port": VIDEO_SERVER_STREAM_PORT,
|
||||
"arstream2_server_control_port": VIDEO_SERVER_CONTROL_PORT,
|
||||
}
|
||||
conn.sendall(json.dumps(reply).encode() + b"\x00")
|
||||
# The burst goes out per controller attach, not once per
|
||||
@@ -337,7 +444,10 @@ class FakeBebop:
|
||||
if buffer_id not in _C2D_BUFFERS or len(body) < COMMAND_HEADER.size:
|
||||
return
|
||||
ids = COMMAND_HEADER.unpack_from(body)
|
||||
self.received.append(Received(ids, body[COMMAND_HEADER.size :], buffer_id, int(data_type), int(seq)))
|
||||
args = body[COMMAND_HEADER.size :]
|
||||
self.received.append(Received(ids, args, buffer_id, int(data_type), int(seq)))
|
||||
if ids == _VIDEO_ENABLE:
|
||||
self._set_video(bool(args[0]) if args else False)
|
||||
if ids in (_ALL_STATES, _ALL_SETTINGS):
|
||||
# The real drone answers these with a burst of its current state,
|
||||
# which is what makes request_full_state worth calling.
|
||||
@@ -349,7 +459,12 @@ class FakeBebop:
|
||||
self.emit(
|
||||
"common.SettingsState.ProductVersionChanged", acked=True, software="4.7.1", hardware="HW_05"
|
||||
)
|
||||
self.emit("common.SettingsState.ProductSerialHighChanged", acked=True, high="PI04")
|
||||
# Deliberately nonsense, and please keep it that way. A real Bebop 2
|
||||
# serial is the aircraft's identity: it appears in its SSID and in the
|
||||
# ckcm system log beside the CPU id and the last GPS fix. A simulator
|
||||
# that volunteered a realistic-looking one would get quoted into a bug
|
||||
# report as if it were a specimen, so this one reads "not a real sim".
|
||||
self.emit("common.SettingsState.ProductSerialHighChanged", acked=True, high="N0TAREAL")
|
||||
self.emit("common.SettingsState.ProductSerialLowChanged", acked=True, low="0000000SIM")
|
||||
self.emit("ardrone3.SettingsState.MotorFlightsStatusChanged", acked=True,
|
||||
nbFlights=42, lastFlightDuration=611, totalFlightDuration=26_340) # fmt: skip
|
||||
@@ -385,6 +500,81 @@ class FakeBebop:
|
||||
self.emit("common.CommonState.SensorsStatesListChanged", acked=True,
|
||||
sensorName=sensor, sensorState=0 if sensor == _FAULTY_SENSOR else 1) # fmt: skip
|
||||
|
||||
# -- video -----------------------------------------------------------
|
||||
def _set_video(self, wanted: bool) -> None:
|
||||
"""Answer VideoEnable, and start or stop the stream.
|
||||
|
||||
The event goes out whether or not there is anything to stream: a
|
||||
client checking that its VideoEnable took effect is reading the
|
||||
aircraft's state, not its bitrate.
|
||||
"""
|
||||
self.emit(
|
||||
"ardrone3.MediaStreamingState.VideoEnableChanged",
|
||||
acked=True,
|
||||
enabled="enabled" if wanted else "disabled",
|
||||
)
|
||||
if self._video is None:
|
||||
if wanted:
|
||||
log.debug("VideoEnable(1) with no video source; nothing to stream")
|
||||
return
|
||||
if wanted:
|
||||
self._video_wanted.set()
|
||||
else:
|
||||
self._video_wanted.clear()
|
||||
|
||||
def _video_socket(self) -> socket.socket:
|
||||
if self._video_udp is not None:
|
||||
return self._video_udp
|
||||
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
try:
|
||||
sock.bind((self.host, self.video_source_port))
|
||||
except OSError as exc:
|
||||
log.warning(
|
||||
"cannot send video from port %d (%s); using an ephemeral one instead",
|
||||
self.video_source_port,
|
||||
exc,
|
||||
)
|
||||
sock.bind((self.host, 0))
|
||||
self._video_udp = sock
|
||||
return sock
|
||||
|
||||
def _video_loop(self) -> None:
|
||||
"""Push RTP for as long as the controller wants it.
|
||||
|
||||
One source object for the life of the sim, so its sequence numbers
|
||||
and timestamps keep advancing across both the loop point in the file
|
||||
and a disable/enable cycle. A decoder handed a timestamp that went
|
||||
backwards treats the stream as corrupt and stays that way.
|
||||
"""
|
||||
assert self._video is not None
|
||||
while not self._stop.is_set():
|
||||
if not self._video_wanted.wait(0.1):
|
||||
continue
|
||||
target = self._video_target
|
||||
if target is None:
|
||||
self._video_wanted.clear()
|
||||
continue
|
||||
sock = self._video_socket()
|
||||
log.info("streaming video to %s:%d: %s", *target, self._video.describe)
|
||||
deadline = time.monotonic()
|
||||
for delay, datagram in self._video.packets():
|
||||
if self._stop.is_set() or not self._video_wanted.is_set():
|
||||
break
|
||||
if delay > 0:
|
||||
deadline = max(deadline + delay, time.monotonic())
|
||||
now = time.monotonic()
|
||||
if deadline > now:
|
||||
time.sleep(deadline - now)
|
||||
try:
|
||||
sock.sendto(datagram, target)
|
||||
except OSError as exc: # the socket can close under us at teardown
|
||||
log.debug("video send failed: %s", exc)
|
||||
break
|
||||
self.video_packets_sent += 1
|
||||
self.video_bytes_sent += len(datagram)
|
||||
log.info("video stopped after %d packets", self.video_packets_sent)
|
||||
|
||||
def _stream_loop(self) -> None:
|
||||
while self._d2c is None and not self._stop.is_set():
|
||||
time.sleep(0.02)
|
||||
@@ -425,13 +615,29 @@ class FakeBebop:
|
||||
self.emit("common.CommonState.BatteryStateChanged", acked=True, percent=self._battery)
|
||||
|
||||
|
||||
def serve(seconds: float = 0.0, *, host: str = "127.0.0.1", discovery_port: int = 44444) -> None:
|
||||
def serve(
|
||||
seconds: float = 0.0,
|
||||
*,
|
||||
host: str = "127.0.0.1",
|
||||
discovery_port: int = 44444,
|
||||
video_source: VideoSource | str | Path | None = None,
|
||||
video_fps: float = DEFAULT_FPS,
|
||||
video_start_offset: float | str | None = None,
|
||||
video_seed: int | None = None,
|
||||
) -> None:
|
||||
"""Run a sim until interrupted. Logs; nothing goes to stdout.
|
||||
|
||||
stdout is the MCP server's JSON-RPC transport, and this module is
|
||||
importable from it.
|
||||
"""
|
||||
with FakeBebop(host=host, discovery_port=discovery_port) as sim:
|
||||
with FakeBebop(
|
||||
host=host,
|
||||
discovery_port=discovery_port,
|
||||
video_source=video_source,
|
||||
video_fps=video_fps,
|
||||
video_start_offset=video_start_offset,
|
||||
video_seed=video_seed,
|
||||
) as sim:
|
||||
log.info("fake Bebop 2 on %s:%d", sim.host, sim.discovery_port)
|
||||
end = time.monotonic() + seconds if seconds else None
|
||||
try:
|
||||
@@ -439,3 +645,53 @@ def serve(seconds: float = 0.0, *, host: str = "127.0.0.1", discovery_port: int
|
||||
time.sleep(0.25)
|
||||
except KeyboardInterrupt:
|
||||
pass
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
"""Run the simulator from a shell, which is how a viewer gets developed.
|
||||
|
||||
python -m mcbebop.sim --video clip.h264
|
||||
|
||||
The client then handshakes on 44444 as it would with the aircraft, names
|
||||
its own stream port, and sends VideoEnable to start the RTP.
|
||||
"""
|
||||
import argparse
|
||||
import sys
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="python -m mcbebop.sim", description="A fake Bebop 2 on localhost, optionally with video."
|
||||
)
|
||||
parser.add_argument("--host", default="127.0.0.1")
|
||||
parser.add_argument("--discovery-port", type=int, default=44444)
|
||||
parser.add_argument("--seconds", type=float, default=0.0, help="0 runs until interrupted")
|
||||
parser.add_argument("--video", default=None, help="an Annex-B .h264 file, or a .rtpcap capture")
|
||||
parser.add_argument("--fps", type=float, default=DEFAULT_FPS, help="ignored for a .rtpcap replay")
|
||||
parser.add_argument(
|
||||
"--start-offset",
|
||||
default=None,
|
||||
help="seconds into the stream, or 'random' to begin mid-GOP",
|
||||
)
|
||||
parser.add_argument("--seed", type=int, default=None, help="makes --start-offset=random repeatable")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
offset: float | str | None = args.start_offset
|
||||
if isinstance(offset, str) and offset != "random":
|
||||
offset = float(offset)
|
||||
|
||||
# stderr: stdout is the MCP server's transport and this module is
|
||||
# importable from it.
|
||||
logging.basicConfig(level=logging.INFO, format="%(message)s", stream=sys.stderr)
|
||||
serve(
|
||||
args.seconds,
|
||||
host=args.host,
|
||||
discovery_port=args.discovery_port,
|
||||
video_source=args.video,
|
||||
video_fps=args.fps,
|
||||
video_start_offset=offset,
|
||||
video_seed=args.seed,
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__": # pragma: no cover - a hand-run tool
|
||||
raise SystemExit(main())
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Tool registration.
|
||||
"""Tool and resource registration.
|
||||
|
||||
The one place that knows the full set, so naming stays consistent across the
|
||||
modules that would otherwise each invent their own.
|
||||
@@ -8,13 +8,20 @@ from fastmcp import FastMCP
|
||||
|
||||
from mcbebop.config import Settings
|
||||
from mcbebop.state import AppState
|
||||
from mcbebop.tools import camera, command, connection, files, protocol, safety, state
|
||||
from mcbebop.tools import camera, command, connection, files, logs, protocol, safety, state
|
||||
from mcbebop.tools._common import set_state
|
||||
|
||||
|
||||
def register_all(mcp: FastMCP, settings: Settings) -> AppState:
|
||||
app_state = AppState(settings=settings)
|
||||
set_state(app_state)
|
||||
for module in (connection, protocol, command, state, camera, files, safety):
|
||||
for module in (connection, protocol, command, state, camera, files, logs, safety):
|
||||
module.register(mcp, settings)
|
||||
|
||||
# Imported here rather than at module scope: resources reach back into
|
||||
# `mcbebop.tools` for the shared command and log helpers, and importing it
|
||||
# at the top would close that loop while this package is still loading.
|
||||
from mcbebop import resources
|
||||
|
||||
resources.register(mcp, settings)
|
||||
return app_state
|
||||
|
||||
@@ -2,7 +2,10 @@
|
||||
|
||||
from datetime import UTC, datetime, timedelta
|
||||
|
||||
from fastmcp.exceptions import ToolError
|
||||
|
||||
from mcbebop import errors
|
||||
from mcbebop.config import Settings
|
||||
from mcbebop.state import AppState
|
||||
|
||||
_STATE: AppState | None = None
|
||||
@@ -28,6 +31,21 @@ def require_session():
|
||||
return state.session
|
||||
|
||||
|
||||
def drone_host(settings: Settings) -> str:
|
||||
"""Where to reach the aircraft's own servers.
|
||||
|
||||
FTP and the debug shell talk to the real machine, so the simulator has
|
||||
nothing to answer with and saying that plainly beats a connection refused.
|
||||
"""
|
||||
state = app()
|
||||
if state.target == "sim":
|
||||
raise ToolError(
|
||||
"Files, logs and the debug shell all talk to the real aircraft; the simulator has no "
|
||||
"filesystem. Connect to the drone to use this."
|
||||
)
|
||||
return state.target or settings.drone_ip
|
||||
|
||||
|
||||
def expiry_iso(seconds_left: float) -> str | None:
|
||||
if seconds_left <= 0:
|
||||
return None
|
||||
|
||||
@@ -45,8 +45,14 @@ def register(mcp: FastMCP, settings: Settings) -> None:
|
||||
) -> ConnectionInfo:
|
||||
"""Open a session with the drone. Nothing else works until this succeeds.
|
||||
|
||||
The drone serves one controller at a time, so close any phone app first.
|
||||
This machine must already have joined the drone's own Wi-Fi network.
|
||||
|
||||
Connecting while something else is already controlling the drone takes
|
||||
the link over rather than failing: the aircraft accepts the newcomer and
|
||||
stops sending telemetry to whoever had it, without telling them. So if a
|
||||
phone app is flying it, connect here and the phone goes quiet. Watch
|
||||
last_update_age rather than the connected flag to tell a live link from
|
||||
one that has been taken.
|
||||
"""
|
||||
state = app()
|
||||
async with state.lock:
|
||||
@@ -59,7 +65,7 @@ def register(mcp: FastMCP, settings: Settings) -> None:
|
||||
if target == SIM_TARGET:
|
||||
from mcbebop.sim import FakeBebop
|
||||
|
||||
sim = FakeBebop()
|
||||
sim = FakeBebop(video_source=settings.sim_video_source)
|
||||
sim.__enter__()
|
||||
state.sim = sim
|
||||
session = DroneSession(ip=sim.host, discovery_port=sim.discovery_port)
|
||||
|
||||
@@ -4,28 +4,16 @@ import asyncio
|
||||
from typing import Annotated, Literal
|
||||
|
||||
from fastmcp import Context, FastMCP
|
||||
from fastmcp.exceptions import ToolError
|
||||
from pydantic import Field
|
||||
|
||||
from mcbebop.config import Settings
|
||||
from mcbebop.files import ftp, shell
|
||||
from mcbebop.models import FileEntry
|
||||
from mcbebop.tools._common import app
|
||||
from mcbebop.tools._common import drone_host
|
||||
|
||||
AreaName = Literal["media", "flightplans", "logs"]
|
||||
|
||||
|
||||
def _host(settings: Settings) -> str:
|
||||
"""FTP talks to the aircraft directly, so the simulator has nothing to serve."""
|
||||
state = app()
|
||||
if state.target == "sim":
|
||||
raise ToolError(
|
||||
"File access talks to the real aircraft's FTP server; the simulator has no filesystem. "
|
||||
"Connect to the drone to use this."
|
||||
)
|
||||
return state.target or settings.drone_ip
|
||||
|
||||
|
||||
def register(mcp: FastMCP, settings: Settings) -> None:
|
||||
@mcp.tool(annotations={"readOnlyHint": True, "openWorldHint": True})
|
||||
async def list_files(
|
||||
@@ -46,7 +34,7 @@ def register(mcp: FastMCP, settings: Settings) -> None:
|
||||
Read-only. The firmware-update channel is deliberately not reachable
|
||||
through this tool.
|
||||
"""
|
||||
host = _host(settings)
|
||||
host = drone_host(settings)
|
||||
entries = await asyncio.to_thread(ftp.list_dir, area, path, host=host)
|
||||
return [FileEntry(name=e.name, size=e.size, is_dir=e.is_dir) for e in entries]
|
||||
|
||||
@@ -58,7 +46,7 @@ def register(mcp: FastMCP, settings: Settings) -> None:
|
||||
max_mb: Annotated[int, Field(description="Refuse anything larger.", ge=1, le=512)] = 32,
|
||||
) -> dict:
|
||||
"""Download one file from the drone to this machine, and return where it landed."""
|
||||
host = _host(settings)
|
||||
host = drone_host(settings)
|
||||
dest = await asyncio.to_thread(
|
||||
ftp.fetch,
|
||||
area,
|
||||
@@ -86,6 +74,6 @@ def register(mcp: FastMCP, settings: Settings) -> None:
|
||||
takes an allow-list of command names rather than trying to filter a
|
||||
free-form line.
|
||||
"""
|
||||
host = _host(settings)
|
||||
host = drone_host(settings)
|
||||
result = await asyncio.to_thread(shell.run, command, list(args), host=host)
|
||||
return {"command": result.command, "stdout": result.stdout}
|
||||
|
||||
@@ -0,0 +1,238 @@
|
||||
"""Reading the drone's own system log.
|
||||
|
||||
The aircraft logs far more about itself than it reports over ARSDK. The log is
|
||||
where it says which board it is, which config it read, why a command was
|
||||
refused and what its Wi-Fi driver did. None of that reaches telemetry.
|
||||
|
||||
It is also big: half an hour of uptime is over five thousand entries, which is
|
||||
why everything here filters before it returns and caps what it returns even
|
||||
then.
|
||||
|
||||
Only the live `ckcm/ckcm.bin` is reachable from here, and that is deliberate.
|
||||
`system.conf` caps it at a megabyte, after which the drone rolls it into
|
||||
`Debug/archive/debug_NN.tar.lzo`. Those archives are previous *sessions*, and
|
||||
on a second-hand aircraft that means a previous owner's flights: network names
|
||||
and positions that nobody consented to hand to a language model. Reading them
|
||||
would be a decision for a person with the drone on a bench, not something a
|
||||
tool does on the way to answering a question.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import re
|
||||
from pathlib import Path
|
||||
from typing import Annotated, Literal
|
||||
|
||||
from fastmcp import Context, FastMCP
|
||||
from fastmcp.exceptions import ToolError
|
||||
from pydantic import Field
|
||||
|
||||
from mcbebop.config import Settings
|
||||
from mcbebop.files import ftp, ulog
|
||||
from mcbebop.models import LogEntry, LogRead
|
||||
from mcbebop.tools._common import drone_host
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: The log lives in the blackbox tree, which `ftp.AREAS["logs"]` is scoped to.
|
||||
LOG_AREA = "logs"
|
||||
LOG_PATH = "ckcm/ckcm.bin"
|
||||
|
||||
#: The drone rotates the live log at a megabyte, so this has headroom and
|
||||
#: still refuses anything the size of a video.
|
||||
MAX_BYTES = 16 * 1024 * 1024
|
||||
|
||||
#: A five-second USB-role poll that says the same thing every time and can be
|
||||
#: a third of the log. Excluded by default, named in the schema so the caller
|
||||
#: can see it is being excluded and ask for it back.
|
||||
NOISY_TAGS = ("shp_usbmode",)
|
||||
|
||||
Level = Literal["any", "critical", "error", "warning", "info", "debug"]
|
||||
_LEVEL_LETTERS = {name: letter for letter, name in ulog.PRIORITY_NAMES.items()}
|
||||
|
||||
|
||||
async def fetch_and_parse(
|
||||
settings: Settings, path: str = LOG_PATH
|
||||
) -> tuple[Path, int, list, ulog.ParseStats]:
|
||||
"""Pull the log off the aircraft and parse it. Shared with the resources."""
|
||||
host = drone_host(settings)
|
||||
dest = await asyncio.to_thread(
|
||||
ftp.fetch,
|
||||
LOG_AREA,
|
||||
path,
|
||||
host=host,
|
||||
capture_dir=settings.capture_dir,
|
||||
max_bytes=MAX_BYTES,
|
||||
)
|
||||
data = await asyncio.to_thread(dest.read_bytes)
|
||||
stats = ulog.ParseStats()
|
||||
records = list(ulog.parse(data, stats))
|
||||
return dest, len(data), records, stats
|
||||
|
||||
|
||||
def select(
|
||||
records: list,
|
||||
*,
|
||||
tag: str = "",
|
||||
exclude_tags: tuple[str, ...] | list[str] = (),
|
||||
match: str = "",
|
||||
min_level: str = "any",
|
||||
limit: int = 100,
|
||||
) -> tuple[list, int]:
|
||||
"""Filter, then keep the most recent `limit`. Returns `(kept, matched)`.
|
||||
|
||||
The limit always takes from the newest end whatever the display order,
|
||||
because on a log the recent entries are the ones you came for: asking for
|
||||
fifty lines about a failure and getting fifty lines of kernel boot would
|
||||
be an unhelpful reading of "the first fifty".
|
||||
|
||||
An explicit `tag` beats `exclude_tags`. Asking for a tag by name and
|
||||
getting nothing because a default excluded it would be indefensible.
|
||||
"""
|
||||
pattern = None
|
||||
if match:
|
||||
try:
|
||||
pattern = re.compile(match, re.IGNORECASE)
|
||||
except re.error as exc:
|
||||
raise ToolError(
|
||||
f"`match` is read as a regular expression and {match!r} is not one ({exc}). "
|
||||
"Plain text works as-is; escape any of . * + ? [ ] ( ) | \\ you meant literally."
|
||||
) from exc
|
||||
|
||||
want_rank = ulog.rank(_LEVEL_LETTERS[min_level]) if min_level != "any" else ulog.UNKNOWN_RANK
|
||||
tag_lower = tag.lower()
|
||||
excluded = set() if tag else {t.lower() for t in exclude_tags}
|
||||
|
||||
kept = []
|
||||
for record in records:
|
||||
lower = record.tag.lower()
|
||||
if tag and lower != tag_lower:
|
||||
continue
|
||||
if lower in excluded:
|
||||
continue
|
||||
if min_level != "any" and ulog.rank(record.priority) > want_rank:
|
||||
continue
|
||||
if pattern is not None and not pattern.search(record.message):
|
||||
continue
|
||||
kept.append(record)
|
||||
return kept[-limit:] if limit else kept, len(kept)
|
||||
|
||||
|
||||
def to_entry(record) -> LogEntry:
|
||||
return LogEntry(
|
||||
uptime=round(record.uptime, 3),
|
||||
level=record.level,
|
||||
tag=record.tag,
|
||||
source=record.source,
|
||||
message=record.message,
|
||||
)
|
||||
|
||||
|
||||
def register(mcp: FastMCP, settings: Settings) -> None:
|
||||
@mcp.tool(annotations={"readOnlyHint": True, "openWorldHint": True})
|
||||
async def read_log(
|
||||
ctx: Context,
|
||||
tag: Annotated[
|
||||
str,
|
||||
Field(
|
||||
description=(
|
||||
"Keep only this tag, exact and case-insensitive. KERNEL is the Linux log, "
|
||||
"COMMANDS is the drone's own account of the ARSDK traffic it handled, NETWORK "
|
||||
"and NETMON are the link, SETTINGS and colibry are the flight config. "
|
||||
"Empty keeps every tag; the result lists them all with counts either way."
|
||||
)
|
||||
),
|
||||
] = "",
|
||||
exclude_tags: Annotated[
|
||||
list[str],
|
||||
Field(
|
||||
description=(
|
||||
f"Tags to drop, case-insensitive. Defaults to {list(NOISY_TAGS)}, a five-second "
|
||||
"USB-role poll that repeats itself and can be a third of the log. Pass an empty "
|
||||
"list for everything. Ignored when `tag` names a tag explicitly."
|
||||
)
|
||||
),
|
||||
] = list(NOISY_TAGS), # noqa: B006 - read into the schema by FastMCP; never mutated
|
||||
match: Annotated[
|
||||
str,
|
||||
Field(
|
||||
description=(
|
||||
"Regular expression the message must contain, case-insensitive. "
|
||||
"Plain text works as a substring search."
|
||||
)
|
||||
),
|
||||
] = "",
|
||||
min_level: Annotated[
|
||||
Level,
|
||||
Field(description="Keep this severity and worse. 'error' is the quickest way to find trouble."),
|
||||
] = "any",
|
||||
limit: Annotated[
|
||||
int, Field(description="How many entries to return, counting back from the newest.", ge=1, le=500)
|
||||
] = 100,
|
||||
newest_first: Annotated[
|
||||
bool,
|
||||
Field(
|
||||
description="Order of the returned entries. The limit takes from the newest end either way."
|
||||
),
|
||||
] = True,
|
||||
path: Annotated[
|
||||
str, Field(description="A different file in the log area. The default is the system log.")
|
||||
] = LOG_PATH,
|
||||
) -> LogRead:
|
||||
"""Read the drone's system log: kernel, Wi-Fi, camera, flight config, command handling.
|
||||
|
||||
Fetches the whole log over FTP and parses Parrot's ulogcat framing, so
|
||||
it needs the aircraft reachable but no debug shell and no button press.
|
||||
Reach for it when telemetry is not enough to explain something: a
|
||||
refused command, a link that dropped, a sensor that failed its
|
||||
self-test, or which hardware variant this airframe actually is.
|
||||
|
||||
There is no wall-clock time in the log. Entries are stamped with
|
||||
seconds since the drone booted, and the drone does not know the date
|
||||
unless a phone has told it.
|
||||
|
||||
The log is live and the drone appends to it while this runs, so the
|
||||
newest entry is whatever had been written when the fetch landed. It is
|
||||
also only the current file: the drone rotates it at a megabyte and the
|
||||
archives it rolls into are not read from here.
|
||||
|
||||
Under `COMMANDS` the drone names things the way its own firmware does,
|
||||
which is close to Parrot's XML names without matching them, so look a
|
||||
name up with list_commands() rather than assuming the log's spelling.
|
||||
"""
|
||||
dest, size, records, stats = await fetch_and_parse(settings, path)
|
||||
kept, matched = select(
|
||||
records,
|
||||
tag=tag,
|
||||
exclude_tags=exclude_tags,
|
||||
match=match,
|
||||
min_level=min_level,
|
||||
limit=limit,
|
||||
)
|
||||
entries = [to_entry(r) for r in (reversed(kept) if newest_first else kept)]
|
||||
|
||||
notes = []
|
||||
if stats.truncated:
|
||||
notes.append("the fetch landed mid-record, which is normal while the drone is writing")
|
||||
if stats.resyncs:
|
||||
notes.append(f"{stats.resyncs} frames did not decode and were skipped")
|
||||
if not tag:
|
||||
present = {k.lower() for k in stats.tags}
|
||||
dropped = [t for t in exclude_tags if t.lower() in present]
|
||||
if dropped:
|
||||
notes.append(f"excluded: {', '.join(dropped)} (pass exclude_tags=[] to see them)")
|
||||
if tag and not matched:
|
||||
known = ", ".join(sorted(stats.tags)[:12])
|
||||
notes.append(f"no entry carries the tag {tag!r}; present tags include {known}")
|
||||
|
||||
return LogRead(
|
||||
path=str(dest),
|
||||
size=size,
|
||||
parsed=stats.entries,
|
||||
matched=matched,
|
||||
returned=len(entries),
|
||||
newest_first=newest_first,
|
||||
tags=dict(sorted(stats.tags.items(), key=lambda kv: -kv[1])),
|
||||
note="; ".join(notes),
|
||||
entries=entries,
|
||||
)
|
||||
@@ -11,7 +11,7 @@ from mcbebop.models import ArgumentInfo, CommandDetail, CommandSummary
|
||||
from mcbebop.protocol import xml_index
|
||||
|
||||
|
||||
def _summary(spec) -> CommandSummary:
|
||||
def summary(spec) -> CommandSummary:
|
||||
return CommandSummary(
|
||||
name=spec.full_name,
|
||||
title=spec.title,
|
||||
@@ -22,6 +22,29 @@ def _summary(spec) -> CommandSummary:
|
||||
)
|
||||
|
||||
|
||||
def detail(spec) -> CommandDetail:
|
||||
"""Everything the XML says about one command, expectations resolved to names."""
|
||||
confirmed = []
|
||||
for exp in spec.expectations:
|
||||
other = xml_index.by_ids(exp.ids)
|
||||
label = other.full_name if other else "-".join(str(i) for i in exp.ids)
|
||||
if exp.fields:
|
||||
label += " where " + ", ".join(f"{k}={v}" for k, v in exp.fields.items())
|
||||
confirmed.append(label)
|
||||
|
||||
return CommandDetail(
|
||||
**summary(spec).model_dump(),
|
||||
ids=list(spec.ids),
|
||||
doc=spec.doc,
|
||||
buffer=str(spec.buffer),
|
||||
confirmed_by=confirmed,
|
||||
args=[
|
||||
ArgumentInfo(name=a.name, type=a.type, doc=a.doc, enum_values=[m.name for m in a.members])
|
||||
for a in spec.args
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
def register(mcp: FastMCP, settings: Settings) -> None:
|
||||
@mcp.tool(annotations={"readOnlyHint": True, "openWorldHint": False})
|
||||
async def list_commands(
|
||||
@@ -53,7 +76,7 @@ def register(mcp: FastMCP, settings: Settings) -> None:
|
||||
continue
|
||||
if bebop2_only and xml_index.supports_bebop2(spec.support) is False:
|
||||
continue
|
||||
out.append(_summary(spec))
|
||||
out.append(summary(spec))
|
||||
return sorted(out, key=lambda c: c.name)
|
||||
|
||||
@mcp.tool(annotations={"readOnlyHint": True, "openWorldHint": False})
|
||||
@@ -66,28 +89,4 @@ def register(mcp: FastMCP, settings: Settings) -> None:
|
||||
if spec is None:
|
||||
near = [c.full_name for c in xml_index.search(name)][:5]
|
||||
raise errors.unknown_command(name, near)
|
||||
|
||||
confirmed = []
|
||||
for exp in spec.expectations:
|
||||
other = xml_index.by_ids(exp.ids)
|
||||
label = other.full_name if other else "-".join(str(i) for i in exp.ids)
|
||||
if exp.fields:
|
||||
label += " where " + ", ".join(f"{k}={v}" for k, v in exp.fields.items())
|
||||
confirmed.append(label)
|
||||
|
||||
return CommandDetail(
|
||||
**_summary(spec).model_dump(),
|
||||
ids=list(spec.ids),
|
||||
doc=spec.doc,
|
||||
buffer=str(spec.buffer),
|
||||
confirmed_by=confirmed,
|
||||
args=[
|
||||
ArgumentInfo(
|
||||
name=a.name,
|
||||
type=a.type,
|
||||
doc=a.doc,
|
||||
enum_values=[m.name for m in a.members],
|
||||
)
|
||||
for a in spec.args
|
||||
],
|
||||
)
|
||||
return detail(spec)
|
||||
|
||||
@@ -28,6 +28,40 @@ def _sensor_results(values: dict) -> dict[str, bool]:
|
||||
return out
|
||||
|
||||
|
||||
def _battery(session) -> tuple[int | None, str]:
|
||||
"""The battery percentage, and how much to trust it.
|
||||
|
||||
This aircraft sends `BatteryStateChanged` from two places with different
|
||||
values: the real figure, which its own log corroborates and which tracks a
|
||||
smooth discharge, and a constant 0 that appears in no log line. Taking the
|
||||
newest frame therefore reports a healthy drone as flat.
|
||||
|
||||
The tie is broken on physical grounds rather than preference: an aircraft
|
||||
that is powered, holding a link and streaming telemetry is not at 0%, so a
|
||||
zero from a live drone is not a credible reading while a non-zero one is.
|
||||
The disagreement is reported either way, because a caller should know the
|
||||
number was contested.
|
||||
"""
|
||||
raw = session.state(["BatteryStateChanged_percent"]).get("BatteryStateChanged_percent")
|
||||
if raw is None:
|
||||
return None, "not reported"
|
||||
|
||||
sources = raw.get("sources") or {}
|
||||
distinct = {v for v in sources.values()}
|
||||
if len(distinct) < 2:
|
||||
return raw["value"], f"{raw['value']}%"
|
||||
|
||||
credible = sorted(v for v in distinct if isinstance(v, int) and v > 0)
|
||||
if not credible:
|
||||
return None, f"sources disagree but none is credible: {sources}"
|
||||
best = credible[-1]
|
||||
others = sorted(distinct - {best})
|
||||
return best, (
|
||||
f"{best}%, contested: the aircraft also reports {others} on another channel. "
|
||||
"A drone that is powered and holding a link is not at 0%, so that reading is disregarded."
|
||||
)
|
||||
|
||||
|
||||
def register(mcp: FastMCP, settings: Settings) -> None:
|
||||
@mcp.tool(annotations={"readOnlyHint": True, "openWorldHint": False})
|
||||
async def get_state(
|
||||
@@ -49,7 +83,10 @@ def register(mcp: FastMCP, settings: Settings) -> None:
|
||||
"""
|
||||
session = require_session()
|
||||
raw = session.state(keys or None)
|
||||
return {k: StateValue(value=v["value"], age=round(v["age"], 2)) for k, v in raw.items()}
|
||||
return {
|
||||
k: StateValue(value=v["value"], age=round(v["age"], 2), sources=v.get("sources"))
|
||||
for k, v in raw.items()
|
||||
}
|
||||
|
||||
@mcp.tool(annotations={"readOnlyHint": True, "openWorldHint": False})
|
||||
async def watch_state(
|
||||
@@ -96,13 +133,8 @@ def register(mcp: FastMCP, settings: Settings) -> None:
|
||||
# one that says nothing at all.
|
||||
(blocking if blocks else advisory).append(f"{name}: not reported")
|
||||
|
||||
battery = v.get("BatteryStateChanged_percent")
|
||||
add(
|
||||
"battery",
|
||||
None if battery is None else battery >= 30,
|
||||
"not reported" if battery is None else f"{battery}%",
|
||||
blocks=True,
|
||||
)
|
||||
battery, battery_detail = _battery(session)
|
||||
add("battery", None if battery is None else battery >= 30, battery_detail, blocks=True)
|
||||
|
||||
rssi = v.get("WifiSignalChanged_rssi")
|
||||
add("link", None if rssi is None else rssi > -75, "not reported" if rssi is None else f"{rssi} dBm")
|
||||
|
||||
@@ -119,7 +119,14 @@ async def test_handshake_describes_this_controller(sim, session):
|
||||
assert request["d2c_port"] == session.d2c_port != 0
|
||||
|
||||
|
||||
async def test_second_controller_is_refused(sim, session):
|
||||
async def test_second_controller_takes_the_link_over(sim, session):
|
||||
"""The aircraft accepts a newcomer and stops talking to whoever had it.
|
||||
|
||||
Measured on the real drone 2026-10-02: the first session's frame count
|
||||
froze while the second received, and the first went on reporting
|
||||
connected = True. That silent failure is why a client should judge a link
|
||||
by telemetry freshness rather than by its connected flag.
|
||||
"""
|
||||
other = DroneSession(
|
||||
"127.0.0.1",
|
||||
discovery_port=sim.discovery_port,
|
||||
@@ -128,9 +135,13 @@ async def test_second_controller_is_refused(sim, session):
|
||||
encoder=fake_encode,
|
||||
decoder=fake_decode,
|
||||
)
|
||||
with pytest.raises(HandshakeError, match="one controller"):
|
||||
await other.connect()
|
||||
assert not other.connected
|
||||
try:
|
||||
assert other.connected
|
||||
# The loser keeps its socket and its optimism; only the data stops.
|
||||
assert session.connected
|
||||
finally:
|
||||
await other.disconnect()
|
||||
|
||||
|
||||
async def test_unreachable_address_fails_fast_rather_than_hanging():
|
||||
@@ -393,10 +404,12 @@ def test_probe_gives_up_on_a_closed_port_quickly():
|
||||
def test_an_explicit_address_is_trusted_without_probing():
|
||||
# A caller who names an address wants the connection error, not a
|
||||
# discovery verdict, so find() must not quietly return None here.
|
||||
found = discovery.find("10.1.2.3")
|
||||
# 192.0.2.0/24 is RFC 5737 documentation space, so this cannot name a host
|
||||
# on whatever network the suite happens to run on.
|
||||
found = discovery.find("192.0.2.7")
|
||||
assert found is not None
|
||||
assert (found.ip, found.via) == ("10.1.2.3", "given")
|
||||
assert found.address == ("10.1.2.3", 44444)
|
||||
assert (found.ip, found.via) == ("192.0.2.7", "given")
|
||||
assert found.address == ("192.0.2.7", 44444)
|
||||
|
||||
|
||||
# -- an event we cannot name ---------------------------------------------
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
"""The aircraft reports its battery twice, with different values.
|
||||
|
||||
Observed on the live drone 2026-10-02: `BatteryStateChanged` (0-5-1) arrives as
|
||||
`00 05 01 00 40` on buffer 126 and `00 05 01 00 00` on buffer 127, repeatedly.
|
||||
The 64 is corroborated by the drone's own log and tracks a smooth discharge; the
|
||||
zeros appear in no log line. A store that keeps the newest frame therefore
|
||||
reported a healthy aircraft as flat, and preflight refused on it.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
|
||||
from mcbebop.arsdk.session import Reading
|
||||
from mcbebop.arsdk.types import Event
|
||||
from mcbebop.tools.state import _battery
|
||||
|
||||
KEY = "BatteryStateChanged_percent"
|
||||
|
||||
|
||||
class FakeSession:
|
||||
def __init__(self, entry):
|
||||
self.entry = entry
|
||||
|
||||
def state(self, keys):
|
||||
return {KEY: self.entry} if self.entry is not None else {}
|
||||
|
||||
|
||||
def test_reading_detects_disagreement():
|
||||
assert Reading(0, 1.0, 4, {126: 64, 127: 0}).conflicting
|
||||
assert not Reading(64, 1.0, 1, {126: 64}).conflicting
|
||||
assert not Reading(64, 1.0, 2, {126: 64, 127: 64}).conflicting
|
||||
|
||||
|
||||
def test_event_carries_its_buffer():
|
||||
"""Without the source, the store cannot tell two reporters apart."""
|
||||
assert Event(ids=(0, 5, 1), name="x", values={}, at=0.0).buffer_id == 0
|
||||
assert Event(ids=(0, 5, 1), name="x", values={}, at=0.0, buffer_id=127).buffer_id == 127
|
||||
|
||||
|
||||
def test_the_real_conflict_resolves_to_the_credible_value():
|
||||
pct, detail = _battery(FakeSession({"value": 0, "age": 0.1, "sources": {126: 64, 127: 0}}))
|
||||
assert pct == 64, "the newest frame said 0; the credible one is 64"
|
||||
assert "contested" in detail and "0" in detail, "the caller must learn the number was disputed"
|
||||
|
||||
|
||||
def test_a_lone_reading_is_not_described_as_contested():
|
||||
pct, detail = _battery(FakeSession({"value": 64, "age": 0.1, "sources": {126: 64}}))
|
||||
assert pct == 64 and detail == "64%"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("entry", "expected"),
|
||||
[
|
||||
({"value": 58, "age": 0.1}, 58),
|
||||
({"value": 0, "age": 0.1, "sources": {126: 0, 127: 0}}, 0),
|
||||
(None, None),
|
||||
],
|
||||
)
|
||||
def test_other_shapes(entry, expected):
|
||||
"""A genuine zero from every source still reports zero, and still blocks."""
|
||||
assert _battery(FakeSession(entry))[0] == expected
|
||||
@@ -55,3 +55,19 @@ def test_acknowledgement_is_decided_by_data_type_not_buffer():
|
||||
frame = Frame(DataType.DATA_WITH_ACK, buffer_id, 9, b"\x00\x05\x00\x00")
|
||||
assert Frame.decode_all(frame.encode())[0].buffer_id == buffer_id
|
||||
assert 0 <= BufferId.ack_for(buffer_id) <= 255
|
||||
|
||||
|
||||
def test_simulator_models_takeover_not_refusal_by_default():
|
||||
"""The aircraft accepts a second controller; it does not refuse one.
|
||||
|
||||
Tested on the real drone 2026-10-02: session A's frames froze at 264 while
|
||||
session B took over, and A went on reporting connected = True. The
|
||||
simulator defaulted to refusing, which is the same unverified assumption
|
||||
the client carried, so no test could catch the difference.
|
||||
"""
|
||||
import dataclasses
|
||||
|
||||
from mcbebop.sim import FakeBebop
|
||||
|
||||
field = next(f for f in dataclasses.fields(FakeBebop) if f.name == "single_controller")
|
||||
assert field.default is False, "the default must model the aircraft, not the folklore"
|
||||
|
||||
@@ -0,0 +1,159 @@
|
||||
"""`read_log`: the filters, the defaults, and what it says about a live log.
|
||||
|
||||
The log is built here from the ulogcat format rather than fetched, so none of
|
||||
this touches the aircraft or the captured file.
|
||||
"""
|
||||
|
||||
import pytest
|
||||
from fastmcp import Client
|
||||
from test_ulog import entry
|
||||
|
||||
from mcbebop.config import Settings
|
||||
from mcbebop.server import build_server
|
||||
from mcbebop.tools import _common, logs
|
||||
|
||||
|
||||
def synthetic_log() -> bytes:
|
||||
return b"".join(
|
||||
[
|
||||
entry("Booting Linux on physical CPU 0", tag="KERNEL", uptime_us=0),
|
||||
entry("AM I HOST mode for 5 s", tag="shp_usbmode", uptime_us=1_000_000),
|
||||
entry("AM I HOST mode for 5 s", tag="shp_usbmode", uptime_us=6_000_000),
|
||||
entry("Cmd Tx : battery level <62%>", tag="COMMANDS", uptime_us=10_000_000),
|
||||
entry("Magneto calibration state (1-required) : 0", tag="COMMANDS", uptime_us=11_000_000),
|
||||
entry("eth0: no IPv6 routers present", tag="KERNEL", priority="W", uptime_us=12_000_000),
|
||||
entry("fileOpen: error: cannot open file", tag="colibry", priority="E", uptime_us=13_000_000),
|
||||
]
|
||||
)
|
||||
|
||||
|
||||
def records():
|
||||
from mcbebop.files import ulog
|
||||
|
||||
return list(ulog.parse(synthetic_log()))
|
||||
|
||||
|
||||
# --- the filters, without a server ------------------------------------------
|
||||
|
||||
|
||||
def test_a_tag_filter_is_exact_and_case_insensitive():
|
||||
kept, matched = logs.select(records(), tag="kernel")
|
||||
assert matched == 2
|
||||
assert {r.tag for r in kept} == {"KERNEL"}
|
||||
|
||||
|
||||
def test_a_substring_is_a_valid_regex_and_behaves_like_one():
|
||||
kept, _ = logs.select(records(), match="battery")
|
||||
assert [r.tag for r in kept] == ["COMMANDS"]
|
||||
|
||||
|
||||
def test_a_broken_regex_says_it_was_read_as_a_regex():
|
||||
from fastmcp.exceptions import ToolError
|
||||
|
||||
with pytest.raises(ToolError, match="regular expression"):
|
||||
logs.select(records(), match="unclosed (")
|
||||
|
||||
|
||||
def test_a_severity_filter_keeps_that_level_and_worse():
|
||||
kept, _ = logs.select(records(), min_level="warning")
|
||||
assert [r.level for r in kept] == ["warning", "error"]
|
||||
kept, _ = logs.select(records(), min_level="error")
|
||||
assert [r.level for r in kept] == ["error"]
|
||||
|
||||
|
||||
def test_the_limit_takes_the_newest_matches_not_the_first():
|
||||
kept, matched = logs.select(records(), limit=2)
|
||||
assert matched == 7
|
||||
assert [r.uptime_us for r in kept] == [12_000_000, 13_000_000]
|
||||
|
||||
|
||||
def test_the_noisy_poll_is_excluded_by_default_and_can_be_asked_for():
|
||||
assert "shp_usbmode" in logs.NOISY_TAGS
|
||||
kept, _ = logs.select(records(), exclude_tags=logs.NOISY_TAGS)
|
||||
assert "shp_usbmode" not in {r.tag for r in kept}
|
||||
kept, _ = logs.select(records(), exclude_tags=[])
|
||||
assert "shp_usbmode" in {r.tag for r in kept}
|
||||
|
||||
|
||||
def test_naming_a_tag_explicitly_beats_the_default_exclusion():
|
||||
# Asking for a tag by name and getting nothing because a default hid it
|
||||
# would be the worst kind of helpful.
|
||||
kept, matched = logs.select(records(), tag="shp_usbmode", exclude_tags=logs.NOISY_TAGS)
|
||||
assert matched == 2
|
||||
assert {r.tag for r in kept} == {"shp_usbmode"}
|
||||
|
||||
|
||||
# --- through the server -----------------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def served(monkeypatch, tmp_path):
|
||||
"""A server whose FTP fetch hands back a synthetic log."""
|
||||
log = tmp_path / "ckcm.bin"
|
||||
log.write_bytes(synthetic_log())
|
||||
asked: dict = {}
|
||||
|
||||
def fake_fetch(area, path, *, host, capture_dir, max_bytes=0, timeout=0.0):
|
||||
asked.update(area=area, path=path, host=host)
|
||||
return log
|
||||
|
||||
monkeypatch.setattr("mcbebop.files.ftp.fetch", fake_fetch)
|
||||
return build_server(Settings(capture_dir=tmp_path)), asked
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def client(served):
|
||||
server, asked = served
|
||||
async with Client(server) as c:
|
||||
# No connect(): the log comes off the aircraft's FTP server, which
|
||||
# answers whether or not a command session is open.
|
||||
_common.app().target = "192.0.2.1"
|
||||
yield c, asked
|
||||
|
||||
|
||||
async def test_read_log_fetches_from_the_log_area_and_returns_parsed_entries(client):
|
||||
c, asked = client
|
||||
result = await c.call_tool("read_log", {})
|
||||
body = result.structured_content
|
||||
assert asked == {"area": logs.LOG_AREA, "path": logs.LOG_PATH, "host": "192.0.2.1"}
|
||||
assert body["parsed"] == 7
|
||||
assert body["returned"] == 5, "the usb poll should be excluded by default"
|
||||
assert body["entries"][0]["message"].startswith("fileOpen"), "newest first by default"
|
||||
assert body["tags"]["KERNEL"] == 2
|
||||
|
||||
|
||||
async def test_read_log_reports_which_tags_it_hid(client):
|
||||
c, _ = client
|
||||
body = (await c.call_tool("read_log", {})).structured_content
|
||||
assert "shp_usbmode" in body["note"]
|
||||
assert "exclude_tags=[]" in body["note"]
|
||||
|
||||
|
||||
async def test_read_log_can_order_oldest_first_while_still_taking_the_newest(client):
|
||||
c, _ = client
|
||||
body = (await c.call_tool("read_log", {"limit": 2, "newest_first": False})).structured_content
|
||||
assert [e["uptime"] for e in body["entries"]] == [12.0, 13.0]
|
||||
|
||||
|
||||
async def test_the_whole_file_tag_census_survives_the_filters(client):
|
||||
c, _ = client
|
||||
body = (await c.call_tool("read_log", {"tag": "colibry"})).structured_content
|
||||
assert body["matched"] == 1
|
||||
assert body["parsed"] == 7, "parsed counts the file, not the filter"
|
||||
assert set(body["tags"]) == {"KERNEL", "shp_usbmode", "COMMANDS", "colibry"}
|
||||
|
||||
|
||||
async def test_an_absent_tag_says_which_tags_are_present(client):
|
||||
c, _ = client
|
||||
body = (await c.call_tool("read_log", {"tag": "NETMON"})).structured_content
|
||||
assert body["returned"] == 0
|
||||
assert "NETMON" in body["note"] and "KERNEL" in body["note"]
|
||||
|
||||
|
||||
async def test_the_log_is_not_reachable_on_the_simulator(client):
|
||||
from fastmcp.exceptions import ToolError
|
||||
|
||||
c, _ = client
|
||||
_common.app().target = "sim"
|
||||
with pytest.raises(ToolError, match="simulator"):
|
||||
await c.call_tool("read_log", {})
|
||||
+100
-2
@@ -1,14 +1,61 @@
|
||||
"""Guards that the package stays coherent as the streams land."""
|
||||
"""Guards that the package stays coherent as the streams land.
|
||||
|
||||
The second half of this file is a privacy guard rather than a coherence one.
|
||||
This package was written against a real aircraft parked at a real address, so
|
||||
the things that identify it -- the serial, the SoC CPU id, the Wi-Fi BSSID, the
|
||||
first GPS fix -- all passed through this working tree at some point. PyPI is
|
||||
immutable per version and sdists are mirrored within minutes, so the audit that
|
||||
catches a stray identifier has to run before every publish, not after one.
|
||||
"""
|
||||
|
||||
import re
|
||||
import tomllib
|
||||
from pathlib import Path
|
||||
|
||||
import mcbebop
|
||||
from mcbebop.server import build_server
|
||||
|
||||
ROOT = Path(__file__).parent.parent
|
||||
|
||||
# Every pattern here is a shape that identifies the owner's aircraft or
|
||||
# network. The comment on each says what it is, so a future hit is diagnosable
|
||||
# rather than mysterious.
|
||||
_FORBIDDEN = (
|
||||
# A Bebop 2 serial's high half. The full serial also appears in the
|
||||
# aircraft's SSID, which makes it a locator and not just a label.
|
||||
(re.compile(r"PI04"), "a real Bebop 2 serial prefix"),
|
||||
# The P7 SoC CPU id: 'P7' then a long uppercase run. 'P7ID', the command
|
||||
# name in ardrone3.xml, is too short to match and is meant to be here.
|
||||
(re.compile(r"P7[0-9A-Z]{8,}"), "a P7 SoC CPU id"),
|
||||
# Any MAC address at all, not just a Parrot OUI.
|
||||
(re.compile(r"\b[0-9A-Fa-f]{2}(?::[0-9A-Fa-f]{2}){5}\b"), "a MAC address"),
|
||||
# A plausible residential coordinate. The simulator reports ARSDK's 500.0
|
||||
# no-fix sentinel instead, which has no decimal places to match.
|
||||
(re.compile(r"\b[0-9]{1,3}\.[0-9]{6,}"), "a high-precision coordinate"),
|
||||
# An absolute build path leaks a username and the local layout.
|
||||
(re.compile(r"/home/[a-z]"), "an absolute home-directory path"),
|
||||
)
|
||||
|
||||
# 192.168.42.0/24 is the drone's own fixed network and is in Parrot's public
|
||||
# documentation, so it is expected. Any other private address is somebody's
|
||||
# real LAN; use RFC 5737 documentation space in examples instead.
|
||||
_PRIVATE_IP = re.compile(
|
||||
r"\b(?:10\.\d{1,3}\.\d{1,3}\.\d{1,3}"
|
||||
r"|172\.(?:1[6-9]|2\d|3[01])\.\d{1,3}\.\d{1,3}"
|
||||
r"|192\.168\.\d{1,3}\.\d{1,3})\b"
|
||||
)
|
||||
_ALLOWED_IP = re.compile(r"\b192\.168\.42\.\d{1,3}\b")
|
||||
|
||||
|
||||
def _shipped_files() -> list[Path]:
|
||||
"""Every text file that reaches an artifact, which is src/ plus the docs."""
|
||||
files = [ROOT / "README.md", ROOT / "pyproject.toml"]
|
||||
files += [p for p in (ROOT / "src").rglob("*") if p.is_file() and "__pycache__" not in p.parts]
|
||||
return files
|
||||
|
||||
|
||||
def _pyproject() -> dict:
|
||||
return tomllib.loads((Path(__file__).parent.parent / "pyproject.toml").read_text())
|
||||
return tomllib.loads((ROOT / "pyproject.toml").read_text())
|
||||
|
||||
|
||||
def test_version_matches_pyproject():
|
||||
@@ -38,3 +85,54 @@ def test_vendored_xml_present_and_parses():
|
||||
assert proj.find("class") is not None, f"{name} has no <class> elements"
|
||||
total += sum(len(c.findall("cmd")) for c in proj.iter("class"))
|
||||
assert total == 264, f"expected 264 commands, found {total}"
|
||||
|
||||
|
||||
# -- privacy guards ------------------------------------------------------
|
||||
def test_no_shipped_file_carries_an_aircraft_identifier():
|
||||
hits = []
|
||||
for path in _shipped_files():
|
||||
text = path.read_text(encoding="utf-8", errors="replace")
|
||||
for lineno, line in enumerate(text.splitlines(), 1):
|
||||
for pattern, what in _FORBIDDEN:
|
||||
if pattern.search(line):
|
||||
hits.append(f"{path.relative_to(ROOT)}:{lineno} looks like {what}: {line.strip()[:90]}")
|
||||
assert not hits, "identifying data in a file that ships:\n" + "\n".join(hits)
|
||||
|
||||
|
||||
def test_no_shipped_file_names_a_private_network_but_the_drones_own():
|
||||
hits = []
|
||||
for path in _shipped_files():
|
||||
text = path.read_text(encoding="utf-8", errors="replace")
|
||||
for lineno, line in enumerate(text.splitlines(), 1):
|
||||
for found in _PRIVATE_IP.finditer(line):
|
||||
if not _ALLOWED_IP.fullmatch(found.group()):
|
||||
hits.append(f"{path.relative_to(ROOT)}:{lineno} names {found.group()}")
|
||||
assert not hits, (
|
||||
"a private address other than the drone's own 192.168.42.0/24; "
|
||||
"use RFC 5737 documentation space:\n" + "\n".join(hits)
|
||||
)
|
||||
|
||||
|
||||
def test_sdist_excludes_captures_and_dev_trees():
|
||||
"""The sdist exclusions are a separate mechanism from .gitignore.
|
||||
|
||||
A file can be gitignored and still swept into an sdist, which is how
|
||||
captures reach PyPI. Assert the directories that hold real data off the
|
||||
aircraft are named in both places.
|
||||
"""
|
||||
excluded = set(_pyproject()["tool"]["uv"]["build-backend"]["source-exclude"])
|
||||
ignored = (ROOT / ".gitignore").read_text().split()
|
||||
for name in ("captures", "tests"):
|
||||
assert name in excluded or f"{name}/**" in excluded, f"{name} is not excluded from the sdist"
|
||||
for name in ("captures",):
|
||||
assert f"{name}/" in ignored or name in ignored, f"{name} is not gitignored"
|
||||
|
||||
|
||||
def test_declared_licence_files_exist():
|
||||
"""A licence named in metadata but absent from the tree ships a lie."""
|
||||
declared = _pyproject()["project"]["license-files"]
|
||||
assert declared, "license-files must name the licence texts so they ship"
|
||||
for name in declared:
|
||||
assert (ROOT / name).is_file(), f"{name} is declared in license-files but missing"
|
||||
# The vendored XML is Parrot's and the expression has to say so.
|
||||
assert "BSD-3-Clause" in _pyproject()["project"]["license"]
|
||||
|
||||
@@ -0,0 +1,198 @@
|
||||
"""The MCP resources, driven through a real client.
|
||||
|
||||
The point of the split under test here is that the command catalogue needs no
|
||||
aircraft while everything else does, and that the ones that do degrade into an
|
||||
explanation rather than an exception.
|
||||
"""
|
||||
|
||||
import json
|
||||
|
||||
import pytest
|
||||
from fastmcp import Client
|
||||
from test_ulog import entry
|
||||
|
||||
from mcbebop.config import Settings
|
||||
from mcbebop.protocol import xml_index
|
||||
from mcbebop.server import build_server
|
||||
from mcbebop.tools import _common, logs
|
||||
|
||||
|
||||
def body(result) -> dict:
|
||||
return json.loads(result[0].text)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def client():
|
||||
async with Client(build_server()) as c:
|
||||
yield c
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def connected(client):
|
||||
await client.call_tool("connect", {"target": "sim"})
|
||||
yield client
|
||||
await client.call_tool("disconnect", {})
|
||||
|
||||
|
||||
# --- what is registered ------------------------------------------------------
|
||||
|
||||
|
||||
async def test_the_static_resources_are_registered(client):
|
||||
uris = {str(r.uri) for r in await client.list_resources()}
|
||||
assert uris == {"bebop://commands", "bebop://state"}
|
||||
|
||||
|
||||
async def test_the_templated_resources_are_registered(client):
|
||||
uris = {t.uri_template for t in await client.list_resource_templates()}
|
||||
assert uris == {
|
||||
"bebop://commands/{name}",
|
||||
"bebop://state/{key}",
|
||||
"bebop://files/{area}",
|
||||
"bebop://log/{tag}",
|
||||
}
|
||||
|
||||
|
||||
# --- the catalogue, which needs no drone -------------------------------------
|
||||
|
||||
|
||||
async def test_the_whole_catalogue_reads_without_a_drone(client):
|
||||
data = body(await client.read_resource("bebop://commands"))
|
||||
assert data["count"] == len(xml_index.all_commands())
|
||||
names = {c["name"] for c in data["commands"]}
|
||||
assert "ardrone3.Piloting.TakeOff" in names
|
||||
assert all({"tier", "direction", "bebop2"} <= set(c) for c in data["commands"])
|
||||
|
||||
|
||||
async def test_one_command_reads_in_full_with_its_dotted_name(client):
|
||||
data = body(await client.read_resource("bebop://commands/ardrone3.Camera.OrientationV2"))
|
||||
assert data["name"] == "ardrone3.Camera.OrientationV2"
|
||||
assert [a["name"] for a in data["args"]] == ["tilt", "pan"]
|
||||
assert data["ids"] and data["buffer"]
|
||||
|
||||
|
||||
async def test_an_enum_argument_arrives_with_its_allowed_names(client):
|
||||
# Without these a caller has to guess at the string an argument accepts,
|
||||
# which is the main reason to read a command rather than send and see.
|
||||
data = body(await client.read_resource("bebop://commands/common.Mavlink.Start"))
|
||||
(arg,) = [a for a in data["args"] if a["name"] == "type"]
|
||||
assert arg["enum_values"] == ["flightPlan", "mapMyHouse"]
|
||||
|
||||
|
||||
async def test_an_unknown_command_suggests_rather_than_failing(client):
|
||||
data = body(await client.read_resource("bebop://commands/ardrone3.Piloting.Takeof"))
|
||||
assert data["available"] is False
|
||||
assert "ardrone3.Piloting.TakeOff" in data["did_you_mean"]
|
||||
|
||||
|
||||
# --- the ones that need a drone ----------------------------------------------
|
||||
|
||||
|
||||
async def test_state_explains_itself_when_nothing_is_connected(client):
|
||||
data = body(await client.read_resource("bebop://state"))
|
||||
assert data["available"] is False
|
||||
assert "connect" in data["reason"].lower()
|
||||
|
||||
|
||||
async def test_one_state_key_explains_itself_when_nothing_is_connected(client):
|
||||
data = body(await client.read_resource("bebop://state/BatteryStateChanged_percent"))
|
||||
assert data["available"] is False
|
||||
|
||||
|
||||
async def test_state_carries_every_reported_value_with_its_age(connected):
|
||||
data = body(await connected.read_resource("bebop://state"))
|
||||
assert data["available"] is True
|
||||
assert data["keys"] > 20
|
||||
battery = data["values"]["BatteryStateChanged_percent"]
|
||||
assert isinstance(battery["value"], int) and battery["age"] >= 0
|
||||
|
||||
|
||||
async def test_state_reports_link_freshness_because_connected_is_not_evidence(connected):
|
||||
# The aircraft accepts a second controller and silently redirects
|
||||
# telemetry to it, leaving this session reporting connected with nothing
|
||||
# arriving. The age of the newest event is the only tell.
|
||||
data = body(await connected.read_resource("bebop://state"))
|
||||
assert data["last_event_age"] is not None and data["last_event_age"] < 10
|
||||
|
||||
|
||||
async def test_a_state_key_matches_on_a_prefix(connected):
|
||||
data = body(await connected.read_resource("bebop://state/BatteryStateChanged"))
|
||||
assert data["available"] is True
|
||||
assert "BatteryStateChanged_percent" in data["values"]
|
||||
|
||||
|
||||
async def test_a_state_key_the_drone_never_sent_says_so(connected):
|
||||
data = body(await connected.read_resource("bebop://state/NoSuchThing"))
|
||||
assert data["available"] is False
|
||||
assert "bebop://state" in data["reason"]
|
||||
|
||||
|
||||
async def test_a_file_listing_on_the_simulator_explains_itself(connected):
|
||||
data = body(await connected.read_resource("bebop://files/media"))
|
||||
assert data["available"] is False
|
||||
assert "simulator" in data["reason"]
|
||||
|
||||
|
||||
async def test_an_unknown_file_area_lists_the_real_ones(client):
|
||||
data = body(await client.read_resource("bebop://files/update"))
|
||||
assert data["available"] is False
|
||||
assert "flightplans" in data["reason"]
|
||||
|
||||
|
||||
async def test_the_log_resource_explains_itself_with_no_aircraft(connected):
|
||||
data = body(await connected.read_resource("bebop://log/KERNEL"))
|
||||
assert data["available"] is False
|
||||
assert "simulator" in data["reason"]
|
||||
|
||||
|
||||
# --- the log resource against a synthetic log --------------------------------
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
async def with_log(monkeypatch, tmp_path):
|
||||
log = tmp_path / "ckcm.bin"
|
||||
log.write_bytes(
|
||||
entry("Machine: Milos board", tag="KERNEL", uptime_us=0)
|
||||
+ entry("Cmd Tx : battery level <62%>", tag="COMMANDS", uptime_us=9_000_000)
|
||||
+ entry("eth0: no IPv6 routers present", tag="KERNEL", priority="W", uptime_us=12_000_000)
|
||||
)
|
||||
monkeypatch.setattr("mcbebop.files.ftp.fetch", lambda *a, **k: log)
|
||||
async with Client(build_server(Settings(capture_dir=tmp_path))) as c:
|
||||
_common.app().target = "192.0.2.1"
|
||||
yield c
|
||||
|
||||
|
||||
async def test_the_log_resource_returns_recent_entries_for_one_tag(with_log):
|
||||
data = body(await with_log.read_resource("bebop://log/KERNEL"))
|
||||
assert data["available"] is True
|
||||
assert data["matched"] == 2
|
||||
assert data["parsed"] == 3
|
||||
assert data["entries"][0]["message"].startswith("eth0"), "newest first"
|
||||
|
||||
|
||||
async def test_the_log_resource_shows_the_whole_tag_census(with_log):
|
||||
data = body(await with_log.read_resource("bebop://log/COMMANDS"))
|
||||
assert data["tags"] == {"KERNEL": 2, "COMMANDS": 1}
|
||||
|
||||
|
||||
async def test_the_log_resource_caps_what_it_returns(with_log):
|
||||
from mcbebop import resources
|
||||
|
||||
assert resources.LOG_RESOURCE_LIMIT <= 100, "a resource read must not flood a context"
|
||||
data = body(await with_log.read_resource("bebop://log/KERNEL"))
|
||||
assert len(data["entries"]) <= resources.LOG_RESOURCE_LIMIT
|
||||
|
||||
|
||||
async def test_a_tag_the_log_does_not_carry_is_empty_not_an_error(with_log):
|
||||
data = body(await with_log.read_resource("bebop://log/NETMON"))
|
||||
assert data["available"] is True
|
||||
assert data["matched"] == 0
|
||||
assert data["entries"] == []
|
||||
|
||||
|
||||
async def test_the_log_resource_does_not_apply_the_tools_default_exclusions(with_log):
|
||||
# It addresses one tag by name, so there is nothing to protect the caller
|
||||
# from; a resource that silently returned nothing for a tag it was asked
|
||||
# for would be worse than a long answer.
|
||||
assert logs.NOISY_TAGS, "the tool excludes something by default"
|
||||
data = body(await with_log.read_resource("bebop://log/KERNEL"))
|
||||
assert data["matched"] == 2
|
||||
@@ -0,0 +1,475 @@
|
||||
"""The RTP packetiser and the capture format, against bytes.
|
||||
|
||||
Synthetic NALs throughout: a NAL is a header byte and a payload as far as
|
||||
RFC 6184 is concerned, so real H.264 would only make the assertions harder to
|
||||
read. The one test that needs real video is the ffmpeg decode in
|
||||
`test_sim_video.py`, which is the only one that can tell whether any of this
|
||||
is actually correct.
|
||||
"""
|
||||
|
||||
import random
|
||||
import struct
|
||||
from itertools import pairwise
|
||||
|
||||
import pytest
|
||||
|
||||
from mcbebop.media import capture, rtp
|
||||
|
||||
# header bytes: NRI 3 is "most important", which parameter sets carry.
|
||||
SPS = bytes([0x67]) + b"\x42\xc0\x1e"
|
||||
PPS = bytes([0x68]) + b"\xce\x3c\x80"
|
||||
SEI = bytes([0x06]) + b"\x05\x02\x00\x00\x80"
|
||||
# A slice whose first bit is set, so first_mb_in_slice is 0 and it opens a
|
||||
# picture. 0x65 is an IDR at NRI 3, 0x41 a non-IDR at NRI 2.
|
||||
IDR = bytes([0x65, 0x88]) + b"\xaa" * 40
|
||||
SLICE = bytes([0x41, 0x9A]) + b"\xbb" * 40
|
||||
# first bit clear, so first_mb_in_slice is not 0: a continuation slice.
|
||||
SLICE_2 = bytes([0x41, 0x1A]) + b"\xcc" * 40
|
||||
|
||||
|
||||
def annex_b(*nals: bytes, four_byte: bool = False) -> bytes:
|
||||
code = b"\x00\x00\x00\x01" if four_byte else b"\x00\x00\x01"
|
||||
return b"".join(code + nal for nal in nals)
|
||||
|
||||
|
||||
# -- Annex-B parsing -----------------------------------------------------
|
||||
def test_three_and_four_byte_start_codes_both_split():
|
||||
assert list(rtp.iter_nals(annex_b(SPS, PPS))) == [SPS, PPS]
|
||||
assert list(rtp.iter_nals(annex_b(SPS, PPS, four_byte=True))) == [SPS, PPS]
|
||||
|
||||
|
||||
def test_trailing_zeros_belong_to_the_start_code_not_the_nal():
|
||||
# A fourth zero before 00 00 01 is a trailing byte of the previous NAL.
|
||||
# Counting it would change the length and so change where the MTU falls.
|
||||
data = b"\x00\x00\x01" + SPS + b"\x00\x00\x00\x00\x01" + PPS
|
||||
assert list(rtp.iter_nals(data)) == [SPS, PPS]
|
||||
|
||||
|
||||
def test_a_stream_with_no_start_code_yields_nothing():
|
||||
assert list(rtp.iter_nals(b"\xde\xad\xbe\xef")) == []
|
||||
with pytest.raises(ValueError, match="Annex-B"):
|
||||
rtp.AnnexBStream.from_bytes(b"\xde\xad\xbe\xef")
|
||||
|
||||
|
||||
def test_leading_garbage_before_the_first_start_code_is_skipped():
|
||||
assert list(rtp.iter_nals(b"junk" + annex_b(SPS))) == [SPS]
|
||||
|
||||
|
||||
def test_nal_type_reads_the_low_five_bits():
|
||||
assert rtp.nal_type(SPS) == 7
|
||||
assert rtp.nal_type(PPS) == 8
|
||||
assert rtp.nal_type(IDR) == 5
|
||||
assert rtp.nal_type(SLICE) == 1
|
||||
|
||||
|
||||
# -- access units --------------------------------------------------------
|
||||
def test_parameter_sets_attach_to_the_picture_that_follows_them():
|
||||
units = rtp.access_units([SPS, PPS, IDR, SLICE])
|
||||
assert [u.nals for u in units] == [(SPS, PPS, IDR), (SLICE,)]
|
||||
assert units[0].is_idr and units[0].carries_parameter_sets
|
||||
assert not units[1].is_idr and not units[1].carries_parameter_sets
|
||||
|
||||
|
||||
def test_a_continuation_slice_stays_in_the_same_access_unit():
|
||||
# Splitting a multi-slice picture per slice would put the marker bit and
|
||||
# the timestamp step in the wrong places.
|
||||
units = rtp.access_units([IDR, SLICE_2, SLICE])
|
||||
assert [u.nals for u in units] == [(IDR, SLICE_2), (SLICE,)]
|
||||
|
||||
|
||||
def test_the_stream_keeps_the_parameter_sets_for_reuse():
|
||||
stream = rtp.AnnexBStream.from_bytes(annex_b(SPS, PPS, IDR, SLICE, SLICE))
|
||||
assert stream.parameter_sets == (SPS, PPS)
|
||||
assert len(stream.units) == 3
|
||||
|
||||
|
||||
def test_a_stream_without_parameter_sets_warns(caplog):
|
||||
rtp.AnnexBStream.from_bytes(annex_b(IDR))
|
||||
assert "joining late" in caplog.text
|
||||
|
||||
|
||||
# -- RTP headers ---------------------------------------------------------
|
||||
def unpack(packet):
|
||||
return rtp.parse_packet(packet)
|
||||
|
||||
|
||||
def test_a_small_nal_becomes_exactly_one_single_nal_packet():
|
||||
p = rtp.Packetiser(mtu=1400, ssrc=0x11223344, seq=7, timestamp=9000)
|
||||
packets = p.packetise([IDR])
|
||||
assert len(packets) == 1
|
||||
pt, seq, ts, ssrc, marker, payload = unpack(packets[0])
|
||||
assert (pt, seq, ts, ssrc, marker) == (96, 7, 9000, 0x11223344, True)
|
||||
assert payload == IDR # the NAL travels whole, header byte included
|
||||
|
||||
|
||||
def test_the_header_is_twelve_bytes_of_version_two():
|
||||
packets = rtp.Packetiser().packetise([IDR])
|
||||
assert packets[0][0] == 0x80 # V=2, no padding, no extension, no CSRC
|
||||
assert len(packets[0]) == 12 + len(IDR)
|
||||
|
||||
|
||||
def test_parse_packet_rejects_anything_that_is_not_version_two():
|
||||
with pytest.raises(ValueError, match="version 2"):
|
||||
rtp.parse_packet(b"\x40" + b"\x00" * 15)
|
||||
with pytest.raises(ValueError, match="at least"):
|
||||
rtp.parse_packet(b"\x80\x60")
|
||||
|
||||
|
||||
# -- FU-A ----------------------------------------------------------------
|
||||
def big_nal(size: int, header: int = 0x65) -> bytes:
|
||||
return bytes([header]) + bytes((i % 251) + 1 for i in range(size - 1))
|
||||
|
||||
|
||||
def test_an_oversized_nal_fragments_into_well_formed_fu_a():
|
||||
nal = big_nal(4000)
|
||||
p = rtp.Packetiser(mtu=1400)
|
||||
packets = p.packetise([nal])
|
||||
payloads = [unpack(pkt)[5] for pkt in packets]
|
||||
assert len(payloads) == 3 # 1398 payload bytes per fragment, 3999 to carry
|
||||
|
||||
for payload in payloads:
|
||||
assert payload[0] & 0x1F == rtp.NAL_FU_A
|
||||
assert payload[0] & 0x60 == nal[0] & 0x60 # NRI is preserved
|
||||
assert payload[1] & 0x1F == rtp.nal_type(nal) # the carried type
|
||||
assert payload[1] & 0x20 == 0 # the reserved bit must be zero
|
||||
|
||||
starts = [bool(p[1] & 0x80) for p in payloads]
|
||||
ends = [bool(p[1] & 0x40) for p in payloads]
|
||||
assert starts == [True, False, False]
|
||||
assert ends == [False, False, True]
|
||||
|
||||
|
||||
def test_fu_a_fragments_reassemble_into_the_original_nal():
|
||||
nal = big_nal(4000)
|
||||
payloads = [unpack(p)[5] for p in rtp.Packetiser(mtu=1400).packetise([nal])]
|
||||
rebuilt = bytes([(payloads[0][0] & 0xE0) | (payloads[0][1] & 0x1F)])
|
||||
rebuilt += b"".join(p[2:] for p in payloads)
|
||||
assert rebuilt == nal
|
||||
|
||||
|
||||
def test_no_fragment_exceeds_the_mtu():
|
||||
for mtu in (3, 50, 1400):
|
||||
packets = rtp.Packetiser(mtu=mtu).packetise([big_nal(4000)])
|
||||
assert max(len(unpack(p)[5]) for p in packets) <= mtu
|
||||
|
||||
|
||||
def test_a_nal_exactly_at_the_mtu_is_not_fragmented():
|
||||
nal = big_nal(100)
|
||||
assert len(rtp.Packetiser(mtu=100, aggregate=False).packetise([nal])) == 1
|
||||
assert len(rtp.Packetiser(mtu=99, aggregate=False).packetise([nal])) == 2
|
||||
|
||||
|
||||
def test_an_mtu_too_small_to_carry_a_payload_byte_is_refused():
|
||||
with pytest.raises(ValueError, match="at least 3"):
|
||||
rtp.Packetiser(mtu=2)
|
||||
|
||||
|
||||
# -- STAP-A --------------------------------------------------------------
|
||||
def test_parameter_sets_aggregate_into_one_stap_a():
|
||||
packets = rtp.Packetiser(mtu=1400).packetise([SPS, PPS, SEI, IDR])
|
||||
assert len(packets) == 2 # one STAP-A for the headers, one for the picture
|
||||
payload = unpack(packets[0])[5]
|
||||
assert payload[0] & 0x1F == rtp.NAL_STAP_A
|
||||
assert payload[0] & 0x60 == 0x60 # the highest NRI of what it carries
|
||||
|
||||
carried, pos = [], 1
|
||||
while pos < len(payload):
|
||||
(length,) = struct.unpack_from(">H", payload, pos)
|
||||
carried.append(payload[pos + 2 : pos + 2 + length])
|
||||
pos += 2 + length
|
||||
assert carried == [SPS, PPS, SEI]
|
||||
|
||||
|
||||
def test_a_picture_is_never_aggregated():
|
||||
# A STAP-A carrying a slice would be legal, but it is not what this does,
|
||||
# and the docstring says so. Two small slices stay two packets.
|
||||
packets = rtp.Packetiser(mtu=1400).packetise([IDR, SLICE_2])
|
||||
assert len(packets) == 2
|
||||
assert all(unpack(p)[5][0] & 0x1F != rtp.NAL_STAP_A for p in packets)
|
||||
|
||||
|
||||
def test_a_lone_parameter_set_goes_out_as_a_single_nal_not_a_stap_a():
|
||||
(packet,) = rtp.Packetiser(mtu=1400).packetise([SPS])
|
||||
assert unpack(packet)[5] == SPS
|
||||
|
||||
|
||||
def test_aggregation_can_be_switched_off():
|
||||
packets = rtp.Packetiser(mtu=1400, aggregate=False).packetise([SPS, PPS, SEI, IDR])
|
||||
assert len(packets) == 4
|
||||
assert [unpack(p)[5] for p in packets] == [SPS, PPS, SEI, IDR]
|
||||
|
||||
|
||||
def test_aggregation_stops_at_the_mtu_rather_than_overflowing():
|
||||
sets = [big_nal(300, header=0x68) for _ in range(5)]
|
||||
packets = rtp.Packetiser(mtu=700).packetise(sets)
|
||||
assert len(packets) == 3 # 2 + 2 + 1 at 302 bytes of cost each
|
||||
assert all(len(unpack(p)[5]) <= 700 for p in packets)
|
||||
|
||||
|
||||
# -- sequence, marker, clock ---------------------------------------------
|
||||
def test_sequence_numbers_advance_by_one_per_packet():
|
||||
p = rtp.Packetiser(mtu=1400, seq=100)
|
||||
packets = p.packetise([SPS, PPS, IDR]) + p.packetise([SLICE])
|
||||
assert [unpack(pkt)[1] for pkt in packets] == [100, 101, 102]
|
||||
assert p.seq == 103
|
||||
|
||||
|
||||
def test_sequence_numbers_wrap_at_sixteen_bits():
|
||||
p = rtp.Packetiser(mtu=1400, seq=0xFFFE)
|
||||
packets = p.packetise([IDR]) + p.packetise([IDR]) + p.packetise([IDR])
|
||||
assert [unpack(pkt)[1] for pkt in packets] == [0xFFFE, 0xFFFF, 0]
|
||||
|
||||
|
||||
def test_the_marker_bit_lands_only_on_the_last_packet_of_an_access_unit():
|
||||
p = rtp.Packetiser(mtu=1400)
|
||||
packets = p.packetise([SPS, PPS, big_nal(4000)])
|
||||
markers = [unpack(pkt)[4] for pkt in packets]
|
||||
assert markers == [False, False, False, True]
|
||||
|
||||
|
||||
def test_the_timestamp_is_one_per_access_unit_and_advances_between_them():
|
||||
p = rtp.Packetiser(mtu=1400, timestamp=1000)
|
||||
first = p.packetise([SPS, PPS, big_nal(4000)], advance=3000)
|
||||
second = p.packetise([SLICE], advance=3000)
|
||||
assert {unpack(pkt)[2] for pkt in first} == {1000}
|
||||
assert {unpack(pkt)[2] for pkt in second} == {4000}
|
||||
|
||||
|
||||
def test_the_timestamp_wraps_at_thirty_two_bits():
|
||||
p = rtp.Packetiser(timestamp=(1 << 32) - 1000)
|
||||
p.packetise([IDR], advance=3000)
|
||||
assert p.timestamp == 2000
|
||||
|
||||
|
||||
def test_the_ssrc_is_the_same_on_every_packet():
|
||||
p = rtp.Packetiser(mtu=1400)
|
||||
packets = p.packetise([SPS, PPS, big_nal(4000)]) + p.packetise([SLICE])
|
||||
assert len({unpack(pkt)[3] for pkt in packets}) == 1
|
||||
|
||||
|
||||
# -- start offsets -------------------------------------------------------
|
||||
def test_no_offset_starts_at_the_beginning():
|
||||
assert rtp.resolve_start_offset(None, 10.0, random.Random(0)) == 0.0
|
||||
|
||||
|
||||
def test_a_random_offset_lands_inside_the_stream_and_repeats_with_a_seed():
|
||||
first = rtp.resolve_start_offset("random", 10.0, random.Random(5))
|
||||
second = rtp.resolve_start_offset("random", 10.0, random.Random(5))
|
||||
assert first == second
|
||||
assert 0.0 <= first < 10.0
|
||||
|
||||
|
||||
def test_an_offset_past_the_end_wraps_rather_than_falling_off():
|
||||
assert rtp.resolve_start_offset(12.5, 10.0, random.Random(0)) == 2.5
|
||||
|
||||
|
||||
def test_a_nonsense_offset_is_an_error():
|
||||
with pytest.raises(ValueError, match="'random'"):
|
||||
rtp.resolve_start_offset("middle", 10.0, random.Random(0))
|
||||
with pytest.raises(ValueError, match="negative"):
|
||||
rtp.resolve_start_offset(-1.0, 10.0, random.Random(0))
|
||||
|
||||
|
||||
# -- the packetised source ----------------------------------------------
|
||||
@pytest.fixture
|
||||
def stream():
|
||||
# Ten pictures, parameter sets only at the front, which is the case that
|
||||
# makes repetition matter.
|
||||
nals = [SPS, PPS, IDR] + [SLICE] * 9
|
||||
return rtp.AnnexBStream.from_bytes(annex_b(*nals))
|
||||
|
||||
|
||||
def take(source, n):
|
||||
out = []
|
||||
for item in source.packets():
|
||||
out.append(item)
|
||||
if len(out) >= n:
|
||||
break
|
||||
return out
|
||||
|
||||
|
||||
def test_the_source_paces_one_period_per_frame_and_nothing_between_packets(stream):
|
||||
source = rtp.PacketisedSource(stream, fps=30.0)
|
||||
delays = [delay for delay, _ in take(source, 6)]
|
||||
assert delays[0] == pytest.approx(1 / 30)
|
||||
assert delays[1] == 0.0 # the second packet of the same frame
|
||||
assert sum(1 for d in delays if d > 0) == len([d for d in delays if d > 0])
|
||||
|
||||
|
||||
def test_the_source_loops_without_the_clock_going_backwards(stream):
|
||||
source = rtp.PacketisedSource(stream, fps=30.0, parameter_set_period=0)
|
||||
packets = [data for _delay, data in take(source, 40)]
|
||||
stamps = [unpack(p)[2] for p in packets]
|
||||
assert stamps == sorted(stamps), "a timestamp that goes back makes a decoder give up"
|
||||
seqs = [unpack(p)[1] for p in packets]
|
||||
assert seqs == list(range(seqs[0], seqs[0] + len(seqs)))
|
||||
assert len(packets) > len(stream.units), "it must have passed the loop point"
|
||||
|
||||
|
||||
def test_parameter_sets_recur_so_a_viewer_can_join_late(stream):
|
||||
source = rtp.PacketisedSource(stream, fps=30.0, parameter_set_period=4)
|
||||
payloads = [unpack(data)[5] for _delay, data in take(source, 30)]
|
||||
stap_a = [p for p in payloads if p[0] & 0x1F == rtp.NAL_STAP_A]
|
||||
# One at the file's own head plus one every fourth frame after it.
|
||||
assert len(stap_a) >= 5
|
||||
assert all(SPS in p and PPS in p for p in stap_a)
|
||||
|
||||
|
||||
def test_turning_repetition_off_sends_the_parameter_sets_once_per_pass(stream):
|
||||
source = rtp.PacketisedSource(stream, fps=30.0, parameter_set_period=0)
|
||||
payloads = [unpack(data)[5] for _delay, data in take(source, 25)]
|
||||
assert sum(1 for p in payloads if p[0] & 0x1F == rtp.NAL_STAP_A) == 3
|
||||
|
||||
|
||||
def test_a_random_start_offset_begins_mid_stream_and_is_reproducible(stream):
|
||||
a = rtp.PacketisedSource(stream, fps=30.0, start_offset="random", seed=11)
|
||||
b = rtp.PacketisedSource(stream, fps=30.0, start_offset="random", seed=11)
|
||||
assert a.start_index == b.start_index
|
||||
assert 0 <= a.start_index < len(stream.units)
|
||||
assert rtp.PacketisedSource(stream, fps=30.0, start_offset=0.1).start_index == 3
|
||||
|
||||
|
||||
def test_a_source_needs_a_positive_frame_rate(stream):
|
||||
with pytest.raises(ValueError, match="fps"):
|
||||
rtp.PacketisedSource(stream, fps=0)
|
||||
|
||||
|
||||
# -- the capture format --------------------------------------------------
|
||||
def rtp_packet(seq, timestamp, ssrc=0xAABBCCDD, payload=b"\x41\x9a\xff"):
|
||||
return struct.pack(">BBHII", 0x80, 96, seq, timestamp, ssrc) + payload
|
||||
|
||||
|
||||
def test_a_capture_round_trips(tmp_path):
|
||||
path = tmp_path / "c.rtpcap"
|
||||
packets = [(0.0, rtp_packet(1, 9000)), (0.033, rtp_packet(2, 12000))]
|
||||
assert capture.write_capture(path, packets) == 2
|
||||
assert list(capture.read_capture(path)) == packets
|
||||
|
||||
|
||||
def test_a_capture_creates_its_parent_directory(tmp_path):
|
||||
path = tmp_path / "deep" / "c.rtpcap"
|
||||
capture.write_capture(path, [(0.0, rtp_packet(1, 0))])
|
||||
assert path.is_file()
|
||||
|
||||
|
||||
def test_something_that_is_not_a_capture_is_rejected(tmp_path):
|
||||
path = tmp_path / "nope.rtpcap"
|
||||
path.write_bytes(b"not a capture at all")
|
||||
with pytest.raises(capture.CaptureFormatError, match="does not start"):
|
||||
list(capture.read_capture(path))
|
||||
short = tmp_path / "short.rtpcap"
|
||||
short.write_bytes(b"BEBO")
|
||||
with pytest.raises(capture.CaptureFormatError, match="too short"):
|
||||
list(capture.read_capture(short))
|
||||
|
||||
|
||||
def test_a_future_capture_version_is_refused_rather_than_misread(tmp_path):
|
||||
path = tmp_path / "future.rtpcap"
|
||||
path.write_bytes(struct.pack("<8sHH", capture.MAGIC, 99, 0))
|
||||
with pytest.raises(capture.CaptureFormatError, match="version 99"):
|
||||
list(capture.read_capture(path))
|
||||
|
||||
|
||||
def test_a_capture_cut_short_still_reads_up_to_the_break(tmp_path, caplog):
|
||||
path = tmp_path / "cut.rtpcap"
|
||||
capture.write_capture(path, [(0.0, rtp_packet(1, 0)), (0.033, rtp_packet(2, 3000))])
|
||||
path.write_bytes(path.read_bytes()[:-5]) # a Ctrl-C mid-write
|
||||
got = list(capture.read_capture(path))
|
||||
assert len(got) == 1
|
||||
assert "mid-packet" in caplog.text
|
||||
|
||||
|
||||
# -- replay --------------------------------------------------------------
|
||||
@pytest.fixture
|
||||
def capture_file(tmp_path):
|
||||
"""Three frames of two packets each, with a gap only between frames."""
|
||||
packets = []
|
||||
for frame in range(3):
|
||||
at = frame * 0.04
|
||||
packets.append((at, rtp_packet(frame * 2, 9000 + frame * 3000)))
|
||||
packets.append((at + 0.001, rtp_packet(frame * 2 + 1, 9000 + frame * 3000)))
|
||||
path = tmp_path / "flight.rtpcap"
|
||||
capture.write_capture(path, packets)
|
||||
return path
|
||||
|
||||
|
||||
def test_a_replay_reproduces_the_recorded_gaps(capture_file):
|
||||
source = capture.ReplaySource.from_path(capture_file)
|
||||
delays = [delay for delay, _ in take(source, 6)]
|
||||
assert delays[0] == 0.0
|
||||
assert delays[1] == pytest.approx(0.001)
|
||||
assert delays[2] == pytest.approx(0.039)
|
||||
assert delays[3] == pytest.approx(0.001)
|
||||
|
||||
|
||||
def test_a_replay_restamps_the_ssrc_and_renumbers_continuously(capture_file):
|
||||
source = capture.ReplaySource.from_path(capture_file, seed=3)
|
||||
packets = [data for _delay, data in take(source, 10)]
|
||||
assert {unpack(p)[3] for p in packets} == {source.ssrc}
|
||||
assert source.ssrc != 0xAABBCCDD
|
||||
seqs = [unpack(p)[1] for p in packets]
|
||||
assert seqs == list(range(seqs[0], seqs[0] + len(seqs)))
|
||||
|
||||
|
||||
def test_a_replay_keeps_the_recorded_timestamp_deltas(capture_file):
|
||||
source = capture.ReplaySource.from_path(capture_file)
|
||||
stamps = [unpack(data)[2] for _delay, data in take(source, 6)]
|
||||
# Two packets per frame share a timestamp; frames are 3000 ticks apart.
|
||||
assert [b - a for a, b in pairwise(stamps)] == [0, 3000, 0, 3000, 0]
|
||||
|
||||
|
||||
def test_a_replay_carries_the_clock_past_the_loop_point(capture_file):
|
||||
source = capture.ReplaySource.from_path(capture_file)
|
||||
items = take(source, 14) # six recorded packets, then round again
|
||||
stamps = [unpack(data)[2] for _delay, data in items]
|
||||
assert stamps == sorted(stamps), "looping must not rewind the clock"
|
||||
# The gap the file cannot know: filled with its own median, one frame.
|
||||
assert items[6][0] == pytest.approx(0.039)
|
||||
assert stamps[6] - stamps[5] == 3000
|
||||
|
||||
|
||||
def test_a_replay_can_start_part_way_in(capture_file):
|
||||
source = capture.ReplaySource.from_path(capture_file, start_offset=0.04)
|
||||
assert source.start_index == 2
|
||||
assert capture.ReplaySource.from_path(capture_file, start_offset="random", seed=2).start_index >= 0
|
||||
|
||||
|
||||
def test_an_empty_capture_is_an_error_not_a_silent_stall(tmp_path):
|
||||
path = tmp_path / "empty.rtpcap"
|
||||
capture.write_capture(path, [])
|
||||
with pytest.raises(capture.CaptureFormatError, match="no packets"):
|
||||
capture.ReplaySource.from_path(path)
|
||||
|
||||
|
||||
def test_the_recorder_writes_what_arrives_on_the_port_it_binds(tmp_path):
|
||||
import socket
|
||||
import threading
|
||||
|
||||
path = tmp_path / "live.rtpcap"
|
||||
port = 0
|
||||
probe = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||
probe.bind(("127.0.0.1", 0))
|
||||
port = probe.getsockname()[1]
|
||||
probe.close()
|
||||
|
||||
done = threading.Event()
|
||||
count = []
|
||||
|
||||
def run():
|
||||
count.append(capture.record(path, port=port, host="127.0.0.1", max_packets=2))
|
||||
done.set()
|
||||
|
||||
thread = threading.Thread(target=run, daemon=True)
|
||||
thread.start()
|
||||
sender = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||
for i in range(2):
|
||||
for _attempt in range(20):
|
||||
sender.sendto(rtp_packet(i, i * 3000), ("127.0.0.1", port))
|
||||
if done.wait(0.05) or len(count) or i == 0:
|
||||
break
|
||||
done.wait(5)
|
||||
sender.close()
|
||||
assert count == [2]
|
||||
got = list(capture.read_capture(path))
|
||||
assert len(got) == 2
|
||||
assert got[0][0] == 0.0 # times are relative to the first packet
|
||||
+31
-2
@@ -74,6 +74,16 @@ def sim():
|
||||
yield fake
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def sim_factory():
|
||||
"""For tests that need the refusing simulator rather than the default.
|
||||
|
||||
The default models the aircraft: a second controller is accepted and the
|
||||
first is starved. Refusal is the opt-in mode.
|
||||
"""
|
||||
return FakeBebop
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def controller(sim):
|
||||
client = Controller(sim)
|
||||
@@ -165,15 +175,34 @@ def test_handshake_is_accepted_and_names_a_c2d_port(sim, controller):
|
||||
assert sim.handshakes[0]["controller_name"] == "test"
|
||||
|
||||
|
||||
def test_a_second_controller_is_refused(sim, controller):
|
||||
def test_a_second_controller_is_accepted_by_default(sim, controller):
|
||||
"""Matching the aircraft, which accepts a newcomer and starves the first.
|
||||
|
||||
Tested on the real drone 2026-10-02. The refusal this test used to assert
|
||||
was folklore inherited from pyparrot's error text.
|
||||
"""
|
||||
second = Controller(sim)
|
||||
try:
|
||||
assert second.reply["status"] == 1
|
||||
assert second.reply["status"] == 0
|
||||
finally:
|
||||
second.close()
|
||||
|
||||
|
||||
def test_a_second_controller_can_be_refused_when_asked(sim_factory):
|
||||
"""Kept because a client must still handle a non-zero status."""
|
||||
with sim_factory(single_controller=True) as strict:
|
||||
first = Controller(strict)
|
||||
second = Controller(strict)
|
||||
try:
|
||||
assert first.reply["status"] == 0
|
||||
assert second.reply["status"] == 1
|
||||
finally:
|
||||
second.close()
|
||||
first.close()
|
||||
|
||||
|
||||
def test_the_slot_is_free_again_after_release(sim, controller):
|
||||
"""Only meaningful in the strict mode; the default never withholds a slot."""
|
||||
assert sim.occupied
|
||||
sim.release()
|
||||
third = Controller(sim)
|
||||
|
||||
@@ -0,0 +1,490 @@
|
||||
"""The simulator's video path, end to end over real sockets.
|
||||
|
||||
The assertion that counts is at the bottom: ffmpeg is pointed at the
|
||||
simulator and has to produce frames at 856x480. Everything above it checks
|
||||
behaviour the decode test cannot distinguish, such as whether the stream
|
||||
actually stops when told to.
|
||||
|
||||
Synthetic NALs for everything except the decode, so the suite still runs on a
|
||||
machine without ffmpeg. The clip for the decode is generated at setup and
|
||||
never committed: a video file in the repository would be a 2 MB answer to a
|
||||
question ffmpeg answers in three seconds.
|
||||
"""
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import socket
|
||||
import struct
|
||||
import subprocess
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
from PIL import Image
|
||||
|
||||
from mcbebop.arsdk.types import COMMAND_HEADER, BufferId, DataType, Frame
|
||||
from mcbebop.media import rtp, video
|
||||
from mcbebop.sim import FakeBebop, load_specs, video_source_for
|
||||
|
||||
SPECS = load_specs()
|
||||
VIDEO_ENABLE_CHANGED = SPECS["ardrone3.MediaStreamingState.VideoEnableChanged"].ids
|
||||
|
||||
WIDTH, HEIGHT, FPS = 856, 480, 30
|
||||
|
||||
|
||||
def free_port() -> int:
|
||||
"""A port nothing holds, for something else to bind in a moment.
|
||||
|
||||
Racy in principle. The alternative is the real 55004, which is worse:
|
||||
one left-behind ffmpeg and every run of this file fails.
|
||||
"""
|
||||
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||
sock.bind(("127.0.0.1", 0))
|
||||
port = sock.getsockname()[1]
|
||||
sock.close()
|
||||
return port
|
||||
|
||||
|
||||
class Controller:
|
||||
"""A hand-rolled controller that also binds its own video port."""
|
||||
|
||||
def __init__(self, sim: FakeBebop, *, stream_port: int | None = None) -> None:
|
||||
self.udp = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||
self.udp.bind(("127.0.0.1", 0))
|
||||
self.udp.settimeout(0.3)
|
||||
self.rtp: socket.socket | None = None
|
||||
if stream_port is None:
|
||||
self.rtp = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||
self.rtp.bind(("127.0.0.1", 0))
|
||||
self.rtp.settimeout(0.3)
|
||||
stream_port = self.rtp.getsockname()[1]
|
||||
self.stream_port = stream_port
|
||||
request = {
|
||||
"d2c_port": self.udp.getsockname()[1],
|
||||
"controller_type": "computer",
|
||||
"controller_name": "video-test",
|
||||
"arstream2_client_stream_port": stream_port,
|
||||
"arstream2_client_control_port": stream_port + 1,
|
||||
}
|
||||
with socket.create_connection(("127.0.0.1", sim.discovery_port), timeout=2) as tcp:
|
||||
tcp.sendall(json.dumps(request).encode())
|
||||
raw = b""
|
||||
while b"\x00" not in raw:
|
||||
raw += tcp.recv(4096)
|
||||
self.reply = json.loads(raw.split(b"\x00")[0].decode())
|
||||
self.c2d = ("127.0.0.1", self.reply["c2d_port"])
|
||||
self._seq = 0
|
||||
|
||||
def video_enable(self, on: bool) -> None:
|
||||
self._seq += 1
|
||||
payload = COMMAND_HEADER.pack(1, 21, 0) + bytes([1 if on else 0])
|
||||
frame = Frame(DataType.DATA_WITH_ACK, BufferId.C2D_ACK, self._seq, payload)
|
||||
self.udp.sendto(frame.encode(), self.c2d)
|
||||
|
||||
def rtp_packets(self, seconds: float = 1.0) -> list[bytes]:
|
||||
assert self.rtp is not None
|
||||
out: list[bytes] = []
|
||||
deadline = time.monotonic() + seconds
|
||||
while time.monotonic() < deadline:
|
||||
try:
|
||||
out.append(self.rtp.recv(65535))
|
||||
except TimeoutError:
|
||||
continue
|
||||
return out
|
||||
|
||||
def wait_for_rtp(self, seconds: float = 3.0) -> list[bytes]:
|
||||
deadline = time.monotonic() + seconds
|
||||
while time.monotonic() < deadline:
|
||||
got = self.rtp_packets(0.3)
|
||||
if got:
|
||||
return got
|
||||
pytest.fail("no RTP arrived on the port the handshake named")
|
||||
|
||||
def events(self, seconds: float = 1.0) -> list[tuple[tuple[int, int, int], bytes]]:
|
||||
out = []
|
||||
deadline = time.monotonic() + seconds
|
||||
while time.monotonic() < deadline:
|
||||
try:
|
||||
data = self.udp.recv(65535)
|
||||
except TimeoutError:
|
||||
continue
|
||||
for frame in Frame.decode_all(data):
|
||||
if (
|
||||
frame.buffer_id in (BufferId.D2C_ACK, BufferId.D2C_NON_ACK)
|
||||
and len(frame.payload) >= COMMAND_HEADER.size
|
||||
):
|
||||
ids = COMMAND_HEADER.unpack_from(frame.payload)
|
||||
out.append((ids, frame.payload[COMMAND_HEADER.size :]))
|
||||
return out
|
||||
|
||||
def close(self) -> None:
|
||||
self.udp.close()
|
||||
if self.rtp is not None:
|
||||
self.rtp.close()
|
||||
|
||||
|
||||
def decode_argv(sdp: Path, pattern: Path, frames: int = 10) -> list[str]:
|
||||
"""ffmpeg reading our SDP. rtp and udp have to be whitelisted explicitly
|
||||
or ffmpeg refuses the file with an error that reads like a bad path."""
|
||||
return [
|
||||
"ffmpeg", "-y", "-hide_banner", "-loglevel", "error",
|
||||
"-protocol_whitelist", "file,rtp,udp", "-i", str(sdp),
|
||||
"-frames:v", str(frames), "-fps_mode", "passthrough", str(pattern),
|
||||
] # fmt: skip
|
||||
|
||||
|
||||
# -- sources -------------------------------------------------------------
|
||||
SPS = bytes([0x67]) + b"\x42\xc0\x1e"
|
||||
PPS = bytes([0x68]) + b"\xce\x3c\x80"
|
||||
IDR = bytes([0x65, 0x88]) + b"\xaa" * 2000 # big enough to need FU-A
|
||||
SLICE = bytes([0x41, 0x9A]) + b"\xbb" * 400
|
||||
|
||||
|
||||
def synthetic_annex_b(frames: int = 10) -> bytes:
|
||||
nals = [SPS, PPS, IDR] + [SLICE] * (frames - 1)
|
||||
return b"".join(b"\x00\x00\x00\x01" + nal for nal in nals)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fake_source():
|
||||
stream = rtp.AnnexBStream.from_bytes(synthetic_annex_b())
|
||||
return rtp.PacketisedSource(stream, fps=FPS, parameter_set_period=5)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def sim(fake_source):
|
||||
# Port 0 rather than the aircraft's 5004: a receiver binds, so it never
|
||||
# looks at where a packet came from, and a fixed port makes two sims in
|
||||
# one test session fight.
|
||||
with FakeBebop(video_source=fake_source, video_source_port=0) as fake:
|
||||
yield fake
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def controller(sim):
|
||||
client = Controller(sim)
|
||||
yield client
|
||||
client.close()
|
||||
|
||||
|
||||
# -- no video source: unchanged ------------------------------------------
|
||||
def test_a_sim_without_video_has_no_video_thread():
|
||||
with FakeBebop() as plain:
|
||||
assert not [t for t in plain._threads if t.name == "sim-video"]
|
||||
assert plain.video_streaming is False
|
||||
|
||||
|
||||
def test_video_enable_is_answered_even_with_nothing_to_stream():
|
||||
# The event reports the aircraft's state, not its bitrate. A client that
|
||||
# waits for the confirmation must not hang because the sim has no file.
|
||||
with FakeBebop() as plain:
|
||||
client = Controller(plain)
|
||||
try:
|
||||
client.events(0.5) # drain the identity burst
|
||||
client.video_enable(True)
|
||||
enabled = [args for ids, args in client.events(1.0) if ids == VIDEO_ENABLE_CHANGED]
|
||||
assert enabled == [struct.pack("<i", 0)] # the "enabled" member
|
||||
assert plain.video_streaming is False
|
||||
assert plain.video_packets_sent == 0
|
||||
finally:
|
||||
client.close()
|
||||
|
||||
|
||||
def test_a_video_source_that_does_not_exist_fails_at_construction(tmp_path):
|
||||
# Better here than at VideoEnable time, where it would look like a drone
|
||||
# that accepted the command and then quietly sent nothing.
|
||||
with pytest.raises(FileNotFoundError):
|
||||
FakeBebop(video_source=tmp_path / "nope.h264")
|
||||
|
||||
|
||||
def test_the_factory_reads_the_suffix(tmp_path):
|
||||
from mcbebop.media import capture
|
||||
|
||||
h264 = tmp_path / "clip.h264"
|
||||
h264.write_bytes(synthetic_annex_b())
|
||||
assert isinstance(video_source_for(h264), rtp.PacketisedSource)
|
||||
|
||||
cap = tmp_path / "flight.rtpcap"
|
||||
packet = struct.pack(">BBHII", 0x80, 96, 1, 9000, 0x1234) + b"\x41\x9a"
|
||||
capture.write_capture(cap, [(0.0, packet)])
|
||||
assert isinstance(video_source_for(cap), capture.ReplaySource)
|
||||
|
||||
|
||||
# -- with a source -------------------------------------------------------
|
||||
def test_nothing_streams_until_video_enable_arrives(sim, controller):
|
||||
assert controller.rtp_packets(0.5) == []
|
||||
assert sim.video_packets_sent == 0
|
||||
|
||||
|
||||
def test_video_enable_starts_rtp_on_the_port_the_handshake_named(sim, controller):
|
||||
assert controller.reply["arstream2_server_stream_port"] == 5004
|
||||
controller.video_enable(True)
|
||||
packets = controller.wait_for_rtp()
|
||||
for packet in packets:
|
||||
payload_type, _seq, _ts, _ssrc, _marker, body = rtp.parse_packet(packet)
|
||||
assert payload_type == 96
|
||||
assert body
|
||||
assert any(rtp.parse_packet(p)[4] for p in packets), "no frame was ever marked complete"
|
||||
assert sim.video_streaming is True
|
||||
|
||||
|
||||
def test_the_stream_reports_itself_enabled_then_disabled(sim, controller):
|
||||
controller.events(0.4)
|
||||
controller.video_enable(True)
|
||||
controller.wait_for_rtp()
|
||||
controller.video_enable(False)
|
||||
reported = [args for ids, args in controller.events(1.0) if ids == VIDEO_ENABLE_CHANGED]
|
||||
assert struct.pack("<i", 1) in reported # "disabled"
|
||||
|
||||
|
||||
def test_video_enable_zero_stops_the_stream(sim, controller):
|
||||
controller.video_enable(True)
|
||||
controller.wait_for_rtp()
|
||||
controller.video_enable(False)
|
||||
deadline = time.monotonic() + 2.0
|
||||
while time.monotonic() < deadline and sim.video_streaming:
|
||||
time.sleep(0.05)
|
||||
assert sim.video_streaming is False
|
||||
controller.rtp_packets(0.4) # drain whatever was already in flight
|
||||
assert controller.rtp_packets(0.5) == []
|
||||
|
||||
|
||||
def test_losing_the_controller_stops_the_stream(sim, controller):
|
||||
# The aircraft's stream dies with the link, which a viewer has to survive.
|
||||
controller.video_enable(True)
|
||||
controller.wait_for_rtp()
|
||||
sim.release()
|
||||
deadline = time.monotonic() + 2.0
|
||||
while time.monotonic() < deadline and sim.video_streaming:
|
||||
time.sleep(0.05)
|
||||
controller.rtp_packets(0.4)
|
||||
assert controller.rtp_packets(0.5) == []
|
||||
|
||||
|
||||
def test_the_clock_and_the_sequence_carry_across_a_stop_and_restart(sim, controller):
|
||||
controller.video_enable(True)
|
||||
first = controller.wait_for_rtp()
|
||||
controller.video_enable(False)
|
||||
time.sleep(0.3)
|
||||
controller.rtp_packets(0.3)
|
||||
controller.video_enable(True)
|
||||
second = controller.wait_for_rtp()
|
||||
|
||||
last = rtp.parse_packet(first[-1])
|
||||
resumed = rtp.parse_packet(second[0])
|
||||
assert ((resumed[1] - last[1]) & 0xFFFF) < 1000, "the sequence number restarted"
|
||||
assert ((resumed[2] - last[2]) % (1 << 32)) < 90_000, "the clock restarted"
|
||||
assert resumed[3] == last[3], "the SSRC changed mid-session"
|
||||
|
||||
|
||||
def test_shutdown_stops_the_stream_rather_than_leaking_a_thread(fake_source):
|
||||
fake = FakeBebop(video_source=fake_source, video_source_port=0)
|
||||
fake.start()
|
||||
client = Controller(fake)
|
||||
try:
|
||||
client.video_enable(True)
|
||||
client.wait_for_rtp()
|
||||
finally:
|
||||
client.close()
|
||||
fake.stop()
|
||||
assert not any(t.is_alive() for t in fake._threads)
|
||||
|
||||
|
||||
def test_the_stream_is_paced_in_real_time_rather_than_blasted(sim, controller):
|
||||
# Counting marker bits rather than packets: one per access unit, so this
|
||||
# measures frames per second directly and does not move when the
|
||||
# packetisation of the test clip changes. Blasting the file would show
|
||||
# thousands.
|
||||
controller.video_enable(True)
|
||||
controller.wait_for_rtp()
|
||||
frames = sum(1 for p in controller.rtp_packets(1.0) if rtp.parse_packet(p)[4])
|
||||
assert 20 < frames < 45, f"{frames} frames in a second, asked for {FPS}"
|
||||
|
||||
|
||||
def test_a_random_start_offset_reaches_the_sim(tmp_path):
|
||||
path = tmp_path / "clip.h264"
|
||||
path.write_bytes(synthetic_annex_b(frames=40))
|
||||
first = FakeBebop(video_source=path, video_start_offset="random", video_seed=4, video_source_port=0)
|
||||
second = FakeBebop(video_source=path, video_start_offset="random", video_seed=4, video_source_port=0)
|
||||
try:
|
||||
assert first._video.start_index == second._video.start_index
|
||||
assert first._video.start_index > 0
|
||||
finally:
|
||||
first.stop()
|
||||
second.stop()
|
||||
|
||||
|
||||
# -- the test that proves it ---------------------------------------------
|
||||
@pytest.fixture(scope="session")
|
||||
def clip(tmp_path_factory):
|
||||
"""Three seconds of H.264 at the resolution the aircraft streams.
|
||||
|
||||
Generated, not committed. `-f h264` already writes Annex-B, so no
|
||||
bitstream filter is needed: `h264_mp4toannexb` is for the other
|
||||
direction and ffmpeg rejects it on an input that is already Annex-B.
|
||||
"""
|
||||
if shutil.which("ffmpeg") is None:
|
||||
pytest.skip("no ffmpeg")
|
||||
out = tmp_path_factory.mktemp("clip") / "testsrc.h264"
|
||||
argv = [
|
||||
"ffmpeg", "-y", "-hide_banner", "-loglevel", "error",
|
||||
"-f", "lavfi", "-i", f"testsrc=size={WIDTH}x{HEIGHT}:rate={FPS}",
|
||||
"-t", "3", "-c:v", "libx264", "-preset", "ultrafast", "-pix_fmt", "yuv420p",
|
||||
"-g", "15", "-f", "h264", str(out),
|
||||
] # fmt: skip
|
||||
subprocess.run(
|
||||
argv,
|
||||
check=True,
|
||||
capture_output=True,
|
||||
timeout=120,
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
@pytest.mark.skipif(shutil.which("ffmpeg") is None, reason="needs ffmpeg to decode")
|
||||
def test_ffmpeg_decodes_the_simulated_stream_at_the_right_size(clip, tmp_path):
|
||||
"""If this fails the feature does not work, whatever the unit tests say."""
|
||||
port = free_port()
|
||||
sdp = tmp_path / "sim.sdp"
|
||||
sdp.write_text(video.sdp_text(port=port))
|
||||
pattern = tmp_path / "frame%03d.png"
|
||||
|
||||
with FakeBebop(video_source=clip, video_fps=FPS, video_source_port=0) as sim:
|
||||
client = Controller(sim, stream_port=port)
|
||||
# ffmpeg binds before the stream starts: RTP is connectionless, so
|
||||
# anything sent before it is listening is simply gone.
|
||||
proc = subprocess.Popen(
|
||||
decode_argv(sdp, pattern),
|
||||
stderr=subprocess.PIPE,
|
||||
)
|
||||
try:
|
||||
time.sleep(1.0) # let it bind and read the SDP
|
||||
client.video_enable(True)
|
||||
_out, err = proc.communicate(timeout=60)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
_out, err = proc.communicate()
|
||||
finally:
|
||||
client.close()
|
||||
|
||||
frames = sorted(tmp_path.glob("frame*.png"))
|
||||
assert len(frames) >= 5, f"ffmpeg decoded {len(frames)} frames; stderr was {err.decode()[-2000:]}"
|
||||
for frame in frames:
|
||||
with Image.open(frame) as im:
|
||||
assert im.size == (WIDTH, HEIGHT)
|
||||
assert sim.video_packets_sent > len(frames)
|
||||
|
||||
|
||||
@pytest.mark.skipif(shutil.which("ffmpeg") is None, reason="needs ffmpeg to decode")
|
||||
def test_ffmpeg_can_join_a_stream_already_in_progress(clip, tmp_path):
|
||||
"""The customer's actual case: goggles switched on mid-flight.
|
||||
|
||||
Nothing but the repeated parameter sets makes this work. A stream that
|
||||
sent its SPS and PPS once at the start would leave a decoder that joined
|
||||
later with no way to size a frame, and it would never recover.
|
||||
"""
|
||||
port = free_port()
|
||||
sdp = tmp_path / "late.sdp"
|
||||
sdp.write_text(video.sdp_text(port=port))
|
||||
pattern = tmp_path / "late%03d.png"
|
||||
|
||||
with FakeBebop(
|
||||
video_source=clip,
|
||||
video_fps=FPS,
|
||||
video_source_port=0,
|
||||
video_start_offset="random",
|
||||
video_seed=19,
|
||||
) as sim:
|
||||
client = Controller(sim, stream_port=port)
|
||||
client.video_enable(True)
|
||||
time.sleep(2.0) # the drone has been flying a while
|
||||
proc = subprocess.Popen(
|
||||
decode_argv(sdp, pattern),
|
||||
stderr=subprocess.PIPE,
|
||||
)
|
||||
try:
|
||||
_out, err = proc.communicate(timeout=60)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
_out, err = proc.communicate()
|
||||
finally:
|
||||
client.close()
|
||||
|
||||
frames = sorted(tmp_path.glob("late*.png"))
|
||||
assert len(frames) >= 5, f"joining late decoded {len(frames)}; stderr was {err.decode()[-2000:]}"
|
||||
with Image.open(frames[0]) as im:
|
||||
assert im.size == (WIDTH, HEIGHT)
|
||||
|
||||
|
||||
@pytest.mark.skipif(shutil.which("ffmpeg") is None, reason="needs ffmpeg to decode")
|
||||
def test_a_capture_of_our_own_stream_replays_and_still_decodes(clip, tmp_path):
|
||||
"""Proves the replay path on a capture we can actually make.
|
||||
|
||||
A capture off the aircraft would be better and we have none, so this
|
||||
records the simulator's own output instead. It exercises the file format,
|
||||
the restamping and the pacing; what it cannot exercise is the aircraft's
|
||||
bursts, which is the whole reason the replay path exists.
|
||||
"""
|
||||
from mcbebop.media import capture as cap
|
||||
|
||||
record_port = free_port()
|
||||
recorded: list[tuple[float, bytes]] = []
|
||||
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
|
||||
sock.bind(("127.0.0.1", record_port))
|
||||
sock.settimeout(0.3)
|
||||
|
||||
with FakeBebop(video_source=clip, video_fps=FPS, video_source_port=0) as sim:
|
||||
client = Controller(sim, stream_port=record_port)
|
||||
client.video_enable(True)
|
||||
first = None
|
||||
deadline = time.monotonic() + 6.0
|
||||
while time.monotonic() < deadline and len(recorded) < 400:
|
||||
try:
|
||||
data = sock.recv(65535)
|
||||
except TimeoutError:
|
||||
continue
|
||||
now = time.monotonic()
|
||||
first = now if first is None else first
|
||||
recorded.append((now - first, data))
|
||||
client.close()
|
||||
sock.close()
|
||||
assert len(recorded) > 50, "nothing to replay"
|
||||
|
||||
path = tmp_path / "own.rtpcap"
|
||||
cap.write_capture(path, recorded)
|
||||
|
||||
port = free_port()
|
||||
sdp = tmp_path / "replay.sdp"
|
||||
sdp.write_text(video.sdp_text(port=port))
|
||||
pattern = tmp_path / "replay%03d.png"
|
||||
|
||||
with FakeBebop(video_source=path, video_source_port=0) as replay:
|
||||
client = Controller(replay, stream_port=port)
|
||||
proc = subprocess.Popen(
|
||||
decode_argv(sdp, pattern),
|
||||
stderr=subprocess.PIPE,
|
||||
)
|
||||
try:
|
||||
time.sleep(1.0)
|
||||
client.video_enable(True)
|
||||
_out, err = proc.communicate(timeout=60)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
_out, err = proc.communicate()
|
||||
finally:
|
||||
client.close()
|
||||
|
||||
frames = sorted(tmp_path.glob("replay*.png"))
|
||||
assert len(frames) >= 5, f"the replay decoded {len(frames)}; stderr was {err.decode()[-2000:]}"
|
||||
with Image.open(frames[0]) as im:
|
||||
assert im.size == (WIDTH, HEIGHT)
|
||||
|
||||
|
||||
def test_nothing_in_the_repository_is_a_video_file():
|
||||
# The clip above is generated at setup for exactly this reason.
|
||||
root = Path(__file__).resolve().parents[1]
|
||||
tracked = subprocess.run(
|
||||
["git", "-C", str(root), "ls-files"], capture_output=True, text=True, check=True
|
||||
).stdout.split()
|
||||
bad = [f for f in tracked if f.endswith((".h264", ".264", ".mp4", ".rtpcap", ".ts"))]
|
||||
assert bad == []
|
||||
+1
-1
@@ -32,7 +32,7 @@ async def test_every_tool_is_registered(client):
|
||||
assert names == {
|
||||
"connect", "disconnect", "connection_status", "list_commands", "command_info",
|
||||
"send_command", "get_state", "watch_state", "preflight_check", "camera_snapshot",
|
||||
"camera_record", "list_files", "fetch_file", "shell_read", "arm", "disarm",
|
||||
"camera_record", "list_files", "fetch_file", "shell_read", "read_log", "arm", "disarm",
|
||||
} # fmt: skip
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,276 @@
|
||||
"""The ulogcat parser: framing, the optional fields, and what a live log does to it.
|
||||
|
||||
Frames are built here from the format rather than copied from the aircraft,
|
||||
because the aircraft's log is operational data. The one test that reads the
|
||||
real capture is skipped wherever that capture is not present, which is
|
||||
everywhere but the machine it was fetched on.
|
||||
"""
|
||||
|
||||
import struct
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from mcbebop.files import ulog
|
||||
|
||||
SAMPLE = Path(__file__).resolve().parents[1] / "captures" / "logs" / "ckcm.bin"
|
||||
|
||||
|
||||
def pstr(text: str) -> bytes:
|
||||
raw = text.encode()
|
||||
assert len(raw) < 256
|
||||
return bytes([len(raw)]) + raw
|
||||
|
||||
|
||||
def frame(kind: int, body: bytes) -> bytes:
|
||||
return ulog.FRAME_START + bytes([kind]) + body + ulog.FRAME_END
|
||||
|
||||
|
||||
def header(
|
||||
*,
|
||||
priority: str = "I",
|
||||
uptime_us: int = 1_000_000,
|
||||
tid: int = 42,
|
||||
source: str | None = None,
|
||||
tag: str | None = "TEST",
|
||||
colour: bytes | None = None,
|
||||
) -> bytes:
|
||||
flags = ulog.FLAG_DATE | ulog.FLAG_TID
|
||||
if colour is not None:
|
||||
flags |= ulog.FLAG_COLOUR
|
||||
if source is not None:
|
||||
flags |= ulog.FLAG_NAME
|
||||
if tag is not None:
|
||||
flags |= ulog.FLAG_TAG
|
||||
body = bytes([flags, ord(priority)])
|
||||
if colour is not None:
|
||||
body += colour
|
||||
body += struct.pack("<Qi", uptime_us, tid)
|
||||
if source is not None:
|
||||
body += pstr(source)
|
||||
if tag is not None:
|
||||
body += pstr(tag)
|
||||
return frame(ulog.KIND_HEADER, body)
|
||||
|
||||
|
||||
def message(text: str, colour: bytes | None = None) -> bytes:
|
||||
if colour is None:
|
||||
return frame(ulog.KIND_MESSAGE, pstr(text))
|
||||
return frame(ulog.KIND_MESSAGE_COLOURED, colour + bytes([ulog.KIND_MESSAGE]) + pstr(text))
|
||||
|
||||
|
||||
def entry(text: str = "hello", **kw) -> bytes:
|
||||
return header(**kw) + message(text, colour=kw.get("colour"))
|
||||
|
||||
|
||||
# --- the framing -------------------------------------------------------------
|
||||
|
||||
|
||||
def test_an_entry_is_a_header_frame_and_a_message_frame():
|
||||
(record,) = ulog.parse(entry("Machine: Milos board", tag="KERNEL", priority="W"))
|
||||
assert record.tag == "KERNEL"
|
||||
assert record.priority == "W"
|
||||
assert record.level == "warning"
|
||||
assert record.message == "Machine: Milos board"
|
||||
|
||||
|
||||
def test_the_timestamp_is_microseconds_since_boot():
|
||||
(record,) = ulog.parse(entry(uptime_us=2_067_211_079))
|
||||
assert record.uptime_us == 2_067_211_079
|
||||
assert record.uptime == pytest.approx(2067.211079)
|
||||
|
||||
|
||||
def test_both_name_fields_are_optional_and_independent():
|
||||
data = (
|
||||
entry("has both", source="dragon-prog/Behaviour", tag="COMMANDS")
|
||||
+ entry("tag only", source=None, tag="KERNEL")
|
||||
+ entry("source only", source="ephemerisd", tag=None)
|
||||
)
|
||||
records = list(ulog.parse(data))
|
||||
assert [(r.source, r.tag) for r in records] == [
|
||||
("dragon-prog/Behaviour", "COMMANDS"),
|
||||
("", "KERNEL"),
|
||||
("ephemerisd", ""),
|
||||
]
|
||||
|
||||
|
||||
def test_the_thread_id_comes_through():
|
||||
(record,) = ulog.parse(entry(tid=1200))
|
||||
assert record.tid == 1200
|
||||
|
||||
|
||||
def test_a_coloured_entry_keeps_the_headers_colour_and_still_reads_its_text():
|
||||
# The data frame clamps each channel to a minimum of 1, so its copy of the
|
||||
# colour disagrees with the header's for any channel the drone set to 0.
|
||||
# The header's is the true one.
|
||||
data = header(colour=b"\xff\xd0\x00") + message("Switch AWB to AUTO", colour=b"\xff\xd0\x01")
|
||||
(record,) = ulog.parse(data)
|
||||
assert record.colour == b"\xff\xd0\x00"
|
||||
assert record.message == "Switch AWB to AUTO"
|
||||
|
||||
|
||||
def test_an_uncoloured_entry_has_no_colour():
|
||||
(record,) = ulog.parse(entry("plain"))
|
||||
assert record.colour is None
|
||||
|
||||
|
||||
def test_a_message_of_the_maximum_length_round_trips():
|
||||
text = "x" * 255
|
||||
(record,) = ulog.parse(entry(text))
|
||||
assert record.message == text
|
||||
|
||||
|
||||
def test_the_trailing_newline_the_drone_printed_is_dropped_but_inner_ones_stay():
|
||||
(record,) = ulog.parse(entry("first\nsecond\n"))
|
||||
assert record.message == "first\nsecond"
|
||||
|
||||
|
||||
def test_every_entry_reports_where_in_the_file_it_came_from():
|
||||
first = entry("a")
|
||||
records = list(ulog.parse(first + entry("b")))
|
||||
assert [r.offset for r in records] == [0, len(first)]
|
||||
|
||||
|
||||
# --- a log that is still being written ---------------------------------------
|
||||
|
||||
|
||||
def test_a_fetch_that_lands_mid_record_keeps_everything_before_it():
|
||||
data = entry("complete") + entry("cut in half")
|
||||
for cut in range(len(entry("complete")) + 1, len(data)):
|
||||
stats = ulog.ParseStats()
|
||||
records = list(ulog.parse(data[:cut], stats))
|
||||
assert [r.message for r in records] == ["complete"], f"truncated at {cut}"
|
||||
assert stats.truncated or stats.unpaired, f"truncation at {cut} went unreported"
|
||||
|
||||
|
||||
def test_a_truncation_in_the_very_first_record_yields_nothing_and_does_not_raise():
|
||||
stats = ulog.ParseStats()
|
||||
assert list(ulog.parse(entry("x")[:6], stats)) == []
|
||||
assert stats.truncated
|
||||
|
||||
|
||||
def test_a_fetch_that_starts_mid_record_resyncs_on_the_next_marker():
|
||||
data = entry("first") + entry("second")
|
||||
stats = ulog.ParseStats()
|
||||
records = list(ulog.parse(data[5:], stats))
|
||||
assert [r.message for r in records] == ["second"]
|
||||
|
||||
|
||||
def test_garbage_between_entries_costs_one_resync_and_nothing_else():
|
||||
# Nothing is pending here, so an unreadable frame is damage rather than a
|
||||
# binary payload, and reading it as damage is what stops it swallowing the
|
||||
# header that follows.
|
||||
data = entry("before") + ulog.FRAME_START + b"\x99junk" + ulog.FRAME_END + entry("after")
|
||||
stats = ulog.ParseStats()
|
||||
records = list(ulog.parse(data, stats))
|
||||
assert [r.message for r in records] == ["before", "after"]
|
||||
assert stats.resyncs == 1
|
||||
|
||||
|
||||
def test_a_header_with_no_message_after_it_is_counted_not_guessed():
|
||||
stats = ulog.ParseStats()
|
||||
records = list(ulog.parse(header() + entry("real"), stats))
|
||||
assert [r.message for r in records] == ["real"]
|
||||
assert stats.unpaired == 1
|
||||
|
||||
|
||||
def test_a_message_with_no_header_before_it_is_dropped():
|
||||
stats = ulog.ParseStats()
|
||||
records = list(ulog.parse(message("orphan") + entry("real"), stats))
|
||||
assert [r.message for r in records] == ["real"]
|
||||
assert stats.unpaired == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bit", [ulog.FLAG_PC, ulog.FLAG_THREAD_PRIORITY, 0x80])
|
||||
def test_a_header_whose_opt_bits_describe_a_shape_no_source_documents_is_refused(bit):
|
||||
# The renderer never writes PC or thread-priority, so nothing public says
|
||||
# how wide they are. An unread field makes every field after it wrong, so
|
||||
# skipping the record beats reporting a plausible lie.
|
||||
flags = ulog.FLAG_DATE | ulog.FLAG_TID | ulog.FLAG_TAG | bit
|
||||
bad = ulog.FRAME_START + bytes([ulog.KIND_HEADER, flags, ord("I")])
|
||||
bad += struct.pack("<Qi", 1, 1) + pstr("TAG") + ulog.FRAME_END
|
||||
stats = ulog.ParseStats()
|
||||
records = list(ulog.parse(bad + message("orphaned by the refusal") + entry("real"), stats))
|
||||
assert [r.message for r in records] == ["real"]
|
||||
assert stats.resyncs == 1
|
||||
|
||||
|
||||
def test_a_binary_payload_is_kept_rather_than_losing_the_entry():
|
||||
# A ulog_bin entry's data frame has no kind byte and no length, so there
|
||||
# is nothing to decode and nothing to check it against. Keeping the bytes
|
||||
# and saying so beats dropping the entry and its header with it.
|
||||
payload = b"\x00\x01\x02\x03\x04"
|
||||
data = header(tag="BIN") + ulog.FRAME_START + payload + ulog.FRAME_END
|
||||
stats = ulog.ParseStats()
|
||||
(record,) = ulog.parse(data, stats)
|
||||
assert record.binary == payload
|
||||
assert record.tag == "BIN"
|
||||
assert "binary" in record.message
|
||||
assert stats.resyncs == 0
|
||||
|
||||
|
||||
def test_a_buffer_with_no_marker_at_all_is_empty_rather_than_an_error():
|
||||
stats = ulog.ParseStats()
|
||||
assert list(ulog.parse(b"not a ulogcat file at all", stats)) == []
|
||||
assert not stats.truncated
|
||||
|
||||
|
||||
def test_undecodable_text_does_not_stop_the_parse():
|
||||
body = bytes([ulog.FLAG_TAG, ord("I")]) + struct.pack("<QI", 1, 1) + pstr("T")
|
||||
data = frame(ulog.KIND_HEADER, body) + frame(ulog.KIND_MESSAGE, b"\x03\xff\xfe\xfd")
|
||||
(record,) = ulog.parse(data)
|
||||
assert record.message # replacement characters, not an exception
|
||||
|
||||
|
||||
# --- priorities --------------------------------------------------------------
|
||||
|
||||
|
||||
def test_priority_letters_spell_out_to_syslog_levels():
|
||||
assert ulog.PRIORITY_NAMES["E"] == "error"
|
||||
assert ulog.rank("E") < ulog.rank("W") < ulog.rank("I") < ulog.rank("D")
|
||||
|
||||
|
||||
def test_there_is_no_notice_level_because_the_renderer_writes_notice_as_info():
|
||||
# cprio[] in libulogcat_ckcm.c maps both ULOG_NOTICE and ULOG_INFO to 'I',
|
||||
# with a FIXME saying the CKCM tooling does not know NOTICE. Offering a
|
||||
# notice level here would be offering a filter that can never match.
|
||||
assert "N" not in ulog.PRIORITY_NAMES
|
||||
|
||||
|
||||
def test_an_unknown_priority_letter_survives_and_never_filters_out_silently():
|
||||
(record,) = ulog.parse(entry(priority="Z"))
|
||||
assert record.priority == "Z"
|
||||
assert record.level == "Z"
|
||||
assert ulog.rank("Z") == ulog.UNKNOWN_RANK
|
||||
|
||||
|
||||
# --- statistics --------------------------------------------------------------
|
||||
|
||||
|
||||
def test_the_tag_histogram_covers_everything_parsed():
|
||||
data = entry("a", tag="KERNEL") + entry("b", tag="KERNEL") + entry("c", tag="COMMANDS")
|
||||
stats = ulog.ParseStats()
|
||||
list(ulog.parse(data, stats))
|
||||
assert stats.entries == 3
|
||||
assert stats.tags == {"KERNEL": 2, "COMMANDS": 1}
|
||||
|
||||
|
||||
# --- the real thing ----------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.skipif(not SAMPLE.exists(), reason="no captured log on this machine")
|
||||
def test_the_captured_log_parses_to_the_byte():
|
||||
# The framing was derived from this file, so the bar is total: every
|
||||
# header consumed exactly, nothing skipped, no residue. A resync here
|
||||
# would mean a field shape that was missed.
|
||||
stats = ulog.ParseStats()
|
||||
records = list(ulog.parse(SAMPLE.read_bytes(), stats))
|
||||
assert stats.resyncs == 0
|
||||
assert stats.unpaired == 0
|
||||
assert len(records) == stats.entries > 1000
|
||||
assert all(r.priority in ulog.PRIORITY_NAMES for r in records)
|
||||
assert "KERNEL" in stats.tags
|
||||
# Monotonic apart from the kernel stream, which ulogcat merges in from
|
||||
# /proc/kmsg and which can land a few milliseconds out of order.
|
||||
userspace = [r.uptime_us for r in records if r.tag != "KERNEL"]
|
||||
assert userspace == sorted(userspace)
|
||||
Reference in New Issue
Block a user