Stream video from the simulator, gated on VideoEnable as the drone gates it
The sim takes an optional video source. Nothing flows until ardrone3.MediaStreaming.VideoEnable arrives with 1; RTP then goes from the sim's own 5004 to whatever arstream2_client_stream_port the controller named in its handshake, and stops on a 0, on a link loss, or at shutdown. The MediaStreamingState.VideoEnableChanged reply goes out either way, because that reports the aircraft's state and not its bitrate, so a client waiting on the confirmation must not hang for want of a file to stream. Video stays optional: a FakeBebop() with no source gains no thread and sends no packets. One source object for the life of the sim, so sequence numbers and timestamps keep advancing across both the loop point in the file and a disable/enable cycle. A decoder handed a timestamp that went backwards treats the stream as corrupt and stays that way. The test that proves any of this works is the ffmpeg decode: frames come out at 856x480, including when ffmpeg joins a stream that has already been running for two seconds, which is the case a goggle viewer actually faces. The clip is generated at setup and never committed. Also fixed stop() on a sim that was never started: the sockets bind in __post_init__, so it had ports to release, and joining an unstarted thread raised and left them held.
This commit is contained in:
@@ -30,6 +30,53 @@ land an airborne aircraft is the more dangerous answer.
|
||||
`connect(target="sim")` runs everything against a protocol-accurate simulator,
|
||||
which is where anything involving motion should be rehearsed.
|
||||
|
||||
## The simulator streams video
|
||||
|
||||
The simulator can push RTP/H.264 exactly as the aircraft does, so a viewer's
|
||||
whole video path can be developed and measured without a drone. It answers the
|
||||
handshake with `arstream2_server_stream_port: 5004`, sends nothing until
|
||||
`ardrone3.MediaStreaming.VideoEnable` arrives with 1, then streams from its own
|
||||
5004 to whatever `arstream2_client_stream_port` the client named, and stops on a
|
||||
0, on a link loss, or at shutdown.
|
||||
|
||||
```bash
|
||||
python -m mcbebop.sim --video clip.h264 # steady 30 fps
|
||||
python -m mcbebop.sim --video clip.h264 --start-offset random --seed 7
|
||||
python -m mcbebop.sim --video flight.rtpcap # a real capture, replayed
|
||||
```
|
||||
|
||||
`MCBEBOP_SIM_VIDEO_SOURCE=clip.h264` does the same for `connect(target="sim")`.
|
||||
|
||||
Two kinds of source, and they are **different instruments**:
|
||||
|
||||
| Source | Pacing | Use it for |
|
||||
|---|---|---|
|
||||
| `.h264` Annex-B elementary stream | packetised here, steady frame rate | does the decoder work, does the renderer work |
|
||||
| `.rtpcap` capture off the aircraft | the recorded inter-packet gaps, packet for packet | latency and jitter, bursts, loss behaviour |
|
||||
|
||||
Make the first from any video, at the resolution the aircraft streams:
|
||||
|
||||
```bash
|
||||
ffmpeg -i anything.mp4 -t 10 -vf scale=856:480 -r 30 \
|
||||
-c:v libx264 -preset ultrafast -pix_fmt yuv420p -g 30 -f h264 clip.h264
|
||||
```
|
||||
|
||||
`-f h264` already writes Annex-B, so no bitstream filter is wanted;
|
||||
`h264_mp4toannexb` converts the other direction and ffmpeg rejects it here.
|
||||
|
||||
Make the second from a real drone. Start the recorder first, because RTP is
|
||||
connectionless and anything sent before the bind is gone, then enable video
|
||||
from a session that holds the ARSDK link:
|
||||
|
||||
```bash
|
||||
python -m mcbebop.media.capture flight.rtpcap --seconds 30 # binds 55004
|
||||
```
|
||||
|
||||
`--start-offset random` is worth knowing about. It begins mid-GOP, which is
|
||||
what a viewer switched on while the drone is already flying is handed, and
|
||||
`--seed` makes a failure repeatable. Parameter sets repeat about once a second
|
||||
on the packetised path, which is what lets a late joiner recover at all.
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
|
||||
Reference in New Issue
Block a user