Command: record|play|bag

Author

Affiliation

Paolo Bosetti

University of Trento

Abstract

This set of commands allow to record and replay a full MADS session into a .bag file. This is an alternative to the replay.plugin that can replay a session logged to MongoDB.

Warning

This command has been introduced in MADS v2.4.1.

Description

There are three related commands:

• mads record: to record a session
• mads play: to replay a session
• mads bag: to inspect a session recorded as a bag file

TipDo I have to use a bag file or MongoDB?

MADS adopts MongoDB database as official logging facility. The mads source replay.plugin plugin allows live-replay of sessions logged to a MongoDB database:

• Recording a MADS data flow to MongoDB is a powerful solution, which allows thorough ex-post data analysis and manipulation; if you already store your data to MongoDB, the replay.plugin is a quick and effective way to replay the recorded data;
• If you don’t normally log the data flow, and you need some in-flight quick solution for recording, storing and exchanging the data flow, which does not need a database server, mads record may be simpler and quicker. For example, it can prove particularly useful for automated testing.

Record a session

mads-record is an ordinary MADS agent that subscribes per its sub_topic setting (MQTT-style wildcard filters supported, e.g. sensors/# or sensors/+/x – see CONTEXT.md’s “Settings model” section) and writes every message it receives to a bag file: a bespoke binary format that stores the exact multi-part wire frame (timestamp + topic + parts) byte-for-byte, so JSON payloads are never re-parsed/re-serialized and binary blobs are copied exactly once. Both JSON and blob messages round-trip through a bag byte-identical.

The bag format is documented in full in src/bag.hpp. It carries a trailing index and footer for O(1) mads bag info and fast seeking; if mads-record is killed before it can write that footer (e.g. SIGKILL, a crash, a power loss), the file is still a valid, readable bag – mads-play/ mads bag fall back to a linear scan that recovers every complete record written so far.

The administrative control and agent_event topics are never recorded, even under a catch-all sub_topic (e.g. sub_topic = [“”]), mirroring mads-federate’s own rationale for excluding them from automatic relaying: capturing (and later replaying, via mads-play) a remote-control command such as shutdown would be a footgun, not a feature.

mads-record stops on SIGINT/SIGTERM (or after --count messages, if given), finalizing the bag file (writing its index/footer) before exiting.

Options

-o, --output path

Bag file to write. Required. Overwritten if it already exists.

--no-crc

Disable the per-record CRC32 checksum (enabled by default). Slightly reduces file size and write cost; disables corruption detection on read.

--count n

Stop automatically after recording n messages (default: 0, unlimited – runs until stopped).

-x, --cross

Cross-connect sockets (bind instead of connect on the subscribe side). Mainly useful for testing without a broker.

-n, --name name

Agent/section name. mads-record reads its sub_topic from the settings section named after this (default: record), so several recorders with different filters can run from the same mads.ini.

-i, --agent-id id

Agent ID (has no effect on recorded frames themselves – mads-record never publishes; kept for consistency with every other MADS agent executable).

-s, --settings URI

Settings file path or broker URI. Same as every other MADS agent; mads-record reads its sub_topic from the [record] section (or the name given via --name).

--crypto, --keys_dir, --key_broker, --key_client, --auth_verbose

CURVE encryption options, same as every other MADS agent executable.

-v, --version

Show version information.

-h, --help

Show summary of options.

Examples

Record every message on the default settings/broker into a bag file:

mads-record -o session.bag

On the broker’s mads.ini, select what mads-record subscribes to:

[record]
sub_topic = ["sensors/#"]

Record at most 1000 messages and stop automatically:

mads-record -o sample.bag --count 1000

Inspect or replay the result with mads bag info / mads-play:

mads bag info -f session.bag
mads-play -i session.bag

Replay a session

mads-play reads a bag file written by mads-record (see src/bag.hpp for the on-disk format) and republishes every record exactly as it was captured – topic and parts, byte-for-byte – using the raw wire path (Agent::publish_raw_message()), so JSON payloads are never re-serialized and blob bytes are never re-copied through the JSON path. A subscriber on the network cannot tell a replayed message from the original.

If the bag file is missing its trailing index (e.g. mads-record was killed before it could finalize the file), mads-play falls back to a linear scan that recovers every complete record and prints a warning naming how many records will be replayed.

Options

-i, --input path

Bag file to replay. Required.

--restamp

Best-effort rewrite of the timestamp/timecode fields to the current time before republishing, for records whose payload is the legacy, header-less [topic][snappy(json)] frame (the shape Agent::publish() emits whenever the payload ends up Snappy-compressed – unconditionally under compression = “snappy”, or above ~256 bytes under the default compression = “auto”). Every other field, and every other frame shape (an uncompressed small JSON payload carrying a self-describing header, a MsgPack frame, or a blob’s meta+bytes parts), is republished byte-for-byte unchanged. This is a mads-play-only convenience built on top of publish_raw_message(); it does not change what that function does for any other caller.

--topics pattern

MQTT-style topic filter (+ matches one level, # matches this level and below); only records whose stored topic matches at least one given pattern are replayed. Repeatable. Default: replay every topic in the bag.

--rate factor

Paces replay using the gaps between the recorded timestamps of the records actually being replayed (records skipped by --topics don’t contribute a gap), scaled by factor (2.0 replays twice as fast as originally recorded, 0.5 half as fast). Must be > 0. Default: no pacing – records are republished back-to-back as fast as possible.

-x, --cross

Cross-connect sockets (bind instead of connect on the publish side). Mainly useful for testing without a broker.

-n, --name name

Agent/section name (default: play). Only the endpoint/pub_topic placeholder from this section are used – every record is published under its own recorded topic, not pub_topic.

--agent-id id

Agent ID (no short flag: -i is already --input above). Has no effect on replayed frames themselves, which are republished byte-for-byte; kept for consistency with every other MADS agent executable.

-s, --settings URI

Settings file path or broker URI, same as every other MADS agent.

--crypto, --keys_dir, --key_broker, --key_client, --auth_verbose

CURVE encryption options, same as every other MADS agent executable.

-v, --version

Show version information.

-h, --help

Show summary of options.

Examples

Replay a bag exactly as recorded:

mads-play -i session.bag

Replay only a subset of topics, twice as fast as recorded, with fresh timestamps:

mads-play -i session.bag --topics 'sensors/#' --rate 2.0 --restamp

Inspect a bag file

mads-bag is a wrapper around two subcommands for working with bag files recorded by mads-record (see src/bag.hpp for the on-disk format). Like every other mads-* wrapper it is normally invoked as mads bag subcommand …; mads-bag subcommand … works identically.

info

Prints a summary of a bag file: whether its trailing index/footer is present (giving O(1) access) or was recovered via a linear scan, whether the recovered data is a complete prefix (truncated), whether per-record CRC32 is enabled, the record count, and the first/last recorded timestamp. With --topics, it also scans every record to report a per-topic message count.

export

Streams the bag as JSON Lines (one JSON object per record, in order) to stdout or a file. Each line has the shape:

{"index": 0, "timestamp_ns": 1712345678901234567, "timestamp_iso": "2024-...", "topic": "sensors/acc/x", "parts_base64": ["..."]}

Every part is base64-encoded, regardless of whether it holds JSON text or binary blob bytes: a raw wire part (which may be a compressed payload, a self-describing header, or blob bytes) is not guaranteed to be valid UTF-8, so base64 is the only encoding that is always valid JSON Lines output. To inspect a JSON part with jq, decode it explicitly, e.g. jq -r '.parts_base64[0]' file.jsonl | base64 -d | jq .. --format mcap is not implemented – it is intentionally deferred as an optional future export target (see NEW_FEATURES.md’s P3 section); any format other than jsonl is rejected with an error.

Options

info

-f, --file path

Bag file to inspect. Required.

-t, --topics

Include a per-topic message count. Requires a full scan of the bag (still just N sequential record reads, using the index when present – not the slow linear-scan recovery path).

-j, --json

Print the summary as a single JSON object instead of human-readable text.

-h, --help

Show summary of options.

export

-f, --file path

Bag file to export. Required.

--format jsonl

Export format. Only jsonl is currently implemented (also the default).

-o, --output path

Write to path instead of stdout.

-h, --help

Show summary of options.

Examples

Summarize a bag file:

mads bag info -f session.bag

Same, as JSON, with a per-topic breakdown:

mads bag info -f session.bag -j -t

Export to JSON Lines and inspect one message’s payload with jq:

mads bag export -f session.bag -o session.jsonl
jq -r 'select(.topic == "sensors/acc/x") | .parts_base64[0]' session.jsonl | head -1 | base64 -d | jq .