Local Transport and Shared Memory

Author

Affiliation

Paolo Bosetti

University of Trento

Abstract

MADS Version 3.0.0 speeds up the communication between a broker and the agents running on the same computer: agents now talk to the broker through local ipc sockets, and agents that publish large binary payloads (camera frames, point clouds, big arrays) can hand them over through shared memory instead of pushing the bytes through the broker. This guide explains when to use these features, how to configure and monitor them, and how to write plugins that read and write data without copying it.

Important

This guide describe a feature that is introduced by MADS v3.x, and that is not available on previous versions.

Motivation

In a typical MADS network, every message goes from the publishing agent to the broker and from the broker to every subscriber. When all these agents run on the same computer as the broker (a very common setup: a single PC or an embedded board controlling a machine), going through the network stack is a waste, even over the loopback address 127.0.0.1. And when the messages are big (say, a 2 MB camera frame at 30 fps), the cost of copying the same bytes several times (agent → broker → each subscriber, plus compression and decompression) becomes the bottleneck.

MADS 3.0 attacks this with two independent and complementary features:

Feature	What it does	Needs configuration?	Applies to
Local ipc transport	Agents on the broker’s host connect through Unix-domain sockets (files) instead of TCP	No, it is automatic	All messages
Shared-memory data plane	Blobs are written once in a memory area visible to all local agents; only a small reference travels through the broker	Yes, shm = true on the producer	Binary blobs only

You can use the first without the second, but not the other way around: the shared-memory data plane is only used when the broker is on the same host as the producer.

Note

Jargon, in one sentence each. A socket is the endpoint that a program uses to send and receive messages; ZeroMQ is the messaging library MADS is built on. A Unix-domain socket (ipc:// in ZeroMQ) is a socket that is a file on the local disk and never touches the network card. A blob is the binary payload that can be attached to a MADS message next to its JSON part. Shared memory is a region of RAM that several processes can map in their address space at the same time, so that what one writes the others can read without any copy.

Part 1: the local ipc transport

What it is

Starting from v3.0.0, the broker opens its three sockets (frontend, backend and settings) twice: as usual on TCP ports, and also on Unix-domain socket files, stored in a per-user runtime directory. Agents learn about these extra endpoints at startup, by asking the broker with a new transport command on the settings socket. The reply contains the broker’s host token (hostname and machine id), and the path of the socket files.

If the agent finds out that it runs on the same host as the broker, and it can actually see the socket files, it connects through them. Otherwise it silently keeps using TCP. Remote agents are not affected at all.

Advantages and limitations

Advantages

• lower latency and lower CPU usage than loopback TCP, especially with large messages;
• the broker’s ipc sockets get 1 MiB kernel buffers by default (the stock OS default for Unix sockets is tiny, 8 KiB on macOS, and made ipc slower than TCP for messages of 256 KiB and more);
• no configuration is needed: it is on by default;
• TCP clients, including all remote agents, are unaffected.

Limitations

• agent and broker must be on the same host (same operating system instance). Inside containers, they must share the directory where the socket files live;
• the socket files live in an owner-only directory (mode 0700): only processes of the same OS user as the broker can connect;
• the ip_whitelist option of the broker does not apply to ipc peers, since they have no IP address: the permissions of the runtime directory are their access control. CURVE encryption (--crypto), when enabled, applies to ipc exactly as to TCP;
• a socket path must fit in 100 bytes (the sun_path limit of Unix sockets), so keep ipc_dir short;
• on Windows, ipc uses AF_UNIX sockets and needs Windows 10 or later and a libzmq built against the Windows 10 SDK. Otherwise the broker serves TCP only and agents silently fall back;
• service discovery (--room) still advertises the TCP endpoints only.

Configuration

The default (transport = "auto") is almost always what you want. These are the keys, as documented in mads-broker(1) and in the template mads.ini:

Key	Section	Default	Meaning
transport	[agents] or a specific agent’s section	"auto"	"auto": use ipc if the broker is on this host and the socket files are visible, TCP otherwise. "tcp": always use TCP. "ipc": use ipc, and fail at startup if it cannot be honoured
--transport MODE	command line of every agent	(none)	Same values as above. It overrides the agent’s section, which in turn overrides [agents]
ipc	[broker]	true	Also bind the sockets on ipc endpoints. Set it to false to serve TCP only
ipc_dir	[broker]	$XDG_RUNTIME_DIR/mads, else /tmp/mads-<uid>; %LOCALAPPDATA%\mads\run on Windows	Directory of the socket files
frontend_ipc, backend_ipc, settings_ipc	[broker]	derived from ipc_dir	Explicit ipc://... endpoint for one socket (overrides ipc_dir)
ipc_sndbuf, ipc_rcvbuf	[agents]	1048576 (1 MiB; 0 = OS default)	Kernel buffer size of ipc connections only

The socket files are named broker-<port>-front.ipc, broker-<port>-back.ipc and broker-<port>-settings.ipc, where <port> is the TCP port of the same socket. Hence several brokers can coexist on one host without clashing.

A settings URL of the form ipc://... is also accepted by the -s option of the agents (for example mads source -s ipc:///tmp/mads-1000/broker-9092-settings.ipc ...); this implies ipc for the data sockets.

For example, to force TCP for everybody but one agent that must use ipc:

[agents]
transport = "tcp"

[camera]
transport = "ipc"   # fails at startup if ipc cannot be used

WarningLinux buffer cap

On Linux the kernel silently caps the buffer request at net.core.wmem_max and net.core.rmem_max (about 208 KiB on a stock install). Until you raise the limits, the 1 MiB default has no effect:

sudo sysctl -w net.core.wmem_max=4194304 net.core.rmem_max=4194304

Make it permanent with a file in /etc/sysctl.d/. On macOS the cap is kern.ipc.maxsockbuf (8 MiB by default), so 1 MiB applies as is.

Is it working?

Three places tell you:

1. the broker startup banner prints one extra Binding broker ... at ipc://... line for each of the frontend, backend and settings sockets. The N key of the broker lists the ipc endpoints next to the TCP ones;
2. the info block that each agent prints at startup has a line like Transport: ipc (mode auto);
3. mads doctor (check 8, Local transport) reports whether libzmq supports ipc, whether the runtime directory exists, belongs to you and is owner-only, which endpoints the broker advertises and whether the socket files are visible from here. It also predicts which transport an agent started on this host would select, including per-agent overrides.

Troubleshooting

• An agent on the broker’s host uses TCP. Run mads doctor: it prints the reason (broker too old to know the transport command, ipc = false on the broker, socket file not visible, agent running as another user, container with a different /tmp…). A reason is also printed when transport = "ipc" makes the agent refuse to start.
• The broker warns that an ipc endpoint was skipped. A problem with ipc never affects TCP service. The broker replaces stale socket files left by a crashed broker, but it leaves alone a socket that another live process still serves, and a path that is not a socket. On a clean exit it removes its own socket files.
• Large messages are slower than expected on Linux. See the buffer cap above.
• You want to go back to the 2.x behavior. Set transport = "tcp" in [agents] and, optionally, ipc = false in [broker].

Part 2: the shared-memory data plane

The model, in plain words

When an agent publishes a blob, MADS normally serializes it into a ZeroMQ message, which is copied to the broker, then to every subscriber. With shared memory enabled, the producer instead owns a ring of slots: a fixed number of equally sized buffers in a shared memory segment. Publishing a blob means:

1. take the next slot of the ring;
2. write the bytes in it;
3. publish a tiny message that carries the JSON metadata plus a reference (which segment, which slot, which version) instead of the bytes.

Consumers running on the same host map the same segment and read the bytes in place: no copy at all, whatever the size and the number of consumers. The slot is protected by a sequence counter (a “seqlock”): the producer bumps it before and after writing, so that a reader can detect whether the slot has been overwritten while it was reading.

The ring is lossy: once all slots are used, the oldest one is overwritten, regardless of whether somebody is still reading it. A consumer that finds out that its slot has been overwritten drops that message (it is counted, and never delivered torn or corrupted). This is the right behavior for latest-value streams like camera frames or point clouds: a slow consumer should skip to the freshest frame, not stall the producer or fill up the memory.

Shared memory is opt-in on both sides:

• the producer enables it with shm = true;
• consumers that don’t do anything special (including mads record, and everything using the C API) receive an ordinary copy of the blob, so they need no change at all. Only plugins (and custom agents) that explicitly ask for it get the zero-copy view, see Part 3.

Advantages

• no copy of the payload between producer and consumers, no compression, no ZeroMQ message of several MB through the broker: the broker only forwards a descriptor of a few tens of bytes;
• cost independent of the number of consumers;
• the producer is never slowed down by slow consumers.

Limitations

ImportantEvery consumer must be local, same user, same container

Segments are created owner-only (mode 0600). All consumers of the topic must run on the producer’s host, under the same OS user, and in the same container (same IPC namespace and /dev/shm). A consumer that does not satisfy this cannot map the segment and drops every blob of that topic, reporting it only with rate-limited warnings (the first one names the cause, e.g. permission denied … same OS user). If you cannot guarantee it, leave shm off.

• only blobs go through shared memory. The JSON part of the message always travels through the broker as usual;
• a blob larger than a slot is sent inline (as an ordinary message), with a one-time warning;
• it only takes effect if the broker runs on the producer’s host; otherwise the producer simply keeps publishing ordinary messages;
• the ring is lossy by design, so shared memory is not suitable for streams where every message must be delivered. A slow consumer loses frames when the producer outpaces it by more than shm_slots messages;
• the wire format changes only when shm = true: receivers that predate shared memory (pre-3.0) see an empty blob with the producer’s metadata;
• Windows and POSIX (Linux, macOS) are both supported.

Recorder and player. mads record stores the real bytes of shared-memory blobs, so recordings should be self-contained; if a slot had already been overwritten (or could not be read) by the time the recorder got to it, that payload is lost and mads record reports the number of unrecoverable payloads at exit, with a hint to raise the producer’s shm_slots. Conversely, mads play republishes recorded blobs as ordinary bytes (inline).

Configuration

Three keys, in the section of the producing agent (or in [agents] to set them for all agents): they work for mads source, mads filter and custom agents that read them.

Key	Default	Meaning
shm	false	Publish blobs through a shared-memory ring owned by this agent
shm_slots	8	Number of slots in the ring (at least 2)
shm_slot_size	16777216 (16 MiB)	Size in bytes of each slot. Larger blobs are sent inline

For example, a camera source publishing 1920x1080 RGB frames (6,220,800 bytes) and one filter processing them:

[camera]
shm = true
shm_slots = 8
shm_slot_size = 8388608   # 8 MiB, enough for one frame

[detector]
sub_topic = ["camera"]

The plugin loaders also accept the -o option to set a key from the command line, handy for a quick test without editing mads.ini:

mads source camera.plugin -o shm=true
mads source camera.plugin -o shm=true -o shm_slots=16 -o shm_slot_size=8388608

Invalid geometry (for example a non-integer shm_slots) makes the agent stop at startup with an Error enabling shared memory message.

Sizing

• Slot size must be at least the size of the largest blob you will ever publish, otherwise that blob goes inline (slow path, one warning at the first occurrence). Don’t over-provision either: the memory used is shm_slots x shm_slot_size, allocated by the producer (the default is 8 x 16 MiB = 128 MiB of address space).
• Number of slots controls how much a slow consumer can lag before it starts losing messages: with the default 8, a consumer is safe as long as it finishes reading a message before the producer has published 7 newer ones. For a fast producer and a consumer that is occasionally slow (it does heavy processing, or it is a recorder writing to disk), raise it. If you don’t care about old frames, the default is fine.
• A consumer that reads a message and then keeps the view while waiting for the next one can be overwritten at any time: copy what you need to keep (see Part 3).

Monitoring and cleanup

• Agent info. A plugin loader prints a Shared output: line at startup, with one of three values: zero-copy (plugin writes into the ring), copied into the ring (the plugin does not support shared output, so the loader copies its blob into a slot) or off.
• Torn and failed reads. Consumers count the messages they had to drop: shm_torn() (slot overwritten while reading) and shm_failed() (reference that could not be resolved: wrong host, producer gone, wrong user, already overwritten). They are Agent methods (see Part 4); the plugin loaders log them, rate-limited, as warnings.
• mads doctor, check 9 (POSIX only). Lists the registry in <runtime dir>/shm, where every producer registers its segments with its process id, and counts the orphans: segments whose producer is no longer running (typically after a crash), which hold memory until reboot. Orphans are reported as [WARN], and mads doctor --fix removes them. Windows frees a segment with its last handle and keeps no registry.

Part 3: programming plugins for zero copy

The plugin protocol P9

MADS 3.0 moves the plugin protocol from P8 to P9. Loaders accept exactly the host’s protocol: a plugin built for P8 or older stops with an error asking to rebuild it, and a plugin built for a newer protocol is refused too. The zero-copy API is an addition: P9 only adds virtual methods whose default implementations copy, so a plugin keeps working unchanged. To port an existing plugin:

mads plugin --update path/to/my_plugin    # use --dry-run first, to see what changes
cmake -Bbuild && cmake --build build

mads plugin --update moves the mads_plugin pin in CMakeLists.txt to v2.6-P9 and makes Release the default build type (set before project(), because MSVC otherwise builds Debug with single-config generators). No source change is needed. Zero-copy is opt-in, per method.

Note

Rust plugins (mads plugin --rust) do not have a zero-copy API: the mads-plugin crate has no shared-memory support, and --zero-copy is ignored with a warning for them. Rust plugins therefore only benefit from the local ipc transport; for large payloads, write the plugin in C++.

What the host does for you

The plugin loaders (mads source, mads filter, mads sink) drive the new hooks as follows:

Plugin kind	Hook	Used when	Otherwise
Source	supports_shared_output() + get_output_shared(out, buf)	the plugin returns true and the agent has shm = true (and the broker is local)	get_output(out, &blob)
Filter	supports_shared_output() + process_shared(out, buf)	same	process(out, &blob)
Filter, Sink	load_data_view(data, topic, blob)	a shared-memory blob arrives	default implementation: copies the view and calls load_data(data, topic, &blob)

Two details worth knowing:

• supports_shared_output() is queried once, after set_params(), and must not change afterwards;
• the classic get_output(), process() and load_data() stay mandatory: they are the fallback when shared memory is off, when the blob does not fit a slot, and for consumers that are not on shared memory at all. Your plugin must work on both paths.

Producing without copies

The host passes the plugin a BlobBuffer (defined in mads_plugin’s common.hpp):

struct BlobBuffer {
  unsigned char *data = nullptr; // start of the writable buffer (a ring slot)
  size_t capacity = 0;           // bytes available at `data`
  size_t size = 0;               // bytes written by the plugin
};

The contract is:

1. the host sets data and capacity, and size to 0;
2. write at most capacity bytes at data, then set size to the number of bytes written (size == 0 means no blob, a JSON-only message). Fill out (the JSON part) exactly as in get_output();
3. if the payload does not fit: write nothing, set size to the number of bytes you would need (so size > capacity) and return return_type::retry. The host detects the overflow from size > capacity and, in the same loop iteration, calls get_output() (or process()) to deliver that payload through the ordinary, copying path. For this to work you must check the size before consuming your pending data, so that the fallback call delivers the same payload;
4. buf.data is valid only during the call: never store the pointer.

Tip

Write your data production as a function that takes the destination pointer, like render_into(unsigned char *dst, size_t n), and call it from both get_output() (with the blob vector as destination) and get_output_shared() (with buf.data). The two paths then can’t diverge.

A complete example: a source producing an image

The following source generates a grey gradient frame, whose size is set by parameters. It is a trimmed version of what mads plugin --zero-copy source generates, with the body filled in:

#include <source.hpp>
#include <nlohmann/json.hpp>
#include <pugg/Kernel.h>
#include <vector>

#ifndef PLUGIN_NAME
#define PLUGIN_NAME "gradient"
#endif

using namespace std;
using json = nlohmann::json;

class Gradient : public Source<json> {
public:
  using Source::Source; // inherit constructors

  string kind() override { return PLUGIN_NAME; }
  // Required: shown by the agent when it loads the plugin
  map<string, string> info() override { return {{"Zero-copy", "yes"}}; }

  void set_params(const json &params) override {
    Source::set_params(params);
    _params["width"] = 640;
    _params["height"] = 480;
    _params.merge_patch(params);
    _w = _params["width"].get<size_t>();
    _h = _params["height"].get<size_t>();
    _blob_format = "gray8"; // reported as "format" in the message
  }

  // Classic path: the blob goes in a vector (always required)
  return_type get_output(json &out, vector<unsigned char> *blob = nullptr) override {
    out.clear();
    if (blob) {
      blob->resize(_w * _h);
      render_into(blob->data());
    }
    describe(out);
    return return_type::success;
  }

  // Zero-copy path: opt in...
  bool supports_shared_output() const override { return true; }

  // ...and write straight into the ring slot
  return_type get_output_shared(json &out, BlobBuffer &buf) override {
    out.clear();
    const size_t needed = _w * _h;
    if (needed > buf.capacity) {
      buf.size = needed;           // overflow: report what we need,
      return return_type::retry;   // write nothing; the host calls get_output()
    }
    render_into(buf.data);
    buf.size = needed;             // bytes written
    describe(out);
    return return_type::success;
  }

private:
  size_t _w = 0, _h = 0;
  size_t _frame = 0;

  void render_into(unsigned char *dst) {
    for (size_t y = 0; y < _h; ++y)
      for (size_t x = 0; x < _w; ++x)
        dst[y * _w + x] = static_cast<unsigned char>((x + y + _frame) & 0xff);
    ++_frame;
  }
  void describe(json &out) {
    out["width"] = _w;   // the geometry travels in the JSON part
    out["height"] = _h;
    if (!_agent_id.empty()) out["agent_id"] = _agent_id;
  }
};

MADS_REGISTER_PLUGINS(Gradient)

A few remarks:

• in the overflow case this example does not advance _frame (it does not consume anything), so the fallback get_output() produces the very same frame;
• the loader adds the format field to the message (from blob_format(), here gray8) when the plugin did not set it; geometry like width and height is the plugin’s business and goes in the JSON part, which always travels through the broker;
• to see the zero-copy path, run it with -o shm=true and look for the Shared output: zero-copy line at startup.

Filters

A filter works the same way, with process_shared(out, buf) in place of get_output_shared(). The only difference: on overflow it must leave its loaded input untouched, because the host will call process() on the same input right after. Filters can combine both directions: read their input as a view (load_data_view(), below) and write their output into a slot with process_shared().

Consuming without copies

Filters and sinks can override:

return_type load_data_view(json const &input, string topic, BlobView blob) override;

where BlobView is a plain pair:

struct BlobView {
  const unsigned char *data = nullptr; // first byte of the payload
  size_t size = 0;                     // payload size in bytes
};

The host calls load_data_view() instead of load_data() when the incoming blob lives in a shared-memory ring; messages without a blob still go through load_data(). The default implementation copies the view and calls load_data(), which is why existing plugins keep working. Once you override it, you must respect the contract:

• the view is read-only: never write through blob.data, the memory is shared with other readers (and may be mapped read-only);
• the view is valid only during the call and, for a filter, the following process() / process_shared() call of the same loop iteration; in any case only until the next load_data() or load_data_view(). Never store blob.data in a member. If you need the bytes later, copy them;
• the ring is lossy, so the producer may overwrite the slot at any time, even in the window above. The host re-checks the slot after your call(s) (and after process() for a filter); if it was overwritten, it discards the result and counts a torn read. So you may see garbage while reading, and it will be thrown away: never use data from a view for anything that cannot be safely discarded, such as irreversible side effects (actuating a machine, deleting a file). Compute, then let the host decide whether the result is valid;
• blob.data == nullptr (with size == 0) means no blob.

A complete example: a sink reading a view

This sink computes the average grey level of the frames produced by the source above, without copying them:

#include <sink.hpp>
#include <nlohmann/json.hpp>
#include <pugg/Kernel.h>
#include <iostream>

#ifndef PLUGIN_NAME
#define PLUGIN_NAME "mean_gray"
#endif

using namespace std;
using json = nlohmann::json;

class MeanGray : public Sink<json> {
public:
  using Sink::Sink;

  string kind() override { return PLUGIN_NAME; }
  // Required: shown by the agent when it loads the plugin
  map<string, string> info() override { return {{"Zero-copy", "yes"}}; }

  // Classic path: needed for non-shm producers
  return_type load_data(json const &input, string topic = "",
                        vector<unsigned char> const *blob = nullptr) override {
    if (!blob || blob->empty()) return return_type::retry; // nothing to do
    report(blob->data(), blob->size(), input);
    return return_type::success;
  }

  // Zero-copy path: `blob` points straight into the producer's ring
  return_type load_data_view(json const &input, string topic,
                             BlobView blob) override {
    if (!blob.data) return return_type::retry;
    report(blob.data, blob.size, input);   // read in place, keep nothing
    return return_type::success;
  }

private:
  void report(const unsigned char *p, size_t n, json const &meta) {
    unsigned long long sum = 0;
    for (size_t i = 0; i < n; ++i) sum += p[i];
    cout << meta.value("agent_id", "?") << ": " << meta.value("width", 0)
         << "x" << meta.value("height", 0) << ", mean = "
         << (n ? static_cast<double>(sum) / n : 0.0) << endl;
  }
};

MADS_REGISTER_PLUGINS(MeanGray)

Both entry points share the same report() routine, which only reads. If the producer overwrites the slot while report() is running, the line has already been printed: for a sink that causes an external effect, this is something to consider (do the computation first, and only act at the end of a call that the host will validate, or accept that an occasional torn frame produces a spurious output).

Scaffolding and testing

mads plugin --zero-copy (short -z) generates the hooks for you, with the contract in comments:

mads plugin --zero-copy -t source gradient   # get_output_shared()
mads plugin -z -t filter my_filter           # load_data_view() + process_shared()
mads plugin -z -t sink my_sink               # load_data_view()

The option is valid for C++ only, and without it the generated code carries a one-line comment pointing at the flag. The plugin name is the positional argument (or -n), the type is -t source|filter|sink, and -d sets the directory.

Testing tips:

• the generated main() (used when you run the plugin as an executable) calls the shared hooks with a local std::vector standing in for the ring slot, so you can unit-test the zero-copy code without any agent: call plugin.get_output_shared(out, BlobBuffer{ptr, capacity, 0}) or plugin.load_data_view(in, "", BlobView{ptr, n}) yourself. Test the overflow case too, with a buffer that is too small: you expect retry and buf.size > buf.capacity;
• run the real thing with -o shm=true on the producer and watch the Shared output: line at startup;
• check the consumer for torn warnings: if they are frequent, the ring is too short for that consumer: raise shm_slots;
• test both paths, shared memory on and off (-o shm=false): they must produce the same messages.

Part 4: custom C++ agents

If you write an agent with the Agent class directly (not as a plugin), the same machinery is exposed in agent.hpp. The transport selection needs nothing from you (agent.init() honours transport and shm from the settings). The relevant API:

Method	Purpose
void enable_shared_output(uint32_t slots, size_t slot_size)	Create the ring (slots >= 2, slot_size > 0). Called by init() when shm = true; calling it again replaces the ring. Takes effect only if the broker is local
bool shared_output_enabled() const	Whether blob publishes go through the ring
size_t shared_slot_size() const	Slot capacity in bytes, 0 if disabled
WritableSlot acquire_slot()	Take the next slot (round robin). slot.data and slot.capacity give the writable area. Throws AgentError if shared output is not enabled
void publish_slot(WritableSlot &slot, size_t size, nlohmann::json meta, std::string topic)	Publish the first size bytes of the slot as a shared-memory reference. Consumes the slot
void abandon_slot(WritableSlot &slot)	Give back a slot into which nothing was written: the previous payload of that slot stays valid
void discard_slot(WritableSlot &slot)	Give back a slot that may have been written: its old payload is invalidated
void accept_shared_views(bool)	Consumer: opt in to receive message_type::shared instead of a copied message_type::blob
SharedView const &last_shared() const	The last view received. data(), size(), meta(), and valid()
void record_torn_view()	Count a torn read noticed by the caller
size_t shm_torn() const, size_t shm_failed() const	Counters of dropped payloads
void set_transport_mode(std::string const &), std::string transport_mode() const, std::string transport() const	Force and read the transport ("auto", "tcp", "ipc"; transport() is what is actually in use)

A producer:

Mads::Agent agent("camera", "tcp://localhost:9092");
agent.enable_shared_output(8, 8 * 1024 * 1024);  // or shm = true in mads.ini
agent.init();

// ... in the loop:
if (agent.shared_output_enabled() && frame_bytes <= agent.shared_slot_size()) {
  auto slot = agent.acquire_slot();
  size_t n = grab_frame_into(slot.data, slot.capacity);
  if (n == 0) {
    agent.abandon_slot(slot);       // nothing written
  } else {
    agent.publish_slot(slot, n, {{"format", "gray8"}, {"width", w}, {"height", h}}, "camera");
  }
} else {
  agent.publish(frame_vector, {{"format", "gray8"}}, "camera"); // inline
}

A consumer:

agent.accept_shared_views(true);
// ... in the loop:
switch (agent.receive()) {
case Mads::message_type::shared: {
  auto const &view = agent.last_shared();
  process(view.data(), view.size());   // read in place
  if (!view.valid()) agent.record_torn_view(); // overwritten while reading
  break;
}
case Mads::message_type::blob: /* ordinary copy (inline blob or opted out) */ break;
default: break;
}

Always test valid() after reading the bytes; a copy of a SharedView is cheap and keeps the mapping alive, but not the validity of the data. As in the plugin case, acquire_slot()/publish_slot() are meant to be used from a single thread. If you keep a slot unpublished while acquiring slots more, it gets overwritten: publish promptly.

Note

receive_raw_message() always materializes a shared-memory reference as a normal blob frame, so tools that forward raw messages keep working.

Warning

The Agent API is C++ only and Agent gained members in v3.0.0: code that links MadsCore must be recompiled.

Compatibility and FAQ

Can I mix 2.5.x and 3.0.x components? No. The settings handshake compares MAJOR.MINOR, so a 3.0 broker refuses 2.5 agents and vice-versa. Upgrade the broker and all agents together; recompile anything that links MadsCore and rebuild plugins (mads plugin --update).

Do I need to change my existing plugins? Only rebuild them against P9 (see Part 3). They automatically get ipc, since that is a property of the agent and not of the plugin.

Will a remote agent still work? Yes. TCP is unchanged, and transport = "auto" falls back to TCP when the broker is not on the same host.

I enabled shm = true and one of the consumers receives nothing. It is most probably on another host, running as another user, or in another container: check the first warning in its console, and mads doctor. If you cannot fix that, turn shm off for that topic.

Does shared memory work with encryption (CURVE)? The blob bytes in the ring are not encrypted by CURVE: only the small reference travels through the broker, and access control is that of the OS (owner-only segments). If you need encrypted payloads, leave shm off.

Does the C API or mads record need changes? No. Both receive ordinary blob copies.

Can the JSON part use shared memory? No: only blobs do.

What if my blob is sometimes larger than a slot? It goes inline for that message, with a one-time warning. Raise shm_slot_size if it is frequent.

Where to read more. man mads-broker (LOCAL TRANSPORT), man mads-source (SHARED MEMORY), man mads-doctor (checks 8 and 9), man mads-plugin (--zero-copy, --update), the CHANGES.md file of the v3.0.0 release, and the mads_plugin headers (common.hpp, source.hpp, filter.hpp, sink.hpp), whose comments define the contract of every hook.