NeMo Relay: instrument agent execution and choose an integration

Read support as a binding-and-version matrix

Relay’s versioned 0.9.3 support matrix distinguishes a supported binding from a particular harness integration and from a platform’s packaged CLI. That distinction should choose your first experiment.

Choice	What the reviewed matrix actually establishes
Rust, Python or Node.js application	Maintained binding paths; verify the matching runtime/package
Go or raw C FFI	Experimental, source-first surfaces
Pi integration	Proof of concept scoped to 0.84.x; later minor versions are reported unverified
Intel macOS CLI	No prebuilt CLI asset in this release; source install is a different path
Python worker on Windows ARM64	Unsupported because of the required grpcio wheel

Pi’s upstream release has since reached 1.0.2; that makes the old hook signatures especially important to recheck. The mere presence of Pi on an integration list is not current-version compatibility evidence. Likewise, Relay worker processes are local children, not a security sandbox.

For the first event experiment, the versioned Python quickstart uses an echo tool plus a fake model callback. Check three different outcomes: callback return, matching lifecycle events, and subscriber delivery after flush(). A returned dictionary alone exercises only the callback; events without a successful callback are only execution observations.

The current mutable README names newer binding-package versions than the 0.9.3 repository release. Follow a single package/source coordinate rather than mixing its snippets with the older pinned quickstart. This guide has not installed those newer packages or tested a live harness. Use Pi’s interface guide for its native version boundary and execution finality when correlating tool completion with an outer task.

Choose the execution boundary

NeMo Relay 0.9.3 is the latest published release observed on 4 October 2026, released on 28 September. Its release notes include MCP tool metadata normalization, shared Python worker-plugin support and permission-decision correlation. The 0.8.3 integration evidence below remains a historical snapshot.

Your job	Start with	What you still own
Instrument an application you control	The matching Python, Node.js or Rust binding	Scope lifetime, callbacks, credentials and event delivery
Add Relay to an existing supported harness	That harness’s documented integration	Supported version and integration-specific configuration
Reuse policy or observability behavior	Relay plugin configuration and lifecycle	Trust, configuration and teardown
Expose interchangeable complete harnesses to a product	UHP / HarnessRouter	External tasks, sessions, authentication and artifacts
Send traces to a backend	Relay subscriber/exporter path	Destination credentials and retention

Relay is more than an exporter: middleware can participate before execution, whereas subscribers consume emitted event snapshots. Conversely, adding an exporter does not establish that every harness operation passed through Relay.

Follow one call through the runtime

1. The application or harness integration opens a scope.
2. A managed tool or model call passes through the middleware visible in that scope.
3. The actual callback executes and emits related lifecycle events.
4. A subscriber queue delivers event snapshots to an exporter or in-process observer.

Scopes determine where work belongs, which local behavior applies and what is cleaned up. Events retain parentage after a scope closes. Subscriber delivery is asynchronous; flushing is a delivery barrier, not a substitute for executing the callback. See NVIDIA’s architecture.

A small first workflow

Start from the Python quickstart at 0.9.3 and the matching installation prerequisites. The source example uses local echo callbacks, so it can check instrumentation without a live model request.

1. Create an isolated Python application environment supported by the installation guide.
2. Install the documented package for the example. The 0.9.3 quickstart still pins nemo-relay==0.9.0; record which package you actually install instead of calling that command a 0.9.3 validation.
3. Register an event subscriber, open an Agent scope and run one managed tool callback with nemo_relay.tools.execute.
4. Inspect the callback’s ToolExecutionResult.result and the lifecycle events. A returned echo without events validates only the callback.
5. Close the scope, call nemo_relay.subscribers.flush(), then deregister the subscriber.

This is a source-reviewed workflow, not an executed compatibility test. To progress to a real harness, repeat it through that integration and check tool identity, permissions, cancellation and cleanup. Do not generalize a binding demo into proof that a specific harness integration covers every operation.

Relay and UHP can address different parts of one system

A UHP server can drive a complete harness whose internals use Relay. That composition still needs implementation evidence: neither protocol conformance nor Relay’s presence proves that the outer task and inner event IDs are correlated. The current HarnessRouter reference covers its released backend set; the earlier eight-backend comparison below is dated history.

Historical source notes — baseline reviewed 3 Sep 2026

The following evidence preserves the earlier review and its links. “Current”, “stable” and “main” inside this section refer to that historical cutoff; they are not new release or compatibility claims. Use the guidance above for the current scope.

Status first

NVIDIA published NeMo Relay 0.8.3 on 2 September 2026. The project describes Relay as a shared runtime for scopes, policy, plugins and lifecycle events around agent work. It can sit beside a coding-agent CLI, run inside an application or framework integration, or be embedded directly by a host that owns the real model and tool callbacks.

Stable 0.8.3 is the maturity boundary used for the support claims on this page. Current upstream main has already moved beyond that release: pull request #977, merged as 30b684db on 2 September, adds a post-release Pi routing path for Relay-launched sessions. That source-level work is described separately below and is not attributed to stable 0.8.3.

A fresh integration signal also landed in Hermes on 3 September 2026. Hermes PR #96633, merge 48c0c3a8, upgrades its existing native Relay integration from the 0.7 line to nemo-relay>=0.8.3,<0.9, adapting Relay’s typed tool-result and trace-propagation contracts. Hermes still owns its session, model, tool and streaming callbacks; Relay owns explicitly selected plugin activation.

Caution

NeMo Relay is not a UHP implementation or interoperability protocol. No primary source reviewed for this page establishes native UHP support, a standardized UHP↔Relay binding, or a HarnessRouter NeMo Relay backend. Relay operates around agent execution; UHP standardizes the external client↔server contract for complete-harness execution.

What Relay controls

Relay’s runtime model is built around execution activity rather than UHP task resources:

• Scopes model parent/child relationships for runs, turns, LLM calls, tool calls and subagents and provide lifecycle cleanup and request isolation.
• Lifecycle events and subscribers expose the activity observed by Relay to exporters and other consumers.
• Marks represent point-in-time events such as compaction or skill loading without inventing a start/end pair.
• Managed LLM and tool calls let Relay apply middleware around callbacks it is actually allowed to invoke.
• Middleware can block, sanitize, transform, route, retry or replace managed execution.
• Plugins package reusable policy, observability, guardrail and adaptive behavior.

The control strength depends on the integration boundary. If the real LLM or tool call is routed through Relay, middleware can participate in managed execution. If a framework exposes only before/after hooks, Relay can observe that lifecycle but does not gain authority it does not actually own. If provider traffic is routed through the Relay gateway, Relay becomes part of that production request path.

Stable 0.8.3 integration surface

The tagged 0.8.3 support matrix distinguishes several integration styles.

Integration path	Stable 0.8.3 evidence	Important boundary
Claude Code CLI	Minimum 2.1.121; persistent install, transparent run, lifecycle hooks, local gateway routing and pre-tool security	Claude desktop/web/application sessions are outside this claim unless they expose the same local controls.
Codex CLI	Minimum 0.143.0; persistent install, transparent run, ten supported plugin hooks, local gateway routing and pre-tool security	Cloud/remote work that bypasses the local machine has partial or no LLM capture; encrypted Codex multi-agent v2 payloads cannot be reliably linked.
Hermes Agent	Native in-process integration; no separate Relay CLI or observability plugin is required	Hermes owns the actual session/model/tool/streaming loop; Relay supplies the selected runtime/plugin layer.
LangChain / LangGraph / Deep Agents	Maintained integrations with observability, security and optimization support	Framework integration is not a new external agent protocol.
OpenClaw	Maintained integration with observability and partial security support	Public hooks do not expose the managed-execution rewrite authority required for Relay optimization.

Relay also supports native and local gRPC worker plugins. The project explicitly warns that the worker process boundary isolates crashes and dependencies but is not a security sandbox.

Trajectory and telemetry boundary

Relay records its canonical lifecycle stream as Agent Trajectory Observability Format (ATOF) events. Exporters can project completed runs into Agent Trajectory Interchange Format (ATIF) trajectories or translate runtime activity to OpenTelemetry/OpenInference output.

That makes Relay relevant to the guide’s execution-evidence coverage, but the layers are different:

• ATOF is Relay’s runtime event model.
• ATIF is a portable completed-run trajectory format.
• OpenTelemetry/OpenInference are observability projections.
• UHP is the live external execution contract for selecting a harness and running/continuing/cancelling work through server task/session semantics.

An ATIF trajectory exported by Relay is therefore evidence about an agent run; it is not a UHP task object or a UHP conformance result.

What changed in the 0.8 line

Release 0.8.0 established several contracts that matter when Relay is embedded into a harness:

• tool callbacks and execution intercepts use the canonical ToolExecutionResult contract across primary bindings and plugin/worker surfaces;
• a provider- or harness-supplied tool-call ID can be preserved on matching lifecycle events;
• managed LLM requests carry a runtime-owned W3C traceparent so downstream provider work can remain attached to the Relay trace;
• Deep Agents can represent orchestrator/subagent activity as nested semantic Agent scopes;
• conditional middleware and event-metadata injection give policy/observability code explicit runtime hooks without changing the agent framework’s own session model.

Patches 0.8.1 and 0.8.2 harden plugin/configuration boundaries, gateway lifecycle, managed integration refresh and observability behavior. 0.8.3 then fixes release-version identity between the Python package and compiled native extension and adds managed coding-agent reliability fixes, including recovery from stale managed plugin state and safe restart of an unresponsive gateway only when Relay can verify that it owns the recorded process.

Hermes 0.8.3 adoption: ownership stays explicit

Hermes PR #96633 is useful architectural evidence because it shows a harness embedding Relay without giving Relay ownership of the harness session itself.

The merged integration updates Hermes to Relay 0.8.3, returns ToolExecutionResult from managed callbacks, forwards Hermes’ provider tool-call ID into Relay correlation, and propagates Relay’s traceparent through SDK/custom request paths that already accept extra headers. Strict native transports do not receive an unsupported SDK-only keyword.

Upstream reports 270 focused Relay tests when isolated by file, a native Anthropic streaming end-to-end test, and a deterministic Hermes CLI smoke covering two managed model calls, streaming, a real file-tool call, shared metrics and export validation. Those are upstream verification results, not a UHP conformance statement.

The reusable boundary is runtime interposition without session-ownership confusion: an execution runtime may observe or govern selected call boundaries while the harness remains authoritative for session state, model/tool invocation semantics and streaming lifecycle.

Post-release current main: Pi named-upstream routing

NeMo Relay current main at this verification cutoff is 30b684db. PR #977 extends Relay-launched Pi sessions so the selected provider can name its original upstream endpoint even when that destination is not statically configured on the gateway.

The trust boundary is narrow:

• the client-supplied destination header is accepted only from a Relay-launched process tree authenticated with its per-invocation proxy credential;
• Relay validates the destination and strips the routing header before forwarding;
• a client-named destination receives only credentials supplied by that caller — Relay does not inject configured or environment provider credentials into an endpoint selected by the client;
• standalone gateways keep static routing because they do not issue the invocation credential.

This is meaningful post-release runtime/security behavior, but it is not part of tagged 0.8.3 and does not establish a Pi UHP implementation or a HarnessRouter integration.

Relay vs UHP

Boundary	NVIDIA NeMo Relay	UHP / HarnessRouter
Primary role	Execution-runtime interposition, policy, lifecycle events and observability	Client↔server contract for selecting and running a complete harness
Main objects	Scopes, lifecycle events, marks, managed calls, middleware/plugins	Harness/model catalogues, tasks, responses, sessions and server capabilities
Typical placement	In-process SDK, framework integration, coding-agent sidecar/hooks, or provider gateway	External HTTP/SSE boundary in front of a selected harness
Control authority	Depends on whether Relay owns the managed call, only observes hooks, or routes provider traffic	Defined by the UHP server/API contract and the selected harness adapter
Evidence output	ATOF events; ATIF/OpenTelemetry/OpenInference projections	UHP response/task/session behavior and conformance checks
Adoption evidence	Native Hermes integration; stable local CLI support for Codex/Claude Code; maintained framework integrations	HarnessRouter separately exposes eight released harness backends through UHP
UHP relationship	No standardized binding or native UHP adoption verified	NeMo Relay is not a released HarnessRouter backend

A system could theoretically place Relay inside a harness that is itself exposed through UHP, but those layers would remain separate. The UHP client should not be assumed to see Relay scopes, plugins, trace state or middleware decisions unless an implementation explicitly maps them into a documented UHP-visible surface.

Why it matters

NeMo Relay is a useful missing layer in the agent-harness architecture map. It is neither just an observability exporter nor another client↔agent protocol: when integrated at the managed-call or gateway boundary, it can participate in policy and execution control while preserving the host harness as the owner of the agent session.

That distinction matters for interoperability analysis. A UHP adapter can standardize access to a complete harness; Relay can independently instrument or govern parts of what happens inside that harness. Treating those as the same layer would overstate both UHP adoption and Relay’s authority.

Related pages

Read harness composition for nested and cross-harness execution topologies, UHP vs ATIF for portable trajectory evidence, UHP and Hermes, UHP and Codex, UHP and Claude Code, and UHP and Pi for the upstream harness boundaries that Relay can touch through separate integrations.

Primary sources

• NVIDIA NeMo Relay 0.8.3 release
• NeMo Relay 0.8.3 overview
• NeMo Relay 0.8.3 support matrix
• NeMo Relay 0.8.3 release notes
• NeMo Relay current-main Pi named-upstream routing — PR #977
• NeMo Relay current-main merge 30b684db
• Hermes Relay 0.8.3 integration — PR #96633
• Hermes merge 48c0c3a8