Skip to content

한국어 | English

Systems Index | Previous: Per-Connection Memory | Next: Core Source Layout

Auto HWM

What this chapter defines — Auto HWM budget calculation and admission contracts, the related context functions, and their internal implementation.

1. Auto HWM Overview

zlink Core automatically calculates the byte limit (HWM) retained by each socket queue from the context memory budget and distributes it among the queues, so the application does not have to set every limit individually. This automatic policy is called the Auto HWM budget.

This document defines the public contract for the inputs used to calculate that budget and how it applies to message admission, the related context functions, and their internal implementation. The functions belong to the context object (zlink_ctx_*), but this document covers them together under the single topic of Auto HWM.

The following documents own related contracts.

Related contract Defining document
Auto HWM option enums and defaults Context
Socket HWM options and observable behavior Socket Common
Short definitions of terms Core Glossary

2. Auto HWM Budget Calculation

This section defines the public calculation and admission contracts. The Internals section of this document owns queue-local state, decoder reservations, and data-path cost boundaries.

Core checks the following inputs in priority order and selects the first usable value. The ZLINK_CTX_OPT_AUTO_HWM_* values below are Context options that the application sets with zlink_ctx_set_data.

  1. A positive ZLINK_CTX_OPT_AUTO_HWM_CORE_BUDGET_BYTES
  2. A positive ZLINK_CTX_OPT_AUTO_HWM_MEMORY_LIMIT_BYTES
  3. A finite process/container hard limit Core detects when the context is created. When several positive limits are detected, the smallest
  4. Physical memory detected by Core

Detection covers Windows Job Object memory limits, finite RLIMIT_AS, and Linux cgroup memory limits. On Linux, Core reads the limits (v2 memory.max, v1 memory.limit_in_bytes) of the current process's cgroup that /proc/self/cgroup names and of all its ancestor cgroups, and uses only positive values below physical memory. Undetected limits are not part of the resolved memory limit.

The first available value among 2–4 is the resolved memory limit. Even when a manual Core budget is set, the resolved memory limit comes from 2–4, and the connection admission check below uses it.

A manual Core budget is used as-is without applying the profile percentage or effective cap. For every other memory input, Core applies the selected profile percentage once and then clamps the result to the effective cap below.

This budget has the following properties.

  • It is the basis for distributing the steady-state HWM of each application directional pipe. It is not a hard cap compared against context-wide usage.
  • Each pipe checks only its own queue bytes and HWM.
  • A pipe that reaches its HWM or uses the empty-pipe oversize exception does not reduce another pipe's HWM or stop that pipe's admission.
Profile Percentage Fixed cap General data role minimum General data role maximum STREAM minimum STREAM maximum
Compact 2% 64 MiB 32 KiB 512 KiB 8 KiB 32 KiB
LowLatency 3% 256 MiB 32 KiB 2 MiB 16 KiB 64 KiB
Balanced 5% 512 MiB 64 KiB 1 MiB 64 KiB 128 KiB
Throughput 8% 1024 MiB 128 KiB 8 MiB 256 KiB 512 KiB

The role-specific bounds are as follows.

  • STREAM role: STREAM bounds.
  • none role: excluded from planning.
  • Every other current role: general data bounds.
  • Exception — the recv_ingress role (SUB/XSUB) under the Balanced profile: 2 MiB instead of the general data maximum.

The profile percentage alone does not determine the budget. The budget is the percentage-based value clamped to the effective cap. The effective cap is the larger of the profile's fixed cap and the floor required for all active application directional queues to reach their role minimums. A fixed cap alone pushes queue-heavy deployments below their queue minimums, while a queue floor alone allows a large host with only a few queues to reserve several GiB.

The following expressions produce the final Core budget (effectiveCoreBudgetBytes). This budget is the basis for distributing the steady-state HWM of each queue described above.

percentShareBytes =                                            // memory limit × profile percentage
    (resolvedMemoryLimitBytes / 100) * profilePercent
  + ((resolvedMemoryLimitBytes % 100) * profilePercent) / 100  // avoid overflow with integer division

effectiveCapBytes =                                            // larger of the two values below
    max (profileFixedCapBytes,                                 //   profile fixed cap
         activeDirectionalQueueCount * perQueueMinimumBytes)   //   total needed for all queues to receive their minimum

effectiveCoreBudgetBytes = min (percentShareBytes, effectiveCapBytes)  // final budget obtained by clamping the percentage value to the cap above

perQueueMinimumBytes is the general data role minimum for that profile. activeDirectionalQueueCount is known only after the physical queue registry has resolved every plannable application direction, so the budget is finalized at that point (context finalize). Setting a manual Core budget skips this entire calculation.

When Core detects a finite hard limit, setting an explicit memory limit or manual Core budget above that limit fails with EINVAL. Physical memory and the finite hard limit are detected only once when the context starts and are not detected again while it runs.

When an Auto HWM option setter succeeds, it stores the configured value and schedules a new calculation through the normal debounce path. zlink_ctx_get_data returns the stored configured value immediately, but the budget snapshot returns the last recorded plan and may therefore contain the previous result until recalculation. Calling zlink_ctx_auto_hwm_recalculate records a new plan immediately.

The ABI v1 planner uses the context's physical directional queue registry. The registry registers each application ypipe direction once and owns the following state.

  • An endpoint-independent stable queue ID and generation
  • Send and receive roles and profile bounds
  • Manual HWM, current applied HWM, and accounted bytes

Even when two endpoints observe the same inproc ypipe, Core counts the same physical direction only once. A new pipe pair uses the following reservation rules for each direction.

A DEALER-ROUTER single connection is registered as one Application pipepair, that is, two Application directional queues. Because there is no separate Completion pipe, no completion directional queue is added.

  • Application direction: atomically reserves the role-specific minimum. If both directions cannot be reserved, Core rejects the entire reservation before publishing the attach and does not register only a subset of the directions. The connection admission rule below decides whether the reservation succeeds. For this the registry keeps the reserved byte total, and returns a direction's reservation when that direction retires and finishes draining.
  • Manual direction: also reserves the role-specific minimum before attach. A finite manual HWM applies to admission immediately and is included in the next plan's manual reservation sum and aggregate HWM statistics.
  • Direction with manual HWM 0: admission remains unlimited, while the next plan uses the role-specific maximum as its calculation reservation and sets the flag that indicates the aggregate HWM is not finite.

Among automatic directions, Core distributes the remaining budget by repeatedly and evenly raising queues that have not yet reached their maximum. This distribution method is called water-filling. An inproc physical ypipe does not add the values from both endpoints. It calculates one final cap using the following rules, then reserves and applies that cap once in the registry.

The judgment uses the set of the two endpoint values, regardless of which endpoint sends or receives.

Set of the two endpoint values Final physical ypipe cap
At least one finite manual value The minimum of the finite manual values
No finite manual value and at least one auto Water-filling result (auto plan)
Both unlimited manual Admission is unlimited; the role-specific maximum is reserved once for calculation

Connection admission. A new pipe pair reserves successfully only when the recorded minimum reservation total plus this pair's per-direction minimums does not exceed the resolved memory limit. The check uses the memory inputs the setters currently store, not the budget. The planner determines budget insufficiency under the manual-reservation and automatic-minimum rule below. What this check limits is the minimum reservation total at the time a new pair is reserved. Changing the memory limit or profile at runtime can leave the minimum total of existing queues above the resolved memory limit; existing queues follow the recalculation and HWM-change contracts below.

If the budget remaining after subtracting manual reservations is less than the sum of the minimums for all automatic directions, Core does not reduce the minimums and sets the insufficient-budget flag. When the budget is sufficient, Core divides the remainder by the number of unique physical queues that have not reached their maximum and repeatedly increases each queue up to its maximum. It assigns division remainders one byte at a time in stable queue ID order. The same registry snapshot and inputs therefore always produce the same result.

When a new input cannot secure the required reservation, Core handles each situation as follows.

Situation Core behavior
A new explicit memory limit or manual Core budget cannot accommodate the current manual HWMs and automatic minimums together Stores the value and schedules recalculation. The planner preserves the automatic minimums and sets the insufficient-budget flag.
A new synchronous inproc attach's minimum reservation exceeds the connection admission limit Fails with ENOBUFS
Physical memory or the hard limit decreases while running Does not change the current context's inputs or plan because they are not detected again after context startup.
A new asynchronous network attach's minimum reservation exceeds the connection admission limit Does not publish the attach and terminates the connection attempt.

When the number of connections changes a per-queue target, Core applies the change as follows.

Change Core behavior
A connection increase lowers the per-queue target The attaching directions record their new target immediately. The lowered target of directions already attached is recorded by the same debounced recalculation path that option changes use. When the last plan can be extended exactly, an attach does not run a synchronous full recalculation; when the incremental extension is not possible, that same attach path falls back to a synchronous full recalculation. Once a new target is recorded, further admission is blocked until current retained bytes drain below it
A connection decrease raises the target Applies it after the cooldown only to a live queue with the same generation
Detached queue When all endpoints are released, clears any remaining provisional and committed charge once and removes the registry entry

The observable behavior of multipart reservation, the empty-queue oversize exception, and dequeue credit is owned by §5 Verification Requirements, while their implementation mechanisms are owned by §4 Internals.

originQueueUsedBytes(queue) = physicalQueueAccountedBytes(queue)

Ordinary admission checks only this origin-local sum and that queue's applied HWM. It does not block other queues merely because the context's current_accounted_bytes exceeds effective_core_budget_bytes.

total_planned_hwm_bytes is the sum of the current targets for application directions, and total_applied_hwm_bytes is the sum of the HWMs actually applied to live application directions. core_queue_accounted_bytes is the number of bytes currently held by Core queues, and current_accounted_bytes equals that value.The ABI-reserved fields runtime_memory_limit_bytes, application_accounted_bytes, outstanding_application_lease_count, deferred_origin_credit_bytes, and retired_queue_count are always zero.

The ROUTER-ROUTER completion progress lane does not apply a byte HWM, LWM, inproc HWM boost, or the legacy 256 KiB floor, and it is excluded from the water-filling denominator above. This lane owns the progress of terminal replies and error replies and also synchronizes receive-flow-state frames between peers. The ROUTER-ROUTER Completion lane is excluded from HWM admission and Core budget reservation, but its current and peak accounted bytes and pending message count are observed separately. These values are included in total_messaging_accounted_bytes and excluded from application water-filling.

Even when an Application pipe is full, valid Completion records and receive-flow-state frames are admitted if the connection remains available and allocation succeeds.

DEALER-ROUTER replies and error replies are bytes in the single Application queue. These bytes are included in core_queue_accounted_bytes, current_accounted_bytes, and, for a multipart record, provisional_accounted_bytes, as well as peak_accounted_bytes and total_messaging_accounted_bytes. application_accounted_bytes remains the reserved value 0. DEALER-ROUTER replies are not included in completion_current_accounted_bytes, completion_peak_accounted_bytes, completion_pending_message_count, or active_completion_directional_queue_count. A reply admission blocked by HWM is counted as an Application admission block.

3. Functions

Immediately reapplies the automatic HWM plan to the entire current context.

ZLINK_EXPORT zlink_config_result_t zlink_ctx_auto_hwm_recalculate(void *context_);

This function immediately recalculates automatic HWM for sockets that still follow the automatic queue and buffer policy. Values the user changed manually remain unchanged, and sockets with Auto HWM disabled remain disabled. Use it after changing the Auto HWM profile or a memory budget input to record a new plan without waiting for the normal refresh path.

Returns: ZLINK_CONFIG_OK on success; otherwise, a zlink_config_result_t value. zlink_errno() preserves the internal errno for diagnostics.

Errors: - EFAULT -- invalid context handle.

Thread safety: Safe to call from any thread.

See also: zlink_ctx_set, zlink_monitor_status

Retrieves the last recorded context-wide Auto HWM plan in a versioned structure.

#define ZLINK_AUTO_HWM_BUDGET_SNAPSHOT_ABI_V1 1u

#define ZLINK_AUTO_HWM_BUDGET_FLAG_PLANNING_ACTIVE       (1u << 0)  // Auto HWM was active in the last plan
#define ZLINK_AUTO_HWM_BUDGET_FLAG_INSUFFICIENT          (1u << 1)  // budget cannot fit manual reservations and required automatic minimums together
#define ZLINK_AUTO_HWM_BUDGET_FLAG_AGGREGATE_HWM_VALID   (1u << 2)  // no unlimited manual direction, so the HWM sum can be interpreted as finite
#define ZLINK_AUTO_HWM_BUDGET_FLAG_AGGREGATE_OVERFLOW    (1u << 3)  // planner/queue aggregation exceeded uint64_t and saturated

typedef struct zlink_auto_hwm_budget_snapshot_t {
  uint32_t abi_version;                         // caller sets to ABI_V1
  uint32_t struct_size;                         // caller sets allocated size → Core returns the full v1 size
  uint64_t budget_generation;                   // increments for each new recorded plan (new context=0)
  uint64_t measurement_epoch;                   // increments for each metrics reset (new context=1)
  uint64_t configured_memory_limit_bytes;       // configured explicit memory limit
  uint64_t runtime_memory_limit_bytes;          // reserved (always 0)
  uint64_t resolved_memory_limit_bytes;         // limit actually used for calculation
  uint64_t configured_core_budget_bytes;        // configured manual Core budget
  uint64_t effective_core_budget_bytes;         // final budget after applying the effective cap
  uint64_t total_planned_hwm_bytes;             // sum of current target HWMs for application directions
  uint64_t total_applied_hwm_bytes;             // sum of HWMs actually applied to live directions
  uint64_t manual_reserved_hwm_bytes;           // sum of reservations for manual directions
  uint64_t core_queue_accounted_bytes;          // accounted bytes owned by Core
  uint64_t application_accounted_bytes;         // reserved (always 0)
  uint64_t current_accounted_bytes;             // equal to core_queue_accounted_bytes
  uint64_t provisional_accounted_bytes;         // reservation for incomplete multipart messages
  uint64_t peak_accounted_bytes;                // maximum observed in the current epoch
  uint64_t completion_current_accounted_bytes;  // current bytes in ROUTER-ROUTER completion lanes
  uint64_t completion_peak_accounted_bytes;     // maximum bytes in ROUTER-ROUTER completion lanes
  uint64_t completion_pending_message_count;    // messages pending in ROUTER-ROUTER completion lanes
  uint64_t total_messaging_accounted_bytes;     // application+completion accounted-byte sum
  uint64_t monitor_queue_applied_hwm_bytes;     // sum of HWMs for open monitor directions
  uint64_t monitor_queue_accounted_bytes;       // sum of bytes held by monitor directions
  uint64_t total_instance_applied_hwm_bytes;    // total_applied + monitor HWM
  uint64_t total_instance_accounted_bytes;      // total_messaging + monitor accounted bytes
  uint64_t oversize_admission_count;            // empty-queue oversize exceptions admitted
  uint64_t largest_oversize_message_bytes;      // largest such message
  uint64_t active_directional_queue_count;      // active application directions (counted once per direction)
  uint64_t active_completion_directional_queue_count;// active ROUTER-ROUTER completion directions
  uint64_t active_send_queue_count;             // send-perspective count in the last plan
  uint64_t active_receive_queue_count;          // receive-perspective count in the last plan
  uint64_t outstanding_application_lease_count; // reserved (always 0)
  uint64_t retired_queue_count;                 // reserved (always 0)
  uint64_t deferred_origin_credit_bytes;        // reserved (always 0)
  uint64_t unlimited_manual_queue_count;        // unlimited manual directions
  uint32_t blocked_ratio_ppm;                   // ratio of first attempts blocked by HWM (ppm)
  uint32_t flags;                               // ZLINK_AUTO_HWM_BUDGET_FLAG_* bits
  uint64_t reserved_u64[8];                     // reserved (all 0)
} zlink_auto_hwm_budget_snapshot_t;

ZLINK_EXPORT zlink_config_result_t
zlink_ctx_get_auto_hwm_budget_snapshot(
  void *context_,
  zlink_auto_hwm_budget_snapshot_t *snapshot_);

The caller zero-initializes the structure, sets abi_version to ZLINK_AUTO_HWM_BUDGET_SNAPSHOT_ABI_V1, and sets struct_size to the allocated byte size. A null snapshot or a size shorter than the two header fields fails with EINVAL; an unsupported version fails with ENOTSUP; an invalid context fails with EFAULT; and a terminating context fails with ETERM. On success, Core writes only the smaller prefix of the caller size and the Core v1 size. The returned struct_size is the full size of the Core v1 structure.

active_directional_queue_count and active_completion_directional_queue_count count each unique physical ypipe direction shared by reader and writer endpoints only once. Completion directions are excluded from the application-direction planning denominator and HWM admission. active_send_queue_count and active_receive_queue_count are the perspective-specific counts from the last socket plan. A new context starts with budget_generation 0 and measurement_epoch 1. budget_generation increments whenever a new plan is recorded, and measurement_epoch increments when metrics are reset.

Physical queue accounting follows these rules.

  • Adds the payload and sizeof(zlink_msg_t) for each frame.
  • Includes incomplete multipart frames in provisional_accounted_bytes; at the final frame, converts the same bytes to committed state without incrementing them again.
  • Read, rollback, hiccup, termination, and conflate replacement return the charge for the frames they actually remove.
  • Completion directions report separate current and peak values and a complete-message pending count.

monitor_queue_applied_hwm_bytes sums the currently applied HWM once for every unique physical ypipe direction of an open monitor. It does not separately add the options copied to the reader and writer endpoints. monitor_queue_accounted_bytes sums the accounted bytes of frames currently retained by those same monitor directions. Neither value is included in application planning, total_applied_hwm_bytes, current_accounted_bytes, or total_messaging_accounted_bytes; they are added only to the following instance totals.

total_instance_applied_hwm_bytes =
    total_applied_hwm_bytes + monitor_queue_applied_hwm_bytes

total_instance_accounted_bytes =
    total_messaging_accounted_bytes + monitor_queue_accounted_bytes

ROUTER-ROUTER Completion queues have no HWM, so they are not added to total_instance_applied_hwm_bytes. If either sum exceeds the uint64_t range, it saturates at UINT64_MAX and sets ZLINK_AUTO_HWM_BUDGET_FLAG_AGGREGATE_OVERFLOW.

peak_accounted_bytes is the largest total observed in the current measurement epoch when a budget snapshot query or Auto HWM recalculation inspects the queues. It does not guarantee that it records values retained only briefly between two inspections.

A snapshot is a consistent registry view that belongs to one budget_generation. It does not mix queue counts, capacity, and accounted counters from different generations. Current counters may change while the snapshot is constructed, but the returned aggregate and component fields must remain mutually consistent at the same snapshot boundary.

Registry-owned fields and sampled fields become visible at different times.

  • Sampling fields — current_accounted_bytes, provisional_accounted_bytes, and peak_accounted_bytes. These values are sampled from per-pipe accounting when the snapshot is constructed.

blocked_ratio_ppm is calculated with the following expression.

floor(first_blocked_admission_attempts * 1,000,000 / total_admission_attempts)

The ratio is 0 when total_admission_attempts is 0. A retry after the same submission wakes is not counted again. Only the first attempt blocked by the target application pipe's HWM is included in the numerator; transport I/O waits and context aggregate usage are excluded.

Changes the current Auto HWM measurement epoch.

ZLINK_EXPORT zlink_config_result_t
zlink_ctx_reset_auto_hwm_budget_metrics(void *context_);

On success, this function increments measurement_epoch by 1, rebases the application and completion peaks to their current accounted bytes, and resets the blocked and total admission-attempt counters, blocked_ratio_ppm, the cumulative oversize count, and the largest oversize value to 0. It does not change current bytes, monitor queue HWM or accounted bytes, the budget, plan, queue counts, or budget_generation. An invalid context fails with EFAULT, and a terminating context fails with ETERM.

4. Internals

This document defines the state that Core maintainers must read and update in the message processing path when implementing Auto HWM memory limits. The Auto HWM Budget Calculation section of this document and the Socket specification own the application-visible budget calculation, HWM scope, and errors.

Value Limited by HWM

Auto HWM distributes a context memory budget among application directional queues to determine each queue's HWM. Message admission is based on the unreturned bytes and applied HWM of the physical queue that will receive the message, not on context-wide usage.

The unreturned bytes of one physical queue are the sum of the following values.

frameCharge = payloadBytes + sizeof(msg_t)

outstandingCharge = provisionalCharge + committedQueueCharge

sizeof(msg_t) is not a measurement of allocator usage. Even a frame with no payload consumes a queue slot and message object, so this fixed value prevents its byte HWM cost from becoming 0. Payload-only accounting cannot impose a memory limit because it allows empty single-part messages or empty multipart frames to remain in a queue indefinitely regardless of HWM.

provisionalCharge is the value reserved before the decoder allocates a payload buffer. committedQueueCharge is the value of frames retained by the queue. When the Core queue dequeues a complete message to the binding, it decreases the committed charge and returns writer credit. The later lifetime of a payload held by the binding or application is outside Core HWM accounting; Core does not convert it into a retained-credit lease.

For example, if the HWM is 1,024 bytes and the unreturned charge is 900 bytes, ordinary admission accepts only a frame whose charge is at most 124 bytes. Another queue being empty or the context-wide sum being below the budget does not change this decision.

Separation of Responsibilities

Processing location Input Result
Budget planner Memory inputs, profile, application queue list Target HWM for each queue
Queue configuration path Target HWM, current applied value, queue generation HWM to apply to the same generation
Message processing path Target queue's unreturned charge, frame charge Admission or backpressure
Snapshot path Per-queue HWM and charge Context query result

The budget planner runs only when an option changes or a queue attaches or detaches. Context-wide sums and snapshot statistics produced by the planner are not used as message admission conditions.

Message Processing Sequence

Ordinary frames are processed in the following order.

%%{init: {'sequence': {'actorFontSize': '18px', 'messageFontSize': '18px', 'noteFontSize': '18px', 'boxMargin': 8, 'width': 140}, 'themeVariables': {'fontSize': '18px'}}}%%
sequenceDiagram
    participant D as Writer/Decoder
    participant Q as Target Queue
    D->>D: candidate charge = payload + fixed frame cost
    Note over D: Reject on uint64_t overflow
    D->>D: Record candidate charge in reservation token before buffer allocation<br/>(skip when submitting a message created by the writer)
    Note over Q: Reject if HWM is nonzero and unreturned+candidate exceeds HWM<br/>(empty-queue oversize exception is separate)
    D->>Q: After enqueue, provisional → committed
    Q-->>D: On complete-message dequeue, decrease committed and return credit to writer
  1. The writer or decoder adds the fixed frame cost to the payload size to calculate the candidate charge.
  2. If the addition exceeds the uint64_t range, it does not admit the frame.
  3. Before allocating the payload buffer, the decoder records the target queue reference, generation, and candidate charge in a reservation token. Accounting for the charge begins at frame write. This step is skipped when the application writer submits a message it has already created.
  4. If the applied HWM is nonzero and adding the candidate charge to the unreturned charge exceeds the HWM, it does not admit the frame. The public empty-queue oversize exception applies separately.
  5. After enqueue completes, it converts the value from provisional state to committed state without adding the reserved value again.
  6. When the queue dequeues a complete message, it decreases the committed values of its frames and returns byte credit to the writer. It does not extend Core charge through binding or application payload lifetime.
  7. Drop, allocation failure, protocol error, and termination each return only the value they actually own, exactly once.

In this sequence, normal frame processing reads and updates only local state created with the queue.

Multipart and Large Messages

A multipart message accumulates the charge of each frame. After reading a frame's payload size from the wire header, the decoder records the frame's charge in a reservation token before buffer allocation. The final frame makes the entire multipart available to read without incrementing the values reserved for earlier frames again.

When a multipart message whose final size is not yet known reaches HWM, Core stops before allocating the buffer for the next MORE frame. However, if the multipart started on an empty queue, it may admit the final frame to complete one record even when that frame exceeds HWM. Eligibility is fixed by whether the queue was empty immediately before the first frame and does not apply to an intermediate MORE frame. If allocation failure or a protocol error discards the multipart, Core returns every charge that the multipart reserved or recorded in the queue.

An empty queue may admit one complete message whose total charge is known at admission even when it exceeds HWM. This exception does not apply to two messages simultaneously and does not bypass the ZLINK_OPT_MAXMSGSIZE check.

Receive Dequeue and Queue Generation

Core HWM charge ends when the Core queue dequeues a complete message to the binding. The queue byte HWM does not track how long the binding or application keeps that payload afterward.

Detaching or reconnecting a queue creates a new generation. HWM replanning and application affect only a live queue of the same generation. When the final endpoint is released, Core clears any remaining provisional and committed charge for that entry and removes it.

HWM Changes

Increasing the HWM applies the new value to the current queue generation. Decreasing the HWM makes the writer's admission limit the new target immediately — a frame whose candidate charge added to the unreturned charge exceeds the new target is not admitted. Frames already admitted are not removed, and the applied value reported by snapshots changes to the new target the moment the retained amount falls to or below it (deferred shrink).

Completion-lane and DEALER-ROUTER reply HWM and accounting follow §2 lane accounting. Monitor queues are also excluded from the queue list used to distribute the application budget.

Synchronization and convergence of the recalculation

Four owners cover the path that produces an Auto-HWM plan: the option lock owns the context options and planning inputs, the state lock owns the generation, the deadline and the last recorded plan, the recalculation lock serializes a full replan and an incremental extension, and the pin lock owns the lifetime pins of the socket list. The lock order is in the synchronization model §6.1.

Full replan. Holding the recalculation lock from start to finish, it snapshots the generation under the state lock, collects only the socket lifetime pins under the pin lock and releases that lock immediately, then reads the inputs under the option lock. It then collects per-socket policies, plans in the registry, applies to sockets and pipes, and records the plan. Because the generation is snapshotted before the inputs, the combination "new generation with old inputs" cannot arise. When the plan is recorded and the applied generation equals the pending one, the pending marker is cleared and the budget generation is raised.

Incremental extension on attach. An attach publishes the pipe into the endpoint list first. If that pipe is an application direction and the socket's Auto-HWM policy is on, it attempts the incremental extension and falls back to a full replan on the spot if that fails. The incremental path holds the recalculation lock and proceeds in this order.

  1. Under the state lock, check that the pending generation equals the last applied generation and copy the applied plan.
  2. Under the option lock, read the current inputs and check they match the inputs of the copied plan.
  3. Under the registry lock, add only the two attaching directions to the plan and update the targets and aggregates.
  4. Release the registry lock and apply that pipe's HWM under the attaching pipe's endpoint lock.
  5. Record the plan and the direction counts under the socket's Auto-HWM lock.
  6. Record the plan and the budget generation under the state lock.

The targets are therefore applied before the plan and the generation are recorded, so a snapshot that observes the new generation only ever sees a plan that is already applied.

Fallback conditions. The incremental extension is abandoned for a full replan when any of the following holds: zero or more than two directions; a disabled plan; a pending generation that differs from the last applied one; copied plan inputs that differ from the current inputs; mixed roles among the automatic directions, or none; a manual direction or an unlimited manual HWM taking part; a new queue id inside the already planned id range; a queue that is not registered or has no endpoint reference; overflow or shortfall in the direction count, the manual reservation, the minimum total, the data budget or the planned aggregate; a terminating context; or a failed allocation.

Convergence. A successful incremental extension arms the debounce deadline on the spot. Only the first caller that finds no deadline arms one and wakes the timer; later attaches in the same burst join the deadline already armed and never push it out. The deadline does not raise the pending generation, so it does not block the incremental path of the attaches that follow. When the deadline passes, one full replan runs and clears the deadline as it records its plan. A deferred shrink that leaves the applied value above the planned one therefore does not make the recalculation repeat.

Applying a target. A planned value is always published with a release store. When the target grows, or shrinks while the currently accounted bytes are already at or below the new target, the applied value is published with it. When it shrinks and more bytes are retained, only the planned value changes and the applied value stays, which is what the snapshot reports as a deferred shrink.

Pending-request admission

The accounted size x of one request is the sum of payloadBytes + sizeof(msg_t) for all Application payload frames. Each physical transport pair applies a 32 MiB logical work budget and an unresolved-request count limit of 16,384. The request's logical work charge C(x) is as follows.

  • x <= 1,024 B: C(x) = x
  • 1,024 B < x <= 32,768 B: C(x) = ceil(x^3 / 1,024^2)
  • x > 32,768 B: C(x) = 32 MiB + 1

C(x) is a size-weighted admission unit for completion liveness; it does not represent allocator bytes or retained payload bytes. A pair whose live work charge and unresolved count are both 0 admits one request whose work charge exceeds the work budget. It admits no subsequent request while that request remains unresolved.

Physical Application HWM limits only frames resident in the queue. When the peer dequeues a request, Core returns its physical-queue charge and writer credit; unresolved correlation neither retains nor reuses that HWM value as a separate lifecycle limit. An HWM change therefore does not alter the correlation admission of a request that already succeeded, and HWM 0 disables only the physical Application-queue limit. The work budget and count limit remain active regardless of the HWM value.

A request that lacks capacity under the work budget or count limit returns ZLINK_SUBMIT_BACKPRESSURED with EAGAIN immediately, regardless of send flags or SNDTIMEO. It publishes no part on the wire and does not invoke its handler. Capacity exhaustion on one pair does not block another pair or ordinary sends on the same pipe. Reply, timeout, disconnect, and close return the work and count reservations together. When a terminal reply or timeout returns the reservations, it wakes request-submit recovery on the pipe owner that held that reservation. When the wait token of a refused DONTWAIT request emits the resource-recovery ZLINK_COMPLETION_WRITABLE, and how that token ends, are owned by the socket README's REQUEST DONTWAIT section — the recovery of the refusing resource is the only wake condition, so this reservation return is that condition.

Message-Path Cost Limits

The send, receive, and decoder admission paths do not perform the following operations.

  • Acquire a context-wide mutex
  • Search a global map for a queue ID or reservation ID
  • Allocate heap memory for each frame reservation
  • Update context-wide current, provisional, or peak sums for each frame
  • Update global atomics for statistics not used in HWM decisions
  • Query the usage of another physical queue

A decoder reservation is represented by inline state owned by the decoder or pipe. The required values are the target queue reference, generation, reserved charge, and whether a reservation exists. The queue lifecycle registry may be used for queue attach, detach, and cleanup of earlier generations, but it is not queried for every ordinary frame admission.

Snapshots and Metrics

A snapshot builds context totals from queue-local state when queried. It may use the required registry lock while constructing the snapshot, but it does not serialize every queue with the same lock used by the message processing path.

Peak statistics record the largest total observed when a snapshot query or Auto HWM recalculation collects values from each queue. Because Core does not update the context-wide sum for every frame, a value retained only between two observation points may not be included in the peak. Resetting statistics or repeatedly querying the snapshot does not change HWM admission results or writer credit.

Implementation Locations

Responsibility Implementation location
Profile bounds and per-queue HWM calculation auto_hwm_policy.*
Context inputs, recalculation, and snapshot API ctx_auto_hwm_*
Physical queue identity and generation ctx_physical_queue_registry.*
Queue-local charge, HWM decision, and byte credit pipe.*
Reservation before allocation zmp_decoder.*, session_base_pipe_io.cpp, pipe.*
Dequeue credit return and detach cleanup pipe.*, ctx_physical_queue_registry.*

5. Implementation and Contract-Test Verification Requirements

This section collects the items that workers must verify. These behaviors are observable only through the public surface—the context options zlink_ctx_set_data/zlink_ctx_get_data, zlink_ctx_get_auto_hwm_budget_snapshot, send and recv admission results, and errno—not through internal implementation. Each item maps to one unit test.

Options and budget - Setting a memory limit or manual Core budget larger than a finite hard limit detected by Core fails with EINVAL. - A valid memory limit or manual Core budget is stored and recalculation is scheduled even when the value is less than the sum of current manual HWMs and automatic minimums. The recalculated snapshot preserves the automatic minimums and sets ZLINK_AUTO_HWM_BUDGET_FLAG_INSUFFICIENT. - Physical memory and the hard limit are detected only once when the context starts. Changing them while the context runs does not trigger detection again. - In a configuration where every direction is automatic, an attach succeeds when the recorded minimum reservation total, including the new pair, does not exceed the resolved memory limit. If the automatic minimum total calculated under the current profile exceeds the budget, the recalculated snapshot preserves the automatic minimums and sets ZLINK_AUTO_HWM_BUDGET_FLAG_INSUFFICIENT. - A new pair whose minimum reservation total would exceed the resolved memory limit is not registered. A synchronous inproc attach fails with ENOBUFS, and a network connection attempt ends without publishing. - Calling zlink_ctx_set_data/zlink_ctx_get_data for an Auto HWM byte option with a size other than exactly sizeof(uint64_t) fails with EINVAL and leaves the value unchanged. - With the same connection configuration and inputs, the snapshot's effective_core_budget_bytes is always the same (deterministic). - A new pipe pair first reserves the role-specific minimum for each direction regardless of the manual HWM size. A finite manual HWM applies to admission immediately and is included in the next snapshot's manual_reserved_hwm_bytes and aggregate HWM statistics.

Admission (byte accounting) - Frame, multipart, and oversize admission verification refer to Message Processing Sequence; pending work and count verification refer to Pending-request admission.

Credit, dequeue, and generation - After receiving a complete message, Core queue charge ends and the sender can send again. The snapshot's application_accounted_bytes remains zero even if the application keeps the payload. - After socket detach or reconnect, accounting from the previous generation does not reduce the new generation's charge or increase its writer credit (snapshot).

HWM changes - Lowering the HWM preserves frames already admitted, admission of new frames is refused against the new target immediately, and the snapshot's applied value reports the new target only after the retained amount has fallen to or below it.

Reply accounting by topology - When a controlled DEALER-ROUTER reply remains queued, its byte delta appears in core_queue_accounted_bytes, current_accounted_bytes, and, for a multipart record, provisional_accounted_bytes, as well as peak_accounted_bytes and total_messaging_accounted_bytes. application_accounted_bytes remains 0. - The same DEALER-ROUTER reply does not appear in completion_current_accounted_bytes, completion_peak_accounted_bytes, completion_pending_message_count, or active_completion_directional_queue_count. The Application direction count reports only Application directional queues. - A ROUTER-ROUTER reply appears in Completion current, peak, pending, and direction counts and is excluded from the Application water-filling denominator and HWM admission. - A DEALER-ROUTER reply that is not admitted because of HWM or PAUSED is observed as Application admission backpressure. - Receive-flow control and monitor traffic do not change application send admission results or the denominator of the snapshot's total_planned_hwm_bytes. - zlink_auto_hwm_budget_snapshot_t uses ABI v1 and its defined field layout.

Snapshot invariance - Calling zlink_ctx_get_auto_hwm_budget_snapshot or zlink_ctx_reset_auto_hwm_budget_metrics does not change the admission or rejection result for the same send sequence. - An unsupported abi_version fails with ENOTSUP, and a terminating context fails with ETERM.

Systems Index | Previous: Per-Connection Memory | Next: Core Source Layout