Execution & Concurrency

How Nexus bounds concurrent work — a global semaphore queue, a per-worker concurrency cap, a per-tenant request cap, and the bridge connection governor — and why it is a queue that delays work rather than a limiter that rejects it.

This page is for capacity planners and platform engineers who need to know how many things run at once inside a Nexus tenant, what the defaults are, and — critically — what happens when demand exceeds them. The short version: Nexus queues excess work and drains it as capacity frees up. It does not reject a run because the system is busy. Read on for exactly where each bound sits.

The layers of concurrency control

Concurrency is bounded at several layers, each with a different job. They compose: a run must pass every applicable layer to be executing at a given moment.

Global semaphore queue (default max 5)

At the heart of execution is a semaphore that admits a bounded number of runs into the "executing" state at once — five by default. Think of it as five lanes. When all five lanes are occupied, the sixth run does not fail; it waits in the durable run queue until a lane opens, then proceeds. This is a smoothing mechanism: it flattens a burst of triggers into a steady execution rate instead of letting a spike stampede your data sources and your Postgres connection pool.

The important mental model is delay, not rejection. A queued run's latency increases under load; its success rate does not drop because the queue was busy. There is no hard backpressure rejection at this layer — nothing throws a "system busy, try later" error at the run.

Worker max concurrent (10)

Each worker process will drive up to ten runs concurrently (they are async coroutines, so a single worker interleaves I/O-bound runs efficiently). This cap protects a single replica from over-subscribing itself. In a multi-replica deployment, workers coordinate through the shared Postgres queue — each claims distinct runs via FOR UPDATE SKIP LOCKED — so adding replicas adds worker capacity without double-claiming work.

Per-tenant request cap (HTTP 429)

Separate from run execution, inbound API pressure is bounded per tenant. When a caller drives the API harder than the configured cap, the server responds with 429 Too Many Requests to that caller. This is the one place the platform does push back with a rejection — but note the distinction: it throttles inbound request rate at the edge, it does not cancel or reject already-accepted runs. A well-behaved client backs off and retries; the run queue itself keeps draining.

Note that ctx.http calls a pipeline makes to external APIs are also per-tenant rate limited on the outbound side, so a runaway pipeline can't hammer a third-party API on your behalf. See Pipeline SDK.

The bridge connection governor

Data adapters talk to your systems (Postgres, SQL Server, NetSuite, and so on) over real connections. Left unbounded, a burst of concurrent runs against the same database could exhaust its connection limit — a classic production failure. The bridge connection governor prevents that:

• It caps the number of live DB connections each replica will hold against a bridge, so total connection pressure scales predictably with replica count rather than with run count.
• It reaps orphaned connections — connections held by work that has since ended — so a leak or an abandoned run cannot slowly consume the budget.
• It applies at the replica level, which composes cleanly with the single-revision rollout model below: you know how many replicas exist, so you know the ceiling on connections your data sources will see.

The governor is why you can point many pipelines at one database without a thundering-herd connection storm. It bounds the fan-out to your systems, not just the fan-in to Nexus.

Single-revision rollout

Nexus deploys with a single-revision rollout model: when you apply an update, traffic moves to the new revision and the old one is retired, rather than running many stacked revisions simultaneously. This matters for concurrency because the alternative — multiple active revisions each spinning up their own workers and connection pools — is a known way to silently multiply your effective concurrency and exhaust a database's connections. Single-revision keeps the concurrency and connection math honest: the caps above are the caps, not the caps times however many revisions happened to be live. See Updates for the rollout mechanics.

Tuning

The defaults (global 5, worker 10) are chosen to be safe on the modest per-tenant infrastructure Nexus provisions, and to keep connection pressure gentle on your data sources. They are operational parameters, not hard architectural limits — if your tenant is sized for more, they can be raised. Raise them deliberately and in step with:

• your Postgres Flexible Server's connection ceiling and compute tier,
• the connection limits of the external systems your pipelines write to, and
• the per-run timeouts and row constraints that bound how long each lane stays occupied.

What this does — and doesn't — do

• Does: bound concurrent execution to a predictable number; smooth bursts by queuing; protect your databases from connection exhaustion; throttle abusive inbound request rate with 429s; keep the concurrency math honest via single-revision rollout.
• Doesn't: reject a run because the system is busy (excess work queues and drains); provide per-run CPU or memory quotas inside the sandbox (there is no cooperative in-runtime CPU/memory cap — the wall-clock timeout and row constraints are the blast-radius controls); guarantee a maximum queue wait time (queue latency rises with sustained overload — monitor it).

See also

• Run Lifecycle
• Timeouts & Limits
• Updates
• Connections & Bridges
• Architecture