Documentation / Trust & Maturity

Testing & Quality

The quality evidence behind Nexus — a large automated server test suite run on every release, independently tested adapter bundles, a gated security-review process, and signed, attested release artifacts.

This page is for an evaluator who wants to know how Nexus keeps quality from regressing as it ships. It describes the concrete quality machinery: the automated test suite, how adapters are tested independently of the server, the gated security-review process changes must pass, and the signing and attestation applied to release artifacts. These are process facts, not marketing adjectives — each one is something you could verify in an evaluation.

The automated test suite

Nexus carries roughly 1,300+ automated server tests across 128 test files, and they run on every release. This is the primary regression net for the FastAPI server that is the heart of the platform — the query engine, the pipeline runtime, the RBAC enforcement points, the scheduler and run queue, the version-control substrate, and the connector integration points all sit behind it.

What the suite covers, in practice:

The query/capability engine — pushdown decisions, the in-engine fallback for operators an adapter cannot push down, WHERE re-application semantics (ADR-0058), LIMIT/OFFSET safety, and the writability gates that block writes to views, formula columns, and read-only collections.
RBAC enforcement — role floors, per-resource grants, execution-role / assume-role behavior, service-bridge vs. user-bridge credential isolation, and the app-token-vs-user-token distinctions. Authorization is enforced at defined checkpoints, and those checkpoints are tested directly.
The pipeline runtime — the sandbox import allowlist and AST-level stripping of exec/eval/file-I/O/unrestricted-network, the SDK surface (ctx.adapters, ctx.secrets, ctx.http, ctx.workspace, and the rest), per-run timeout enforcement, and the row-count constraints.
Scheduling and the run queue — exactly-once cron firing (compare-and-swap claim plus cross-replica advisory locks), the durable run_queue claim semantics, and the heartbeat/orphan reaper.
Version control — commit-once save semantics, history/diff/restore, and multi-replica-safe writes.

A test count is a proxy, not a guarantee — but a suite of this size that gates every release means the common regressions (a permission check silently dropped, a pushdown path returning wrong rows, a scheduler double-fire) are caught before an image ships, not after a customer hits them.

Adapters are tested independently

Connectors are not tested only through the server. Each adapter bundle is independently tested and built in CI, then published as a signed, attested artifact — a local developer build of a security-critical bundle is refused by policy. That independence matters for two reasons:

An adapter's capability declarations (what it can push down, what it can write) are validated against its actual behavior, so the engine's pushdown decisions rest on declarations that have been exercised, not assumed.
A change to one connector cannot silently regress another, because each bundle carries its own test surface and its own signed manifest.

The per-connector capability facts — full pushdown for the SQL family and the internal workspace, filter/order-only for QuickBooks and Monday with aggregation in-engine, NetSuite's dual-surface behavior, and Stripe as preview/pre-1.0 — are documented in the Connector Capability Matrix. Those are the declarations the adapter tests hold the connectors to.

The gated security-review process

Changes to the sensitive surfaces of Nexus do not merge on a maintainer's say-so. There is a gated security-review process for changes touching authentication, request signing, the Key Vault / secrets path, token handling, RBAC enforcement, and the network egress controls. The review is a checklist against the platform's security rules, and it is a required gate — a change to an auth-gating flag or a signature-required endpoint list does not ship without passing it.

This is deliberately more than a code review. It exists because the highest-severity failures in a platform like this are not crashes — they are silent authorization or isolation failures that a functional test would happily pass. The security-review gate is where a human specifically asks "does this change widen who can do what, and is that intended?" before the change reaches an image. The broader secure-development and vulnerability-management narrative — including how findings are tracked in an internal register and how red-team exercises feed back into the process — is described under Secure Development & Vulnerability Management. Detailed evidence from that program is available to evaluators under NDA (see below).

Signed and attested release artifacts

Quality is not only "does it work" — it is "can you prove that what you deployed is what we built." Nexus's release artifacts are signed and attested:

Adapter bundles ship with a signed manifest plus an SLSA-style attestation and a transparency-log entry. Each bundle is dual-signed with ES256 by two separate Azure Key Vaults, and the server verifies the signatures fail-closed before loading an adapter — an unverifiable bundle does not load.
Server and web-UI images are built in CI and version-pinned; the running image is identifiable by a bare-semver tag, and the baked web UI and system skills are pinned to it so there is no client/server skew.

The supply-chain mechanics are covered in depth under Supply-Chain Integrity. The point for this page: the artifacts that make up a Nexus deployment are verifiable, not opaque.

Where this holds up — and where a buyer should push

Being candid about the limits of the quality story:

We publish a test count, not a coverage percentage or a third-party audit. The ~1,300+ tests are real and gate every release, but we do not claim an externally-audited coverage figure or a third-party penetration test we have not commissioned. Do not read "1,300 tests" as "certified."
Some register items are open. The security program tracks findings in an internal register; some are still open or in progress at any given time. We describe the program and architecture rather than implying every control is universally GA on every tenant. Process isolation, for instance, is live on the commercial tenant and rolling out — see Shipped vs. Roadmap.
The strongest independent signal available to you is a proof-of-concept in your own tenant. Because Nexus deploys into your subscription, you can exercise the real thing against your real data and your own security tooling. That is a better quality signal than any number we could print. See References & Evaluation.