# CAF v2 Build-versus-Reuse Decision Memo

**Decision:** Build only the CAF-specific compatibility and evidence boundary. Reuse the host’s container, identity, policy, scheduling, inference, telemetry, retention, and incident-response capabilities through versioned provider interfaces. Treat Pi Coding Agent as the first conformance target, not as the architecture. The first strategic proof after MVP should be a second runtime—not a broader Pi feature set.

## Decision context

This recommendation uses the product baseline as stated:

- CAF is a thin, replaceable, developer-first container/runtime adapter for managed HPC.
- Selected tool and inference operations route through CAF adapters.
- The host owns hard enforcement, identity, scheduling, network, mount and resource controls, telemetry, retention, and incident response.
- CAF records application-level evidence and links managed runs to scheduler jobs.
- The MVP uses Pi Coding Agent; the product should progress toward explicit runtime admission.

The implication is a narrow product category: CAF is an **agent-runtime compatibility layer for managed HPC**, not an agent framework, scheduler, policy engine, security platform, inference server, or observability system. Its defensible value is the stable semantic bridge between rapidly changing agent runtimes and site-managed HPC services.

## Capability-layer decisions

| Capability layer | Recommendation | CAF should uniquely own | Adopt or integrate | Reasoning |
|---|---|---|---|---|
| Product contract and versioning | **Build** | CAF run/call identity, operation envelope, execution-profile schema, compatibility rules, error taxonomy, version negotiation, deprecation policy | JSON Schema or another open schema format | This is the durable product boundary. If another component defines these semantics, CAF becomes a packaging script for that component. |
| Runtime adapter SDK | **Build** | Minimal runtime-facing SPI for registering/replacing selected tools, routing inference, propagating run context, returning structured outcomes, and declaring bypass limitations | Runtime-native extension mechanisms; protocol adapters where useful | The SDK must preserve a common CAF contract while allowing runtime-specific code. It should not implement reasoning, memory, planning, or general agent orchestration. |
| Runtime admission and conformance | **Build** | Admission manifest, declared effective tool surface, automated conformance suite, negative/bypass tests, evidence tests, compatibility matrix, signed test result reference | Existing CI, image-scanning, signing, and artifact systems | Runtime admission is the path from a Pi-based MVP to a portfolio. It is CAF’s central quality mechanism and should be independent of any one runtime. |
| Pi reference runtime | **Integrate; do not fork** | Thin Pi adapter and pinned, reproducible profile | Pi’s reasoning loop and extension points | Pi is a useful first implementation but must remain replaceable. Pi-specific types must stop at the adapter boundary. |
| Container artifact and launch | **Reuse with a CAF profile** | CAF image composition rules, required labels/metadata, adapter/runtime compatibility declaration, launch handshake | OCI image tooling and the site-selected runtime; HPC container platforms where selected by the site | OCI already separates image, runtime, and distribution specifications. CAF should package to this boundary, not build a container runtime. Apptainer is an example of an HPC-oriented container platform and should be a host integration option, not a CAF dependency. |
| Image trust and host admission | **Host-owned integration** | Required claims in the CAF manifest and a way to bind the launched digest/profile to the CAF run | Host allowlisting, signature/attestation verification, registry, admission controller | Admission is hard enforcement and therefore belongs to the host. CAF may emit identity and provenance inputs but must not declare itself admitted. |
| Execution profile | **Build the portable core; host maps and enforces it** | Small, declarative intent schema: runtime/image identity, requested workspace class, inference class, scheduler capability, evidence mode, and resource class | Site policy and platform-specific launch specifications | Avoid copying every Kubernetes, Slurm, or container-runtime knob into CAF. The profile should express portable intent and return the host’s resolved decision/reference. |
| Policy decisions | **Integrate** | A vendor-neutral request/decision envelope and decision reference recorded with the operation | Host policy engine, including OPA-like APIs where deployed | CAF can request and apply a decision to routed operations, but the host owns policy and actual access. CAF must not carry an independent policy language that can drift from host enforcement. |
| Identity and credentials | **Integrate** | Propagation fields for CAF run, call, user/workload, and delegated context; no credential persistence | Site identity, credential broker, workload identity, secrets system | Workload identity systems such as SPIFFE illustrate a reusable identity boundary. CAF should accept opaque host-issued identity and credential references rather than mint identities or manage secrets. |
| Tool transport | **Adopt selectively behind CAF semantics** | CAF operation identity, policy linkage, result status, scheduler linkage, and evidence obligations | Runtime-native tools; MCP-compatible transport or servers when useful | MCP defines tool discovery and calls, but it does not replace CAF’s host execution profile, HPC scheduling semantics, evidence requirements, or runtime admission. Make MCP one provider, not the CAF contract itself. |
| Workspace/file operations | **Build only the adapter** | Normalized operation/result semantics and application-level evidence | Site filesystems, object stores, workspace services, mount policy | CAF must not create a parallel storage layer. The host controls mounts and access; CAF records only operations routed through its selected tools. |
| Scheduler/job operations | **Build the CAF scheduling facade; reuse scheduler authority** | Portable submit/status/cancel/result vocabulary, idempotency key, run-to-job linkage, normalized terminal states | Slurm CLI/REST or another site scheduler API | Slurm exposes a REST interface, but scheduler-specific fields and lifecycle states will vary. Keep native details in an extension map and preserve the original scheduler job ID. |
| Inference routing | **Build the adapter; reuse model serving** | Provider-neutral request metadata, run/request correlation, endpoint/model references, timing/status/token-use fields when available, content-capture policy flag | Site inference gateway or model server; standardized inference protocols where suitable | CAF should not serve models or enforce model guardrails. Predictive protocols such as KServe V2 can be provider options, but generative APIs and metadata require an adapter and should not leak into the core contract. |
| Application-level evidence | **Build schema and emission; reuse storage/export** | CAF event types, IDs, causality, redaction classification, write ordering, scheduler linkage, schema evolution | OpenTelemetry-compatible context/export where useful; host log/audit/SIEM storage and retention | Host telemetry sees effects; CAF adds application semantics. OpenTelemetry supplies reusable logs/traces conventions and correlation mechanisms, but CAF needs its own semantic conventions for agent tool, inference, and scheduler events. CAF should export, not become the retention system. |
| Hard isolation, telemetry, retention, response | **Do not build** | References that let operators correlate CAF evidence with host records | Host container controls, network/mount/resource enforcement, monitoring, retention, security analytics, incident response | Owning these would contradict the baseline, duplicate mature system functions, and turn CAF into a control plane. |
| Developer experience | **Build a thin CLI/SDK** | Validate profile, inspect admitted capabilities, launch, show run/job references, explain adapter failures, export a portable run summary | Existing shell, CI, notebooks, portals, and workflow systems | Developer-first value requires clear errors and a stable local contract. Avoid building an IDE, chat UI, or workflow orchestrator in the MVP. |

## Target architecture and seams

CAF should expose four explicit seams and keep all product code on one side of a documented interface:

1. **Runtime SPI (northbound).** A runtime adapter registers the effective tool surface, intercepts designated tool and inference requests, propagates CAF context, and returns structured outcomes. Runtime-specific objects never enter the CAF core event schema.
2. **Host provider interfaces (southbound).** Separate providers cover launch/profile resolution, workspace operations, inference, scheduling, policy decisions, identity/credentials, and evidence export. Providers can be local commands, Unix sockets, HTTP/gRPC services, or embedded libraries, but their semantics remain stable.
3. **Evidence/export interface (eastbound).** CAF emits immutable event envelopes with schema version, run ID, call ID, parent ID, timestamps, operation class, decision reference, status, and external references. Content is omitted by default. Host-controlled exporters own transport, buffering, storage, retention, and access.
4. **Admission/conformance interface (lifecycle).** A runtime package declares adapter and contract versions, selected capabilities, unsupported paths, image identity, and conformance results. Admission is based on reproducible tests and declared limitations, not a brand-name allowlist.

A useful internal architecture is a small runtime-neutral CAF core plus runtime and host plug-ins. Keep the runtime adapter in-process where interception requires runtime hooks, but move common normalization and evidence logic into the runtime-neutral core. Do not require a long-running CAF control plane. If a helper process is needed for credential isolation or reliable evidence delivery, define it as an optional host-provided endpoint rather than making every CAF deployment depend on a CAF service.

### Contract design rules

- Use globally unique CAF `run_id` and `call_id`; preserve native runtime IDs and scheduler job IDs as separate external references.
- Make requests idempotent where repeated submission could create side effects, especially scheduler submission.
- Return explicit `allowed`, `denied`, `unsupported`, `unavailable`, `failed`, and `unknown` outcomes; do not collapse host denial into runtime failure.
- Separate the requested execution profile from the host-resolved profile and record the resolution reference.
- Provide a namespaced extension field for scheduler, runtime, and inference-provider details; never promote one provider’s fields into the core without evidence from at least two implementations.
- Version schemas independently from adapter implementations. Support at least one prior minor version and fail closed on incompatible major versions.
- Record evidence delivery state separately from operation outcome. A successful tool call with failed evidence export is not a fully compliant managed operation.
- Document observation scope precisely: CAF can attest to requests and results routed through its interfaces, not to all effects produced by a process or scheduler job.

## Conceptual fit with market building blocks

The market already supplies most lower-layer mechanisms CAF needs:

- **Packaging/runtime:** OCI defines image, runtime, and distribution specifications; this supports a portable CAF artifact without selecting one host runtime. Apptainer illustrates an HPC-focused container integration option.
- **Scheduling:** Slurm’s REST interface provides a concrete scheduler-provider target while leaving Slurm in control of job lifecycle and placement.
- **Policy and identity:** OPA exposes policy-decision APIs, while SPIFFE defines workload identity concepts. Both are examples of replaceable host services; neither should become mandatory CAF infrastructure.
- **Tool protocols:** MCP provides open tool listing/calling semantics. CAF can bridge to MCP tools, but CAF still needs HPC execution intent, policy references, runtime admission, and evidence semantics.
- **Inference protocols:** KServe’s V2 protocol demonstrates a model-server-neutral inference interface. It is useful for compatible endpoints but is not sufficient as CAF’s universal generative-inference contract.
- **Observability:** OpenTelemetry supports logs, traces, metrics, resources, and cross-signal correlation. CAF should define semantic conventions and exporters over this ecosystem instead of owning a telemetry backend.
- **Artifact trust:** Sigstore’s Kubernetes policy controller demonstrates image signature and attestation admission. Such enforcement remains a host choice; CAF should supply verifiable artifact metadata and consume the decision.

This comparison supports an **adopt at the mechanism layer, build at the semantic boundary** strategy. CAF should prefer standards-compatible mappings but avoid making optional market components mandatory.

## Lock-in risks and mitigations

| Risk | Failure mode | Mitigation |
|---|---|---|
| Pi lock-in | CAF core mirrors Pi tool objects, callbacks, or conversation lifecycle | Convert at the Pi adapter boundary; make second-runtime conformance the next release gate. |
| HPE host-suite lock-in | Profile, identity, policy, or evidence APIs require SUMS-Agent or another HPE service | Publish provider interfaces and schemas; validate one non-HPE or customer-native integration path. |
| Scheduler lock-in | Slurm states/options become the universal model | Keep a small common lifecycle and namespaced native extensions; test a fake/reference provider and later a second scheduler. |
| Kubernetes lock-in | Admission and launch assume pods, webhooks, or service accounts | Define host-neutral admission results and launcher protocol; implement Kubernetes only as a provider. |
| Evidence-backend lock-in | CAF events depend on one SIEM, database, or telemetry collector | Publish schema; support file/stream and OpenTelemetry-style export; host owns retention and indexing. |
| Policy split brain | CAF and host evaluate different rules, producing contradictory outcomes | Host returns authoritative decisions; CAF only requests, applies to routed calls, and records decision references. |
| Inference API lock-in | One provider’s chat schema becomes the CAF contract | Normalize only stable metadata and lifecycle; put payload-specific details behind provider adapters. |
| MCP overreach | CAF delegates its core semantics to a changing tool protocol | Treat MCP as transport/provider integration; retain CAF identity, evidence, policy, and HPC semantics. |
| Profile-schema sprawl | CAF reproduces all platform knobs and becomes impossible to port | Express intent classes in the core; keep resolved/native details in provider extensions. |
| Security overclaim | Customers infer that adapter capture equals containment or complete audit | Publish bypass tests and observation gaps; state that host controls hard effects and CAF covers only the managed path. |
| Adapter maintenance burden | Every runtime release breaks a CAF integration | Pin supported versions, run conformance in CI, publish a compatibility matrix, and require explicit re-admission for incompatible changes. |

## Phased strategy

### Phase 0 — Freeze the product boundary

Before expanding the MVP, approve a short architecture decision record that assigns ownership exactly as above. Define the versioned run/call envelope, execution-profile core, provider interfaces, and evidence schema. Add a reference host provider that can run in a development environment without pretending to enforce production policy.

**Exit gate:** schemas are published; unsupported and denial behavior is specified; no Pi type appears in the CAF core; host enforcement responsibilities are testable and documented.

### Phase 1 — Pi MVP: prove one managed path

Ship the pinned Pi image and thin adapter with the selected tool profile. Route workspace, inference, and scheduler operations through CAF providers. Produce application-level evidence and link the CAF run to native scheduler job identifiers and returned outputs. Validate one allowed launch, one rejected mismatch, one useful scientific workflow, and documented bypass/observation gaps.

Use existing OCI tooling and site host controls. Do not add a CAF policy engine, telemetry backend, credential store, or scheduler.

**Exit gate:** the baseline acceptance test passes end to end; event export failure is visible; repeated scheduler submission is safe or explicitly rejected; evidence contains no prompt, model response, file content, or command output unless policy explicitly opts in.

### Phase 2 — Turn implementation into runtime admission

Extract a public or partner-facing adapter SDK and automated conformance kit. Admit a second runtime with materially different extension points. Require runtime packages to declare effective tools, inference interception coverage, unsupported paths, adapter/runtime versions, and conformance result references.

This phase is more important than adding many Pi tools: it tests whether CAF is actually thin and replaceable.

**Exit gate:** the same host providers and evidence consumers work unchanged for Pi and the second runtime; only runtime-adapter code changes; failures and bypasses are comparable in a published compatibility matrix.

### Phase 3 — Prove host portability and standards mappings

Add a second host integration dimension—for example, another scheduler provider or container launch environment. Introduce optional mappings for MCP tools, OpenTelemetry export, OPA-style decisions, and a standardized inference endpoint. Keep each optional.

**Exit gate:** the CAF core runs across two runtime adapters and two host profiles without provider-specific core fields; one customer-native or non-HPE service stack is validated.

### Phase 4 — Operational hardening and ecosystem admission

Add signed admission manifests, supply-chain attestations, compatibility lifecycle, upgrade/revocation procedures, backpressure and offline evidence delivery, performance budgets, and administrator-facing diagnostics. Integrate richer host approval, credential, retention, and incident-response services by reference rather than absorbing them.

**Exit gate:** runtime upgrades trigger automated re-conformance; host operators can revoke an admitted artifact/profile; load tests establish adapter overhead and evidence-loss behavior; incident responders can correlate CAF run IDs with host telemetry without CAF ingesting that telemetry.

## Measures that should govern investment

Track measures that validate the boundary rather than vanity adoption metrics:

- adapter code and maintenance effort per admitted runtime;
- percentage of CAF core unchanged when adding the second runtime and host provider;
- routed-operation coverage and documented bypass surface;
- successful run-to-scheduler-job correlation rate;
- evidence completeness, export failure rate, and sensitive-content violations;
- adapter latency/throughput overhead, including at high tool-call volume;
- time to diagnose `denied` versus `unsupported` versus infrastructure failure;
- number of mandatory infrastructure dependencies introduced by CAF (target: zero beyond the host contract).

## Final recommendation

Fund CAF as a **small contract, adapter SDK, conformance program, and reference packaging layer**. Build the semantics that make agent activity portable and attributable in managed HPC; integrate every mechanism already owned by the host or supplied by established infrastructure. Keep Pi pinned for the MVP, then use a second runtime and a second host profile as the decisive architecture tests. If either requires changing the CAF core contract substantially, the design is not yet thin or replaceable.

## Verified external references

- Open Container Initiative overview: https://opencontainers.org/about/overview/
- Apptainer user guide: https://apptainer.org/docs/user/main/introduction.html
- Slurm REST API documentation: https://slurm.schedmd.com/rest.html
- Open Policy Agent REST API: https://openpolicyagent.org/docs/rest-api
- SPIFFE concepts: https://spiffe.io/docs/latest/spiffe-about/spiffe-concepts/
- Model Context Protocol tool specification: https://modelcontextprotocol.io/specification/2025-06-18/server/tools
- KServe Inference Protocol V2: https://kserve.github.io/website/docs/concepts/architecture/data-plane/v2-protocol
- OpenTelemetry documentation: https://opentelemetry.io/docs/
- Sigstore policy-controller overview: https://docs.sigstore.dev/policy-controller/overview/