# ISAES completion: measured qualification and external acceptance

The original redesign requires software implementation, operational evidence,
and representative-researcher usability. A passing automated suite does not
complete every gate. The completion release keeps these results separate in a
machine-readable qualification summary.

`release_tools/isaes_completion_acceptance_contract.json` freezes the supported
workloads, budgets, participant population and task thresholds. It was written
before acceptance measurement. Do not edit its thresholds to make a measured
failure pass. A changed workload or threshold needs a separately identified
contract and new qualification, preserving the earlier attempt.

## Measured developer workloads

Run from the package root using the supported Python 3.12 research interpreter:

```sh
python release_tools/isaes_completion_qualification.py performance --output validation_r34_isaes_ai/performance.json
python release_tools/isaes_completion_qualification.py verify-performance validation_r34_isaes_ai/performance.json
```

The runner requires Linux/Python 3.12 for these frozen developer-executor budgets.
It executes actual reporting calculations and exports in fresh subprocesses:

| Workload | Repetitions | Frozen latency budget | Frozen peak memory budget |
|---|---:|---:|---:|
| Analysis of 100,000 retained ordered observations | 5 | p95 ≤20 seconds | ≤1024 MiB |
| Standard report, exactly two scientific figures and one comparison; HTML/PDF/PPTX/JSON/ZIP exports | 3 | p95 ≤60 seconds | ≤1536 MiB |

Elapsed wall time includes interpreter startup, numerical calculations and
exports. Peak resident memory is the measured subprocess's lifetime high-water
mark. Numeric libraries run with a recorded single-thread policy to avoid
uncontrolled oversubscription. The receipt records runtime/package versions,
hardware/CPU affinity, exact source inventory, contract hash, per-run commands,
observed timings, peak memory, input/output identities and retained artifact
hashes. Before and after checks reject source or runtime changes during the run.

The percentile is the **nearest-rank** statistic `ceil(0.95*n)` of sorted samples.
With three or five observations it is the maximum. These small samples establish
the frozen release workload check; they are not a broad production-latency SLA.
No averaging hides a failing high-water memory or latency observation.

The runner refuses to overwrite an existing measurement directory. Choose a new
output path when repeating an attempt. Failed subprocess logs remain available.
The final release runner performs these measurements after the fresh software
tests and checks the same frozen source afterward.

The live workbench browser runner supplies a separate
`pf.isaes.browser-performance/1` receipt with exactly three measurements of each:

| Browser metric | Frozen p95 budget |
|---|---:|
| First useful view | 10,000 ms |
| Linked selection | 1,000 ms |
| Saved-view restore | 2,000 ms |

Each browser receipt binds the contract and source identity and declares actual
server execution with provider access disabled. Automated browser actions remain
different from a real researcher study. The scope verifier recomputes percentiles
from the observed runs rather than trusting a prefilled summary.

A supplementary `large_array_navigation_100k` gate renders two complete
100,000-point panels in an actual self-contained report served over local HTTP.
It measures three fresh Chromium contexts, with initial chart display, real
Plotly zoom, linked observation selection and saved-state download/restore.
The same frozen 10,000/1,000/2,000 ms budgets apply to initial display,
selection/zoom, and restore. Plotly drawing promises are awaited before stopping
interaction clocks. All retained points and identities remain available; exact
evidence values and unchanged payloads are checked. A sampled browser-reported
JavaScript heap observation is retained separately from process peak RSS.
This gate qualifies the standalone HTML route; the actual authenticated workbench
gate remains separate. Its receipt is
`large-array-browser/isaes-large-array-browser.json` under the validation folder.

## Native Windows execution

After release packaging, collect all new external qualification inputs, receipts,
logs and observations in a sibling directory such as
`../isaes-external-qualification/`, outside the extracted package. The sealed
package inventory includes its existing validation evidence: adding receipts
under `validation_r34_isaes_ai` or elsewhere inside the package changes
that inventory. The helper accepts external paths and still binds the package's
source and acceptance contract. Keep the delivered archive and its original
qualification summary intact; retain later evidence as a separate supplement.
Use a new receipt name for each attempt rather than overwriting an earlier one.

Native Windows qualification cannot run in a Linux container or WSL. Install the
release's pinned supported dependencies, including every reporting/export
dependency, in a native Windows Python 3.12 research environment. The selected
tests also execute a real browser: Node.js must be on `PATH`, Playwright must be
available to Node, and `PF_BROWSER` must name an installed Chromium executable.
Set `PF_PLAYWRIGHT_MODULE` to its absolute module path when it is not available
through ordinary Node resolution. Missing optional exporters or an unconfigured
browser produce skips, which correctly fail this zero-skips qualification.
From the extracted package root, replace the two example installation paths:

```powershell
$env:PF_BROWSER = 'C:\Tools\chromium\chrome.exe'
$env:PF_PLAYWRIGHT_MODULE = 'C:\Tools\isaes-node\node_modules\playwright'
$env:PF_ISAES_PRESENTATION_DIR = [IO.Path]::GetFullPath('..\isaes-external-qualification\windows-browser-001')
Remove-Item Env:PF_ISAES_EXAMPLE_DIR -ErrorAction SilentlyContinue
py -3.12 release_tools/isaes_completion_qualification.py native-windows --output ../isaes-external-qualification/windows-001.json
```

The command refuses other operating systems or Python versions. It runs fixed
scientific-method, quantum-protocol, presentation, lifecycle, actual workflow,
SDK prerequisites, report-assistant/proposal contracts, HTTP authorization,
launcher and installer tests. Assistant protocol tests use explicit local fixtures;
this does not qualify live model accuracy. A passing receipt requires nonzero tests with
zero failures, errors or skips. It retains JUnit and execution logs and binds
source/runtime identities. The verifier checks the actual JUnit counts and fixed
test list; a manually written `status: passed` is insufficient. This selected
native route is separate from hosted containment or real BYOS qualification.

Do not describe Python 3.8 syntax parsing, native Windows launch screenshots or
Linux test results as native Windows execution qualification. The legacy runtime
continues to need its separately scoped validation if that deployment is used.

## Real researcher task study

Use the uncompleted `release_tools/isaes_completion_study_template.json` as a
collection schema. It contains no fabricated participants or passing outcomes.
Export the current source/contract identity before collecting sessions:

```sh
python release_tools/isaes_completion_qualification.py identity --output ../isaes-external-qualification/qualification-identity.json
```

Copy the study template to that external collection directory and keep the
completed manifest and its relative observation files there. Do not edit the
template included in the sealed package.

The frozen study population is at least five researchers in relativity, quantum
research or scientific computing with limited application-specific coding
experience. Every participant must attempt all five frozen tasks:

1. Compare two compatible runs.
2. Investigate a failed scientific check.
3. Trace a literature comparison to its exact source.
4. Change and restore an appropriate view.
5. Export a report and presentation.

The manifest uses `schema_version: "pf.isaes.researcher-study/1"` and, only for
real observed sessions, `evidence_nature: "observed_real_researcher_sessions"`.
Copy the source and contract hashes from the identity receipt. Record study,
protocol and observer identities plus an ISO timestamp including timezone.
Keep participants pseudonymous; each participant contains:

- `participant_id`: unique pseudonymous identity.
- `participant_background`: relevant scientific experience.
- `research_area`: `relativity`, `quantum` or `scientific_computing`.
- `application_specific_coding_experience`: `limited` for this population.
- `tasks`: exactly one attempt per frozen task, using its exact contract text.

Every task record contains the exact `task`, positive `elapsed_seconds`, integer
`assistance_count`, `errors`, `task_outcome` and `observation_artifact`.
Outcomes are `completed_unassisted`, `completed_assisted` or `not_completed`.
Unassisted completion requires zero assistance; assisted completion records
positive assistance. Record errors as objects with `description` and `severity`
(`critical_scientific_misinterpretation`, `operational` or `minor`). An explicit
empty errors array means no observed errors; omitting it is invalid.

`observation_artifact` names a retained relative path. The top-level `evidence`
maps every observation artifact to its SHA256. Observations may be authorized
session notes, task records or recordings. Keep identifiable personal information
and unrelated confidential research out of the distributable receipt.

Acceptance requires **at least 80% unassisted task completion overall and zero
critical scientific misinterpretations**. All attempted tasks count in the
denominator. Per-task outcomes, times and errors remain visible rather than being
hidden by the overall rate. An incomplete, simulated or unrepresentative study
cannot satisfy the contract.

```sh
python release_tools/isaes_completion_qualification.py import-study ../isaes-external-qualification/collected-study.json --output ../isaes-external-qualification/study-import-001.json
```

The importer validates the schema, population declarations, task coverage,
artifact hashes and threshold arithmetic, and copies the original manifest and
observations into a retained evidence directory. Meeting arithmetic thresholds
produces `external_review_required`, **not** automatic usability acceptance.
Independent review must establish that the participants and observations are
real, representative and correctly interpreted. A hash or a self-declared field
cannot prove those facts. Scripts, agents and synthetic unit-test records are
never substituted for representative researchers.

## Hosted production and live BYOS

These gates are conditional on the actual deployment. Unknown deployment scope
remains unqualified; it is not silently marked passed or not applicable. A
standalone-only or regular-storage-only deployment can document why a conditional
gate is outside its scope during release review, without claiming the omitted
service was tested.

The uncompleted deployment template has schema
`pf.isaes.external-deployment/1`. A real probe records `execution_nature:
"live_deployment_probe"`, a target deployment identity, operator, source/contract
hashes and start/completion timestamps. Each check records an actual boolean
`passed`, a concrete `observation`, and one or more `evidence_artifacts` named in
the top-level SHA256 `evidence` inventory.

For `production_container_isolation_if_hosted`, the mandatory checks are:

- `private_files_other_owner_denied`
- `host_filesystem_escape_denied`
- `unapproved_network_egress_denied`
- `resource_limits_enforced`
- `cancellation_stops_worker`
- `publication_rechecks_authorization`
- `rollback_preserves_readers_and_deletion_guards`

For `live_BYOS_if_deployed`, the mandatory checks are:

- `owner_scoped_write_read_hash_match`
- `cross_owner_access_denied`
- `credentials_not_exposed_in_artifacts`
- `interrupted_write_recovery`
- `retained_dependency_blocks_deletion`
- `authorized_deletion_recovery`
- `provider_failure_preserves_retained_evidence`

Use isolated deployment test tenants and disposable, identified probe records.
Do not delete or expose real retained research. The deployment operator must
authorize the probe target, access, resource budget and destructive cleanup of
the probe's own disposable records. An authorized deployment-specific Python
driver can execute the probes and produce the manifest:

```sh
python release_tools/isaes_completion_qualification.py external-probe --driver ../isaes-external-qualification/authorized-deployment-probe.py --gate live_BYOS_if_deployed --output ../isaes-external-qualification/byos-probe-001.json
```

The driver receives `--gate`, `--output`, `--source-identity` and
`--contract-sha256`. It owns the configured deployment authentication. The runner
records driver hash, command and logs; it does not invent a production endpoint
or use a mock service as production evidence. Credentials must not appear in
logs/artifacts. The driver must place probe artifacts alongside its supplied
output path, and any other temporary data outside the sealed package. Use
`--gate production_container_isolation_if_hosted` with a separate output name
for authorized hosted containment probes. Existing independently collected
observations can be imported:

```sh
python release_tools/isaes_completion_qualification.py import-external ../isaes-external-qualification/collected-deployment.json --output ../isaes-external-qualification/deployment-import-001.json
```

Imports remain `external_review_required` after structural and hash checks.
The operator's observations and deployment/containment adequacy require review.
Failed observed behavior remains failed. Neither installed Docker nor local BYOS
fixtures qualify a real deployed boundary by themselves.

## Qualification summary and release integration

During the release build, before sealing the package:

```sh
python release_tools/isaes_completion_qualification.py summarize --performance validation_r34_isaes_ai/performance.json --browser validation_r34_isaes_ai/completion-browser/isaes-completion-browser.json --large-array validation_r34_isaes_ai/large-array-browser/isaes-large-array-browser.json --output validation_r34_isaes_ai/acceptance_status.json
```

After packaging, produce a separate supplemental summary instead. This example
assumes the three named external receipts have been collected; omit any
unavailable optional receipt argument so its gate remains pending:

```sh
python release_tools/isaes_completion_qualification.py summarize --performance validation_r34_isaes_ai/performance.json --browser validation_r34_isaes_ai/completion-browser/isaes-completion-browser.json --large-array validation_r34_isaes_ai/large-array-browser/isaes-large-array-browser.json --windows ../isaes-external-qualification/windows-001.json --study ../isaes-external-qualification/study-import-001.json --external ../isaes-external-qualification/byos-probe-001.json --output ../isaes-external-qualification/acceptance-status-001.json
```

Supply `--windows`, `--study`, and repeatable `--external` receipts when present.
The developer automated workload result can pass while `original_scope_status`
remains pending for missing native execution, real researcher observations or
applicable deployment evidence. The summary lists every missing gate and never
turns imported declarations into an all-scope approval. Final release review
must retain these distinctions and record any independently established external
acceptance separately; changing a receipt's status field is rejected.

The main release validator consumes `verify_performance`, `verify_browser` and
the summary helper. Build-stage receipts and nested run/evidence artifacts belong
to `validation_r34_isaes_ai`, outside the source inventory they bind.
Post-release external supplements remain outside the entire sealed package;
their evidence paths resolve relative to each supplemental receipt's directory.
Historical 527-test ISAES evidence remains historical. Current measurements must
come from the current frozen source, with current commands and runtime recorded.

## Reporting 2.2.0 and optional AI scope

Current package verification uses `python release_tools/validate_r34_isaes_ai.py verify --package`. The preceding 2.1.0 receipts remain unchanged in `validation_r34_isaes_completion`; do not rerun its historical qualification in place. New receipts record the SDK correction, assistant evidence/proposal boundary, reviewed application and existing-provider compatibility. Protocol fixtures are explicitly test doubles, not live provider scientific evaluations.

The report assistant is disabled by default. An operator may set `PF_ISAES_AI_ENABLED=1` for controlled evaluation with existing provider/BYOK settings and institutional controls. This opt-in is not an acceptance receipt. Broad activation still requires intended-environment qualification and representative researcher observations. Evaluate the actual selected provider/model on scientific grounding, unsupported claims, preserved negative findings and instruction-bearing literature; local structural checks do not establish semantic truth. Ordinary reporting and exports must continue to pass with AI access disabled and credentials absent.