# Consolidated r34: redeploy the existing native HF service

This is the native HF upgrade route for the consolidated full r34 release.
It preserves your controller/code/runtime Spaces, Bucket and prefix, tokens,
Variables, Secrets, users, projects and encrypted state. No external PostgreSQL
service is needed. Do not apply the earlier patch ZIPs after this release.
The complete fresh-install procedure remains in
[HUGGING_FACE_NATIVE_DEPLOYMENT.md](HUGGING_FACE_NATIVE_DEPLOYMENT.md).

The release incorporates the approved native latency, startup, response,
runtime, WAL batching, ownership queue and persistence response recovery fixes,
plus applicable shared application improvements. It retains native ownership
fencing and durable acknowledgement; there is no HTTP ownership bypass.
Schema **10** is applied on native startup. Prepare a verified backup before
startup: schema-9 or older application code must not serve the migrated database.
This full release includes both Qualified Lifecycle and Recovery Correction
packages, public email, account-wide admission and execution recovery. See
[R34_PUBLIC_SERVICE_DEPLOYMENT.md](R34_PUBLIC_SERVICE_DEPLOYMENT.md) before opening registration.

## 1. Extract and configure local tools

Check the new archive against its companion `.sha256`, extract the `.xz`, then
extract the `.zip` into a new folder. Preserve the old release and generated
contexts. In **Windows CMD**, set actual paths and existing HF resource names:

```bat
certutil -hashfile "FULL_PATH_TO_NEW_RELEASE.zip.xz" SHA256
set "PF_PROJECT_ROOT=D:\pf-r34-consolidated\privileged-frame-simulation"
set "PF_HF_TOOLS=%USERPROFILE%\pf-hf-tools-r34-20261005"
set "PF_HF_OWNER=YOUR_HF_USERNAME_OR_ORGANIZATION"
set "PF_HF_APP_REPO=%PF_HF_OWNER%/pf-simulation-saas"
set "PF_HF_CODE_REPO=%PF_HF_OWNER%/pf-simulation-runner"
set "PF_HF_RUNTIME_REPO=%PF_HF_OWNER%/pf-runtime-r34"
set "PF_HF_BUCKET_NAME=pf-simulation-state"
set "PF_HF_BACKUP_BUCKET_NAME=pf-simulation-state-backup-REPLACE_WITH_UNIQUE_SUFFIX"
set "PF_HF_UPLOAD=%USERPROFILE%\pf-r34-consolidated-generated"
set "PF_HF_DEPLOY_STATE=%USERPROFILE%\pf-r34-consolidated-deployment"
call "%PF_HF_TOOLS%\Scripts\activate.bat"
cd /d "%PF_PROJECT_ROOT%"
python -c "import sys,huggingface_hub; print(sys.executable); print(sys.version.split()[0]); print(huggingface_hub.__version__)"
hf auth whoami
```

Use the existing Python 3.12 tools environment with `huggingface_hub==1.32.0`
and `truststore>=0.10,<1`; see section 2 of the full guide if preparation is
needed. The generated and deployment paths must not already exist. Choose new
suffixes if they do. Local `set` statements do not change HF Variables.

## 2. Drain, pause and back up existing state

Keep users and scheduled clients out of the maintenance window. Let simulations,
private workflows, assistant work and result collection finish, or cancel through
PF and verify terminal HF Jobs state. Confirm no other controller writes this
Bucket/prefix. Record current controller/code commits, code SHA256, runtime image,
Variables and secure references to stable keys. Do not regenerate keys.

```bat
hf spaces pause "%PF_HF_APP_REPO%"
hf spaces info "%PF_HF_APP_REPO%" --expand sha,runtime
```

Require `PAUSED`. Complete the private Bucket-to-Bucket backup and source/backup
manifest comparison in [section 11.3](HUGGING_FACE_NATIVE_DEPLOYMENT.md#113-create-and-verify-a-private-hf-backup).
Keep the verified copy separate from active state. It includes encrypted quota
usage and reservation history. Preserve `PF_HF_MAX_STORAGE_BYTES` at its approved
value; for the service already raised to 32 GiB it is **34359738368**.
Keep the encrypted meter and Space Variable aligned. Do not reduce it during this upgrade. Deleting files does
not automatically refund conservative lifetime admission.

This backup is the only additional resource in this upgrade procedure; existing
Spaces, production Bucket, tokens and runtime Secrets are reused. If you already
have an equivalent verified, consistent pre-upgrade backup under your recovery
policy, retain its identity and evidence instead of creating a duplicate.

## 3. Generate and qualify matching code

```bat
python deploy\saas\huggingface_native\build_space.py --output "%PF_HF_UPLOAD%"
python deploy\saas\huggingface_native\deploy_release.py verify --bundle "%PF_HF_UPLOAD%"
python deploy\saas\huggingface_native\deploy_release.py --system-ca upload-code --bundle "%PF_HF_UPLOAD%" --repo "%PF_HF_CODE_REPO%"
```

Require `local_bundle_verified`, then `private_code_uploaded`. Save the returned
`code_revision` and `code_sha256`. The latter is the **private code ZIP** hash,
not the full `.zip.xz` release hash. The upload is guarded by the current private
code repository commit and verified against exact remote file identities.

Compare the newly generated `runtime.zip` against the retained context used for
your qualified image, as in section 11.4. Reuse its digest only if those bytes and
image provenance match. Otherwise upload only the generated public `runtime/`,
wait for its build and obtain the new immutable digest using sections 4–5. Keep
code Spaces private. The controller remains Private during maintenance; use
Protected visibility only after the public readiness gates pass.

Set local pins from the verified results:

```bat
set "PF_HF_CODE_REVISION=NEW_40_CHARACTER_COMMIT"
set "PF_HF_CODE_SHA256=NEW_64_CHARACTER_PRIVATE_CODE_SHA256"
set "PF_HF_JOB_IMAGE=YOUR_REGISTRY_IMAGE_AT_SHA256_DIGEST"
python deploy\saas\huggingface_native\qualify.py --namespace "%PF_HF_OWNER%" --runtime-image "%PF_HF_JOB_IMAGE%" --code-repo "%PF_HF_CODE_REPO%" --code-revision "%PF_HF_CODE_REVISION%" --code-sha256 "%PF_HF_CODE_SHA256%" --flavor cpu-basic --system-ca
```

Review the dry-run specification. Run the same command with `--submit` to launch
one billable confinement/scientific-runtime qualification Job. Require
`COMPLETED`, `confinement_probe_passed=true` and
`scientific_runtime_probe_passed=true`. Use your intended supported flavor in
both calls. If submission is uncertain, inspect the existing Job before retrying.
Qualify a distinct assistant image separately against the same new code pins.

## 4. Update code pins, preserving state configuration

In the existing controller's HF **Settings**, set `PF_HF_CODE_REPO`,
`PF_HF_CODE_REVISION`, `PF_HF_CODE_SHA256` and the qualified `PF_HF_JOB_IMAGE`.
Keep the existing namespace, Bucket, prefix, quotas, stable Secrets, email configuration, public
origin and budgets. Preserve `PF_SAAS_DEPLOYMENT_PROFILE=single`; no managed-DB
URL is introduced. Remove `PF_HF_BOOTSTRAP` from both Variables and Secrets for
this initialized service. Do not reinstate first-time administrator credentials.

Review shared bounded settings in
[the shared capacity settings guide](R34_SHARED_CAPACITY_SETTINGS.md); retain
finite admitted capacity within the native controller's connection and worker
limits. Defaults are not a claim of hundreds of thousands of concurrent users.
If editing HF settings restarts the controller, pause it again before proceeding.

## 5. Prepare, apply and verify the controller

Run these **one at a time**, stopping when a command fails:

```bat
python deploy\saas\huggingface_native\deploy_release.py --system-ca prepare --bundle "%PF_HF_UPLOAD%" --space "%PF_HF_APP_REPO%" --state-dir "%PF_HF_DEPLOY_STATE%"
python deploy\saas\huggingface_native\deploy_release.py --system-ca apply --state-dir "%PF_HF_DEPLOY_STATE%"
hf spaces wait "%PF_HF_APP_REPO%" --timeout 25m
python deploy\saas\huggingface_native\deploy_release.py --system-ca status --state-dir "%PF_HF_DEPLOY_STATE%"
hf spaces logs "%PF_HF_APP_REPO%" --tail 200
```

- `prepare` makes no remote changes. It verifies the generated seal, immutable
  private code, current controller revision and public configuration fingerprint.
  It lists exact obsolete release-file deletions, including the old extra
  qualification registries responsible for the earlier clean-check mismatch.
  Unexpected unrelated files stop it.
- `apply` confirms pause, rechecks the plan and submits one parent-guarded commit,
  then requests restart. It does not edit Variables, Secrets, Bucket objects,
  quota values or database rows. The running new application handles schema
  migration during its ordinary native startup.
- `status` verifies exact controller bytes and the running revision. Require
  `full_release_installed_running`, `source_bytes_verified=true` and
  `running_revision_verified=true`. The flags for HTTP readiness and performance
  remain false until separately measured; `RUNNING` alone does not prove them.

If apply has an uncertain response, run `status` before another write. It can
recognize an exact completed commit and record it without repeating it. If that
commit remains `PAUSED`, use the existing `hf spaces restart` command, wait, then
check status. If preparation's revision or configuration is stale, inspect the
change and prepare into a new directory; never force past the guard.

## 6. Accept the new release before reopening access

Sign in using an existing PF account at the direct application URL. Check:

1. Existing organizations, projects, saved runs, attachments, BYOK and BYOS state.
2. Archive and restore a disposable organization and project, recording request
   and visible UI completion timing. Confirm no new ownership-bypass mode.
3. A small simulation through collection, plots/report and authenticated Run ZIP
   download; a separate cancellation through terminal provider state.
4. A new saved record surviving a controlled restart, with the same quota limit,
   preserved usage history and no duplicate provider submission.
5. Representative private workflow and assistant operations with bounded settings.

Retain the full-release checksum, generated manifests, code/runtime/controller
pins and acceptance results. The user's successful earlier native patch tests
are valuable prior evidence; the newly consolidated release still needs this
acceptance run. No live HF deployment or live performance result is implied by
the packaged offline tests.

If startup or acceptance fails, preserve the same state and obtain the first
relevant log error. Do not repeatedly restart, enable bootstrap, reset counters,
change keys/prefix, reapply old patches or switch back to schema-9 or older code against
migrated data. Use a reviewed forward repair or a verified compatible state
restore as described in [the recovery guide](HUGGING_FACE_R34_RECOVERY.md).


## 2026-10-16 assistant patch consolidation

This full release includes the approved Assistant Recovery, Loading and Opening
changes. Do not reapply those controller overlays after deploying the matching
consolidated bundle. The Orbit Viewer Header proposal is not included. Existing
services already running all three assistant patches need no additional patch
merely because a full archive has been produced.

The generated Space README retains `startup_duration_timeout: 2h`. This is a
startup-health allowance, not the expected startup duration. After a restart,
allow database and private-file restoration to complete; repeated restarts can
repeat that work. Require `Application startup complete` and Uvicorn serving in
the current process logs before judging interactive assistant latency. HF's
`RUNNING` stage alone is not a substitute for application readiness.

After an intentional full-release upgrade, check saved conversation reopening,
attachments and response retention, **Open selected run** capture as a separate
action, and workflow authoring. Ordinary opening should not silently capture a
run. Retain settings, Secrets, privacy, state Bucket, prefix and finite capacity
limits. The release itself changes no live configuration or application data.
See [the consolidation record](../R34_ASSISTANT_CONSOLIDATION_20261016.md) for the
operator's reported result and the limits of targeted local validation.