PF research library · 7 min read
Consolidated r34: redeploy the existing native HF service
r34 documentation · native-upgrade.md · Formatted reading copy
On this page
This is the native HF upgrade route for the consolidated full r34 release. It preserves your controller/code/runtime Spaces, Bucket and prefix, tokens, Variables, Secrets, users, projects and encrypted state. No external PostgreSQL service is needed. Do not apply the earlier patch ZIPs after this release. The complete fresh-install procedure remains in HUGGING_FACE_NATIVE_DEPLOYMENT.md.
The release incorporates the approved native latency, startup, response, runtime, WAL batching, ownership queue and persistence response recovery fixes, plus applicable shared application improvements. It retains native ownership fencing and durable acknowledgement; there is no HTTP ownership bypass. Schema 10 is applied on native startup. Prepare a verified backup before startup: schema-9 or older application code must not serve the migrated database. This full release includes both Qualified Lifecycle and Recovery Correction packages, public email, account-wide admission and execution recovery. See R34_PUBLIC_SERVICE_DEPLOYMENT.md before opening registration.
1. Extract and configure local tools
Check the new archive against its companion .sha256, extract the .xz, then
extract the .zip into a new folder. Preserve the old release and generated
contexts. In Windows CMD, set actual paths and existing HF resource names:
certutil -hashfile "FULL_PATH_TO_NEW_RELEASE.zip.xz" SHA256
set "PF_PROJECT_ROOT=D:\pf-r34-consolidated\privileged-frame-simulation"
set "PF_HF_TOOLS=%USERPROFILE%\pf-hf-tools-r34-20261005"
set "PF_HF_OWNER=YOUR_HF_USERNAME_OR_ORGANIZATION"
set "PF_HF_APP_REPO=%PF_HF_OWNER%/pf-simulation-saas"
set "PF_HF_CODE_REPO=%PF_HF_OWNER%/pf-simulation-runner"
set "PF_HF_RUNTIME_REPO=%PF_HF_OWNER%/pf-runtime-r34"
set "PF_HF_BUCKET_NAME=pf-simulation-state"
set "PF_HF_BACKUP_BUCKET_NAME=pf-simulation-state-backup-REPLACE_WITH_UNIQUE_SUFFIX"
set "PF_HF_UPLOAD=%USERPROFILE%\pf-r34-consolidated-generated"
set "PF_HF_DEPLOY_STATE=%USERPROFILE%\pf-r34-consolidated-deployment"
call "%PF_HF_TOOLS%\Scripts\activate.bat"
cd /d "%PF_PROJECT_ROOT%"
python -c "import sys,huggingface_hub; print(sys.executable); print(sys.version.split()[0]); print(huggingface_hub.__version__)"
hf auth whoami
Use the existing Python 3.12 tools environment with huggingface_hub==1.32.0
and truststore>=0.10,<1; see section 2 of the full guide if preparation is
needed. The generated and deployment paths must not already exist. Choose new
suffixes if they do. Local set statements do not change HF Variables.
2. Drain, pause and back up existing state
Keep users and scheduled clients out of the maintenance window. Let simulations, private workflows, assistant work and result collection finish, or cancel through PF and verify terminal HF Jobs state. Confirm no other controller writes this Bucket/prefix. Record current controller/code commits, code SHA256, runtime image, Variables and secure references to stable keys. Do not regenerate keys.
hf spaces pause "%PF_HF_APP_REPO%"
hf spaces info "%PF_HF_APP_REPO%" --expand sha,runtime
Require PAUSED. Complete the private Bucket-to-Bucket backup and source/backup
manifest comparison in section 11.3.
Keep the verified copy separate from active state. It includes encrypted quota
usage and reservation history. Preserve PF_HF_MAX_STORAGE_BYTES at its approved
value; for the service already raised to 32 GiB it is 34359738368.
Keep the encrypted meter and Space Variable aligned. Do not reduce it during this upgrade. Deleting files does
not automatically refund conservative lifetime admission.
This backup is the only additional resource in this upgrade procedure; existing Spaces, production Bucket, tokens and runtime Secrets are reused. If you already have an equivalent verified, consistent pre-upgrade backup under your recovery policy, retain its identity and evidence instead of creating a duplicate.
3. Generate and qualify matching code
python deploy\saas\huggingface_native\build_space.py --output "%PF_HF_UPLOAD%"
python deploy\saas\huggingface_native\deploy_release.py verify --bundle "%PF_HF_UPLOAD%"
python deploy\saas\huggingface_native\deploy_release.py --system-ca upload-code --bundle "%PF_HF_UPLOAD%" --repo "%PF_HF_CODE_REPO%"
Require local_bundle_verified, then private_code_uploaded. Save the returned
code_revision and code_sha256. The latter is the private code ZIP hash,
not the full .zip.xz release hash. The upload is guarded by the current private
code repository commit and verified against exact remote file identities.
Compare the newly generated runtime.zip against the retained context used for
your qualified image, as in section 11.4. Reuse its digest only if those bytes and
image provenance match. Otherwise upload only the generated public runtime/,
wait for its build and obtain the new immutable digest using sections 4–5. Keep
code Spaces private. The controller remains Private during maintenance; use
Protected visibility only after the public readiness gates pass.
Set local pins from the verified results:
set "PF_HF_CODE_REVISION=NEW_40_CHARACTER_COMMIT"
set "PF_HF_CODE_SHA256=NEW_64_CHARACTER_PRIVATE_CODE_SHA256"
set "PF_HF_JOB_IMAGE=YOUR_REGISTRY_IMAGE_AT_SHA256_DIGEST"
python deploy\saas\huggingface_native\qualify.py --namespace "%PF_HF_OWNER%" --runtime-image "%PF_HF_JOB_IMAGE%" --code-repo "%PF_HF_CODE_REPO%" --code-revision "%PF_HF_CODE_REVISION%" --code-sha256 "%PF_HF_CODE_SHA256%" --flavor cpu-basic --system-ca
Review the dry-run specification. Run the same command with --submit to launch
one billable confinement/scientific-runtime qualification Job. Require
COMPLETED, confinement_probe_passed=true and
scientific_runtime_probe_passed=true. Use your intended supported flavor in
both calls. If submission is uncertain, inspect the existing Job before retrying.
Qualify a distinct assistant image separately against the same new code pins.
4. Update code pins, preserving state configuration
In the existing controller's HF Settings, set PF_HF_CODE_REPO,
PF_HF_CODE_REVISION, PF_HF_CODE_SHA256 and the qualified PF_HF_JOB_IMAGE.
Keep the existing namespace, Bucket, prefix, quotas, stable Secrets, email configuration, public
origin and budgets. Preserve PF_SAAS_DEPLOYMENT_PROFILE=single; no managed-DB
URL is introduced. Remove PF_HF_BOOTSTRAP from both Variables and Secrets for
this initialized service. Do not reinstate first-time administrator credentials.
Review shared bounded settings in the shared capacity settings guide; retain finite admitted capacity within the native controller's connection and worker limits. Defaults are not a claim of hundreds of thousands of concurrent users. If editing HF settings restarts the controller, pause it again before proceeding.
5. Prepare, apply and verify the controller
Run these one at a time, stopping when a command fails:
python deploy\saas\huggingface_native\deploy_release.py --system-ca prepare --bundle "%PF_HF_UPLOAD%" --space "%PF_HF_APP_REPO%" --state-dir "%PF_HF_DEPLOY_STATE%"
python deploy\saas\huggingface_native\deploy_release.py --system-ca apply --state-dir "%PF_HF_DEPLOY_STATE%"
hf spaces wait "%PF_HF_APP_REPO%" --timeout 25m
python deploy\saas\huggingface_native\deploy_release.py --system-ca status --state-dir "%PF_HF_DEPLOY_STATE%"
hf spaces logs "%PF_HF_APP_REPO%" --tail 200
preparemakes no remote changes. It verifies the generated seal, immutable private code, current controller revision and public configuration fingerprint. It lists exact obsolete release-file deletions, including the old extra qualification registries responsible for the earlier clean-check mismatch. Unexpected unrelated files stop it.applyconfirms pause, rechecks the plan and submits one parent-guarded commit, then requests restart. It does not edit Variables, Secrets, Bucket objects, quota values or database rows. The running new application handles schema migration during its ordinary native startup.statusverifies exact controller bytes and the running revision. Requirefull_release_installed_running,source_bytes_verified=trueandrunning_revision_verified=true. The flags for HTTP readiness and performance remain false until separately measured;RUNNINGalone does not prove them.
If apply has an uncertain response, run status before another write. It can
recognize an exact completed commit and record it without repeating it. If that
commit remains PAUSED, use the existing hf spaces restart command, wait, then
check status. If preparation's revision or configuration is stale, inspect the
change and prepare into a new directory; never force past the guard.
6. Accept the new release before reopening access
Sign in using an existing PF account at the direct application URL. Check:
- Existing organizations, projects, saved runs, attachments, BYOK and BYOS state.
- Archive and restore a disposable organization and project, recording request and visible UI completion timing. Confirm no new ownership-bypass mode.
- A small simulation through collection, plots/report and authenticated Run ZIP download; a separate cancellation through terminal provider state.
- A new saved record surviving a controlled restart, with the same quota limit, preserved usage history and no duplicate provider submission.
- Representative private workflow and assistant operations with bounded settings.
Retain the full-release checksum, generated manifests, code/runtime/controller pins and acceptance results. The user's successful earlier native patch tests are valuable prior evidence; the newly consolidated release still needs this acceptance run. No live HF deployment or live performance result is implied by the packaged offline tests.
If startup or acceptance fails, preserve the same state and obtain the first relevant log error. Do not repeatedly restart, enable bootstrap, reset counters, change keys/prefix, reapply old patches or switch back to schema-9 or older code against migrated data. Use a reviewed forward repair or a verified compatible state restore as described in the recovery guide.
2026-10-16 assistant patch consolidation
This full release includes the approved Assistant Recovery, Loading and Opening changes. Do not reapply those controller overlays after deploying the matching consolidated bundle. The Orbit Viewer Header proposal is not included. Existing services already running all three assistant patches need no additional patch merely because a full archive has been produced.
The generated Space README retains startup_duration_timeout: 2h. This is a
startup-health allowance, not the expected startup duration. After a restart,
allow database and private-file restoration to complete; repeated restarts can
repeat that work. Require Application startup complete and Uvicorn serving in
the current process logs before judging interactive assistant latency. HF's
RUNNING stage alone is not a substitute for application readiness.
After an intentional full-release upgrade, check saved conversation reopening, attachments and response retention, Open selected run capture as a separate action, and workflow authoring. Ordinary opening should not silently capture a run. Retain settings, Secrets, privacy, state Bucket, prefix and finite capacity limits. The release itself changes no live configuration or application data. See the consolidation record for the operator's reported result and the limits of targeted local validation.