PF research library · 7 min read
r34 public service deployment and scale qualification
r34 documentation · public-service.md · Formatted reading copy
On this page
This release integrates PF_r34_HF_Lifecycle_Qualified_20261009 followed by
PF_r34_HF_Recovery_Correction_20261009 into the full release whose incoming
SHA256 is 140cd2f3bbf55dc48f4095566c0dc9f367d85772666beb5fa60c87dac80d32de.
It also extends public registration, durable email, account-wide admission,
distributed recovery, managed HF accounting and mixed-workload qualification.
Do not install those patch ZIPs again over this release.
The software can be configured from one user upward. This archive does not certify hundreds of thousands of simultaneously active users. That target requires a provisioned fleet, provider allocations and a successful measured campaign with the intended workload, data volume, regions and failure cases. Changing a numeric limit or accepting 200,000 credentials is not that evidence.
Choose the architecture before opening registration
| Deployment | Data authority and execution | Public-service scope |
|---|---|---|
| Standalone | Local application and local files | Preserved; no SaaS infrastructure required |
Native HF, profile single |
One fenced controller, supervised PostgreSQL, encrypted HF recovery, HF Jobs | Existing deployment upgrade; bounded pilot after readiness checks; never run two native controllers against one prefix |
Managed HF, profile hf_managed |
Shared external PostgreSQL, encrypted HF objects, scoped leases, HF Jobs | Multiple coordinated API/worker replicas; requires external database, shared-state design, approved HF capacity and network connectivity |
Linux/Docker, profile distributed |
Shared PostgreSQL, private S3, dedicated Docker workers, private shared assistant/BYOS files | Independent API and worker scaling behind a controlled HTTPS ingress |
The existing ignalum/pf-simulation-saas can first receive the full native
upgrade while retaining its identity, users, keys and storage. This does not
turn its single controller into a fleet. A migration to managed HF or a
distributed Linux/Docker deployment is a separate, explicit architecture step.
Neither the release builder nor its readiness checker provisions resources.
For native upgrade follow the existing-service guide. For managed HF follow the managed migration guide. For Linux/Docker follow the distributed guide.
Maintain privacy while making the application reachable
For HF, a Protected controller Space makes the running application reachable while retaining private repository source. Keep the scientific code Space, Bucket, backups and credential material private. Do not switch the code Space or the full source repository to Public. Protected visibility requires a supporting HF plan; see the official visibility documentation. The deployment tools never change visibility for you.
Use the actual HTTPS application origin as PF_SAAS_PUBLIC_URL, with no path.
Verify direct and embedded access, cookie/CSRF behavior, origin checks and the
exact forwarded-address contract before allowing untrusted users. See
public launch readiness. Never enter 0.0.0.0/0
or an invented HF proxy network to make rate limits appear functional.
If HF does not supply a verifiable ingress contract for the chosen topology,
use a controlled ingress/gateway and prove that its origin cannot be bypassed.
Only platform administrators may open HF accounting/recovery pages or APIs. Organization owners and organization administrators do not acquire this role. Registration creates a non-administrator account and requires email verification. Private assistant workspaces, project grants, BYOK and BYOS remain user scoped.
Public registration and real account email
Keep the existing PF_SAAS_SECRET_KEY: it protects sessions and encrypts queued
account email. Retain the existing HF state key and BYOK/BYOS credential keys.
Do not regenerate them during an upgrade.
For local accounts configure:
PF_SAAS_AUTH_MODE=local
PF_SAAS_REGISTRATION_ENABLED=true
PF_SAAS_PUBLIC_URL=https://YOUR_APPLICATION_ORIGIN
PF_SAAS_MAIL_DELIVERY_WORKER_ENABLED=true
Retain existing encrypted SMTP, or choose the optional Resend HTTPS provider.
For SMTP2GO on HF, preserve PF_SAAS_MAIL_MODE=smtp, host mail.smtp2go.com,
port 443, PF_SAAS_SMTP_SSL=true, PF_SAAS_SMTP_STARTTLS=false, the verified
PF_SAAS_SMTP_FROM and existing SMTP credentials in Secrets. No provider migration
is required. For Resend use PF_SAAS_MAIL_MODE=resend, a verified
PF_SAAS_MAIL_FROM and PF_SAAS_MAIL_API_KEY in the secret manager/HF Secrets,
never a public Variable. The HTTPS provider uses a fixed endpoint and bounded
timeouts. Both transports use the encrypted durable outbox. Registration,
password-reset and invitation tokens
are committed atomically with their queued email. Delivery is asynchronous;
HTTP acceptance and provider acceptance do not prove arrival in an inbox.
Follow email setup, retry and diagnosis, including
real delivery and single-use-token checks on an isolated staging account.
HF documents egress on 80, 443 and 8080. The checker accepts SMTP on these ports with exactly one TLS mode; provider support and real delivery still need testing. Ordinary SMTP on 465/587 is not a reliable HF configuration. Linux/Docker may retain encrypted SMTP on its permitted ports. A managed PostgreSQL endpoint on 5432 likewise needs demonstrated permitted connectivity; choose a supported endpoint/port or provider-approved network arrangement. An HTTPS database administration URL is not a PostgreSQL protocol endpoint. See HF networking.
Admission is shared across organizations
The existing organization limits remain. Add explicit account caps to prevent one person multiplying their allowance by creating or joining organizations. These conservative example values are a starting policy, not a capacity claim:
PF_SAAS_ACCOUNT_MAX_CONCURRENT_JOBS=1
PF_SAAS_ACCOUNT_MAX_QUEUED_JOBS=20
PF_SAAS_ACCOUNT_MONTHLY_COMPUTE_SECONDS=36000
PF_SAAS_ACCOUNT_MAX_STORAGE_BYTES=5368709120
PF_SAAS_ACCOUNT_MAX_OWNED_ORGANIZATIONS=1
PF_SAAS_ACCOUNT_MONTHLY_AI_MESSAGES=1000
PF_SAAS_ACCOUNT_AI_MAX_CONCURRENT_REQUESTS=1
PF_SAAS_ACCOUNT_AI_REQUESTS_PER_MINUTE=30
PF_SAAS_ACCOUNT_AI_TOKENS_PER_MINUTE=250000
PF_SAAS_ACCOUNT_AI_MONTHLY_COST_MICRO_USD=0
PF_SAAS_ACCOUNT_ASSISTANT_MAX_QUEUED_OPERATIONS=4
PF_SAAS_ACCOUNT_ASSISTANT_MAX_CONCURRENT_OPERATIONS=1
The example leaves monetary AI accounting disabled. To enable a cost cap, set
both PF_SAAS_AI_INPUT_COST_PER_MILLION_MICRO_USD and
PF_SAAS_AI_OUTPUT_COST_PER_MILLION_MICRO_USD to reviewed positive conservative
tariffs for the allowed providers/models, then set a positive account cost cap.
Zero tariffs with a positive cost cap are rejected at startup.
New account caps default to zero (disabled) to preserve existing private deployments. Public readiness requires an explicit bounded policy. Configure the same policy on every API, mail and worker process. A billable provider may charge independently of PF's counters; set provider budgets as well.
The migration attributes historical attempts from their jobs and profiles only where retained operations establish a unique user. Unknown ownership is not guessed. Enabling account caps fails closed on unresolved charged usage; retain the source records and resolve attribution through an audited maintenance procedure before public admission. Creating a new organization does not erase the account's existing reservations or retained bytes.
Schema and lifecycle upgrade
The SaaS schema is 10. Drain all API writers, assistant continuations and
workers; confirm corresponding executions terminal, then create and verify a
consistent backup. Native HF applies the additive schema migration while its
single controller starts. Distributed Linux/Docker runs the explicit existing
python -m pf_saas init migration command with the same service configuration
before any new application replica serves. Managed HF uses guarded maintenance
and verifies authority and charged totals before activation.
Do not mix old and new application processes during migration. A rollback is a coordinated restore of the pre-upgrade database, objects and code, with all writers stopped. Restoring only an older code image is not a schema rollback. The additive migration preserves retained files, source accounting records, users, organization/project grants and established execution authority.
The HF storage policy remains conservative_lifetime by default. Platform
administration can switch to safe_lifecycle with a generation check and an
audit reason. Future output reservations settle from verified publication;
physical deletion and refund require recorded ownership, terminal execution,
recovery safety and idempotent receipts. Legacy charges require a reviewed
reconciliation plan and its unchanged digest. Deleting an arbitrary object or
pf-code.zip is not a valid quota reconciliation method.
Keep the currently approved ceiling. For the deployment already raised to
32 GiB this is 34359738368 in both the encrypted meter and
PF_HF_MAX_STORAGE_BYTES. Do not set it back to 16 GiB based on an expectation
of refunds. First measure the reconciled retained usage and required reserve.
Safe accounting prevents avoidable lifetime charges; it cannot guarantee that
real retained data and recovery history fit within 16 GiB.
Managed HF additionally coordinates reclamation with external database backup retention. Its audited recovery floor must establish that no older PITR point or native recovery root can still reference deleted data. See managed accounting. An expired timer alone does not prove backup safety.
Execution recovery runs at startup and periodically. It acquires the owner's fence and proves that the corresponding execution has stopped before marking an operation interrupted and releasing temporary reservations. Retained files remain. Docker child executions receive durable launch/terminal journals; ambiguous legacy executions remain held until evidence resolves them. Recovery is a general lifecycle mechanism, not a hard-coded repair for one operation ID.
Qualify and expand gradually
- Build the complete native/managed contexts from this release and verify the exact private code ZIP and image pins. Check that new mail, recovery, admission and accounting modules are included in the generated contexts.
- Run an isolated staging migration with a protected copy of representative state. Test registration, verification, reset, invitation, private workflows, permissions, concurrent users, cancellation, recovery and accounting.
- Run the offline public readiness checker against configuration and retained ingress/email/isolation/restore evidence.
- Run the mixed-workload qualification campaign on an explicitly marked isolated target. Include real active workflows, provider capacity, BYOK/BYOS, failures, recovery, backlog drainage and a soak. Collect database, API, object-store, worker and provider measurements.
- Approve the observed envelope, configure finite admission and expand public access in stages. At saturation the service must bound/deny work with retry guidance while preserving existing reservations and private results.
The load generator is sharded; the intended user count is separate from executing-job concurrency. For example, 200,000 people browsing or editing is a different compute requirement from 200,000 simultaneous scientific jobs. Both must be represented explicitly in the campaign contract and provisioned within provider limits. The release's local tests and previous isolated HF qualification records cannot substitute for that campaign.