PFPF Simulation
Get started Source code & licensing
Resources / Research guide

PF research library · 7 min read

r34 public service deployment and scale qualification

r34 documentation · public-service.md

Download formatted Markdown
On this page
  1. Choose the architecture before opening registration
  2. Maintain privacy while making the application reachable
  3. Public registration and real account email
  4. Admission is shared across organizations
  5. Schema and lifecycle upgrade
  6. Qualify and expand gradually

This release integrates PF_r34_HF_Lifecycle_Qualified_20261009 followed by PF_r34_HF_Recovery_Correction_20261009 into the full release whose incoming SHA256 is 140cd2f3bbf55dc48f4095566c0dc9f367d85772666beb5fa60c87dac80d32de. It also extends public registration, durable email, account-wide admission, distributed recovery, managed HF accounting and mixed-workload qualification. Do not install those patch ZIPs again over this release.

The software can be configured from one user upward. This archive does not certify hundreds of thousands of simultaneously active users. That target requires a provisioned fleet, provider allocations and a successful measured campaign with the intended workload, data volume, regions and failure cases. Changing a numeric limit or accepting 200,000 credentials is not that evidence.

Choose the architecture before opening registration

Deployment Data authority and execution Public-service scope
Standalone Local application and local files Preserved; no SaaS infrastructure required
Native HF, profile single One fenced controller, supervised PostgreSQL, encrypted HF recovery, HF Jobs Existing deployment upgrade; bounded pilot after readiness checks; never run two native controllers against one prefix
Managed HF, profile hf_managed Shared external PostgreSQL, encrypted HF objects, scoped leases, HF Jobs Multiple coordinated API/worker replicas; requires external database, shared-state design, approved HF capacity and network connectivity
Linux/Docker, profile distributed Shared PostgreSQL, private S3, dedicated Docker workers, private shared assistant/BYOS files Independent API and worker scaling behind a controlled HTTPS ingress

The existing ignalum/pf-simulation-saas can first receive the full native upgrade while retaining its identity, users, keys and storage. This does not turn its single controller into a fleet. A migration to managed HF or a distributed Linux/Docker deployment is a separate, explicit architecture step. Neither the release builder nor its readiness checker provisions resources.

For native upgrade follow the existing-service guide. For managed HF follow the managed migration guide. For Linux/Docker follow the distributed guide.

Maintain privacy while making the application reachable

For HF, a Protected controller Space makes the running application reachable while retaining private repository source. Keep the scientific code Space, Bucket, backups and credential material private. Do not switch the code Space or the full source repository to Public. Protected visibility requires a supporting HF plan; see the official visibility documentation. The deployment tools never change visibility for you.

Use the actual HTTPS application origin as PF_SAAS_PUBLIC_URL, with no path. Verify direct and embedded access, cookie/CSRF behavior, origin checks and the exact forwarded-address contract before allowing untrusted users. See public launch readiness. Never enter 0.0.0.0/0 or an invented HF proxy network to make rate limits appear functional. If HF does not supply a verifiable ingress contract for the chosen topology, use a controlled ingress/gateway and prove that its origin cannot be bypassed.

Only platform administrators may open HF accounting/recovery pages or APIs. Organization owners and organization administrators do not acquire this role. Registration creates a non-administrator account and requires email verification. Private assistant workspaces, project grants, BYOK and BYOS remain user scoped.

Public registration and real account email

Keep the existing PF_SAAS_SECRET_KEY: it protects sessions and encrypts queued account email. Retain the existing HF state key and BYOK/BYOS credential keys. Do not regenerate them during an upgrade.

For local accounts configure:

PF_SAAS_AUTH_MODE=local
PF_SAAS_REGISTRATION_ENABLED=true
PF_SAAS_PUBLIC_URL=https://YOUR_APPLICATION_ORIGIN
PF_SAAS_MAIL_DELIVERY_WORKER_ENABLED=true

Retain existing encrypted SMTP, or choose the optional Resend HTTPS provider. For SMTP2GO on HF, preserve PF_SAAS_MAIL_MODE=smtp, host mail.smtp2go.com, port 443, PF_SAAS_SMTP_SSL=true, PF_SAAS_SMTP_STARTTLS=false, the verified PF_SAAS_SMTP_FROM and existing SMTP credentials in Secrets. No provider migration is required. For Resend use PF_SAAS_MAIL_MODE=resend, a verified PF_SAAS_MAIL_FROM and PF_SAAS_MAIL_API_KEY in the secret manager/HF Secrets, never a public Variable. The HTTPS provider uses a fixed endpoint and bounded timeouts. Both transports use the encrypted durable outbox. Registration, password-reset and invitation tokens are committed atomically with their queued email. Delivery is asynchronous; HTTP acceptance and provider acceptance do not prove arrival in an inbox. Follow email setup, retry and diagnosis, including real delivery and single-use-token checks on an isolated staging account.

HF documents egress on 80, 443 and 8080. The checker accepts SMTP on these ports with exactly one TLS mode; provider support and real delivery still need testing. Ordinary SMTP on 465/587 is not a reliable HF configuration. Linux/Docker may retain encrypted SMTP on its permitted ports. A managed PostgreSQL endpoint on 5432 likewise needs demonstrated permitted connectivity; choose a supported endpoint/port or provider-approved network arrangement. An HTTPS database administration URL is not a PostgreSQL protocol endpoint. See HF networking.

Admission is shared across organizations

The existing organization limits remain. Add explicit account caps to prevent one person multiplying their allowance by creating or joining organizations. These conservative example values are a starting policy, not a capacity claim:

PF_SAAS_ACCOUNT_MAX_CONCURRENT_JOBS=1
PF_SAAS_ACCOUNT_MAX_QUEUED_JOBS=20
PF_SAAS_ACCOUNT_MONTHLY_COMPUTE_SECONDS=36000
PF_SAAS_ACCOUNT_MAX_STORAGE_BYTES=5368709120
PF_SAAS_ACCOUNT_MAX_OWNED_ORGANIZATIONS=1
PF_SAAS_ACCOUNT_MONTHLY_AI_MESSAGES=1000
PF_SAAS_ACCOUNT_AI_MAX_CONCURRENT_REQUESTS=1
PF_SAAS_ACCOUNT_AI_REQUESTS_PER_MINUTE=30
PF_SAAS_ACCOUNT_AI_TOKENS_PER_MINUTE=250000
PF_SAAS_ACCOUNT_AI_MONTHLY_COST_MICRO_USD=0
PF_SAAS_ACCOUNT_ASSISTANT_MAX_QUEUED_OPERATIONS=4
PF_SAAS_ACCOUNT_ASSISTANT_MAX_CONCURRENT_OPERATIONS=1

The example leaves monetary AI accounting disabled. To enable a cost cap, set both PF_SAAS_AI_INPUT_COST_PER_MILLION_MICRO_USD and PF_SAAS_AI_OUTPUT_COST_PER_MILLION_MICRO_USD to reviewed positive conservative tariffs for the allowed providers/models, then set a positive account cost cap. Zero tariffs with a positive cost cap are rejected at startup.

New account caps default to zero (disabled) to preserve existing private deployments. Public readiness requires an explicit bounded policy. Configure the same policy on every API, mail and worker process. A billable provider may charge independently of PF's counters; set provider budgets as well.

The migration attributes historical attempts from their jobs and profiles only where retained operations establish a unique user. Unknown ownership is not guessed. Enabling account caps fails closed on unresolved charged usage; retain the source records and resolve attribution through an audited maintenance procedure before public admission. Creating a new organization does not erase the account's existing reservations or retained bytes.

Schema and lifecycle upgrade

The SaaS schema is 10. Drain all API writers, assistant continuations and workers; confirm corresponding executions terminal, then create and verify a consistent backup. Native HF applies the additive schema migration while its single controller starts. Distributed Linux/Docker runs the explicit existing python -m pf_saas init migration command with the same service configuration before any new application replica serves. Managed HF uses guarded maintenance and verifies authority and charged totals before activation.

Do not mix old and new application processes during migration. A rollback is a coordinated restore of the pre-upgrade database, objects and code, with all writers stopped. Restoring only an older code image is not a schema rollback. The additive migration preserves retained files, source accounting records, users, organization/project grants and established execution authority.

The HF storage policy remains conservative_lifetime by default. Platform administration can switch to safe_lifecycle with a generation check and an audit reason. Future output reservations settle from verified publication; physical deletion and refund require recorded ownership, terminal execution, recovery safety and idempotent receipts. Legacy charges require a reviewed reconciliation plan and its unchanged digest. Deleting an arbitrary object or pf-code.zip is not a valid quota reconciliation method.

Keep the currently approved ceiling. For the deployment already raised to 32 GiB this is 34359738368 in both the encrypted meter and PF_HF_MAX_STORAGE_BYTES. Do not set it back to 16 GiB based on an expectation of refunds. First measure the reconciled retained usage and required reserve. Safe accounting prevents avoidable lifetime charges; it cannot guarantee that real retained data and recovery history fit within 16 GiB.

Managed HF additionally coordinates reclamation with external database backup retention. Its audited recovery floor must establish that no older PITR point or native recovery root can still reference deleted data. See managed accounting. An expired timer alone does not prove backup safety.

Execution recovery runs at startup and periodically. It acquires the owner's fence and proves that the corresponding execution has stopped before marking an operation interrupted and releasing temporary reservations. Retained files remain. Docker child executions receive durable launch/terminal journals; ambiguous legacy executions remain held until evidence resolves them. Recovery is a general lifecycle mechanism, not a hard-coded repair for one operation ID.

Qualify and expand gradually

  1. Build the complete native/managed contexts from this release and verify the exact private code ZIP and image pins. Check that new mail, recovery, admission and accounting modules are included in the generated contexts.
  2. Run an isolated staging migration with a protected copy of representative state. Test registration, verification, reset, invitation, private workflows, permissions, concurrent users, cancellation, recovery and accounting.
  3. Run the offline public readiness checker against configuration and retained ingress/email/isolation/restore evidence.
  4. Run the mixed-workload qualification campaign on an explicitly marked isolated target. Include real active workflows, provider capacity, BYOK/BYOS, failures, recovery, backlog drainage and a soak. Collect database, API, object-store, worker and provider measurements.
  5. Approve the observed envelope, configure finite admission and expand public access in stages. At saturation the service must bound/deny work with retry guidance while preserving existing reservations and private results.

The load generator is sharded; the intended user count is separate from executing-job concurrency. For example, 200,000 people browsing or editing is a different compute requirement from 200,000 simultaneous scientific jobs. Both must be represented explicitly in the campaign contract and provisioned within provider limits. The release's local tests and previous isolated HF qualification records cannot substitute for that campaign.