PF research library · 20 min read
Autonomous ISAES investigations
r34 documentation · isaes-autonomous.md
On this page
- Before enabling unattended work
- Set it up in the workbench
- Default limits
- What an investigation does
- Literature, claims and scientific support
- Results, presentation and retained evidence
- Pause, resume, cancel and failures
- Status and continuation rules
- Privacy, storage and deployment
- Service and authenticated HTTP interfaces
- Investigation and literature tools
- Scientific tools and generated code
- Qualification and related guides
PF Simulation r34 can delegate an unattended investigation of a completed private-workflow run to the configured research assistant. The investigator reads the retained result and report, develops a plan, gathers eligible literature, executes permitted analyses, assesses claims and prepares a presentation. It can revise its approach until it finishes or reaches a recorded control or resource limit. It does not need a chat prompt for each completed run.
This is an explicit, per-workflow choice. Ordinary deterministic ISAES reporting continues to work without an AI provider, credentials or model weights. Existing Discuss report conversations and their review/apply/run proposal sequence retain their own behavior. A standing autonomous policy grants a different, recorded scope of delegation; the investigator does not impersonate a researcher approving an ordinary discussion proposal.
Before enabling unattended work
Use an active private workflow whose program, data contract and scientific question you understand. Declare dataset meaning, units, coordinate frame or basis, sampling and uncertainty where relevant. Attach useful research materials and identify their intended role. Missing information remains a limitation; autonomy does not make a simulation empirically validated or turn unspecified data into independent observations.
Configure the existing assistant provider and a model that supports tool calls, together with its credentials and isolated scientific executor. Standalone settings and hosted BYOK/institutional controls remain in force. The application does not supply a provider account, bypass an institution's external-AI policy, or silently install scientific packages. Generated code uses the configured isolated execution route; it is never imported into the application server.
Review the standing policy before enabling it:
| Choice | What the delegation permits |
|---|---|
enabled |
New autonomous investigations for the selected workflow. Disabled is the default. |
share_with_provider |
Sending the authorized investigation context and tool results to the configured external provider. Enabling unattended provider work requires this explicit choice. |
online_research |
Contextual discovery and bounded retrieval of online sources. Associated retained materials remain a separate evidence source. |
allow_generated_code |
Proposed Python analyses and scientific/presentation code in the isolated executor. The code, execution and output remain inspectable. |
allow_followup_runs |
Separately retained scientific child runs, subject to their own applicable validation and execution controls. |
budgets |
Cumulative limits on steps, provider calls, tokens, elapsed time, sources, downloaded bytes and child runs. |
| Domain allowlist | Restricting which online hosts may be used when online research is enabled. |
The policy belongs to the authenticated owner and workflow. Each saved revision records its actor, and each investigation snapshots the revision it uses. A later policy revision does not silently increase a running job's permissions. Revocation stops work at the next control boundary; it does not recall a request already sent to a provider.
Set it up in the workbench
- Configure the existing assistant provider/model and isolated executor, then start or restart the workbench. Standalone operators use
configure_ai_research_assistant.bator the existing configuration-file/environment route. SaaS users configure permitted BYOK credentials through the existing account controls; deployment policy still governs access. - Open My workflows, select the active workflow and find Autonomous ISAES. Expand Automatic investigation permissions and limits.
- Select Automatically investigate completed runs and Allow sharing investigation evidence with my configured AI provider. Select Allow online literature research, Allow generated analysis programs and Allow follow-up simulation runs for the capabilities you intend to delegate. Review Allowed research domains and each cumulative budget.
- Choose Save automatic investigation settings. The panel shows the saved policy revision and whether provider settings are configured. Configuration does not establish that a live provider request will succeed.
- Run the workflow normally. Its completed scientific run can start the separate investigation without a chat prompt. For an already completed run, use Investigate an existing completed run and Investigate selected completed run.
- Select a record under Retained investigations to inspect progress, usage, decisions, sources, claims, analyses and artifact downloads. Use Pause investigation, Resume investigation, Retry investigation or Cancel investigation as appropriate. Turn off automatic investigations revokes the saved standing delegation.
The automatic trigger is for completed private-workflow runs. Child runs created by an investigation do not recursively start more investigations. The same run and policy revision resolve to the same durable investigation instead of duplicate work. Automatic triggering excludes workflow validation runs and autonomous child runs. Changing the policy creates a new revision and blocks continuing an older job under silently changed authority. To investigate the same run under that new policy, select the completed run again.
Existing provider settings include PF_AI_PROVIDER, PF_AI_MODEL, PF_AI_API_KEY and, when needed, PF_AI_BASE_URL. Supported adapters are OpenAI Responses, OpenAI-compatible Chat Completions, DeepSeek, Gemini and Anthropic; the chosen model must actually support the required tool protocol. Set PF_AI_IMAGE_INPUTS only for an image-capable model when source-page inspection is required. See the provider/setup guide for configuration and credential handling. PF_ISAES_AI_ENABLED controls the separate reviewed report-discussion route; it is not the standing autonomous policy.
Scientific and extraction workers use the configured executor, normally the isolated Docker image selected by PF_AI_DOCKER_IMAGE. Build the current image after updating worker dependencies. Generated-code execution requires the Docker executor in ordinary operation. The developer executor is denied for autonomous generated code unless a trusted internal fixture opt-in is set; the standing user policy cannot grant that bypass. Such internal test execution does not establish production containment. Failure to admit an isolated worker does not authorize a host-execution fallback.
Default limits
All capability flags default to false. Omitting a recognized policy field uses its default; unknown fields and non-boolean flag values are rejected. Budgets are integer values and apply to each investigation across retries and resume.
| Budget field | Default | Allowed maximum |
|---|---|---|
max_steps |
64 | 512 |
max_provider_calls |
24 | 128 |
max_tokens |
120,000 | 2,000,000 |
max_seconds |
1,800 | 86,400 |
max_sources |
24 | 256 |
max_download_bytes |
67,108,864 bytes (64 MiB) | 536,870,912 bytes (512 MiB) |
max_child_runs |
4 | 64 |
Provider token admission reserves an estimate from serialized request bytes, a fixed allowance, the output cap and an image allowance when applicable. Reported provider usage reconciles that reservation; missing usage leaves the reservation charged. This bounds the application's admitted accounting, not every provider's tokenizer or billing behavior, and is not a universal hard monetary-cost cap. The provider configuration must permit at least two rounds and one tool call; smaller limits are rejected for autonomous work.
The minimum is one for each budget except max_child_runs, which may be zero. Other deployment/provider/executor limits can be stricter. At most eight processing attempts are admitted for one investigation, and each workflow retains at most 512 investigations. An interrupted active interval is charged conservatively on recovery; repeatedly restarting the application does not reset its time budget.
domain_allowlist accepts up to 64 DNS hostnames without schemes, URL paths or wildcards. An entry permits that host and its subdomains. An empty list allows eligible public hosts; it does not permit private IP addresses or internal endpoints. Include discovery API hosts as well as publication hosts when restricting online research: api.crossref.org, api.openalex.org, api.datacite.org and export.arxiv.org are the supported discovery endpoints.
What an investigation does
The retained base context identifies the workflow, scientific run, original report and their hashes. The investigator then chooses tools rather than supplying prose in place of execution.
- Read the saved question, results, findings, checks and associated literature. Record gaps and the initial scientific plan.
- Find and inspect contextual evidence when permitted. Preserve source identity, retrieved bytes, extraction details and limitations.
- Compute analyses on retained data, request a new reporting analysis or execute permitted generated code. Where a new scientific run is justified and allowed, retain it as a child with its own identity.
- Assess individual claims against the available evidence. Keep contradictions, incomplete comparisons and methodological qualifications visible.
- Create scientific figures, exact supporting data and a coherent presentation. Record the methods, decisions and source relationships used by that presentation.
- Refine the investigation or finish with a recorded result and limitations. A budget stop or failed tool is a recorded incomplete outcome, not a fabricated success.
Method selection is open-ended through generated code and compatible installed scientific libraries, within the selected capabilities and execution limits. This does not promise arbitrary packages, remote hardware, host devices, network access from generated code or unlimited computation. A method that needs unavailable data, an inaccessible library or an unsupported environment remains unavailable.
Literature, claims and scientific support
Online discovery is contextual: the investigator can search around the retained problem, follow a relevant source, seek a better source for a weak citation, or expand the evidence after a calculation raises a new question. A search result is a discovery lead. A retrieved source, extracted statement and scientific comparison are separate records. Search and retrieval distinguish evidence_expansion, challenge, source_repair and corroboration. Neutral expansion and deliberate challenge are available alongside support-seeking research; repairing an incomplete citation is not new independent support. Linked figures and supplements are discovered but are fetched only when the investigator chooses a permitted targeted retrieval.
Online research needs working DNS and outbound HTTPS access to the selected public services. Live metadata probes in the available release-validation environment failed DNS resolution; fixture coverage does not qualify live retrieval there. See the acceptance matrix for that remaining deployment gate.
Discovery adapters query Crossref, OpenAlex, DataCite and arXiv; they are metadata/discovery services, not a guarantee of accessible full text. Public retrieval accepts credential-free HTTPS on port 443 and rechecks destination restrictions on redirects. It does not sign in, bypass a paywall, send provider credentials to a publication host or access internal/private addresses. A retrieval is bounded to 8 MiB; cumulative downloaded bytes also count against the investigation budget.
Source extraction runs in the isolated worker. PDF text extraction is limited to 500 pages and 650,000 characters; requested raster inspection is limited to eight valid pages per extraction (the initial default is up to four pages). pypdfium2 enables retained page images. Available OCR is approximate; missing OCR or model image support remains an explicit coverage limitation. HTML/XML can retain extracted structural blocks and links; linked figures and supplements need their own retrieval. These bounds are extraction coverage, not a claim that every mathematical or graphical detail has been interpreted.
Inspect the source version and hash as well as the citation. A retrieved-content hash identifies exact downloaded bytes; bibliographic/canonical identity can remain unresolved and must not be treated as a verified publication edition. Inspect equations, tables and figures in the retained source where they matter to a conclusion. Extraction may omit scanned text, layout, equations or plotted values. Approximate values inferred from a figure must be identified as approximate, with their extraction method and uncertainty; they must not be represented as exact measurements supplied by the author. If a claim cannot be traced to inspected evidence, it remains insufficient.
The assessment vocabulary is supports, challenges, mixed, not_comparable and insufficient. An assessment expresses the investigator's evidence judgment and retains that attribution. It does not certify the paper, establish physical validity, or replace independent scientific review. A relevant paper can still use incompatible units, samples, models or experimental conditions. Conflicting findings must remain available.
Independent corroboration needs independent evidence. Restating a model-generated claim, citing the generated report, or citing a derivative source that repeats it is circular support. Post-run selection and adaptive analyses are exploratory unless they meet the separately retained preregistration requirements. Holdout roles must remain protected; choosing a method after inspecting a holdout cannot inherit the earlier study's confirmatory status.
Results, presentation and retained evidence
Investigations retain typed sources, claims, analyses, decisions, usage and artifact records. Generated code is part of the record; a code proposal alone is not a completed calculation. Inspect the execution receipt, returned scientific data, warnings and output identities before relying on a generated result. A child run is a new executed calculation only when its actual run record establishes completion.
The original solver, original run outputs and earlier report bytes remain retained. Autonomous additions have their own identity and relationships to that base. The report and export path must carry the same evidence, source versions and decisions into the available JSON, HTML, PDF, presentation and figure artifacts. A compact presentation does not erase qualifications or contradictory evidence from its retained source material.
Treat private source data and retrieved materials as untrusted inputs, including instructions embedded inside a paper. They provide scientific content; they do not grant permission to change policy, obtain credentials or contact an unrelated endpoint. Only the standing policy and application controls determine what tools may do.
Pause, resume, cancel and failures
The service retains checkpoints at control and tool boundaries. Completed tool receipts and compatible provider checkpoints can be resumed rather than counted as newly executed work. Provider transport retries are disabled within each admitted call so another attempt is charged explicitly; a failed provider operation requires the retained job's retry/resume control. Bounded successful chat segments can continue automatically without a researcher prompt. A restart cannot make a pending external request transactionally disappear: inspect the retained status and any uncertainty before retrying an interrupted operation. Retries and resumed work consume the same cumulative investigation budgets.
Pausing preserves the investigation and its artifacts for continuation. Cancellation persists and stops subsequent work at a checked boundary. Revoking the workflow's policy prevents further delegated work. Closing the browser is not a cancellation request. Storage, owner access, provider admission and executor checks still apply after a restart or resume.
If a job stops, inspect its status, stage, errors, usage and retained decisions. provider_capacity identifies a configured context, checkpoint or resume limit; review those limits before an explicit retry rather than repeatedly resubmitting an unchanged oversized context. When the original report input snapshot could not be retained, resolve reporting prerequisites and start a new investigation under a new policy revision; resuming cannot substitute different inputs into the old identity. Resolve the stated provider, quota, extraction, execution or storage problem. Increasing a limit does not validate an earlier scientific conclusion. Preserve the stopped record and its evidence when starting a different investigation.
Status and continuation rules
| Status | Meaning and next action |
|---|---|
queued / running |
Work is admitted or active. Progress and artifacts are retained. Pause or cancel through the controls. |
paused |
Retained work can be resumed under the same still-enabled policy revision. |
blocked |
Access, policy, snapshot, provider or execution prerequisites prevent progress. Inspect the recorded reason. |
failed |
An operation failed without being substituted by an invented result. Inspect evidence before retrying. |
budget_exhausted |
The investigation reached a cumulative limit or retry cap. Retained work is readable; resume does not grant another budget. |
completed |
The investigator accepted a finish record after retaining a plan, claim assessments and a current complete native presentation. Plain assistant prose alone is not completion. Check scientific support separately. |
cancelled |
Cancellation is retained; the job does not resume. Prior evidence remains available. |
A paused, cancelled, budget-exhausted, failed or blocked investigation retains a bounded partial report in JSON, HTML and ZIP. A worker interrupted for shutdown also retains one when it requeues the work. Use the investigation's existing artifact downloads to read it. This deterministic summary uses only retained state: the stop reason, progress, usage, claims, source and analysis references, missing stages, unresolved findings and limitations. It makes no provider request, retrieves no new sources and performs no new scientific calculation after the stop. Storage failures remain visible; a failed partial-report write does not change an incomplete investigation into a completed one.
A partial report is an incomplete snapshot, separate from the final native scientific presentation and its PDF/PPTX exports. Earlier presentations may be historical or stale; the snapshot identifies them without promoting them to a current final result. Resuming or completing work clears the current partial-report marker while preserving its immutable downloaded artifacts.
After process interruption, restart the standalone application or hosted server. A recovery supervisor starts with the server and checks eligible durable queued/running investigations in bounded batches every 30 seconds; no browser refresh or new chat prompt is required. Temporary owner-lease contention leaves work pending for a later sweep. Paused, cancelled, blocked and failed jobs are not automatically restarted. Use the appropriate explicit resume/retry control after addressing their stated condition.
For hosted recovery, the saved delegation retains trusted user, project and organization identifiers and an owner digest. Every dispatch reconstructs the actor from current database records, rechecks account status and project editing membership, resolves current BYOK/provider access, applies institutional external-AI policy and enters current metered admission. It retains no browser session or API key in that actor binding. Revoked access is not restored by an old delegation. Abandoned metering reservations that require operator reconciliation are not silently waived to force a restart.
The supervisor runs inside the application/server process. It is not a separate always-on distributed scheduler, and a stopped application cannot compute until restarted. The authenticated drain routes below also request bounded recovery for the caller's eligible retained work; they are supplemental to the unattended supervisor.
Privacy, storage and deployment
Sharing with the provider covers the authorized context actually sent and the retained results of subsequent tools. Online research separately discloses each recorded search query to its chosen public metadata service; a context-derived query can itself contain sensitive research terms. That may include private scientific values, report content, associated literature excerpts and generated analyses. Review the provider, model, institutional policy and permitted materials before delegating. Keep credentials in the existing configured credential mechanism; do not place them in workflow data, papers, generated code or policy JSON.
SaaS requests derive the owner from the signed-in context and existing access policy. A caller cannot supply another user's owner identity. Artifact access verifies retained identity and authorization. Investigations and their artifacts participate in the same storage and lifecycle dependency model as the scientific evidence they reference, including applicable BYOS manifests. Removing a current association does not remove a retained investigation's historical dependency.
Archive material that must remain available to retained results. Deletion eligibility depends on the whole retained evidence chain, including sources, claims, analyses, generated code, child runs and presentation outputs. Unknown or inaccessible dependencies cannot be treated as permission to delete scientific evidence.
Unattended means the investigator can make permitted decisions without step-by-step prompts while its configured worker is available. It does not imply that a closed application, stopped server or unavailable executor continues computing. Deployment operators remain responsible for process lifetime, isolated execution, provider access, data retention and applicable operational qualification.
Service and authenticated HTTP interfaces
Application integrations use pf_reporting.autonomy.service.AutonomyService(assistant). Its owner-bound methods are policy, set_policy, enqueue, get, list, process, pause, resume, cancel and artifact. The HTTP adapter dispatches background processing; clients normally poll the retained job rather than keeping a long-running processing request open.
All routes below are relative to /api/assistant/workflows/reporting. Use the edition's existing authenticated request mechanism, including its token or session/project and mutation protections. The owner is never a client-supplied request field.
| Method | Relative route | Request / response |
|---|---|---|
| GET | /workflows/{workflow_id}/autonomy |
Current policy and provider_status. |
| POST | /workflows/{workflow_id}/autonomy |
{"policy": {...}, "expected_revision": 0} for the initial save; subsequent writes use the current revision. Returns saved policy. |
| GET | /workflows/{workflow_id}/investigations |
investigations for the authorized owner/workflow. |
| POST | /autonomy/drain |
Empty JSON object; requests bounded recovery of the authenticated owner's eligible queued/interrupted jobs. |
| POST | /workflows/{workflow_id}/autonomy/drain |
Empty JSON object; requests that bounded recovery for one authorized workflow. Paused work stays paused. |
| POST | /runs/{run_id}/investigate |
Empty JSON object; queues/returns investigation, or null if disabled. |
| GET | /investigations/{id} |
Retained investigation with context, usage, decisions and artifacts. |
| POST | /investigations/{id}/pause |
Empty JSON object; persists pause. |
| POST | /investigations/{id}/resume |
Empty JSON object; resumes eligible work without resetting usage. |
| POST | /investigations/{id}/retry |
Empty JSON object; same eligible continuation semantics as resume. |
| POST | /investigations/{id}/cancel |
Empty JSON object; persists cancellation. |
| GET | /investigations/{id}/artifact?path=... |
Exact registered artifact path, URL encoded; owner- and hash-verified bytes. |
For example, an initial policy can authorize analysis of retained evidence without network research or generated code:
{
"expected_revision": 0,
"policy": {
"schema_version": 1,
"enabled": true,
"share_with_provider": true,
"online_research": false,
"allow_generated_code": false,
"allow_followup_runs": false,
"domain_allowlist": [],
"budgets": {"max_child_runs": 0}
}
}
Fetch the current policy first; never guess expected_revision after the initial save. A conflict means reload and review the current delegation. Credentials, arbitrary host paths, interpreter choices and executor image names are not accepted in these request bodies.
Investigation and literature tools
The dedicated provider purpose is isaes_autonomous. investigate_read reads bounded retained state; investigate_plan retains or revises a scientific plan; investigate_assess binds a specific claim to retained source citations and completed analysis identities; investigate_finish checks that the plan, claims and current presentation satisfy its finish contract. An assessment remains an explicitly attributed LLM judgment. The tool registry does not expose policy editing, credential access, self-approval or the ordinary proposal-review endpoint.
| Literature tool | Evidence it retains or exposes |
|---|---|
literature_associated |
Immutable associated-source snapshots from the base report input. Later association edits are not substituted. |
literature_search |
Contextual query, engine, purpose, response and bounded discovery candidates. Supported engines return a limited result page, not a systematic literature review. |
literature_fetch |
Public source bytes, requested/resolved URL, receipt, content hash, isolated extraction and source-repair/corroboration relationship. |
literature_read |
A retained page or extracted section, paginated with Unicode character offsets (up to 16,000 characters per call), with a read receipt. |
literature_block |
Exact retained HTML/XML table, equation or figure text/markup with an extraction-block locator and citation; a block index is not a printed page number. |
literature_details |
Paginated metadata or safe discovered figure/supplement/full-text URLs (up to ten entries per call); listed links are not automatically fetched or inspected. |
literature_identity |
Resolves a DOI actually present in retained metadata or already-read text against Crossref/DataCite when online access is permitted; distinguishes the source's declared identity from a DOI that may merely cite another work. |
literature_image |
Retained pixels of a requested source page, extracted on demand when necessary. Actual model image support and inspected coverage remain explicit; verified page pixels supplied through the provider image path are limited to 1 MiB. |
literature_cite |
An exact passage already covered by a retained read receipt, with page, offsets, excerpt/source/extraction hashes and optional claim ID. It does not itself establish scientific support. |
Scientific tools and generated code
The investigator can call science_methods, science_read, science_analyze, science_generate, science_present and science_followup. These are provider tool interfaces, not unauthenticated HTTP endpoints. Every call still passes current owner, control and cumulative-budget checks.
science_generate receives a title, Python source and checks_json. Define analyze(data, context) returning a finite JSON object with values and optional figures, notes and tables; optional render(data, result) returns a Matplotlib figure or a list of {title, figure} objects. The context includes retained artifacts, exact already-read citation receipts and source metadata. Checks are declared separately as JSON objects with name, path, expected, atol and rtol; each path is relative to the returned values object. The source limit is 128 KiB, generated numerical JSON is limited to 1 MiB and its scientific evidence artifact to 8 MiB, subject also to the provider tool-envelope and worker limits. At most 24 native figures and 12 custom Matplotlib compositions are accepted; each custom SVG/PNG pair is bounded to 8 MiB and aggregate executor output to 32 MiB. Inspect the exact worker contract and admitted outputs when authoring an extension; arbitrary browser scripts or HTML are not enabled by this interface.
science_analyze executes a registered reporting dependency plan on retained values. New adaptive analysis cannot declare itself confirmatory or preregistered. science_read(section="intent") exposes the original scientific question, reporting intent, units, study design metadata and named method bindings before planning; when holdouts exist, it omits coordinates, numerical references, arbitrary parameters and presentation values. Other science_read sections inspect retained result values, datasets, analyses, method metadata and permitted artifacts with bounded pagination. Declare holdout roles before delegation. When any holdout is declared, adaptive scientific tools receive only explicitly declared non-holdout datasets; unclassified summaries and artifacts are excluded. A protected subpointer inside an artifact excludes that whole artifact, including its other summaries. Follow-up result views inherit the protection. The unchanged solver still executes its original program and the researcher retains the original run/results; the restriction applies to what adaptive tools inspect and reuse. science_followup uses the unchanged active workflow version and reuses retained file inputs, with new bounded parameter values. If the active version differs from the source run, it is blocked. New solver editing is outside that tool's authority.
science_present produces native report JSON, HTML, PDF, PPTX and ZIP artifacts from the retained base and derived evidence. Partial stop snapshots are stored separately under partial-reports/<snapshot-sha256>/report.json, report.html and report.zip; the ZIP contains those JSON/HTML files and a manifest. The job's partial_report marker identifies the current incomplete snapshot. These artifacts do not satisfy investigate_finish. Failed analyses and conflicting claim assessments remain represented; presentation options cannot turn missing evidence into a successful result. Presentation and completion bind the current hashes of claims, sources, decisions, analyses and citation receipts, so later evidence changes require a fresh presentation.
Qualification and related guides
Use the autonomous requirements-to-evidence matrix to inspect the implemented scope, executable checks and remaining acceptance gates. Current commands, counts, runtime and source identity are in validation_r34_autonomous/test_execution.json; verify them with python release_tools/validate_r34_autonomous.py verify --package. Local calculation tests, provider protocol fixtures and browser tests have distinct evidential scopes. None is a live-model scientific evaluation or a representative researcher study.
Native Windows execution, independent expert assessment, live-provider scientific quality and applicable production containment/BYOS qualification may be claimed only from separately recorded execution and review. Historical r34 reporting and documentation receipts retain their original source identities and are not autonomous acceptance results.
Related references: ISAES overview, reviewed report discussion, developer SDK, evidence lifecycle, and private workflow development.