r34 shared classical reporting compatibility reference

The current redesigned system is documented in R34_ISAES_GUIDE.md. This reference explains the original schema-1 interface retained for compatibility. New reports use schema 2, with scientific plans, evidence navigation, separate status dimensions, additional methods, interactive reports and presentation exports. Use the ISAES guide for current installation, supported methods, recovery behavior and validation scope.

The original reporting extension used r34 baseline SHA256 34ebbfeb1d0c6d4ed0c8e7a7e84ea28693a46a0f119791038aeecdb78d16ee4e. The literature lifecycle update builds on the completed reporting release with incoming SHA256 6d8576fb0e70d0c906702e1c0a86d2f5c5f865eb4844660e8a1ca7449bb5c876. It adds shared reporting while preserving the ability to attach private Python workflows and scientific source layers. The reporting pipeline does not require an AI provider, an API key, a modern language model, model weights, or a paid service.

Workbench use

1. Open My workflows. Upload one or more materials under Supplementary research materials. You can upload before selecting a workflow. Papers may be your own or third-party research you are authorized to use.
2. Select a saved workflow. Select the associated materials, optionally describe their roles, then choose Save analysis and literature settings. This single action saves the workflow's analysis/reporting settings together with its selected literature and roles. A document can be used by several private workflows. Existing associations have an immutable revision.
3. Run an activated workflow, or choose Inspect run for a completed saved execution. The workbench prepares and starts a classical report automatically. No physics calculation is repeated for reporting. A read-only member can inspect existing reports but cannot create them.
4. Inspect the report findings, numerical checks, charts and analysis decisions. Download report.html, report.pdf, report.json, or report.zip. The ZIP contains the structured report, self-contained HTML, vector PDF and PNG/SVG figures and exact per-figure JSON. Original scientific outputs remain available in the existing run section.
5. Choose Regenerate report after changing associations or reporting requirements. Earlier report jobs and their captured inputs remain retained. Saved scientific executions remain unchanged.

The save status shows Unsaved changes, Saving..., then Saved after server confirmation. A save error leaves the edits available to correct and retry. The primary button's blue background identifies its action; read the status text for the outcome. Saving settings does not run a workflow or regenerate a report.

A queued or interrupted report can be started/resumed. A failed report offers retry; resolve the displayed missing dependency, data, quota or extension error first. Cancel signals the running isolated worker and discards uncommitted output. After a reporting code/runtime upgrade, regenerate a queued job rather than processing its old binding.

The GUI starts a durable report job through a separate HTTP request and polls progress. It is not an unattended global job scheduler. If the serving process is terminated, queued inputs remain retained. Resume after the existing profile lease has expired or the operator has reconciled a stopped hosted operation. Hosted compute reservations keep the existing administrator crash-reconciliation rules.

How the core chooses work

All components are built into pf_reporting. They are not four mandatory passes. Scalar-only results do not fit PCA or instantiate the language model. Multivariate exploration needs at least eight rows and two nonconstant numeric columns; unsuitable data gets an explicit analysis decision. A private configuration can disable bigram wording selection with parameters.statistical_language_model: false; factual NLG remains available.

A scientific validity check can fail in an otherwise complete report: completeness describes delivery of reporting requirements, not a scientifically favorable result. Missing required analyses, conflicting requirements, unresolved reference mappings or failed renderers produce an incomplete report. Worker errors produce a retryable failed job. No unsupported physical interpretation is fabricated to fill a gap.

Scientific metadata

Existing schema-1 workflow bundles continue to work unchanged. Metrics, series and tables receive default reporting. A new optional reporting object supplies scientific meaning for richer outputs. JSON arrays are not assumed to be density matrices, networks or circuits based on their shape alone.

Example bundle fragment:

{
  "reporting": {
    "schema_version": 1,
    "intent": "State consistency and measured evolution",
    "datasets": [
      {"name": "rho", "kind": "density_matrix", "source": "/data/rho",
       "basis": "Computational basis, q0 least significant",
       "subsystem_dimensions": [2, 2], "tolerance": 1e-8},
      {"name": "energy", "kind": "series", "source": "/series/0",
       "label": "Energy", "x_label": "Time (seconds)", "units": "joules"}
    ],
    "require": ["matrix:rho"]
  }
}

Sources use JSON pointers into the immutable result, or artifact:declared_artifact_name#/pointer for a retained JSON artifact. The service verifies bytes before decoding them. Sources added dynamically by an extension must be included in the bound declared datasets when they need artifact loading; arbitrary host paths are not exposed.

Built-in kinds: scalar, series, trajectory, matrix, density_matrix, distribution, probabilities, samples, table, network, and circuit. A complex matrix uses real and optional imag arrays. A network supplies nodes and pairs in edges. A circuit supplies num_qubits and gates, each with a name and qubits. Diagrams reproduce supplied structure; the core does not infer an executed circuit from an arbitrary program.

For tables, features can name the scientific columns to include in PCA/clustering. Automatic numeric selection is exploratory and may include a time/index column; declare features when that would be inappropriate. Model scaling, loadings, centers, labels, inertia and selection decisions are retained. The model does not need a downloadable ML service.

The updated six-qubit/eight-cycle Floquet example declares the basis and full-state JSON artifact and adds a small private topology analyzer for its actual retained edges. Its original physics program is unchanged. Shared code produces the state checks, heatmaps, basis populations, curves, exploration, narrative and exports.

Private workflows and layers retain control

Analyzer is the abstract calculation interface; Registry provides polymorphic analyzer dispatch and renderer registration. Persistence and execution are encapsulated in ReportingService. The shared renderer consumes common findings and figure specifications, so reporting code does not need to be duplicated in every workflow.

Configuration is combined in this order: bound bundle reporting, saved workbench requirements, returned result.data.reporting, the bound scientific layer hook, and private extension callbacks. Dataset declarations replace by name; parameters merge by key. require, suppress, references and extension lists accumulate by default. Explicit replace lists the fields whose inherited values should be cleared before the current layer's values are applied. For example, {"replace":["require","suppress"],"require":["my_method:observable"]} deliberately replaces inherited requirements. The retained inputs and source hashes make that change inspectable. A surviving requirement that is also suppressed is reported as a conflict.

require and suppress accept analyzer names, dataset names, or analyzer:dataset identifiers. order names finding identifiers in presentation order. The default puts failed checks and reference comparisons ahead of exploratory fits. An invalid custom extension is not silently accepted as a complete report.

A private bundle lists a file/function under extensions, for example reporting.py:configure, and includes that file in its existing files array. The callback can register new kinds, replace a built-in analyzer, register a custom Matplotlib renderer, and return additional reporting configuration:

from pf_reporting.engine import Analyzer, finding, figure

class SquaredObservable(Analyzer):
    name, kinds = "squared_observable", ("my_observable",)

    def analyze(self, dataset, value, context):
        squared = value ** 2
        item = finding(dataset, self.name,
                       "Squared observable calculated from the saved value.",
                       {"square": squared})
        return item, []

def configure(registry, context):
    registry.register(SquaredObservable())
    return {"require": ["squared_observable:observable"]}

Use registry.register(analyzer, replace=True) to replace an existing named analyzer. Use registry.renderer(kind, function, replace=True) for a custom renderer; its function takes a figure specification and returns a Matplotlib Figure. A layer edits pf_research/reporting_overrides.py through the existing source-layer lifecycle. That hook receives the registry and result/artifact/document context before analysis. Both extension mechanisms execute inside the configured isolated worker, with the same resource, dependency, ownership and admission boundaries as private workflows. The catalogue is extensible; it is not a restriction on private scientific methods.

Scientific libraries provide their numerical and plotting machinery inside this runtime. Workflow authors can use installed compatible packages in analyzers and renderers. The framework does not install libraries on demand or grant external networking. The application control plane remains outside editable scientific layers.

Literature and references

PDF, DOCX and UTF-8 TXT/Markdown/TeX/CSV/JSON/BibTeX text is extracted locally. Other formats are retained as hashed originals. Scanned/image-only pages remain associated but have explicit no-text warnings; this version does not perform OCR, interpret figures, solve equations from a PDF or claim automatic understanding of arbitrary literature.

PDF references use physical page numbers. Text documents use form-feed sections, or section 1. DOCX locations are extracted paragraph order within section 1, not printed page numbers. BM25 retrieves short, ranked supplementary passages with document SHA256 and location. Relevance is not applicability.

A numerical literature comparison requires an explicit mapping in references: document_id, page, result source, reference value, nonnegative atol, matching units and output_units, and stated assumptions. Optional equation or table locators can be recorded in the mapping. The core checks the declared numerical agreement; it cannot independently prove that a manually entered reference number faithfully represents the cited paper. Conflicting or unsupported mappings stay visible.

Uploads have immutable original hashes. Association revisions and report snapshots retain the exact index used. Archiving a document hides it from the active list but retains originals required by saved reports. Workflow permanent deletion removes its reports and association revisions, while reusable documents remain separately available. Existing standalone/SaaS ownership, project checks, quotas and BYOS verification apply.

Archive, restore and permanent deletion

The owner's literature collection is shared across their workflows. Archive document is a global status change within that collection, not removal from the selected workflow. It is the normal action for literature already in use: original bytes, extracted indexes and all existing associations remain intact. Archived associated documents remain visible and checked in their workflow, and unrelated selection saves do not detach them. Existing reports and regeneration continue to use their retained evidence.

Use the active list and Archived research materials section to inspect documents and the selected workflow's associations. Choose Restore document before making a new association to an archived item. Archive/restore never edits scientific code, run results, earlier association revisions or report snapshots.

To remove the current selection from one workflow, deselect it and save that workflow's associations. That creates a new revision. Earlier revisions remain dependencies until a separately reviewed administrative association release covers them. Choose Review association release, select workflows under Choose workflows for association release, then confirm Release selected associations. The association preview groups links by workflow name and ID, distinguishes current and historical links and shows protected evidence or inspection errors. Select the workflows to release deliberately; review their identities before confirming.

Association release retires only the selected document's administrative documents links in those workflows. It preserves each original configuration row and adds an auditable, hash-bound receipt. A new current configuration retains the other document selections and reporting settings. Explicit reference mappings in those settings, immutable workflow bundles, run/validation records, saved report inputs, private layers and unreadable evidence cannot be released by this operation. A changed preview, invalid release receipt or inspection error blocks the release request. Retained scientific/report references do not prevent release of administrative links, but they remain untouched and continue to block permanent document deletion. Association release does not delete documents, reports, scientific results or workflows.

Check deletion eligibility inspects all retained dependencies, including archived workflows and administrative history that has not been explicitly released. Permanently delete document remains visible and performs a fresh check; it proceeds only when the inspection finds none and otherwise explains the blockers. The server requires the reviewed state and repeats checks under the same owner coordination used by reporting and associations. A new dependency, changed document, unavailable storage, corrupt input or incomplete inspection prevents deletion. A retained literal document identifier in a layer or workflow also conservatively blocks deletion; private code must declare opaque external resources or computed identifiers in its reporting references/associations.

After releasing administrative associations, inspect deletion eligibility again. Deletion remains a separate explicit operation. Reports and scientific snapshots are never removed to make literature eligible, and there is no cascade into workflow deletion.

Permanent deletion removes only an unreferenced document and its extracted index. It cannot be undone. Regular storage and BYOS use the durable cleanup journal; shared remote bytes remain until their last retained reference is gone. Cleanup operations exposes pending deletion and retry. Backup recovery recognizes literature deletion receipts and refuses to erase a restored document if unexpected dependencies now exist. Independently exported files and backups are outside the live deletion operation.

Library inspection runs only when requested for a document, not during workbench initialization. Documents remain owner-private and hosted edits require the existing editor role and CSRF checks. Cleanup and restoration do not consume scientific execution capacity or call an AI provider.

API and deployment

All routes are under the existing authenticated /api/assistant/workflows/reporting prefix:

Association writes require expected_revision. Reads are private to the workflow owner and project; mutation requires editor access. SaaS keeps CSRF and metered worker admission. Local endpoints keep the existing local access token. No route takes an owner, runtime image, host path or API key from a reporting request.

Install the existing supported research runtime and requirements_reporting.txt; NumPy and Matplotlib remain provided by the core requirements. PDF extraction adds pypdf. Existing research/assistant/hosted Docker dependency builds include these requirements transitively. Rebuild the administrator execution image for the new dependency set, then restart the application. Reporting still uses the existing configured executor (PF_AI_DOCKER_IMAGE is a legacy setting name, not a requirement to configure an AI provider). Never enable the trusted-local test executor for untrusted production workflows.

The extension uses new record kinds in the existing generic private record store and does not require destructive schema changes. Preserve the database and private state together when upgrading. Existing workflow source bindings retain their original identity: new science after an application change may require a new workflow version and validation. Reporting can still use an old run's retained source/layers, with the current reporting engine versioned separately.

Per-document upload limit: 8 MiB; at most 64 associations per workflow and 512 retained documents per owner. Extracted UTF-8 text is bounded to 700 KiB, and a combined report input to 8 MiB. Structured reports have a 1,800 KiB bound. Preview images may be omitted to fit that bound while full figures remain in the ZIP. Existing 32 MiB selected-artifact and worker time/memory/output quotas still apply. Oversized or malformed material yields an explicit error; split materials or use a private specialized analyzer rather than assuming unlimited capacity.

Validation and byte integrity

validation_r34_literature_management contains the focused patch execution, source freeze, browser evidence, reconciliation and current qualification registries. Its tests cover literature management, safe association release, reporting/lifecycle compatibility and nonblocking startup. Inspect its execution record for the exact commands, counts and platform scope. Earlier reporting/literature receipts and the original 527-test ISAES acceptance retain their original source identities and are not newly executed evidence for this patch.

The acceptance environment uses Linux developer-fixture subprocesses. Production Docker containment, a native Windows installation, external object-storage infrastructure and arbitrary third-party scientific programs are not established by those tests. Historical scientific benchmark results and validation totals retain their original evidence and scope; this reporting work does not promote private results to experimental validation.

Verify the delivered package with python release_tools/validate_r34_literature_management.py verify --package. The new inventory is checked against every delivered member. The reconciliation records every added or modified source file and the pinned incoming archive hash. Earlier validation_r34_reporting, validation_r34_literature and validation_r34_isaes records remain historical evidence. Historical validation records are preserved byte for byte rather than relabeled as fresh tests. The outer archive SHA256 is supplied separately to avoid a self-referential hash.