1. Discover datasets
Filter the catalog by phase, asset class, readiness, support level, or training eligibility.
GET /api/v1/benchmarks/catalog
MCP: list_benchmark_datasets
Benchmarks and vintages
Evaluate forecasts without losing the publication clock.
Worldcastr benchmark records pin source vintage, retrieval date, evaluation cutoff, model version, baseline, and allowed use so comparisons remain reproducible and leakage-aware.
- benchmark assets
- 17
- available now
- 9
- training candidates
- 1
Start here
The benchmark workflow
2. Pin a public vintage
Retrieve the exact external forecast values that were available at a documented publication date.
GET /api/v1/benchmarks/vintages
MCP: get_public_forecast_vintage
3. Compare scores
Inspect model, public forecast, and persistence results against the same cutoff and horizon.
GET /api/v1/benchmarks/backtests
MCP: get_backtest_scores
Complete benchmark inventory
Available first, planned and held afterward
17 catalog entries
backtest vintage outputs
Model-vs-persistence cutoff-2016 horizon-8 summary
backtest.model_vs_persistence_cutoff2016_h8Public-safe score summary for the completed 2016-origin Maricopa realized-ACS benchmark. This packages the score-table readout without releasing raw parcel or unit histories.
tract|county-prefixbenchmark onlyphase_1
Provenance and access
Role: realized_backtest_score_table
Formats: json, csv
License: Score summary only. The summarized benchmark is public-safe, while any future release of normalized training panels still requires explicit source-term review.
Snapshot: phase1_anchor_stock_audit/acs_nationwide_all_target_base_source_cutoff2016_h8/20260715_200946_0d4a6952/origin_2016/maricopa_cutoff2016_h8_realized/model_vs_persistence_mdape.csv
backtest vintage outputs
Arizona OEO Maricopa 2022 public forecast vintage
backtest.public_vintage_oeo_maricopa_2022Validated public-vintage population projections from the January 8, 2022 Maricopa County Economic Conditions Report. This is the cleanest currently operationalized same-vintage county control fixture.
countybenchmark onlyphase_1
Provenance and access
Role: same_vintage_county_control
Formats: json, csv
License: Public-source benchmark fixture only. Use as a same-vintage control and provenance-clean comparison table, not as a normalized training panel.
Snapshot: packages/pipeline/scripts/pipeline/training/public_forecast_vintages_az_oeo_maricopa_medium_2022_report.csv
benchmark panels
MAG 2023 public projection fixtures
benchmark.mag_2023_projection_fixturesValidated current MAG 2023 public projection fixtures for MPA, RAZ, and TAZ supports. These are useful benchmark fixtures and planning controls, but they are not 2016 or 2021 same-vintage backtests.
mpa|raz|tazbenchmark onlyphase_1
Provenance and access
Role: validated_current_planning_fixture
Formats: json, csv
License: Validated current planning fixtures only. Keep the publication year and support labels explicit so downstream users do not misread them as historical same-vintage archives.
Snapshot: packages/pipeline/scripts/pipeline/training/arizona_mag_utility_panel_sources.json
benchmark panels
Raw single-source benchmark registry
benchmark.raw_single_source_registryRegistry metadata for the raw single-source benchmark lane spanning Chronos, TimesFM, Nixtla, Prior Labs, and other adjacent model families on source-native panels before enrichment.
mixedbenchmark onlyphase_1
Provenance and access
Role: benchmark_suite_registry
Formats: json, csv
License: Registry metadata only. Underlying raw panels stay governed by source-specific terms and should not be mirrored publicly until the exact source family is reviewed.
Snapshot: packages/pipeline/scripts/pipeline/training/raw_single_source_benchmark_registry.yaml
forecast outputs
Aggregate forecast download artifacts
forecast.aggregate_downloadsCurrent aggregate outlook exports for state and city surfaces. These are the clearest CSV-first public artifacts already close to benchmark and consumer-packaging use.
state|county|tractconsumer feedphase_1
Provenance and access
Role: aggregate_export_artifact
Formats: csv
License: CSV/JSON aggregate exports are in scope first. Parquet remains deferred until each artifact is clearly license-safe.
Snapshot: api:/api/forecasts/download
forecast outputs
Aggregate forecast API surfaces
forecast.aggregate_surfacesCurrent public forecast surfaces across state, county, ZCTA, and tract geographies. This is the aggregate-safe slice of the serving API and the default public floor for downstream forecast consumers.
state|county|zcta|tractconsumer feedphase_1
Provenance and access
Role: public_forecast_surface
Formats: json
License: Public API surface under Worldcastr API terms. Keep benchmark and training reuse at aggregate geography only until parcel and unit provenance review is explicit.
Snapshot: api:/api/v1/forecast/state|county|zcta|tract
provenance manifests
Arizona archive-status ledger
provenance.arizona_archive_status_ledgerValidated-versus-pending ledger for Arizona OEO, MAG, and Phoenix public-vintage discovery. This is the core provenance record for what is ready, pending, or rejected.
county|mpa|raz|tazbenchmark onlyphase_1
Provenance and access
Role: provenance_ledger
Formats: json, csv
License: Metadata only. Publishing the ledger is safe and useful because it exposes discovery status and provenance boundaries without mirroring raw source extracts.
Snapshot: packages/pipeline/scripts/pipeline/training/arizona_mag_public_archive_status.json
provenance manifests
Arizona enrichment source catalog
provenance.arizona_enrichment_source_catalogPublic-source catalog for Phoenix and Maricopa growth, utility, parcel, and environmental context sources. This is useful as a provenance manifest and as a future benchmark and training candidate map.
parcel|tract|mpa|raz|tazbenchmark onlyphase_1
Provenance and access
Role: source_catalog
Formats: json, csv
License: Catalog only. The public URLs and metadata are safe to expose, but mirrored raw pulls and unclear-license extracts remain out of scope.
Snapshot: packages/pipeline/scripts/pipeline/training/arizona_phoenix_enrichment_source_catalog.json
provenance manifests
Arizona utility-panel source manifest
provenance.arizona_utility_panel_sourcesSource manifest for the Maricopa utility-siting benchmark lane, including validated fixtures, archive-pending targets, and required output artifacts.
tract|mpa|raz|tazbenchmark onlyphase_1
Provenance and access
Role: source_manifest
Formats: json, csv
License: Metadata only. The manifest is public-safe because it describes sources and required outputs without exposing raw parcel or unit histories.
Snapshot: packages/pipeline/scripts/pipeline/training/arizona_mag_utility_panel_sources.json
benchmark panels
Normalized public training panels candidate
benchmark.normalized_training_panels_candidatePhase-3 candidate for selectively released normalized training panels. The default path is aggregate-safe, provenance-safe panels first before any parcel-level or assessor-derived release.
tract|county|statetraining candidatephase_3
Provenance and access
Role: phase_3_training_candidate
Formats:
License: Do not release as a training panel until source terms, normalization semantics, and provenance safety are documented for each included source family.
Snapshot: plan:aggregate_normalized_training_panels
forecast outputs
Narrative forecast surfaces
forecast.narrativesAI-generated market narratives keyed by county, tract, city, or state. This is a useful downstream consumer feed, but it should stay phase-2 until copy QA and versioning are formalized.
county|tract|city|stateconsumer feedphase_2
Provenance and access
Role: consumer_narrative_surface
Formats: json
License: Narrative text can be public, but release should stay behind explicit copy QA, versioning, and provenance notes so benchmark users do not confuse narrative packaging with numeric fixtures.
Snapshot: api:/api/v1/narratives
forecast outputs
Tract target-metric forecast bridge
forecast.target_metric_tractAggregate tract-level target metric forecasts such as population and housing targets. This is the phase-2 bridge between Worldcastr outputs and downstream benchmark and training consumers.
tractconsumer feedphase_2
Provenance and access
Role: target_metric_bridge
Formats: json
License: Public aggregate tract forecasts only. This surface is appropriate for benchmark consumers, but it should not be presented as a parcel-level training panel.
Snapshot: api:/api/v1/forecast/tract/metric
forecast outputs
Property-level forecast surfaces
forecast.property_level_surfacesParcel and unit forecast endpoints already exist, but they stay outside the conservative public-data floor until source terms and parcel or unit provenance are reviewed explicitly.
parcel|unitholdbackphase_3
Provenance and access
Role: property_level_holdback
Formats: json
License: Parcel and unit outputs remain out of scope for phase 1 public-data packaging pending explicit source and license review.
Snapshot: api:/api/v1/forecast/lot|unit
holdback assets
Analytics and session data
holdback.analytics_session_dataAggregated product analytics and recent session event views remain internal-only and are not public benchmark material.
internalholdbackhold
Provenance and access
Role: internal_usage_holdback
Formats:
License: Internal-only analytics surface. Keep behind non-demo API key gates and exclude from the public-data pack.
Snapshot: api:/api/v1/analytics/events
holdback assets
Source-unclear assessor extracts
holdback.assessor_extracts_license_unclearAssessor extracts with unresolved source or license status remain blocked from public release even if they are analytically useful.
parcelholdbackphase_3
Provenance and access
Role: license_unclear_holdback
Formats:
License: Blocked until source and license review is explicit. Do not publish mirrored raw extracts or derived training panels from this class.
Snapshot: internal:assessor_extracts_license_unclear
holdback assets
Raw parcel and unit histories
holdback.raw_parcel_unit_historiesRaw parcel and unit histories exceed the conservative public floor and should stay private until source terms, privacy boundaries, and normalization rules are explicit.
parcel|unitholdbackphase_3
Provenance and access
Role: raw_history_holdback
Formats:
License: Not public-safe for phase 1. Limit release to tract, ZCTA, county, and state aggregates until raw parcel and unit source terms are reviewed explicitly.
Snapshot: internal:raw_parcel_unit_histories
holdback assets
Training cost history
holdback.training_cost_historyTraining run durations, instance classes, and compute-cost history remain internal operational data and are not part of the public benchmark floor.
internalholdbackhold
Provenance and access
Role: internal_ops_holdback
Formats:
License: Internal-only operational history. Keep behind authenticated internal surfaces and out of the public benchmark namespace.
Snapshot: api:/api/v1/training-runs
Documentation
Start with the job you need to do, then move into the complete interface reference.
Interfaces
REST, MCP, and typed client references.