/reference/index.html Reference material for the current data contract, CLI surface, metrics, and result artifacts. Hugo en-gb /reference/data-contract/index.html Mon, 01 Jan 0001 00:00:00 +0000 /reference/data-contract/index.html The backtest harness needs enough information to replay the same model surface consistently across folds and across repos. Minimum Required Inputs For a runnable pilot manifest, the repo needs: a weekly source table a date column a KPI / response column the locked model formula priors boundaries repo targets and repo paths fit settings and seed policy If recommendation stability is in scope, the repo also needs: /reference/results-and-artifacts/index.html Mon, 01 Jan 0001 00:00:00 +0000 /reference/results-and-artifacts/index.html Backtest outputs are written under run-scoped result trees so filtered reruns do not overwrite earlier summaries. Result Tree Shape Typical layout: results.../ <dataset_id>/ <comparison_label>/ run_id=.../ experiment_manifest.yaml fold_manifest.csv summary/ run_status.csv holdout_scores.csv holdout_summary.csv parameter_stability_summary.csv recommendation_stability_summary.csv repo_target=<repo>/ fold_id=01/ run_manifest.yaml run_status.json fit_payload.rds prediction_payload.rds holdout_scores.csv recommendations.csv stability/ parameter_drift.csv parameter_drift_summary.csv recommendation_drift.csv recommendation_drift_summary.csv Most Important Summary Files summary/run_status.csv Fold-by-fold execution state. /reference/cli/index.html Mon, 01 Jan 0001 00:00:00 +0000 /reference/cli/index.html The main entrypoint is: Rscript scripts/dsambayes-backtest.R <command> [options] validate Validate a pilot manifest and print the planned run scope. Rscript scripts/dsambayes-backtest.R validate \ --manifest .planning/research/pilot_manifest.yaml plan Build the run matrix for the active manifest. Rscript scripts/dsambayes-backtest.R plan \ --manifest .planning/research/pilot_manifest.yaml run Execute a batch or write a dry-run result tree. /reference/metrics/index.html Mon, 01 Jan 0001 00:00:00 +0000 /reference/metrics/index.html Forward Holdout Metrics RMSE Root mean squared error on the observed KPI scale. WMAPE Weighted mean absolute percentage error on the observed KPI scale. Mean error Signed bias on the observed KPI scale. SMAPE Secondary holdout metric on the observed KPI scale. Holdout ELPD / log score Secondary probabilistic diagnostics when compatible posterior outputs are available.