18. Reproducibility¶

Use this guide when you want runs that are easy to rerun, compare, and audit. In ModSSC, reproducibility comes from a combination of fixed seeds, saved configs, stable fingerprints, and disciplined cache usage.

18.1 What ModSSC fingerprints for you¶

Datasets are identified and cached from their resolved provider, dataset ID, version, and resolved options.
Sampling outputs are driven by the sampling plan plus the split seed.
Preprocess cache entries depend on the dataset fingerprint, resolved preprocess plan, fit subset, and preprocess seed.
Graph cache entries depend on the dataset fingerprint, preprocess fingerprint, graph spec, and graph seed.

This means cache reuse is precise when the inputs match and intentionally broken when a meaningful upstream input changes.

18.2 Seed strategy¶

Use run.seed as the root seed in benchmark configs.
When you need stage-specific control, set sampling.seed, preprocess.seed, views.seed, graph.seed, augmentation.seed, and search.seed explicitly.
For multi-seed sweeps, keep one shared cache root and let the fingerprints separate seed-specific artifacts.

For strict comparisons, prefer one committed config per experiment family and vary only one dimension at a time.

18.3 Strong vs best-effort determinism¶

Sampling is the strongest reproducibility layer because it is fully driven by deterministic plans and explicit seeds.
Torch-based inductive methods use a best-effort deterministic seeding helper that also enables deterministic algorithms when supported.
Torch-based transductive GNN helpers seed torch, but backend- and device-specific kernels can still introduce small variations.
For regression checks where exact reruns matter, prefer CPU runs or the same hardware/backend stack.

18.4 Cache discipline¶

Reuse one shared MODSSC_CACHE_ROOT when you want fast reruns of the same experiment family.
Use a fresh MODSSC_CACHE_ROOT only when you explicitly want a clean-room comparison.
If you change code inside preprocess or graph implementations while keeping the same config, clear the impacted cache folders before comparing outputs.
Treat benchmark configs and caches as trusted local artifacts only.

18.5 Practical checklist¶

Pin the same commit, Python version, and dependency profile.
Save the benchmark config.yaml next to the produced run.json.
Keep run.seed fixed unless you are intentionally doing a seed sweep.
Save sampling artifacts when you need to reuse the exact same split outside the bench runner.
Prefer CPU or a fixed accelerator stack for exact comparisons.
Start from a known-good config from the Bench config cookbook.

Sources