17. Skip to content

17. Bench config cookbook

Use this guide when you want a concrete starting config instead of building YAML from scratch. The goal is simple: start from the nearest working config, change as little as possible, and only widen the search space once the baseline runs.

17.1 Fastest starting points

Goal Config Command
Smallest inductive walkthrough bench/configs/experiments/toy_inductive.yaml python -m bench.main --config bench/configs/experiments/toy_inductive.yaml
Smallest transductive walkthrough bench/configs/experiments/toy_transductive.yaml python -m bench.main --config bench/configs/experiments/toy_transductive.yaml
Smallest HPO walkthrough bench/configs/experiments/toy_inductive_hpo.yaml python -m bench.main --config bench/configs/experiments/toy_inductive_hpo.yaml

17.2 Minimal configs by modality

17.2.1 Inductive

Modality Config Typical extras
Audio bench/configs/experiments/minimal/inductive/pseudo_label/audio/speechcommands.yaml audio, inductive-torch
Graph bench/configs/experiments/minimal/inductive/pseudo_label/graph/cora.yaml graph, inductive-torch
Tabular bench/configs/experiments/minimal/inductive/pseudo_label/tabular/iris.yaml openml, inductive-torch
Text bench/configs/experiments/minimal/inductive/pseudo_label/text/imdb.yaml hf, inductive-torch
Vision bench/configs/experiments/minimal/inductive/pseudo_label/vision/mnist.yaml vision, inductive-torch

17.2.2 Transductive

Modality Config Typical extras
Audio bench/configs/experiments/minimal/transductive/gcn/audio/speechcommands.yaml audio, transductive-torch
Graph bench/configs/experiments/minimal/transductive/gcn/graph/cora.yaml graph, transductive-torch
Tabular bench/configs/experiments/minimal/transductive/gcn/tabular/iris.yaml openml, transductive-torch
Text bench/configs/experiments/minimal/transductive/gcn/text/imdb.yaml hf, transductive-torch
Vision bench/configs/experiments/minimal/transductive/gcn/vision/mnist.yaml vision, transductive-torch

17.3 Which config should you copy first

  • Use the toy configs when you want the fastest local validation of the full pipeline.
  • Use the minimal modality configs when you want a real provider-backed dataset but still want a small surface area.
  • Use the HPO toy config when you want to validate the search workflow before tuning a larger model.
  • Use the minimal co-training config when you specifically need views.plan: bench/configs/experiments/minimal/inductive/co_training/text/imdb.yaml.

17.4 Safe editing strategy

When you fork a working config, change fields in this order:

  1. run.name and run.output_dir
  2. dataset.id and dataset.options
  3. sampling.plan
  4. preprocess.plan
  5. graph.spec if the method is transductive
  6. method.id and method.params
  7. evaluation.metrics

Keep one known-good baseline config untouched so you always have a rollback point.

17.5 Common patterns

17.5.1 First local smoke

Start with one of the toy configs, then rerun with --log-level detailed only if you need deeper diagnostics.

17.5.2 First real dataset run

Start with the matching minimal config instead of copying a large benchmark suite from bench/configs/best/.

17.5.3 Clean-room reproducibility check

Reuse the same config, set a dedicated MODSSC_CACHE_ROOT, and keep the copied config.yaml and run.json together under runs/.

Sources
  1. bench/configs/experiments/minimal/README.md
  2. bench/configs/experiments/toy_inductive.yaml
  3. bench/configs/experiments/toy_transductive.yaml
  4. bench/configs/experiments/toy_inductive_hpo.yaml
  5. bench/schema.py