amap-ml/blockpilot

BlockPilot is a small collection of Python scripts for running benchmarks and predictions around something the README calls block sizes and acceptance lengths, without ever explaining what task or model these blocks belong to. The repository has no description or topic tags on GitHub, and the README itself is just a quickstart guide, so the underlying purpose has to be inferred from the script names and their behavior. Setup is a standard Python virtual environment created with venv, followed by installing dependencies from a requirements file. From there, one script named run_benchmark_acceptance_lengths.sh runs across multiple datasets and multiple block sizes, writing out JSON files that record an acceptance length for each combination. A second script, run_dataset_txt.sh, wraps a Python file called dataset.py and takes a plain text file of prompts as input, with options to choose which block sizes to test, how many samples to use, and a random seed. Two more scripts round out the toolkit. utils/predict.py loads a saved dataset and a saved model, both stored as PyTorch files, and predicts block ids for a task, writing the results to a JSON file. map_selected_block_values.py then takes those predicted ids and looks up the corresponding value from separate block-specific JSON files, translating a chosen id back into whatever value it represents. Because the README only documents commands and file paths rather than explaining what the model does, what a block size represents, or what the acceptance length measures, this project reads as an internal benchmarking toolkit for a larger, unpublished piece of research rather than a general-purpose tool ready for outside use. Anyone approaching it would need to already understand the surrounding project to make sense of these scripts.