facebookresearch/lilo

Analysis updated 2026-07-17 · repo last pushed 2026-05-12

★ 12PythonAudience · researcherComplexity · 4/5MaintainedSetup · moderate

Mindmap

mindmap
  root((lilo))
    What it does
      Compares candidates via LLM
      Learns preferences from feedback
      Suggests next best options
    Tech stack
      Python
      LLM API
      YAML config
    Use cases
      Tune ML hyperparameters
      Optimize summarizer settings
      Compare design candidates
    Audience
      ML researchers
      Product teams
    Setup
      Bring your own LLM key
      Use synthetic test envs
      Configure via YAML

mindmap root((lilo)) What it does Compares candidates via LLM Learns preferences from feedback Suggests next best options Tech stack Python LLM API YAML config Use cases Tune ML hyperparameters Optimize summarizer settings Compare design candidates Audience ML researchers Product teams Setup Bring your own LLM key Use synthetic test envs Configure via YAML

What do people build with it?

USE CASE 1

Tune machine learning hyperparameters using natural-language feedback instead of manual scoring functions.

USE CASE 2

Optimize a text summarizer's tradeoffs between accuracy, length, and readability by describing preferences in words.

USE CASE 3

Compare candidate designs (like product settings) by having an LLM judge which one better fits your stated goals.

What is it built with?

PythonYAML

How does it compare?

	facebookresearch/lilo	aim-uofa/reasonmatch	airbone42/360-data-athlete
Stars	12	12	12
Language	Python	Python	Python
Last pushed	2026-05-12	—	—
Maintenance	Maintained	—	—
Setup difficulty	moderate	hard	hard
Complexity	4/5	5/5	4/5
Audience	researcher	researcher	general

Figures from each repo's GitHub metadata at analysis time.

How do you get it running?

Difficulty · moderate Time to first run · 1h+

Requires bringing your own LLM API key (OpenAI, Anthropic, etc.) to run experiments.

Copy-paste prompts

Prompt 1

Help me plug my Anthropic API key into LILO's LLM template so I can run its optimization loop.

Prompt 2

Explain how LILO turns pairwise LLM comparisons into a model that predicts what I care about.

Prompt 3

Set up a LILO YAML config to optimize hyperparameters for my machine learning model.

Frequently asked questions

What is lilo?

A Bayesian optimization tool that uses an LLM's plain-English judgments of candidate solutions to automatically find the option that best matches your stated preferences.

What language is lilo written in?

Mainly Python. The stack also includes Python, YAML.

Is lilo actively maintained?

Maintained — commit in last 6 months (last push 2026-05-12).

How hard is lilo to set up?

Setup difficulty is rated moderate, with roughly 1h+ to a first successful run.

Who is lilo for?

Mainly researcher.

Open on GitHub → Explain another repo

This repo across BitVibe Labs

Verify against the repo before relying on details.