okaybroda/hrm-embed

HRM-Embed is a research model that turns text into a list of numbers, called an embedding, which can then be used to search for similar or related documents. What makes it unusual is the underlying design. Instead of the typical transformer architecture most embedding models use, it is built on top of a different architecture called HRM, which loops over its own internal state multiple times rather than processing text in a single pass, an approach originally designed for reasoning tasks. The author's idea was that this looping behavior might help with a benchmark called BRIGHT, which tests retrieval that requires several logical steps rather than just matching similar wording. That hypothesis partly held up: the model does better on reasoning heavy searches. However, its accuracy is limited by the fact that its base model was trained on a fairly narrow slice of text, so it lacks broad general knowledge, which hurts it on knowledge heavy searches. The README explains that a plain query alone performs weakly, scoring 18.1 on the benchmark's scoring system, but performance jumps to 34.3 when an AI model first rewrites the query into a clearer form before searching. This rewriting step helps almost every subject area except code search, where it can actually hurt results. The project includes the full evaluation code so results can be reproduced. Using the model requires Python and the transformers library, plus code the author includes to pull the embedding out manually since it does not work with the standard sentence-transformers tool most embedding models use. It was trained entirely on a single consumer graphics card. The model builds on an existing open source HRM text model and is released under the Apache 2.0 license.