Analysis updated 2026-05-18
Study how a rule-based register classifier detects formality in Bengali and Hindi text.
Run baseline comparisons of direct versus English-pivoted machine translation quality.
Try a small demo classifier that labels Hindi pronouns as formal, familiar, or intimate.
| kanishkpaul/register-obstruction | 0-bingwu-0/live-interpreter | 0cm-labs/tokenizer-benchmark | |
|---|---|---|---|
| Stars | 2 | 2 | 2 |
| Language | Python | Python | Python |
| Setup difficulty | easy | moderate | moderate |
| Complexity | 3/5 | 2/5 | 2/5 |
| Audience | researcher | general | researcher |
Figures from each repo's GitHub metadata at analysis time.
The public demo classifier installs with pip and runs with no external dependencies, but the full research dataset and instrument are not public.
register-obstruction is a research project studying a specific problem in machine translation between Bengali and Hindi. Both languages grammatically force a speaker to choose a level of formality, called register, when addressing someone: formal, familiar, or intimate. English has no equivalent grammatical marking, so when a translation system routes Bengali to Hindi by passing through English as a middle step, that formality signal gets lost and the system has to guess it when producing the final Hindi text. The project asks whether this guessing causes measurably worse translations than translating directly between Bengali and Hindi. The author frames this as an active, ongoing research preview rather than a finished result. Part of the work, a rule-based classifier that detects register from pronouns and verb endings, has been built and validated, along with baseline measurements comparing direct versus English-pivoted translation across several models. The more ambitious part of the study, a mathematical way of scoring how much information is obstructed by the pivot, has been planned and preregistered but not yet run. The README is explicit that some data and results are being withheld until a future writeup, including the frozen test sentences and the full internal classifier details. The validation results show the register classifier performs well on both languages, though the author is careful to note the validation labels were produced by Claude rather than independent human reviewers, so the numbers reflect agreement between the classifier and an AI model, not confirmed human judgment. A separate finding noted along the way is that certain Hindi verb conjugations are ambiguous across multiple grammatical persons, meaning any register classifier for Hindi inherits that built-in uncertainty. Baseline results comparing direct and pivoted translation across five different translation systems are included as a results table, showing that dedicated translation models lose most register information even without the English pivot, while general-purpose language models show a much larger drop in accuracy specifically when routed through English. Everything in the study runs locally on a single Mac with sixteen gigabytes of memory, using quantized local models, with no cloud services or paid human annotation involved. A small demo classifier for Hindi honorific pronouns can be tried locally after installing the package with pip and running the included example script and tests.
A research project measuring whether translating Bengali to Hindi through English loses honorific formality information that neither language marks the same way.
Mainly Python. The stack also includes Python, MLX, llama.cpp.
Use freely for any purpose, including commercial use, as long as you keep the copyright notice.
Setup difficulty is rated easy, with roughly 5min to a first successful run.
Mainly researcher.
This repo across BitVibe Labs
Verify against the repo before relying on details.