LLM Evaluator is a small, fully-offline tool for a very specific job: measuring how good one text generation is against a reference. Give it two sentences — a ground-truth and a candidate — and it scores the candidate with four standard statistical metrics: BLEU and ROUGE for n-gram precision and recall, METEOR for unigram alignment with stemming and synonymy, and a character-level perplexity model. A composite 0–1 score blends the strongest signals into a single at-a-glance verdict.
The clever part is what it deliberately does not do. All metrics are implemented in pure Python and NLTK with no neural weights, so the tool runs deterministically, offline, in a tiny image. It is a clean, inspectable reference for teams that want ground-truth text metrics without standing up a model-serving pipeline — and an LLM-as-a-Judge panel is built in and ready to be switched on when needed.
Check out the demo here:
