Edwin Peter - BoredDigest

2 months ago 2 min read

Serving LLMs

Summary of notes, taken from Efficiently Serving LLMs from deeplearning.ai Autoregressive Generation: How does a Causal Language Model work? 1) Loading the LLM & Tokenizer from transformers import AutoModelForCausalLM, AutoTokenizer model_name

2 months ago 1 min read

LLM Evaluator — offline text-quality metrics with no neural weights

LLM Evaluator is a small, fully-offline tool for a very specific job: measuring how good one text generation is against a reference. Give it two sentences — a ground-truth and a candidate — and it

2 months ago 1 min read

CoinRAG — when chunk-level retrieval is too coarse

CoinRAG is a side-by-side demo of what the research paper calls nugget-level retrieval — a finer-grained alternative to traditional chunk-based RAG. Instead of retrieving whole chunks and stuffing them into context, the system first

2 months ago 1 min read

AI Command Hub — a firm-wide view of every model in your organization

AI Command Hub is a full-stack web application built to answer a hard practical question for enterprise AI leaders: how do you keep track of every model your organization runs, at once? It

2 months ago 1 min read

Dynamic Reader — document mapping without the black box

Dynamic Reader is a working experiment in a simple question: can we make document mapping as intuitive as writing a prompt? Instead of training a classifier over labelled examples, it treats extraction and