Lab component 15

Which passage would get cited, and why

Ask a question. This page splits itself into passages, scores each one with real BM25, and checks whether the winner can stand on its own once it is lifted out of context. That second test is the one most content fails.

Passage Retrieval Scorer

Real BM25 Chunks this page live No model call

What a retrieval system would assemble

Ask a question above.

Scoring is Okapi BM25 with the standard parameters, k1 at 1.5 and b at 0.75, computed over the passages of this document at runtime. Term frequency is saturated so repeating a keyword stops helping, inverse document frequency is calculated across the passages themselves, and length normalisation penalises long rambling sections. This is a real ranking function, not a keyword count, and it is roughly the lexical half of what a hybrid retrieval pipeline does before any model sees the text.

You are not ranking a page any more, you are ranking a passage

A generative surface does not cite documents. It retrieves chunks, and then it either uses one or it does not. That changes what a well written section looks like.

01

Self containment beats flow

A paragraph that opens with this, it, or they reads beautifully in sequence and scores badly in isolation, because the referent is in the paragraph above and the retriever never fetched that one.

02

Length is a real penalty

BM25 normalises by passage length. A 400 word section that answers one question will lose to a 90 word section that answers the same question, every time.

03

Keyword stuffing stops working at three

Term frequency saturation means the fourth mention of a phrase adds almost nothing. The curve flattens, which is exactly why the tactic died and why nobody noticed the mechanism.