Retrieval you can inspect.

This evaluates the portfolio demo, not the full RAGOps platform or production traffic.

What executes

BM25 runs for every query: k1 = 1.2, b = 0.75, document titles weighted three times. Four example questions also run cosine similarity against 24 paragraph vectors. Each channel contributes up to 10 candidates to equal-weight reciprocal rank fusion: sum of 1 / (60 + rank), with ranks starting at 1. The top 5 become retrieved evidence. There is no reranker.

What is precomputed

Open-source MiniLM embeddings are generated offline using a pinned quantized ONNX model, attention-mask mean pooling and L2 normalization. Document inputs contain the title and paragraph. Query embeddings are saved for four examples and the separate evaluation dataset. Free-text queries have no query embedding and use BM25 only. No model or API runs in the public site.

Pinned model · Apache 2.0 · Vectors and generation provenance · Embedding generator

Labels and metrics

The 24 authored evaluation paraphrases differ from the four demo questions. Each has one manually designated relevant paragraph. This small diagnostic set is not a held-out benchmark; judgments are not exhaustive. Recall@5 is relevant labeled chunks retrieved divided by labeled relevant chunks, averaged over queries. MRR@5 is the average reciprocal rank of the first labeled relevant chunk within the first five results, or zero when absent. No answer quality or faithfulness is scored.

The corpus, sample questions and replay answers remain in English in both interface languages, so the query inputs, rankings and evaluation remain comparable.

Reproduce and inspect

Run pnpm eval in the portfolio checkout using the saved document and evaluation-query vectors. pnpm eval:check recomputes rankings and checks the report, source hashes and public copies. Builds reject stale evaluation artifacts.

Answer boundary

Answers are authored fixture replay. A fixture is shown only if all its required citation chunks occur in the actual top-five results. Otherwise it is withheld. Retrieval never inserts a passage to make an answer appear supported.