Paste ranked results and the relevant ids for each query. Get precision@k, recall@k, MRR, MAP, nDCG@k and hit rate per query, or compare two runs side by side.
To evaluate retrieval, compare each query's ranked list with the ids you know are relevant and average a few numbers over all queries. Precision@k is the share of the top k results that are relevant. Recall@k is the share of all relevant ids that appear in the top k. Hit rate@k is the share of queries with at least one relevant result in the top k. Reciprocal rank is 1 divided by the rank of the first relevant result, and MRR is its mean over queries, so a first relevant result at rank 2 scores 0.5. Average precision adds up the precision at every rank that holds a relevant result and divides by the number of relevant ids, and MAP is its mean. nDCG@k discounts each gain by log2 of its rank plus 1 and divides by the best possible ordering, so it is 1 for a perfect ranking and can use graded relevance.
One query returns d3, d1, d7, d2, d9, and the relevant ids are d1 and d2. At k = 5, two of five results are relevant, so precision@5 is 0.4, and both relevant ids were found, so recall@5 is 1. The first relevant result is at rank 2, so the reciprocal rank is 0.5. Relevant results sit at ranks 2 and 4, so average precision is (1/2 + 2/4) / 2 = 0.5. For nDCG, DCG is 1/log2(3) + 1/log2(5) = 1.0616, the ideal DCG with both relevant ids first is 1 + 1/log2(3) = 1.6309, and nDCG@5 is 0.6509. The sample on this page contains this query.
Use one line per query with tabs between three columns: the query, the ranked ids, and the relevant ids, with ids separated by spaces or commas. You can also paste JSON, either keyed by query, as in {"q1": {"ranked": [...], "relevant": [...]}}, or as a list of objects. Relevant can be an object that maps ids to grades, such as {"d1": 3, "d2": 1}, for graded nDCG, with a linear gain by default or 2^grade - 1 if you choose it. Precision divides by k even when a query returned fewer than k results. A repeated id in a ranked list counts once, at its first rank, and the repeat still takes its place, so later ids keep the rank they were listed at. Queries with no relevant ids are left out of the averages and counted. MAP and MRR look at the whole list, not only the top k.
Paste a second system's ranked ids in the Run B box, matched to run A by query text, and the page shows both averages, the change, and a per-query table of nDCG, reciprocal rank and recall for each run. Both runs are always scored against run A's relevant ids, so the comparison is fair; relevant ids typed into the Run B box are ignored, and the page says so when they differ. A query that run B lacks scores 0 there, and the page says how many. The tool reports numbers only: it does not run a significance test, and on a few dozen queries a gap of 0.02 can be chance. It also does not judge relevance for you or call a search system. You supply the ranked ids and the labels.
Add gain / log2(rank + 1) for each relevant result in the top k to get DCG, then divide by the DCG of the best possible order. For ranked d3, d1, d7, d2, d9 with d1 and d2 relevant, DCG@5 is 1.0616, ideal DCG is 1.6309 and nDCG@5 is 0.6509.
MRR looks only at the first relevant result: rank 2 gives 0.5. MAP averages the precision at every relevant result, so it also rewards finding the second and third relevant ids early. With relevant results at ranks 2 and 4, reciprocal rank is 0.5 and average precision is 0.5 too, but they diverge on other lists.
Precision divides by k. If a query has one relevant id and it is at rank 1, precision@5 is 1/5 = 0.2, however good the ranking is. Use hit rate, MRR or nDCG to judge a single-answer task.
Give relevant as an object of grades in JSON, for example {"d1": 3, "d2": 1}. The default gain is the grade itself, and the 2^grade - 1 option weights top grades more. Yes or no labels are grade 1, and both gains then agree.
A query with no relevant ids has no recall, MAP or nDCG, so it is left out and counted in the notice. Add at least one relevant id, or remove the line.
No. They are processed in your browser, in this tab, and no server receives them. Your cut-off and gain choice are saved in this browser, the queries and ids are not.