Visualize you have a new system that appears at every doc_id and exact_query, and afterwards calculates the probability of click For each and every doc for every query. You see that its behavior is nearly just like your recent method in both side by sides in addition to a/B tests, https://best-woman-award54233.rimmablog.com/32043801/machine-learning-convention-can-be-fun-for-anyone