PPreference
stated categoryprofile weightstored state
What the user explicitly stored
Auditing the Preference–Exposure–Consumption Gap
An interactive companion to the paper“When Exposure Is Not Attention: Auditing the Preference–Exposure–Consumption Gap in Personalized News Recommenders”Compare preference profiles, recommendation logs, app-surface pathways, and clicks—then examine what each trace can establish and where interpretation must stop.
Measurement map
The gap lives between traces.
Select a relation to inspect the gap between traces.
ELogged exposure
article IDrankuser-day list
What the system retained
SSurface
home / feedsearchcategory
Where the click journey entered
CConsumption
clicktimestamparticle category
What the click log observed
P ↔ C
Does the stored profile align with later clicking?
Compare dominant categories, active sets, and weighted distributions. Each statistic answers a different version of “alignment.”
argmax match1[argmax(P) = argmax(C)] · coarse dominant-category recovery.
Interpretation boundaryAgreement between P and C does not establish satisfaction, total interest, or causal personalization success.
User journey
The same click can leave different traces.
Select a scenario. Each card reports a stored trace, not an assumed stage in a perfect funnel.
Climate
The stored profile assigns climate a high weight.
Present · rank 3
The article appears in the retained same-day recommendation list.
Home / feed
The click event records a feed entry pathway.
Clicked
A click is observed. Reading depth is not.
Supported observation
This article is present in the retained list and later appears as a click event.
Still outside the data
The logs do not establish that the user visually attended to the list entry or read the article deeply.
Synthetic preference simulator
Compare the profile with later clicks.
Adjust P and C. Top match compares the largest categories, Jaccard compares active sets, and cosine compares their weights.
P = stored stated preference. C = clicked-article categories. Neither represents total interest.
CategoryP · profileC · clicks
Politics
Technology
Climate
Culture
Top-category matchNoPolitics ≠ Technology
Set overlap · Jaccard0.750active categories shared by P and C
Weighted similarity · cosine0.000directional similarity of the two vectors
Formulas and the current calculation
Top-category match
argmax(P) = argmax(C)Politics ≠ Technology → no match
Jaccard set overlap
J = |P⁺ ∩ C⁺| / |P⁺ ∪ C⁺|3 shared / 4 active = 0.750
Cosine similarity
A = (P · C) / (‖P‖₂ ‖C‖₂)weighted vector similarity = 0.835
Jaccard ignores weight magnitude and compares only active category sets. Cosine uses the weights, but two equally shaped vectors can score highly even when their absolute volumes differ.
Traceability lab
What does an exposure–click match actually measure?
Toggle the synthetic clicks, follow the denominator, and compare traceability with what the log still cannot establish.
Synthetic click setFound in the retained same-day list?
Live calculation · synthetic
Htrace = 28 = 25.0%
2 matched clicks support same-day list–click co-occurrence. The other 6 are untraced by this retained list.
SupportsThe article appears in retained E and later in C.
Does not establishVisual attention, causal influence, or no exposure elsewhere.
Paper evidence · comparable click events
The retained list carries signal above a date-matched candidate-pool baseline, while still tracing only a small share of the observed journey.
Observed hit11.29%retained list
Baseline9.13%date-matched pool
Difference+2.16 pp11.29 − 9.13
Top-5 lift1.47×Hlogged@5 / Hbase@5
Interpretation: above-baseline overlap shows that the retained recommendations are informative. Low absolute coverage shows that they are not a complete record of attention or access.
Diversity calculator
Change the distribution, change the conclusion.
Adjust four category counts. Exposure and clicks remain separate distributions throughout the calculation.
Distribution profile
Exposure Clicks
Concentrated · 0Entropy / evenness1 · Even
Exposure
0.000 · 0.00 effective categories
Clicks
0.000 · 0.00 effective categories
Dots are the exact category probabilities. The smoothed silhouettes are visual guides: a flatter, broader profile has higher entropy; a sharper peak has lower entropy. Category positions are labels, not a continuous scale.
Exposure normalized entropy0.000
Click normalized entropy0.000
HHI gap · clicks − exposure0.000
JS divergence0.000
The current click distribution is more concentrated than the logged exposure distribution.
EntropyHow evenly mass spreads
HHIHow strongly mass concentrates
JS divergenceHow category composition differs
Metric formulas and the current calculation
Normalized entropy
Hₙ(X) = −Σ pₓ log(pₓ) / log|X⁺|Exposure 0.995 · Clicks 0.805
HHI concentration
HHI(X) = Σ pₓ²HHI(C) − HHI(E) = +0.144
JS divergence
JS(P,Q) = ½KL(P‖M) + ½KL(Q‖M)M = (P + Q) / 2 → 0.069
Reading the direction
entropy ↓ · HHI ↑ = concentration ↑JS measures compositional difference and has no “more diverse” direction by itself.
X⁺ is the set of categories with nonzero mass. Probabilities are computed separately within exposure and clicks, so the comparison concerns their shapes rather than raw event totals.
Paper evidence
Read each result with its denominator and boundary.
Switch questions and conditions to see what changes—and what the result does not establish.
Claim checker
Match each claim to its evidence.
Select a statement to see the strongest interpretation supported by the available trace.
Supported
E
Directly observed in E
The retained recommendation log can establish list membership and recorded rank for that list. It does not establish visual attention.
Observed trace
Item identity, user-day list membership, and recorded rank in E.
Evidence still missing
A verified impression or any measure of visual attention.
Safer wording“The article appears in the retained recommendation list.”
Lessons from the PEC audit
Separate the trace before interpreting the user.
Do not collapse P, E, S, and C.
Stated preference, logged lists, interface pathways, and clicks are different measurements. Agreement between two layers does not make them interchangeable.
Name the missing mechanism.
A logged item does not establish visual attention. A click does not establish reading depth. A same-day match does not establish that the recommendation caused the click.
Make the denominator part of the result.
Count matching, click thresholds, retained-list coverage, and eligible-user rules can change the apparent gap. Report them beside every metric.
Treat absence as bounded uncertainty.
Not finding a click in one retained list means “not traceable here.” It does not prove that the platform never surfaced the article elsewhere.
The paper’s message: responsible recommender evaluation begins by distinguishing what the system recorded, what the analyst inferred, and what the data cannot reveal.
Continue with the paper and public artifact.
The tutorial provides the measurement logic; the paper documents the protocol, cohorts, uncertainty, and complete empirical results.
Questions and feedback on the paper are always welcome. I would be happy to connect with others working on recommender systems, ranking, information retrieval, or algorithmic auditing. Discuss the paper → Author website →