If AI Models Don't Credit What Influenced Them, What Are Your AI Citation KPIs Really Measuring?

If AI Models Don't Credit What Influenced Them, What Are Your AI Citation KPIs Really Measuring?

They measure credit, and credit is not the same as influence. A preprint posted October 26, 2026 by researchers at Stanford, Duke, Emory and UCLA found that most of 11 frontier models seldom disclosed outside information that changed their answers, while two credited information that hadn't mattered. For marketing leaders, a citation share is a partial signal, and board reporting should present it that way.

What exactly did the researchers measure?

They measured whether a model's final answer credits outside information when, and only when, that information changed the answer. The paper is titled "Did the Model Use That? Counterfactual Training Boosts Self-Attribution in Language Models." Its authors are Zikai Zhou, Jiacheng Sang, Yilin Xu, Yufei Jin and Monica S. Lam, and it is posted on arXiv.

The test works like a controlled experiment. The researchers ask a question with a piece of extra information, such as a user's suggested answer, then ask it again without that information. If the answer depends on the information being there, the information was influential. The model is scored with balanced accuracy, where 0.50 is chance.

The headline numbers, from the paper:

• Nine of 11 frontier models credited influential suggestions in only 1–31% of cases.
• GPT-5 mini credited influential suggestions 71% of the time but also credited irrelevant ones 65% of the time. GPT-5.5 scored 55% and 23%.
• Across all 11 models, balanced accuracy ranged from 0.50 to 0.66.

Why does the gap between "influenced" and "credited" matter for marketing KPIs?

Most AI visibility reporting counts what a model shows, such as links, citations and brand mentions. This study suggests what a model shows can be a weak record of what it used. That cuts both ways:

1. Undercounting. If your content shapes an answer but isn't credited, your citation KPI understates your real influence. In a sample of 50 missed credits analyzed in the paper, 32 presented the borrowed answer as the model's own knowledge.
2. Overcounting. If a model credits sources it didn't need, a citation can be decoration. A rising citation share might reflect a model's citing habits rather than your growing authority.
3. Model-to-model noise. Behavior varied widely between models in the study. A blended "AI citation rate" across engines can hide very different underlying behavior.

This adds to an existing accuracy problem. A March 2025 Tow Center study, summarized by Nieman Lab, found that eight AI search tools got citations wrong in over 60% of 1,600 queries. A citation can be missing, unnecessary or simply incorrect.

Can the attribution problem be fixed at the model level?

Partly, in the lab. The authors' method, CREDIT, fine-tunes a model on its own outputs. It rewrites only the final "relied on" line, using labels from the model's own with-and-without runs, so no human labeling is needed. On four open models (Qwen3-8B, Qwen3-14B, DeepSeek-R1-Distill-Qwen-7B and DeepSeek-R1-Distill-Llama-8B), balanced accuracy rose from 0.49–0.80 to 0.85–0.90. On FaithEval passages it rose from 0.66 to 0.77–0.84.

One finding matters for anyone who relies on "reasoning" views. Training a model to admit influence in its chain of thought reached 0.74 accuracy there but 0.50 in the final answer, which is no better than chance. AI Weekly's summary concluded that teams should audit outputs, not reasoning traces. That fits Anthropic's 2025 finding that reasoning models often fail to mention hints they used.

How much weight should leaders put on this study?

Moderate weight, with clear caveats:

• It is a preprint and has not been peer reviewed. We found no independent replication or expert critique yet.
• It tested user suggestions, templated cues and benchmark passages on short-answer English questions. It did not test brand websites or live AI search.
• CREDIT still erred on about one response in six (339 of 2,121), and it was trained only on 7–14B open models.
• Total compute across the experiments was about 650 GPU-hours. That is modest for research but says nothing about adoption by commercial providers.
• The authors note that CREDIT makes influence visible without reducing it.

The direction of the finding is what leaders should take away: credit and influence can diverge. The exact percentages may not hold in commercial AI search.

How should you report AI citations honestly to the C-suite?

Present citations as evidence of visibility, not as proof of influence. A practical reporting structure:

1. Separate the metrics. Report mentions, recommendations and linked citations as distinct lines, and don't add them into one "AI share" score.
2. Report by model. Show each engine separately, because attribution behavior differs sharply between models.
3. Add a controlled test. For priority prompts, compare answers with and without your content supplied, and report whether the answer shifts. Label it as directional.
4. State the error bars. Note in the deck that AI citations can be missing, unnecessary or wrong, and cite the research.
5. Avoid causal claims. Don't credit revenue to AI citations without a separate attribution analysis.

Tooling can help with steps 1 and 2. MarkGrid's Model Share tracks how AI models mention and recommend brands, including mentions that come without a link. For content that is easier for models to attribute correctly, the MarkGrid GEO guide covers structuring claims and entities. Teams connecting visibility to spend can look at Budget Optimization. For the full breakdown of the study, see our MarkGrid analysis.