When AI Visibility Changes, Ask What Actually Moved

Photo credit: Johna Burke

As organizations rush to measure visibility across LLMs and AI-powered search, communication leaders risk mistaking changes in the data for changes in reputation or impact.

Watch the shadows. Don’t chase them.

A blood moon offers a useful analogy for one of the most important challenges emerging in communication measurement and evaluation: what we see can change dramatically even when the thing we are observing has not.

That matters as organizations begin tracking what large language models say about them, which sources are retrieved or cited, how prominently brands appear and how those results change across prompts, platforms and time. A brand may disappear from an answer, a competitor may suddenly appear, a different source may be cited or visibility may rise and fall from one observation to the next.

Those changes deserve attention. However, before declaring success, diagnosing a problem or changing strategy, communication leaders should ask a more important question:

What actually moved?

Was it your communication, reputation or audience behavior? Was there a change in competitor activity, earned coverage or the available source material? Did the platform, model, prompt or methodology change? Or did the conditions surrounding the observation change?

This is the measurement challenge underlying much of the current excitement about generative engine optimization (GEO), answer engine optimization (AEO) and AI-mediated discovery. We have new things to measure, but that does not mean we should abandon what we already know about measuring well.

An Observation Is Not Yet a Conclusion

An LLM response captured at a particular moment tells us what was produced under a particular set of conditions. That observation can be useful. Repeated systematically, it can contribute to evidence of a pattern. Tested against other credible signals and considered over time, it can help inform a decision.

What it cannot do is tell us everything we need to know about visibility, reputation or impact. One prompt, one platform, one score or one result should not become a conclusion simply because it is measurable.

This is why the discipline behind the AMEC GEO Principles emphasizes repeatability, documented methodology, transparency and testing across tools, prompts, markets, languages and time. AI outputs can provide directional evidence, but their value increases considerably when interpreted in context and triangulated against other relevant signals.

The technology may be new. The analytical responsibility is not.

That is an important distinction for our profession. Generative AI is changing the environment in which information is discovered, interpreted and shared, but it has not suspended the principles of good measurement. If anything, an environment that produces more data, more quickly and under less visible conditions makes those principles even more essential.

The Moon Didn’t Change

During a lunar eclipse, Earth moves between the sun and the moon. Sunlight passing through Earth’s atmosphere reaches the lunar surface differently, producing the coppery red appearance we call a blood moon.

The moon itself has not suddenly turned red. The conditions surrounding how we see it have changed.

Communication data can behave in much the same way. A spike in sentiment, a drop in engagement, a shift in search visibility or an unexpected reputation signal may represent something important. Equally, the underlying subject may be relatively stable while something in the environment surrounding our observation has changed.

A snapshot tells us what something looked like at a particular moment, but it does not necessarily tell us why. That distinction is becoming more important, not less, as AI gives us additional observable data points and increasingly sophisticated ways of collecting them.

The temptation will be to measure everything that becomes visible. The discipline lies in determining what it means.

Context Explains and Triangulation Tests

When AI visibility changes, the objective should not be to react immediately to the movement. It should be to understand what the signal represents.

Can the result be replicated, and does it persist over time? Does it appear across platforms, prompts, markets or languages? Are changes in earned coverage, source prominence, competitor activity or reputation indicators pointing in the same direction? Are independent forms of evidence supporting the same interpretation?

These questions matter because one observation may be interesting without being significant. Multiple credible signals moving consistently provide a much stronger basis for deciding whether to act.

Over time, evidence helps establish persistence, context helps explain meaning and triangulation helps test whether an interpretation is sufficiently credible to inform a decision. Analytical distance can also improve our perspective, particularly when a result is unexpected or emotionally charged.

This is why persistent measurement and evaluation are so important. Longitudinal evidence helps us distinguish a trend from a spike, an anomaly from an early indicator and meaningful change from temporary distortion. It allows us to watch what is changing without assuming that every movement deserves pursuit.

An anomaly should never be dismissed simply because it might be temporary. Watch it and interrogate it, but do not chase it before understanding what it represents.

Sometimes the issue is not the data at all. It is the perspective through which we are interpreting it.

When the Shadow Becomes the Whole Picture

People closest to an organization, campaign, issue or crisis possess institutional knowledge that an outsider may never fully replicate. That context is enormously valuable, but proximity can also influence interpretation.

An internal concern can begin to feel universal. A sudden negative signal can appear overwhelming. A metric watched for years can continue to command attention even when other evidence suggests that its significance has changed. Gradually, the shadow can begin to look like the whole picture.

This is why diversified analytical perspective matters. Internal teams bring organizational knowledge and an understanding of history, relationships and strategic intent. External analysts, research partners and measurement specialists may bring comparative evidence, distance and different assumptions. External does not automatically mean objective, just as internal does not automatically mean biased. The value comes from deliberately testing those perspectives against one another.

The reverse matters, too. An external analyst may identify a developing pattern that an internal team has gradually normalized through repeated exposure. Equally, an outsider may misunderstand a signal because they lack the institutional context required to interpret it responsibly.

Neither perspective should automatically prevail. Instead, ask what each can see that the other cannot and whether independent evidence supports the interpretation.

In other words, triangulate the data and triangulate the perspective. Then ask the question that makes analysis considerably more useful:

What evidence would prove us wrong?

Credible analysis should challenge assumptions, expose blind spots and distinguish between what feels dominant and what the evidence demonstrates is significant. Sometimes distance does not make the shadow disappear. It simply allows us to see that it has edges.

Better Data Should Produce Better Decisions

None of this is an argument for waiting until every uncertainty disappears. That would make measurement an obstacle rather than a management tool.

The purpose of rigorous measurement and evaluation is to give leaders greater confidence about when the evidence is strong enough to act and when it is telling us to keep watching. That distinction will become critical as organizations receive increasing volumes of data about their visibility across search, LLMs and other AI-mediated environments.

More dashboards will not solve the interpretation problem, and neither will compressing a complex information environment into another isolated score. The opportunity is much greater than that.

We can establish meaningful baselines, maintain comparable data over time and interrogate anomalies rather than simply reporting them. We can test emerging signals, introduce relevant context, triangulate findings and challenge the assumptions through which those findings are interpreted. Most importantly, we can make the decision that the evidence supports rather than the decision that the most dramatic movement seems to demand.

I was reminded of one final part of this while photographing the blood moon myself. The photographs showed the same eclipse under changing conditions. Exposure, framing, focal length and available light affected what my camera could capture and how prominently different parts of the scene appeared. The physical event did not change because I changed the way I photographed it, but the lens changed what I could see.

Data has lenses, too. Knowing what we can see, what we cannot see and what may be influencing the view is part of interpreting it responsibly.

Because whether we are looking through a camera, a dashboard or an LLM, context completes the picture. Sometimes the most dramatic movement in our data is not a change in what we are measuring.

Sometimes, the shadow moved. Watch the shadows. Don’t chase them.