How AI Decides What Is Worth Showing You in AR
An AR system may have access to labels, directions, messages, warnings and live data about almost everything around you.
The difficult question is not how to display all of it. It is how to decide what should remain hidden.
This five-part series explains how AI maps physical spaces, combines human signals, filters digital information, handles spatial errors and changes the way people interact with computers.
A useful AR interface is partly an information-removal system. It must prevent available data from covering the physical world the user is trying to see.
A car contains far more information than its dashboard displays.
Sensors may measure engine temperature, battery voltage, wheel speed and many other conditions. The driver usually sees only the information needed for the current situation.
A warning becomes prominent when action is required. Other measurements remain in the background.
Augmented reality faces a similar design problem on a larger scale.
Detecting information is easier than presenting it well
Imagine walking through a railway station while wearing an AR device.
The system could potentially show:
- directions to every platform
- departure times
- shop names
- advertisements
- translations
- messages from other people
- accessibility information
- safety warnings
Displaying everything would make the system less useful.
Labels would overlap. Important signs might be covered. The user would spend more effort reading the interface than understanding the environment.
The central task is therefore selection.
The system starts with possible items
Before anything can be ranked, the system needs a set of display candidates.
Some candidates may come from object-recognition models. Others may come from maps, business databases, calendars, navigation software or manually written rules.
A maintenance application might receive:
- the identity of nearby machines
- their current status
- the technician’s assigned job
- inspection records
- active safety alerts
AI is only one part of this wider system. A learned model may recognise equipment or estimate relevance while ordinary software retrieves records and applies fixed safety rules.
Relevance depends on the current task
The same physical object may require different information for different people.
A visitor near a factory machine may need a warning to stay outside a marked area. A technician may need the maintenance checklist. A supervisor may need production status.
The object has not changed. The task and user role have.
A relevance system can consider signals such as:
- the user’s stated goal
- their location and direction of movement
- the objects currently visible
- their role and permissions
- the stage of an active workflow
- the urgency of an alert
These signals help rank possible overlays.
Ranking does not mean understanding everything
The system may assign each candidate a score.
An emergency warning could receive a high score because it is urgent. A maintenance instruction may rank highly because it matches the current task. A café offer may rank poorly because it is unrelated.
The system can then display the highest-priority item, delay another or hide the rest.
This resembles ranking in search engines and recommendation systems, but the physical setting creates extra constraints.
An irrelevant search result is inconvenient. An irrelevant AR panel can block the user’s view of a doorway, tool or moving person.
Attention is limited physical space
AR overlays compete with the real world for the user’s attention.
A bright animated label can draw the eyes away from a nearby hazard. A large panel can hide an object. Too many indicators can make it difficult to identify which one matters.
This means the system must consider more than informational relevance.
It may also need to estimate:
- whether the user is already reading something
- whether an overlay would cover an important object
- whether the information can wait
- whether audio or haptic feedback would be less distracting
- whether the user is moving through a complex area
The best display method depends on the situation.
A useful overlay should have a reason to exist
Consider a technician approaching five machines.
Only one machine is included in the current work order. The system can highlight that machine and leave the others unmarked.
When the technician reaches it, the interface can display the first inspection step. Later steps remain hidden until they become relevant.
This reduces clutter and creates a sequence the user can follow.
The system is not revealing everything it knows. It is choosing a small action-relevant subset.
Rules and models may work together
Not every display decision should be left to a predictive model.
A safety warning may follow a fixed rule: if a protected area is active and an unauthorised user approaches, show the warning.
A model may still help identify the person’s location or recognise the relevant object. The final presentation rule can remain explicit.
This mixed approach is often more practical than asking one model to decide everything.
Rules provide consistency for known high-priority cases. Models help handle uncertain perception and changing context.
Personalisation creates another risk
An interface may learn which information a user usually opens and which alerts they dismiss.
That can reduce repeated clutter, but it can also hide something important.
A user who ignores several low-priority notices should not automatically stop receiving a later safety warning. A past preference should not override the purpose of the current message.
Personalisation therefore needs limits. The system should distinguish between convenience settings and information that must remain visible.
Minimal does not mean silent
A minimal interface should not hide its own uncertainty.
If the system is unsure which machine the user means, it can show two subtle outlines and ask for a choice. If a spatial anchor is unstable, it can indicate that the placement may be inaccurate.
Removing clutter should improve understanding rather than create false confidence.
The central lesson is:
The hardest AR design problem may not be producing more information. It may be deciding which information deserves to enter the user’s view.
- Urgent information must remain visible even when personalisation would normally hide it.
- Overlays should not cover objects the user needs to see.
- The user needs control over how much information appears and when.
Comments
Post a Comment