Augmented Reality (AR) & Virtual Reality (VR) — Systems, Rendering, Tracking, and Use Cases

System Components

AR/VR systems combine optics (HMD lenses, pancake or Fresnel), displays (OLED/LCD/LCOS), low-latency graphics pipelines, inertial and optical tracking, audio rendering, and input devices (hand controllers, hand-tracking, eye-tracking).

Tracking & SLAM for AR

AR requires robust 6-DoF tracking. Visual–inertial odometry (VIO) fuses IMU measurements with camera frames. SLAM systems (feature-based or direct) build local maps; loop-closure reduces drift. Key metrics: pose latency, pose jitter, and tracking robustness under dynamic lighting.

Rendering Pipeline & Latency

Motion-to-photon latency is critical; techniques include asynchronous reprojection, late latching, foveated rendering with eye tracking, and multi-resolution shading (tiled rendering). For distributed AR, network latency and synchronization are additional constraints.

Interaction Models

Interactions range from device-based controllers to direct hand/gesture and gaze. Haptics and audio spatialization enhance presence. UX constraints: minimizing simulator sickness, maintaining stable world-locked anchors, and ensuring comfortable ergonomics.

Applications & Deployment

• Enterprise training and remote assistance (AR overlays).
• Industrial visualization, maintenance, and design review.
• Immersive entertainment, VR simulation, and social VR.
• Medical simulation and telesurgery assistance (requires deterministic low-latency networks).

References

1. SLAM and VIO literature; SIGGRAPH and IEEE VR proceedings for rendering and human factors.
2. ETSI/3GPP for MEC/low-latency networking enabling AR/VR at the edge.