How do waveguides and combiners compare in building augmented reality displays? originally appeared on Quora - the place to gain and share knowledge, empowering people to learn from others and better understand the world.
See-through optics has a sort of inaccessible mysticism thanks to sci-fi (Iron Man, Star Trek, etc.) and dense hardware vocabulary. But it's super easy to get started. So this will be an accessible, layman’s introduction to the topic.
Let's begin with basics. We have these partially transparent displays that mix digital images with the real world. Light rays need to bounce off something to redirect into your eye. From the real world, we are already getting redirected rays. From the digital world, we need to create artificial light (e.g. from LEDs, OLEDs) and then redirect them. The optical device that combines this computer-generated image with the real world is called a "combiner." Essentially, a combiner works like a partial mirror that redirects the display light and selectively lets light in from the real world. Pretty simple.
Like the question suggests, the optical hardware solutions can break down into two categories: conventional HMD optical combiners and emerging waveguide combiners. And hopefully obviously, both of these are very different, and have very different tradeoffs from opaque virtual reality displays.
There is an extensive history of see-through displays since the late 1960s (I’ll add some good places to get more technical background at the end). Consequently, there is a huge range of optical technologies, but it all boils down to fundamental tradeoffs between resolution, field of view, eye box, image quality, hardware weight/fit, aesthetic form factor, and other features. Ideally, everyone wants stylish, easy-to-use glasses with 200x100 degree FOV (like human eyes) and perfect image quality invented by Tony Stark from Iron Man. But physical and optical limitations of HMDs/NEDs, e.g. how we bounce and bend light with actually wearable hardware, make that unrealistic in the foreseeable future. So we need to figure out which tradeoffs we care about.
Optical hardware is entirely about tradeoffs.
Traditional combiners produce reasonable see-through and imaging quality with consistent performance and affordable materials owed to decades of supply chain development. We can cover two popular varieties of the implementations: polarized beam combiners as an example of flat combiners, and off-axis combiners as an example of curved combiners.
Examples of polarized beam combiners include Google Glass as well as the smart glasses from Epson (Japan), Rockchip (China) and ITRI (Taiwan). Beam splitters can be polarized using LCOS microdisplays, like Google Glass does, or just by using simple half-tone mirrors. Unfortunately, PBC-based displays tend to be small due to weight and size constraints for the combiner, and there may be additional blurriness from the beam split. Google Glass gets 13-degree FOV, and Epson BT-300 gets 23-degree FOV with 1280x720 resolution. Both are on the low end of the acceptable range for consumer displays. However, larger FOV and resolution would require much bigger and heavier hardware.
Pros: light, small, relatively affordable ($500–700ish) Cons: very limited FOV and display, difficult to improve.
The best modern example of an off-axis, semi-spherical combiner is the Meta 2. Unlike other varieties of small and light combiners, Meta went the other direction for larger FOV and display resolution. They tout a single OLED flat panel to support an “almost 90-degree FOV” and a 2560x1440 pixel display split between both eyes. However, their hardware is bulkier and comparable to VR headsets like the Oculus and HTC Vive. Additional concerns include their low angular resolution (less detailed/crisp images) and how the plastic material of their combiner maintains its quality (e.g. minor perturbations become emphasized and strain over time may lead to eventual visual artifacts) — which are choices they made to lower costs. An older example of curved combiners is Link's Advanced Helmet Mounted Display.
Pros: High FOV and resolution, relatively affordable ($900ish).
Cons: Bulky, low angular resolution, material quality risk.
Trying to improve traditional combiners with FOV and resolution means a smaller eye box, thicker combiner optics, larger combiner further away, and poor imaging quality. It’s not about computational performance limits as much as physical ones from how light behaves with the hardware.
After addressing this difficult tradeoff problem, newer technology is now pushing into non-conventional techniques like holographic and diffractive optics. These methods use what’s called waveguide grating or waveguide hologram to progressively extract a collimated image guided by total internal reflection (TIR) in a waveguide pipe. The pipe is a thin sheet of glass or plastic that the light bounces through. In effect, you can think of a waveguide like a router that transmits the image at your eye.
Waveguides are the most technically sophisticated type of see-through optics, and they are equally hard to design. These ideas, however, are not new, and folks have been exploring waveguides for optics since the early 80s. Companies like Sony , Konica/Minolta , Nokia/Microsoft , Magic Leap, and many others have all worked on various waveguide combiners.
Surface relief slanted sub-wavelength gratings, for example, are the commonly presumed implementation used for Microsoft Hololens. Here, the waveguide has a series of very delicate structures in a linear pattern (fine on the order of the wavelengths of light). This diffraction grating acts as a lens to bend the light through the TIR until it exits towards the eye. A pleasant result from this process is “pupil expansion”; the egressing light can slightly diffuse to increase its FOV.
All in all, state-of-the-art waveguide techniques might get you something close to 32Hx18V degree FOV at 1920x1080, potentially without as much of the bulk and weight of traditional combiner workarounds. Magic Leap patents suggest its technology aspires to get close to 120Hx80V degree horizontal FOV but may end up closer to 50–55-degree FOV. The improved FOV could be more promising, or at least more glorified than traditional approaches — but little of the promise has been demonstrating itself so far. Moreover, waveguide combiners have their set of challenges.
First, waveguides require a lot of precision and are finicky — volume holographic media like photopolymers, DCG, silver halides, etc. can change based on environmental temperature, humidity, and pressure. Second, the angular resolution weakens with more diffusion (i.e. FOV vs. imaging details tradeoff). And finally, the supply chain has not been readily established for the technologies, so mass production is difficult and expensive, it is noteworthy to mention the substantial $1B+ ongoing R&D cost by both companies.
Pros: Potentially better FOV and resolution on medium sized devices.
Cons: Expensive technology (estimated $3k+), still actively in development, maybe black magic.
In summary, this discussion is mostly between a proven and relatively well-explored space with physical tradeoff limitations (traditional combiners) compared to a much-hyped area of experimental technologies that could bypass these physical limitations (waveguide combiners). Personally, I think the distrust in waveguide technology is not entirely unmerited; no public demo has shown any work better than traditional combiners. On the other hand, I also think these massive investments make complete sense.
Consumers, calibrated on sci-fi expectations, have been underwhelmed at most/all traditional combiner hardware. In the past five decades of optics, AR existed as niche products. There may still be new improvements in conventional combiners (like ODG glasses), but for Microsoft and Magic Leap, waveguides are moonshot projects for AR optics promising enough to be accepted by the mass market.
Hopefully, this article can serve as a starting point for directions to better resources.
High-level tradeoffs:
Technical history:
Karl Guttag’s blog on graphics and display optics:
Hardware/market breakdowns:
Understanding waveguides/Hololens: http://www.theverge.com/2016/4/6/11376442/microsoft-hololens-holograms-parts-teardown-photos-hands-on
This question originally appeared on Quora - the place to gain and share knowledge, empowering people to learn from others and better understand the world. You can follow Quora on Twitter, Facebook, and Google+. More questions:
- Augmented Reality: What are some great ideas for new augmented reality games (like Pokémon GO)?
- Optics: Why do so many video games have an aversion to using working mirrors in their environments?
- Optical Computing: What is preventing us from building optical computers that use light instead of solid-state hardware?