Apple Glasses: What's the story so far on Apple's AR headset?

For at least a decade, Apple has been working on augmented reality and virtual reality technologies.

We know this thanks to statements from the company, predictions from analysts, numerous patent filings, and Apple's interest in growing its ARKit platform for AR apps and games. It has also hired AR/VR experts, and it's acquired multiple AR/VR companies. Add it all up, and the rumour mill suggests Apple might be developing a headset that supports AR and possibly even VR.

It supposedly has a secret research unit comprised of hundreds of employees who are working on multiple virtual and augmented reality headset prototypes. Here's everything you need to know about this product, including when it'll arrive.

When will Apple Glasses arrive?

This is hard to pinpoint exactly when Apple Glasses will arrive due to conflicting reports last year predictions were pointing to 2022. But in early 2021, reports are suggesting that the company is set to enter the "second phase of development" and Apple might release its AR Glasses at some point in 2021.

Conversely in July 2019, DigiTimes claimed Apple halted development on all its headsets. The team developing the prototypes was dissolved in May and reassigned to other products. At the same time, it's been claimed Apple moved Kim Vorrath, a software executive, over to the augmented reality headset division to bring "order" to the team.

Vorrath's hiring indicates Apple has not killed its AR headset team, as previously thought. References to an AR headset have also been found in iOS 13 - supposedly codenamed Garta. Code in the latest version of Xcode further indicates Apple is developing an AR headset. There are even references to codenamed test devices.

And after having previously also claimed that the headset will reportedly enter mass production by the end of 2019 and might launch in early 2020, Apple analyst Ming-Chi Kuo suggested in May 2020 that they won't launch until 2022 at the earliest. 

How will Apple Glasses work?

Here's everything we've heard so far about Apple Glasses and what they'll be able to do if they ever come to market...

AR stereo

9to5Mac said Apple is working on stereo AR in iOS that supports a face-mounted AR experience. It's been in internal testing with two Apple devices (codenamed Luck and Franc) and a third-party device, called HoloKit, which is a cardboard AR headset kit. Stereo AR apps can reportedly work in "held mode" (normal AR) or a "worn mode" when used with an external device such as a headset.

AR-only or AR/VR?

It's not yet clear if Apple's smart glasses will be an AR-only headset or an AR/VR headset. It could be the same project or two different projects. It's also unknown if they are indeed "smart glasses" or a full-blown headset. Respected Apple analyst Ming-Chi Kuo expects the AR glasses to be pitched as an iPhone accessory - offloading computing, networking, etc, to the iPhone - like Google's Daydream.

Bloomberg said Apple is developing an AR product with a display, processor, and a new "rOS" or operating system. Supposedly, rOS is based on iOS. Apple is also reportedly developing a "system-on-a-package" chip for this AR headset. It's considering using touch panels, voice activation, and head gestures for input control. Virtual rooms and 360-degree video playback are also being considered.

Other rumours

The latest reports claimed Apple is working on a powerful AR/VR headset that features an 8K display for each eye. It's untethered from a computer or smartphone and should work with both VR and AR apps. This headset can connect to a "dedicated box" over a high-speed, short-range wireless technology called 60GHz WiGig. It features a 5-nanometer Apple processor and resembles a PC tower.

All the technology will be built into the headset and the box.

In the middle of 2019, The Information had was reporting on the company's AR glasses noting info on the design:

"The lenses use a polarized system, similar to the technology in 3D movie glasses, which create the illusion of depth using stereoscopic images, the person said. The technology is similar to that in other AR and VR devices already on the market from Microsoft, Magic Leap and Facebook…

The person said making the AR lenses is especially challenging because they are composed of multiple, extremely thin layers of different synthetic materials, each of which is susceptible to bubbles, scratches and other marks. To reduce defects, the lenses must be manufactured in dust-free zones known as clean rooms."

It's also said that due to concerns about battery life, the Apple AR Glasses likely won't include a camera. 

Has Apple ever spoken about Apple Glasses?

Apple CEO Tim Cook has said the technology does not exist to create AR glasses in a quality way. "But today I can tell you the technology itself doesn't exist to do that in a quality way," he explained. "The display technology required, as well as putting enough stuff around your face - there are huge challenges with that. The field of view, the quality of the display itself, it's not there yet."

Keep in mind, in 2016, he also said: "AR can be really great... We have been and continue to invest a lot in this. We're high on AR in the long run... I think AR is big and profound. This is one of those huge things that we'll look back at and marvel at the start of it."

Who is part of the Apple Glasses team?

Apple has hired several employees with expertise in AR/VR technology, including computer science professor Doug Bowman, who led the Virginia Tech's Center for Human-Computer Interaction. He focuses on three-dimensional user interface design. Apple also hired employees that have worked on Microsoft's HoloLens team, and at Lytro, a company working on a VR camera.

Apple also hired Zeyu Li, who served as a principal computer vision engineer at Magic Leap, as a "Senior Computer Vision Algorithm Engineer." Another hire is Yury Petrov, a former research scientist at Facebook-owned Oculus, who is now serving as a "research scientist" at Apple. Augmented reality expert Jeff Norris also joined Apple in April 2017 as a senior manager. He worked at NASA.

In May 2018, Apple hired Sterling Crispin, who developed a painting app for mobile VR headsets called Cyber Paint. In December 2018, it hired former senior Tesla and Microsoft HoloLens designer Andrew Kim. There's also Jaunt VR founder Arthur van Hoff, who joined Apple as a senior architect in April 2019. Keep in mind Apple's AR glasses/headset team has hundreds of other employees.

In late June 2020 it came to light that Apple is hiring a lot of Magic Leap's employees after the failure of that AR headset. And according to Bloomberg, Apple now has over 1,000 employees working on the project. 

Which AR/VR companies has Apple bought?

Not only has Apple hired AR/VR experts, but it's also acquired companies that specialise in this area. In 2017, it purchased Vrvana, a company that developed a mixed reality headset called Totem. Around that time, it also bought Akonia Holographics, a company that makes lenses for AR smart glasses. Apple also purchased Israeli-based 3D body sensing firm PrimeSense way back in 2013.

Other companies it's purchased include Metaio, Faceshift, Emotient, Flyby Media, and RealFace.

from Pocket-lint https://ift.tt/2Blx2Ur
via IFTTT