Apple Glasses: What's the story so far on Apple's AR headset?

For at least a decade, Apple has been working on augmented reality and virtual reality technologies.

We know this thanks to statements from the company, predictions from analysts, numerous patent filings, and Apple's interest in growing its ARKit platform for AR apps and games.

It has also hired AR/VR experts, and it's acquired multiple AR/VR companies. Add it all up, and the rumour mill suggests Apple might be developing a headset. Indeed the latest reports suggest there will be a VR headset to whet our appetites before following this up with an AR headset. 

Apple supposedly has a secret research unit comprised of hundreds of employees who are working on multiple virtual and augmented reality headset prototypes. Here's everything you need to know. 

When will Apple Glasses arrive?

This is hard to pinpoint exactly when Apple Glasses will arrive due to conflicting reports last year.  Predictions were pointing to 2022. In early 2021, reports suggested that the company is set to enter the "second phase of development". And again in early 2021, it emerged, via a detailed Bloomberg report, that Apple will most likely introduce a niche VR headset before a pair of AR glasses appear. This could be as early as 2022. 

The source Bloomberg spoke to says the VR headset is codenamed N301, with the AR glasses being N421. The glasses are, according to the source, an "architecture" which Bloomberg says means Apple is still looking at underlying technologies and so would be with us by 2023 at the very earliest. Bloomberg also says development slowed during 2020 because of the inability for all engineers being able to be in the office at the same time.

Conversely in July 2019, DigiTimes claimed Apple halted development on all its headsets. The team developing the prototypes was dissolved in May and reassigned to other products. At the same time, it's been claimed Apple moved Kim Vorrath, a software executive, over to the augmented reality headset division to bring "order" to the team.

Vorrath's hiring indicates Apple has not killed its AR headset team, as previously thought. References to an AR headset have also been found in iOS 13 - supposedly codenamed Garta. Code in the latest version of Xcode further indicates Apple is developing an AR headset. There are even references to codenamed test devices.

And after having previously also claimed that the headset will reportedly enter mass production by the end of 2019 and might launch in early 2020, Apple analyst Ming-Chi Kuo suggested in May 2020 that they won't launch until 2022 at the earliest. 

How will Apple Glasses or a VR headset work?

Here's everything we've heard so far about Apple Glasses and what they'll be able to do if they ever come to market...

Displays

According to the early 2021 Bloomberg source, the headset will use higher-resolution displays than those currently in use in rival headsets such as Oculus. The headset could also feature an 8K display for each eye. It's untethered from a computer or smartphone and should work with both VR and AR apps. This headset can connect to a "dedicated box" over a high-speed, short-range wireless technology called 60GHz WiGig. It features a 5-nanometer Apple processor and resembles a PC tower.

AR stereo

9to5Mac said Apple is working on stereo AR in iOS that supports a face-mounted AR experience. It's been in internal testing with two Apple devices (codenamed Luck and Franc) and a third-party device, called HoloKit, which is a cardboard AR headset kit. Stereo AR apps can reportedly work in "held mode" (normal AR) or a "worn mode" when used with an external device such as a headset.

According to the early 2021 Bloomberg source, the CPU in use is apparently more powerful than Apple's M1 chips used in new Macs, so could be the A15 coming to 2021's iPhones or an M2 that is likely to come to more powerful Pro Macs this year. The CPU and GPU will be inside the headset rather than offloaded to another device (like a Mac), which is also the model Oculus has increasingly followed. And they will have a fan for cooling, too. 

AR-only or AR/VR?

It sounds like there are two different projects at play here. Respected Apple analyst Ming-Chi Kuo expects the AR glasses to be pitched as an iPhone accessory - offloading computing, networking, etc, to the iPhone - like Google's Daydream.

In 2020, Bloomberg said Apple is developing an AR product with a display, processor, and a new "rOS" or operating system. Supposedly, rOS is based on iOS. Apple is also reportedly developing a "system-on-a-package" chip for this AR headset. It's considering using touch panels, voice activation, and head gestures for input control. Virtual rooms and 360-degree video playback are also being considered.

Design 

According to the early 2021 Bloomberg source the first headset will be a pro-level fabric-covered device. It was also a weighty during initial testing, but apparently, it is now the same size as an Oculus Quest. The goal with the headset will be to get developers on board in time for the launch of consumer-friendly AR glasses. 

In the middle of 2019, The Information had was reporting on the company's AR glasses noting info on the design: "The lenses use a polarized system, similar to the technology in 3D movie glasses, which create the illusion of depth using stereoscopic images, the person said. The technology is similar to that in other AR and VR devices already on the market from Microsoft, Magic Leap and Facebook…

On the lenses

The person The Information spoke to said making the AR lenses is especially challenging because they are composed of multiple, extremely thin layers of different synthetic materials, each of which is susceptible to bubbles, scratches and other marks. To reduce defects, the lenses must be manufactured in dust-free zones known as clean rooms."

Apparently the headset doesn't have space for prescription glasses according to the early 2021 Bloomberg report - instead custom lenses can be inserted into the headset. Apple is also testing the built-in cameras for hand tracking and is also said to be working on a system to input text by 'typing' in the air. 

Has Apple ever spoken about Apple Glasses?

Apple CEO Tim Cook has said the technology does not exist to create AR glasses in a quality way. "But today I can tell you the technology itself doesn't exist to do that in a quality way," he explained. "The display technology required, as well as putting enough stuff around your face - there are huge challenges with that. The field of view, the quality of the display itself, it's not there yet."

Keep in mind, in 2016, he also said: "AR can be really great... We have been and continue to invest a lot in this. We're high on AR in the long run... I think AR is big and profound. This is one of those huge things that we'll look back at and marvel at the start of it."

Who is part of the Apple Glasses team?

Apple has hired several employees with expertise in AR/VR technology, including computer science professor Doug Bowman, who led the Virginia Tech's Center for Human-Computer Interaction. He focuses on three-dimensional user interface design. Apple also hired employees that have worked on Microsoft's HoloLens team, and at Lytro, a company working on a VR camera.

Apple also hired Zeyu Li, who served as a principal computer vision engineer at Magic Leap, as a "Senior Computer Vision Algorithm Engineer." Another hire is Yury Petrov, a former research scientist at Facebook-owned Oculus, who is now serving as a "research scientist" at Apple. Augmented reality expert Jeff Norris also joined Apple in April 2017 as a senior manager. He worked at NASA.

In May 2018, Apple hired Sterling Crispin, who developed a painting app for mobile VR headsets called Cyber Paint. In December 2018, it hired former senior Tesla and Microsoft HoloLens designer Andrew Kim. There's also Jaunt VR founder Arthur van Hoff, who joined Apple as a senior architect in April 2019. Keep in mind Apple's AR glasses/headset team has hundreds of other employees.

In late June 2020 it came to light that Apple is hiring a lot of Magic Leap's employees after the failure of that AR headset. And according to Bloomberg, Apple now has over 1,000 employees working on the project. 

Which AR/VR companies has Apple bought?

Not only has Apple hired AR/VR experts, but it's also acquired companies that specialise in this area. In 2017, it purchased Vrvana, a company that developed a mixed reality headset called Totem. Around that time, it also bought Akonia Holographics, a company that makes lenses for AR smart glasses. Apple also purchased Israeli-based 3D body sensing firm PrimeSense way back in 2013.

And in 2020, Apple bought NextVR, a company which gave sports and concerts a VR platform. The company had partnered with Fox Sports, Wimbledon and the NBA among others. 

Other companies it's purchased include Metaio, Faceshift, Emotient, Flyby Media, and RealFace.

from Pocket-lint https://ift.tt/2Blx2Ur
via IFTTT