Rendering by Antonia De Rosa. Edited by 9to5Mac.
In a follow-up report on the challenges Apple has been facing in creating its mixed reality headset, The Information corroborates details we’ve learned from Bloomberg in recent years. But it also digs deeper into the problems the project has faced from Jony Ive to the wearable’s power, battery, heat, cost, and even an alleged lack of discipline from the AR/VR team.
Earlier this week, The Information shared part one of this report. According to sources that have worked on the mixed reality headset team, early problems starting as far back as 2016 have included CEO Tim Cook not being a “champion” of the project, and Jony Ive shutting down the idea of making a VR headset pivoting it to the current AR/VR design.
Part two of The Information’s report corroborates what Bloomberg reported in 2020, that Jony Ive also made the decision in 2019 for the headset team to ditch a design that worked with a base station to go with a less powerful but simpler headset that worked as a standalone device.
Apple CEO Tim Cook and then–Chief Design Officer Jonathan Ive were among the executives who viewed VR demos on prototype headsets that simulated how the two approaches would differ, according to two people familiar with the demos. The headset that worked with a base station had superior graphics, including photorealistic avatars, while the stand-alone version depicted its avatars more like cartoon characters. Mike Rockwell, the Apple vice president in charge of the company’s AR/VR team, favored the headset with the base station, believing that Apple’s top brass wouldn’t accept the stand-alone version’s lower-quality visuals, according to the two people.
He was wrong. Ive had pushed for the stand-alone version of the headset since the early days of the project, according to a person familiar with it. Ultimately, Apple’s senior executives sided with Ive. Despite that, Rockwell still assured them he could make a great product. The choice has had lasting repercussions for the repeatedly delayed headset, which goes by the internal code-name of N301.
That decision has allegedly created many struggles as the headset team has worked to balance “battery life and performance while minimizing the heat generated so people don’t get singed while wearing the device.”
Sources say the failure of Mike Rockwell – the headset lead – delivering “the high-quality mixed-reality experience he told Apple’s executives he would” is the primary reason the product has been delayed so many times.
Apple’s leaders expect an AR experience far beyond what competitors like Meta Platforms, parent company of Facebook, are offering in terms of graphics, body tracking and latency—the lag between a user’s movements and what they see on their display, according to three people familiar with the matter. A delay of even a few tenths of a second between a user’s head movements and the corresponding perspective changes inside a headset can create nausea.
There has also been division amongst the headset team as to what the target customer should be. Some say Jony Ive changing the course from a more powerful headset with a base station to a standalone device was the wrong move.
Some of those people place the blame on Ive, who they say fundamentally changed the purpose of the headset from a product that creatives and professionals would use at a desk to a portable device for consumers. Those people argue that Apple should have first developed a product for professionals to encourage them to make content for the headset before releasing one for consumers.
The Information’s new report shares details like integrating 14 cameras, solving video stream issues, and processor problems also being roadblocks.
Another design decision that has greatly added to the technical challenges for the Apple headset has been the inclusion of its 14 cameras, which allow it to capture everything from images of the outside world to facial expressions and body gestures.
Apple had to build the Bora image signal processor to process the bounty of imagery. But Apple’s engineers have faced technical challenges getting Bora to work with the headset’s main processor, code-named Staten. The back-and-forth communication between the two chips increases latency, which can create nausea for people wearing the headset.
That meant Apple had to build another custom piece to this puzzle, a streaming codec. And reportedly that hasn’t been fully ironed out.
Even more challenges for the project have been a lack of discipline.
Prior to 2019, it had a freewheeling culture, operating almost like a startup within Apple, said four people familiar with the team. Employees brainstormed features and experimented with ideas that might never see the light of day.
Apple ended up bringing in its veteran Kim Vorrath to reign in the project and later assigned hardware chief Dan Riccio to oversee the mixed reality headset.
Vorrath brought more structure to the group, requiring individual teams to come up with defining features for the headset’s software to motivate them and create more accountability. After she joined, engineers were introduced to a concept she had used in software engineering known as the “six-week sprint,” said two people familiar with the matter.
Interestingly, even after Jony Ive departed Apple, some headset team members had to go to his home to get “approval on changes.”
One person familiar with the matter said Ive’s consulting work for Apple since he left includes the headset, adding that he is often brought in to help his former team push through their preferences in areas such as battery, camera placement and ergonomics over those of engineers. Two people said even after Ive left Apple, some employees on the headset project were still required to make the trek from Cupertino to San Francisco, where Ive has a home, to get his approval on changes.
Another tidbit, Ive has tweaked the headset design over the years and his most recent preference is for the headset to have a battery tethered and worn alongside the headset, instead of it being integrated into the headband.
Finally, another challenge has been cost. That’s one of the main reasons Apple assigned executive Dan Riccio to the project.
Reports from Bloomberg and The Information have suggested Apple is currently been looking at a price from above $2,000 to $3,000.
For all the details on Apple’s headset challenges, check out the full report from The Information.
FTC: We use income earning auto affiliate links. More.
Comments