Wednesday, October 11, 2017

Tim Cook on Augmented Reality pervasiveness and glasses →

TIm Cook sat down with Andrew Griffin from The Independent to talk about Augmented Reality.

On how widespread AR will become:

“Think back to 2008, when the App Store went live. There was the initial round of apps and people looked at them and said, ‘this is not anything, mobile apps are not going to take off’.”

“And then step by step things start to move. And it is sort of a curve, it was just exponential – and now you couldn’t imagine your life without apps. Your health is on one app, your financials, your shopping, your news, your entertainment – it’s everything.”

“AR is like that. It will be that dramatic.”

I think Tim’s right in terms of AR eventually being everywhere — this is the stuff of science fiction! That said, it’s going to take truly transformative experiences for the masses to jump on board. While furniture apps are cool, there’s a possibility they could be seen as the fart apps of AR after a while.

On AR glasses:

“There are rumours and stuff about companies working on those – we obviously don’t talk about what we’re working on.”

“But today I can tell you the technology itself doesn’t exist to do that in a quality way. The display technology required, as well as putting enough stuff around your face – there’s huge challenges with that.”

“The field of view, the quality of the display itself, it’s not there yet,”

That’s a nice way of saying Google Glass without actually saying it. Don’t get me wrong, Glass was pretty cool, but it was nowhere near ready for the masses, and why it has been relegated to factories. True AR glasses (or whatever form they come in) are going to be game changing.

On AirPods/audio as part of the AR experience:

I asked Cook whether he saw Apple’s AirPods – the wireless earphones that also allow their wearer to talk to Siri and hear directions – as a kind of augmented reality technology. He didn’t, but said that he can “envision audio becoming a key part of the AR experience”, referencing a game we had played that was soundtracked by the beautiful and dynamic twinkling of a Japanese rock garden.

I have been saying the AirPods (and Watch) could potentially be part of Apple’s AR strategy. If you notice, Tim’s distinction here is that AirPods are not AR, but that doesn’t mean they can’t still play a role. As for the Watch, I could see it providing motion data for games or possibly used as an input device.

Tuesday, August 29, 2017

Google announces ARCore, an Augmented Reality development kit

Today, Google announced a new Augmented Reality SDK preview for developers dubbed ARCore. Based off work it has accomplished with Project Tango and similar to Apple’s ARKit, ARCore allow developers to implement Augmented Reality into their apps starting today.

From the Google Blog:

ARCore will run on millions of devices, starting today with the Pixel and Samsung’s S8, running 7.0 Nougat and above. We’re targeting 100 million devices at the end of the preview. We’re working with manufacturers like Samsung, Huawei, LG, ASUS and others to make this possible with a consistent bar for quality and high performance.

Google has even launched its own AR Experiments showcase to highlight example uses of ARCore. This is similar to the third-party run Made With ARKit website, which serves the same purpose for iOS.

As for the SDK name, I think Google could’ve chosen something less Apple-like as the branding is highly similar.

Either way, Augmented Reality is going to be huge. Pedestrian implementations are already loved by the masses (see: Pokémon Go, Snapchat filters). We’re all going to be blown away by what AR will do for us, and it won’t take that much longer to reap tangible rewards.

Wednesday, July 19, 2017

Apple’s Machine Learning Journal →

Today, Apple announced a new journal (read: blog) to catalog their machine learning findings.

Welcome to the Apple Machine Learning Journal. Here, you can read posts written by Apple engineers about their work using machine learning technologies to help build innovative products for millions of people around the world. If you’re a machine learning researcher or student, an engineer or developer, we’d love to hear your questions and feedback. Write us at [email protected]

In the first entry, they discuss improving the realism of synthetic images by using large, diverse, and accurately annotated training sets.

Most successful examples of neural nets today are trained with supervision. However, to achieve high accuracy, the training sets need to be large, diverse, and accurately annotated, which is costly. An alternative to labelling huge amounts of data is to use synthetic images from a simulator. This is cheap as there is no labeling cost, but the synthetic images may not be realistic enough, resulting in poor generalization on real test images. To help close this performance gap, we’ve developed a method for refining synthetic images to make them look more realistic. We show that training models on these refined images leads to significant improvements in accuracy on various machine learning tasks.

They go into explaining the challenges and methods used to refine synthetic images, demonstrated by the example figure below.

Unlabeled Real Images
Figure 1. The task is to learn a model that improves the realism of synthetic images from a simulator using unlabeled real data, while preserving the annotation information.

The post is fascinating. Alternate reality and machine learning are the next frontier for computing, and a growing focus for Apple. This is demonstrated by iOS 11’s ARKit and CoreML, which allows developers to easily implement these technologies into their apps. In a recent interview with Bloomberg, Tim Cook talked about autonomous systems and Apple’s focus on them, including software for self-driving cars, calling it “the mother of all AI projects”.

Some are worried Apple is limiting themselves in these areas because of their privacy and security standpoints. It’s a self-imposed limitation, yes, but that could be why they are being more open about publishing their findings on efforts in this space—to attract like-minded individuals who have the same passion and belief system. For example, all machine learning features on iOS right now are done on-device. No identifiable data is sent back to iCloud and analyzed by a super computer to suggest similar faces in the Photos app, for instance. It’s all done by your iPhone or iPad. Mark Gurman even reported back in May that Apple is developing an ‘AI’ chip to specifically handle these tasks, similar to how the motion co-processor handles all motion data. Makes total sense.

I would much rather have the comfort knowing my device is doing all the work if it comes at a cost of speed to market. Besides, it’s only an inevitability that our machines will do more for us on their own. Apple may take a little more time to get there, but that’s their M.O. iPhone wasn’t the first smartphone, Apple Watch wasn’t the first smartwatch, but both products are now the benchmark in their markets. Apple will do this right, as opposed to other companies who live on getting their hands on your data, and it will be the benchmark for machine learning privacy.

Apple is indeed a secretive company, but under Tim Cook’s direction we are seeing them embrace the ability to be more open. One prior example is the open sourcing of Swift. It makes me excited to see what will come next as a result of this openness.

Tuesday, July 18, 2017

Google Glass Enterprise Edition →

Alphabet and X1 announced Glass Enterprise Edition today, the successor to the Google Glass consumer product discontinued in 2015. This new version is aimed to improve the efficiency of factories.

Steven Levy for Wired:

That’s what Erickson wears every day. She works for AGCO, an agricultural equipment manufacturer that is an early adopter of Glass EE. For about two years, Glass EE has been quietly in use in dozens of workplaces, slipping under the radar of gadget bloggers, analysts, and self-appointed futurists. Yes, the population of those using the vaunted consumer version of Glass has dwindled, tired of being driven out of lounges by cocktail-fork-wielding patrons fearing unwelcome YouTube cameos. Meanwhile, Alphabet has been selling hundreds of units of EE, an improved version of the product that originally shipped in a so-called Explorer Edition in 2013. Companies testing EE—including giants like GE, Boeing, DHL, and Volkswagen—have measured huge gains in productivity and noticeable improvements in quality. What started as pilot projects are now morphing into plans for widespread adoption in these corporations. Other businesses, like medical practices, are introducing Enterprise Edition in their workplaces to transform previously cumbersome tasks.

Makes a lot of sense. To me, Glass is more of a HUD than a real AR experience, which is still cool. I could easily see how having a HUD while working on cars, machinery, even on people in the operating room would be extremely helpful. This is Glass’s rightful place.

The difference between the original Glass and the Enterprise edition could be summarized neatly by two images. The first is the iconic photo of Brin alongside designer Diane von Furstenberg at a fashion show, both wearing the tell-tale wraparound headband with display stub. The second image is what I saw at the factory where Erickson works, just above the Iowa state line and 90 miles from Sioux Falls, South Dakota. Workers at each station on the tractor assembly line—sporting eyewear that doesn’t look much different from the safety frames required by OSHA—begin their tasks by saying, “OK, Glass, Proceed.” When they go home, they leave their glasses behind.

Quite a contrast.
Left: Frazer Harrison/Getty Images. Right: Courtesy of AGCO.

Quite a contrast. Truly laughable, but necessary. Hopefully it taught us a lesson that not all wearables have to be trendy or fashionable. Maybe some wearables just need to be utilitarian. Although, if you want people to wear them everywhere, they better look good. Apple is rumored to be working on AR glasses in some form, and they would never make anything as ugly as Glass for the masses.


  1. Alphabet’s moonshot entity. 

Sunday, May 7, 2017

iPhone 8 Rumors: screen, Touch ID, and Augmented Reality

It’s iPhone rumor season, and leaks have been coming out of the woodwork the past couple weeks. Apple is highly thought to be releasing two iPhone models this year–an iPhone 7S, with modest upgrades from the iPhone 7, and a special iPhone 8 to commemorate the iPhone’s 10th year anniversary.

Aside from an upgraded processor (A11), more memory, and better battery life, here’s a breakdown of a few more exciting features the current leaks are pointing to for the iPhone 8.

Read on