Envisioning the Future of Augmented Reality
An optimistic concept about the bright future of consumer Augmented Reality technology.
With Augmented Reality, we seek to imagine the line between the virtual and the real to be non-existent. However, this is not the case with current AR experiences. Using AR with a phone's camera is severely lacking immersion. With Apple Glasses, which are fashionable and consumer-oriented, we can solve that problem. Through its operating system realityOS, it creates an unprecedented experience that is the future of AR.
Augmented Reality is mainly used with the internal camera whereas the experience itself occurs on the smartphone display. This is not only inconvenient and exhausting when holding up your phone, the degree of reality of these experiences is also very low. How might we make AR experiences feel more immersive while also making them more comfortable to use?
By leveraging groundbreaking technology, we can build a new wearable product that is capable of immersive Augmented Reality. This new product should be intended for the mass and have the potential to establish itself as a new product category besides iPhone and Mac. If so, there is serious potential in becoming the next consumer product revolution, as the product may facilitate many people's lives in new and unprecedented ways.
Introducing Apple Glasses, the world’s most beautiful, true-AR consumer smart glasses. With its groundbreaking design and ingenious technology, Apple Glasses lay the foundation for an augmented future. Sophisticated apps focus on your entertainment, productivity or health, whichever you prefer. Thanks to a cooperation with Carl Zeiss AG, Apple Glasses feature changeable lenses and can easily replace your daily pair of glasses. You can also change it to sun glasses in the glimpse of a second. A depth sensor and dot projector enable tracking-based augmented reality. The same way we know it, just better. Apple Glasses are ready for numerous different lifestyles, as it is featuring water resistance according to IP68 and an overall light weight that makes you forget you even wear them.
Apple Glasses reveal trailer
In the following, I will explain my strategy and design process behind my concept for Apple Glasses. I will go into detail about the research that went into it and how I came up with the idea as well as the process of modeling the final product.
The Process: Strategy
Understanding, Ideating, and Concepting
In order to become a successful product that is widely distributed in society, I assume there are two main factors which have to be combined, as shown in the image below. Simply put, the glasses have to look great and be comfortable while maintaining the best possible functionality.
Factors of success for Apple Glasses
What are socially accepted and unaccepted designs?
The overall design of the glasses has to be accepted by society. In terms of appearance, this would mark the minimum that needs to be achieved if the functionality augmented our everyday life in such a meaningful and convincing way that people would still wear them on a daily basis. Even better would be the fact that people wear Apple Glasses not only because of its functionality but also for its great-looking design. In this regard, an acceptable design could be a fashionable design. But design does not only include appearance, it also includes other aspects of the user experience such as general comfort, intuitive control, etc.
What are socially accepted glasses these days?
They are usually rather small, non-bulky, light-weight and have mostly two lenses in an overall slim design.
What are socially unacceptable glasses these days?
Obviously the opposite; very big, bulky glasses that would make yourself look like a cyborg.
Socially accepted glasses designs
Above are shown examples of socially acceptable designs. Of course, this is, to some extent, a subjective view and taste greatly varies. Still, some of the designs or a slightly modified version of them are widely distributed in society. Others are less distributed, but its design is still accepted. The range of acceptance is a process that is steadily increasing with more and more approaches to bold designs, especially when it comes to sunglasses. In addition, these glasses are lightweight and comfortable to wear.
As proof, I interviewed 10 people, their age ranging from 20 to 81 years old:
Would you comfortably wear one of the glasses above in public?
Everyone agreed to wear the glasses above in public.
Socially unaccepted designs
The designs shown above are assumed to not be suitable as everyday glasses, due to their bulky design, which very much differentiates them from models consumers are used to. In general, it has to be considered where the product is going to be used. In a new media environment at home or at an event, these designs could be accepted as great-looking design, whereas as a choice of glasses worn on a daily basis for many hours, they may be unacceptable.
Again, as proof, I interviewed the same 10 people, their age ranging from 20 to 81 years old:
Would you comfortably wear one of the glasses above in public?
Only 30% agreed to wear the glasses above in public.
Ultimately, it is not surprising that the approval rate drastically declined. But you may ask yourself, how would 30% even comfortably wear these glasses in public? The approval rate comes from technophiles who stated, even though these glasses very much differentiate in terms of portability and size, if there was a significant added value, they would wear it. As a technophile myself, although I did not count myself towards the 30%, I would have also agreed to wear them, if the user experience would remarkably improve my life.
In general, acceptance is simply a subjective restriction in our minds. It depends not only on your personal character traits but also on society and culture. For example, there is a stereotype stating that Germans are usually very obstinate people who are strictly bound to their daily routine. If something goes against it, it will be rejected. Of course, this does not apply to everyone, but for some people or cultures, it takes longer to “adapt” to something new than for others.
How can AR be more immersive?
How can reality be realistically augmented by 3D elements or, in other words, how can immersion be improved? Several factors have been discussed in the literature by Slater and Wilbur (1997)
The field of view (FOV) should be as panoramic as possible. Consequently, this also means that distractions in the field of view limit immersion, thus making the user experience worse. These distractions include when the bridge of the glasses can be perceived in the user’s peripheral vision happening with specific shapes, such as the ones shown below.
Less ideal glasses shapes
This is a common problem of shapes with two separate lenses. However, when both lenses are disproportionately big, the problem may be negligible, but this is obviously not an ideal solution. The ideal solution is to use a single lens, which, as described previously, is also a common and socially accepted shape of glasses.
A more ideal, panoramic glasses shape
With single lens glasses, instead of a bridge, there is a nose resting area, whereas the lens is continous from eye to eye.
The more sensory modalities are accomodated, the more immersive the experience can be. For this reason, the glasses should have speakers to incorporate the sound modality which may improve the experience in certain simulations.
Resolution & Display
Obviously, a higher resolution leads to a more immersive user experience. This also applies to the size of the display, which should ideally cover as much of the user’s vision as technologically possible.
What about people who do not wear glasses?
Apple Glasses are also, but not only intended for people who usually wear glasses. Whereas previous research was about social comfortability, the following research is about weighing up functionality against one's personal inconvenience of having to wear glasses. The design of the glasses should not be taken into account. 10 people who do not wear glasses for vision correction have been asked. 6 of them have been used for previous questions. The other 4 people could not contribute to this question as they wear glasses.
As someone who does not wear glasses for vision correction, are you willing to wear Apple Glasses as your daily pair of glasses given they provide you enough value?
60% of respondents would consider wearing Apple Glasses if enough value is provided.
Again, the relatively high approval rate of 60% can be explained with the significant added value they would hope to benefit from. This does not mean that the remaining 40% would never wear a pair of AR smart glasses. Instead, they might be unsure to decide and would have to try out a physical product in order to be sure about their position. But still, there are always people who strictly refuse to wear glasses.
Technical feasibility and limits
How do AR glasses actually work? This is very important and has to be researched before the design process can start. Let’s take a look at some earlier-shown, non-consumer products and how they function.
Microsoft HoloLens (left) and Magic Leap One (right)
The one on the left is called Microsoft HoloLens and the one on the right is named Magic Leap One. They work pretty similar in terms of experience and functionality, yet the HoloLens was released 2 years earlier in 2016. Both use a new form of optics technology, diffractive and reflective waveguides, to project images into the wearer’s eye.
How most Augmented Reality glasses work these days
The display source is mostly mounted on the side and uses either a so-called Liquid Crystal on Silicon (LCOS) or DLP display, which shoot collimated light rays through a diffraction grating. The diffraction grating basically redirects light to create an expanded image which is ultimately projected into the eye.
What are the trade-offs of using this technology?
Relatively high resolution (720p+)
No ghosting (unwanted images)
Rather expensive ($2000+)
Small FOV (Around 40 degrees)
Requires many sensors
The biggest trade-off is definitely the small FOV, but it will likely increase in upcoming products. Patents filed by Microsoft show, they found a solution to double the FOV through image splitting, then projecting the image and combining the components. The price is also a factor that reduces suitability for the mass, yet in the next months, with new emerging manufacturing methods arising, this is decreasing.
Are there any alternatives?
Yes, there are other techniques, but they all use projection and optics as well, e.g. the birdbath optical design or the curved mirror optical design, and they have even worse trade-offs like unwanted images or transparent 3D objects. That said, I don’t believe, either of the two technologies will eventually prevail.
What about a fullscreen transparent OLED display instead of projecting the image?
It is not as easy as it sounds: The human eye can only focus at a minimum distance of at least 10cm to 20cm. When wearing glasses, the display would be much closer at around 2cm to 4cm distance. For this reason, virtual reality head-mounted displays use lenses, which make the display appear further away. As with AR glasses, lenses cannot be used, because otherwise the real world image would be distorted. Of course, this is a very simplified explanation and maybe, optical engineers may still figure out a better solution in the near future.
If Apple’s optical engineers found a solution to increase the FOV to at least 90 degrees, which would be the equivalent to the FOV of a virtual reality headset, I could imagine them using waveguide technology.
I assume that Apple Glasses or AR glasses in general will evolve to become the next technical revolution after mobile phones and I am convinced, once they have established, the growth of augmented reality will sooner or later overtake the market size of virtual reality. After some research, this has been confirmed by other market researchers, as shown below.
As both of these statistics show, you can clearly see a tendency that AR will experience a major boom in the next few years while virtual reality has still a healthy growth. The source of the VR statistics can be found here
, while the AR statistics are accessible here
Product Target Group
Because of the fact that I assume this product to be very widely distributed and become the next category of products which eventually revolutionize our lives, it is intended for a vast range of customers.
Teenagers (13 to 19 years old)
Teenagers primarily use Apple Glasses as an entertainment device. Besides watching movies and using AR social media, the primary usage is gaming. It may help teenagers to do sports and stay fit in a playful and immersive manner.
Adults (20 to 55 years old)
Adults use Apple Glasses as a tool to facilitate everyday routines, for example by getting directions, using (video) calls, and in general communicating. Younger adults also use Apple Glasses for gaming but it is not as much in the focus as opposed to teenagers. Females may also use Apple Glasses as a fashion accessory besides its useful functions. Business professionals may use Apple Glasses for efficient, more easy work and to improve communicating with clients.
Elders (56 to 75 years old)
Elders use Apple Glasses mainly for the sake of accessibility and as everyday help. Apple Glasses may provide useful information such as reminders, help them with reading, etc., but it may also help them train their memory through certain simulations. For the Elderly, Apple Glasses also serves as their intelligent medical assistant, e.g. it may visually display information for those who suffer from eye disorders which results in vision loss.
Teenager Persona - Meet Jack
Actual State Scenario
Normally, Jack comes home from school at 5 pm, then learns for school for an additional hour. Once he’s finished he plays online video games with his friends at home. Occasionally, his mother comes in and reminds him to do sports every now and then. He’s annoyed of himself, that it would require further motivation in a fun and playful manner so that he would get up to do sports after a hard and exhausting day of school. Shortly after, he represses the thought and continues to play.
Target State Scenario
Now, Jack comes home from school at 5 pm, then learns for school for an additional hour. Once he’s finished, as opposed to playing online video games at home, he meets up with his friends outside to play a location-based, augmented reality, multiplayer treasure hunt game with his friends. It was the immersive feeling that finally convinced him to go out and play instead of staying at home. His conviction ultimately led him to sell his console, as he thinks it’s boring in comparison to what Apple Glasses are capable of. He’s happy that, while having the most fun ever playing a game, he’s still doing sports and is improving his fitness.
Adult Persona - Meet Thomas
Actual State Scenario
Normally, Thomas always schedules 20-minute breaks where he is on the go and prepares for his next meeting. He’s upset, that he does not have additional monitors while being on the move, as this would increase his efficiency. Being stressed and annoyed, he needs to drive to his next appointment by car but, as always, New York is very crowded. To get to his destination, he needs to use the navigation device of his car. As he always gets new clients, he often drives different routes and has to look at the navigation device to know the exact exit or direction, which often distracts him a lot. On weekends, Thomas usually takes the subway to get around New York, typically at rush hour where it is extremely crowded. He’s annoyed that he cannot get his phone out and watch Netflix to let time lapse away as there is barely room to fit in the subway.
Target State Scenario
Now, as opposed to 20-minute breaks, he only schedules 10-minute breaks as his Apple Glasses are capable of displaying additional virtual monitors for his laptop which make his work remarkably efficient allowing him to accept more clients and appointments as before while stress is reduced. When he drives to his next appointment by car, he uses his Apple Glasses to get directions. An arrow on the road generated using augmented reality shows him the exact way allowing him to pay attention to the road the entire time. On weekends, when Thomas usually takes the subway to get around New York at rush hour, he conveniently uses Apple Glasses to watch Netflix having both hands free in an extremely crowded subway.
Elder Persona - Meet Libby
Actual State Scenario
Normally, Libby has big problems in everyday life. When she wakes up and looks at the clock, she can barely see the time due to her central vision loss. Once awake, she normally reads the newspaper during breakfast but she can barely read anymore.
This day, her family members want to visit her, so she is supposed to bake a cake. Due to her dementia, she ended up forgetting to bake the cake, but also, in general, that she was expecting visitors.
When her family members come, she is surprised but happy. Often, Libby has problems to distinguish her grandson Douglas from Danny up to the point where she is sometimes confused that she even has a grandson. When her son notices the confusion, he addresses the subject that Libby urgently needs a caregiver.
Target State Scenario
Now, when Libby wakes up, she puts on her Apple Glasses which strongly support her in everyday life. They have special accessibility features for central vision loss. When she looks at the clock, Apple Glasses recognize the time and display it in her peripheral vision where Libby can more easily read it. The same happens when she wants to read the newspaper. Words in the newspaper could either be recognized using OCR (Optical Character Recognition) or she may access the information through an internet connection. Later, Apple Glasses remind her to bake a cake, as her family members will visit. When her family members come, Apple Glasses help her to recognize her grandsons using facial recognition and displaying his name as a label which sticks to them using augmented reality. In addition, it also shows her the relationship between certain family members. Libby feels more independent than ever and does not need a caregiver anymore. Every evening, she uses a simulation app to train her memory, hoping to eventually be less dependent on technical assistance.
The Process: Design
Based on my research, I kicked off the design process with some rough sketches. Selected ones are shown below.
Passion projects help me to focus on my true passions and are a great way to balance other work. The Apple Glasses project was an individual, voluntary project by myself made in my spare time. This is something I do from time to time, to think of projects I am deeply interested in as a way to try out something new and become better at my profession.
If you still ask yourself why I made this project, my abstract answer would be to create meaning. Now, if you think about it, my intention was mainly to improve my experience design skills, or should I say ours? Because I share my skills, I can create even more meaning and change people’s lives even further by working on myself. Because I am not trying to achieve a victory over other people, I am well aware that having a big piece of the pie does not mean there is less for everyone else. By sharing my skills, I am seeking to work together to achieve something we would not have been able to achieve independently. To improve the outcome of this, I am tirelessly striving to broaden my horizon, expand my skills and I am endlessly driven by unlimited energy that creates a deep, insatiable hunger for more because I believe we as humans have unlimited potential and can grow inexhaustibly. That is also the reason why I dived into new applications and technologies for this project, as a way to continuously develop. In addition, it was a key project to allow myself to learn who I really am, what my passions are as well as the energy that keeps me committed and continuously drives me what I love to do in life, or how I want to spend my life in service.