Smartglasses

From Wikipedia, the free encyclopedia

Using the touch pad built on the side of the 2013 Google Glass to communicate with the user's phone using Bluetooth.

 

Man wearing a 1998 EyeTap, Digital Eye Glass.

Smartglasses or smart glasses are wearable computer glasses that add information alongside or to what the wearer sees. Alternatively smartglasses are sometimes defined as wearable computer glasses that are able to change their optical properties at runtime. Smart sunglasses which are programmed to change tint by electronic means are an example of the latter type of smartglasses. Superimposing information onto a field of view is achieved through an optical head-mounted display (OHMD) or embedded wireless glasses with transparent heads-up display (HUD) or augmented reality (AR) overlay that has the capability of reflecting projected digital images as well as allowing the user to see through it, or see better with it. While early models can perform basic tasks, such as just serve as a front end display for a remote system, as in the case of smartglasses utilizing cellular technology or Wi-Fi, modern smart glasses are effectively wearable computers which can run self-contained mobile apps. Some are handsfree that can communicate with the Internet via natural language voice commands, while other use touch buttons.

Like other computers, smartglasses may collect information from internal or external sensors. It may control or retrieve data from other instruments or computers. It may support wireless technologies like Bluetooth, Wi-Fi, and GPS. While a smaller number of models run a mobile operating system and function as portable media players to send audio and video files to the user via a Bluetooth or WiFi headset. Some smartglasses models, also feature full lifelogging and activity tracker capability.

Such smartglasses devices may also have all the features of a smartphone. Some also have activity tracker functionality features (also known as "fitness tracker") as seen in some GPS watches.

Features and applications

As with other lifelogging and activity tracking devices, the GPS tracking unit and digital camera of some smartglasses can be used to record historical data. For example, after the completion of a workout, data can be uploaded onto a computer or online to create a log of exercise activities for analysis. Some smart watches can serve as full GPS navigation devices, displaying maps and current coordinates. Users can "mark" their current location and then edit the entry's name and coordinates, which enables navigation to those new coordinates.

Although some smartglasses models manufactured in the 21st century are completely functional as standalone products, most manufacturers recommend or even require that consumers purchase mobile phone handsets that run the same operating system so that the two devices can be synchronized for additional and enhanced functionality. The smartglasses can work as an extension, for head-up display (HUD) or remote control of the phone and alert the user to communication data such as calls, SMS messages, emails, and calendar invites.

Security applications

Smart glasses could be used as a body camera. In 2018, Chinese police in Zhengzhou and Beijing were using smart glasses to take photos which are compared against a government database using facial recognition to identify suspects, retrieve an address, and track people moving beyond their home areas.

Healthcare applications

Several proofs of concept for Google Glasses have been proposed in healthcare. In July 2013, Lucien Engelen started research on the usability and impact of Google Glass in health care. Engelen, who is based at Singularity University and in Europe at Radboud University Medical Center, is participating in the Glass Explorer program.

Key findings of Engelen's research included:

1. The quality of pictures and video are usable for healthcare education, reference, and remote consultation.The camera needs to be tilted to different angle for most of the operative procedures
2. Tele-consultation is possible—depending on the available bandwidth—during operative procedures.
3. A stabilizer should be added to the video function to prevent choppy transmission when a surgeon looks to screens or colleagues.
4. Battery life can be easily extended with the use of an external battery.
5. Controlling the device and/or programs from another device is needed for some features because of sterile environment.
6. Text-to-speech ("Take a Note" to Evernote) exhibited a correction rate of 60 percent, without the addition of a medical thesaurus.
7. A protocol or checklist displayed on the screen of Google Glass can be helpful during procedures.

Dr. Phil Haslam and Dr. Sebastian Mafeld demonstrated the first concept for Google Glass in the field of interventional radiology. They demonstrated the manner in which the concept of Google Glass could assist a liver biopsy and fistulaplasty, and the pair stated that Google Glass has the potential to improve patient safety, operator comfort, and procedure efficiency in the field of interventional radiology. In June 2013, surgeon Dr. Rafael Grossmann was the first person to integrate Google Glass into the operating theater, when he wore the device during a PEG (percutaneous endoscopic gastrostomy) procedure. In August 2013, Google Glass was also used at Wexner Medical Center at Ohio State University. Surgeon Dr. Christopher Kaeding used Google Glass to consult with a colleague in a distant part of Columbus, Ohio. A group of students at The Ohio State University College of Medicine also observed the operation on their laptop computers. Following the procedure, Kaeding stated, "To be honest, once we got into the surgery, I often forgot the device was there. It just seemed very intuitive and fit seamlessly."

The November 16, 2013, in Santiago de Chile, the maxillofacial team led by Dr.gn Antonio Marino conducted the first orthognathic surgery assisted with Google Glass in Latin America, interacting with them and working with simultaneous three-dimensional navigation. The surgical team was interviewed by the ADN radio and the LUN newspaper. In January 2014, Indian Orthopedic Surgeon Selene G. Parekh conducted the foot and ankle surgery using Google Glass in Jaipur, which was broadcast live on Google website via the internet. The surgery was held during a three-day annual Indo-US conference attended by a team of experts from the US, and co-organized by Ashish Sharma. Sharma said Google Glass allows looking at an X-Ray or MRI without taking the eye off of the patient, and allows a doctor to communicate with a patient's family or friends during a procedure.

Baby Eve with Georgia for the Breastfeeding Support Project

 

In Australia, during January 2014, Melbourne tech startup Small World Social collaborated with the Australian Breastfeeding Association to create the first hands-free breastfeeding Google Glass application for new mothers. The application, named Google Glass Breastfeeding app trial, allows mothers to nurse their baby while viewing instructions about common breastfeeding issues (latching on, posture etc.) or call a lactation consultant via a secure Google Hangout, who can view the issue through the mother's Google Glass camera. The trial was successfully concluded in Melbourne in April 2014, and 100% of participants were breastfeeding confidently.

Display types

Various techniques have existed for see-through HMDs. Most of these techniques can be summarized into two main families: “Curved Mirror” (or Curved Combiner) based and “Waveguide” or "Light-guide" based. The mirror technique has been used in EyeTaps, by Meta in their Meta 1, by Vuzix in their Star 1200 product, by Olympus, and by Laster Technologies. 

Various waveguide techniques have existed for some time. These techniques include diffraction optics, holographic optics, polarized optics, reflective optics, and projection:

• Diffractive waveguide – slanted diffraction grating elements (nanometric 10E-9). Nokia technique now licensed to Vuzix.
• Holographic waveguide – 3 holographic optical elements (HOE) sandwiched together (RGB). Used by Sony and Konica Minolta.
• Reflective waveguide – thick light guide with single semi reflective mirror. This technique is used by Epson in their Moverio product.
• Virtual retinal display (VRD) – Also known as a retinal scan display (RSD) or retinal projector (RP), is a display technology that draws a raster display (like a television) directly onto the retina of the eye - developed by MicroVision, Inc.

The Technical Illusions castAR uses a different technique with clear glass. The glasses have a projector, and the image is returned to the eye by a reflective surface.

Smart sunglasses

Smart sunglasses which are able to change their light filtering properties at runtime generally use liquid crystal technology. As lighting conditions change, for example when the user goes from indoors to outdoors, the brightness ratio also changes and can cause undesirable vision impairment. An attractive solution for overcoming this issue is to incorporate dimming filters into smart sunglasses which control the amount of ambient light reaching the eye. An innovative liquid crystal based component for use in the lenses of smart sunglasses is PolarView by LC-Tec. PolarView offers analog dimming control, with the level of dimming being adjusted by an applied drive voltage. 

Another type of smart sunglasses uses adaptive polarization filtering (ADF). ADF-type smart sunglasses can change their polarization filtering characteristics at runtime. For example, ADF-type smart sunglasses can change from horizontal polarization filtering to vertical polarization filtering at the touch of a button.

The lenses of smart sunglasses can be manufactured out of multiple adaptive cells, therefore different parts of the lens can exhibit different optical properties. For example the top of the lens can be electronically configured to have different polarization filter characteristics and different opacity than the lower part of the lens.

Human Computer Interface (HCI) control input

Head-mounted displays are not designed to be workstations, and traditional input devices such as keyboard and mouse do not support the concept of smartglasses. Instead Human Computer Interface (HCI) control input needs to be methods lend themselves to mobility and/or hands-free use are good candidates, for example:

• Touchpad or buttons
• Compatible devices (e.g. smartphones or control unit) for remote control
• Speech recognition
• Gesture recognition
• Eye tracking
• Brain–computer interface

Notable products

In development

• AMA Xperteye - Advanced Mobile Applications (AMA Studios) software for off the shelf customizable smart glasses interface
• b.g. (Beyond Glasses) by Meganesuper Co., Ltd. – adjustable wearable display than can be attached to regular prescription glasses
• castAR by Technical Illusions – wearable AR device for gaming

Current

• Airscouter, a virtual retinal display made by Brother Industries
• Epiphany Eyewear - smart glasses developed by Vergence Labs, a subsidiary of Snap Inc.
• Epson Moverio BT-300 and Moverio Pro BT-2000/2200 – augmented reality smartglasses by Epson.
• Everysight Raptor – smart glasses for cyclists.
• EyeTap – eye-mounted camera and head-up display (HUD).
• Golden-i Infinity – a wearable smart screen for Android or Win10 host devices.
• Google Glass – optical head-mounted display.
• Magic Leap
• Microsoft HoloLens - a pair of mixed reality smart glasses with high-definition 3D optical head-mounted display and spatial sound developed and manufactured by Microsoft, using the Windows Holographic platform.
• Pivothead SMART – "Simple Modular Application-Ready Technology", released in October 2014
• SixthSense – wearable AR device.
• Spectacles - sunglasses with an embedded wearable camera by Snap Inc.
• Vuzix – Augmented reality glasses for 3D gaming, manufacturing training, and military applications.

Discontinued

• Looxcie – ear-mounted streaming video camera

2010s

2012

• On 17 April 2012, Oakley's CEO Colin Baden stated that the company has been working on a way to project information directly onto lenses since 1997, and has 600 patents related to the technology, many of which apply to optical specifications.
• On 18 June 2012, Canon announced the MR (Mixed Reality) System which simultaneously merges virtual objects with the real world at full scale and in 3D. Unlike the Google Glass, the MR System is aimed for professional use with a price tag for the headset and accompanying system is $125,000, with $25,000 in expected annual maintenance.

2013

• At MWC 2013, the Japanese company Brilliant Service introduced the Viking OS, an operating system for HMD's which was written in Objective-C and relies on gesture control as a primary form of input. It includes a facial recognition system and was demonstrated on a revamp version of Vuzix STAR 1200XL glasses ($4,999) which combined a generic RGB camera and a PMD CamBoard nano depth camera.
• At Maker Faire 2013, the startup company Technical Illusions unveiled CastAR augmented reality glasses which are well equipped for an AR experience: infrared LEDs on the surface detect the motion of an interactive infrared wand, and a set of coils at its base are used to detect RFID chip loaded objects placed on top of it; it uses dual projectors at a frame rate of 120 Hz and a retro reflective screen providing a 3D image that can be seen from all directions by the user; a camera sitting on top of the prototype glasses is incorporated for position detection, thus the virtual image changes accordingly as a user walks around the CastAR surface.
• At D11 Conference 2013, the startup company Atheer Labs unveild its 3D augmented reality glasses prototype. The prototype includes binicular lens, 3D images support, a rechargeable battery, WiFi, Bluetooth 4.0, accelerometer, gyro and an IR. User can interact with the device by voice commands and the mounted camera allows the users to interact naturally with the device with gestures.

2014

• The Orlando Magic, Indiana Pacers, and other NBA teams used Google Glass on the CrowdOptic platform to enhance the in-game experience for fans.
• Rhode Island Hospital's Emergency Department became the first emergency department to experiment with Google Glass applications.

2018

• Intel announces Vaunt, a set of smart glasses that are designed to appear like conventional glasses and are display-only, using retinal projection.

Market structure

Analytics company IHS has estimated that the shipments of smart glasses may rise from just 50,000 units in 2012 to as high as 6.6 million units in 2016. According to a survey of more than 4,600 U.S. adults conducted by Forrester Research, around 12 percent of respondents are willing to wear Google Glass or other similar device if it offers a service that piques their interest. Business Insider's BI Intelligence expects an annual sales of 21 million Google Glass units by 2018. Samsung and Microsoft are expected to develop their own version of Google Glass within six months with a price range of $200 to $500. Samsung has reportedly bought lenses from Lumus, a company based in Israel. Another source says Microsoft is negotiating with Vuzix. In 2006, Apple filed patent for its own HMD device. In July 2013, APX Labs founder and CEO Brian Ballard stated that he knows of 25 to 30 hardware companies which are working on their own versions of smartglasses, some of which APX is working with.

In fact, there were only about 150K AR glasses shipped to customers through the world in 2016 despite strong opinion of CEOs of leading tech companies that AR is entering our life. This outlines some serious technical limitations that prevent OEMs from offering a product that would balance functionality and customers’ desire not to wear daily a massive facial/cephalic device. The solution could be in transfer of battery, processing power and connectivity from the AR glasses frame to an external wire-connected device such as smart necklace. This could allow development of AR glasses serving as display only – lite, cheap and stylish.

Public reception for commercial usage

Critical reception

In November 2012, Google Glass received recognition by Time Magazine as one of the "Best Inventions of the Year 2012", alongside inventions such as the Curiosity Rover. After a visit to the University of Cambridge by Google's chairman Eric Schmidt in February 2013, Wolfson College professor John Naughton praised the Google Glass and compared it with the achievements of hardware and networking pioneer Douglas Engelbart. Naughton wrote that Engelbart believed that machines "should do what machines do best, thereby freeing up humans to do what they do best". Lisa A. Goldstein, a freelance journalist who was born profoundly deaf, tested the product on behalf of people with disabilities and published a review on August 6, 2013. In her review, Goldstein states that Google Glass does not accommodate hearing aids and is not suitable for people who cannot understand speech. Goldstein also explained the limited options for customer support, as telephone contact was her only means of communication.

In December 2013, David Datuna became the first artist to incorporate Google Glass into a contemporary work of art. The artwork debuted at a private event at The New World Symphony in Miami Beach, Florida, US and was moved to the Miami Design District for the public debut. Over 1500 people used Google Glass to experience Datuna's American flag from his "Viewpoint of Billions" series.

After negative public reaction, the retail availability of Google Glass ended in January 2015, and the company moved to focus on business customers in 2017.

Privacy concerns

The EyeTap's functionality and minimalist appearance have been compared to Steve Mann's EyeTap, also known as "Glass" or "Digital Eye Glass", although Google Glass is a "Generation-1 Glass" compared to EyeTap, which is a "Generation-4 Glass". According to Mann, both devices affect both privacy and secrecy by introducing a two-sided surveillance and sousveillance. Concerns have been raised by various sources regarding the intrusion of privacy, and the etiquette and ethics of using the device in public and recording people without their permission. There is controversy that Google Glass would violate privacy rights due to security problems and others.

Privacy advocates are concerned that people wearing such eyewear may be able to identify strangers in public using facial recognition, or surreptitiously record and broadcast private conversations. Some companies in the U.S. have posted anti-Google Glass signs in their establishments. In July 2013, prior to the official release of the product, Stephen Balaban, co-founder of software company Lambda Labs, circumvented Google’s facial recognition app block by building his own, non-Google-approved operating system. Balaban then installed face-scanning Glassware that creates a summary of commonalities shared by the scanned person and the Glass wearer, such as mutual friends and interests. Additionally, Michael DiGiovanni created Winky, a program that allows a Google Glass user to take a photo with a wink of an eye, while Marc Rogers, a principal security researcher at Lookout, discovered that Glass can be hijacked if a user could be tricked into taking a picture of a malicious QR code.

Other concerns have been raised regarding legality of Google Glass in a number of countries, particularly in Russia, Ukraine, and other post-USSR countries. In February 2013, a Google+ user noticed legal issues with Google Glass and posted in the Google Glass community about the issues, stating that the device may be illegal to use according to the current legislation in Russia and Ukraine, which prohibits use of spy gadgets that can record video, audio or take photographs in an inconspicuous manner. Concerns were also raised in regard to the privacy and security of Google Glass users in the event that the device is stolen or lost, an issue that was raised by a US congressional committee. As part of its response to the governmental committee, Google stated in early July that is working on a locking system and raised awareness of the ability of users to remotely reset Google Glass from the web interface in the event of loss. Several facilities have banned the use of Google Glass before its release to the general public, citing concerns over potential privacy-violating capabilities. Other facilities, such as Las Vegas casinos, banned Google Glass, citing their desire to comply with Nevada state law and common gaming regulations which ban the use of recording devices near gambling areas.

Safety considerations

Concerns have also been raised on operating motor vehicles while wearing the device. On 31 July 2013 it was reported that driving while wearing Google Glass is likely to be banned in the UK, being deemed careless driving, therefore a fixed penalty offense, following a decision by the Department for Transport. In the U.S., West Virginia state representative Gary G. Howell introduced an amendment in March 2013 to the state's law against texting while driving that would include bans against "using a wearable computer with head mounted display." In an interview, Howell stated, "The primary thing is a safety concern, it [the glass headset] could project text or video into your field of vision. I think there's a lot of potential for distraction."

In October 2013, a driver in California was ticketed for "driving with monitor visible to driver (Google Glass)" after being pulled over for speeding by a San Diego Police Department officer. The driver was reportedly the first to be ticketed for driving while wearing a Google Glass. While the judge noted that 'Google Glass fell under "the purview and intent" of the ban on driving with a monitor', the case was thrown out of court due to lack of proof the device was on at the time.

Functionality considerations

Today most AR devices look bulky, and applications such as navigation, a real-time tourist guide, and recording, can drain smart glasses' batteries in about 1–4 hours. Battery life might be improved by using lower-power display systems (as with the Vaunt), wearing a battery pack elsewhere on the body (such as a belt pack or companion smart necklace).