The New and Improved Reality

Mrs. Green wants to give her home a new look. To do this, she needs to purchase a couch, coffee table, and a few decorative vases. She has identified the pieces of furniture she wants to buy by locating them on the internet. However, she would like to see how they look when placed in her living room. She quickly downloads a few programs and slips on her specially designed goggles. Immediately, she sees the couch and coffee table arranged attractively in her living room and the delicate vase perched on a nearby desk. Mrs. Green is pleased with the arrangement and immediately orders the pieces.

Within a matter of minutes, Mrs. Green was able to find the right items, see them in her living room, and purchase them without stepping foot outside of her home. One might wonder if this type of shopping is a reality. Although it is not now, in the near future it will be. The goggles Mrs. Green would have used are very much real and have been under development for over five years; it is the technology of Augmented Reality. Augmented Reality is a system where computer-generated images are directly projected onto the user’s view of the real world. Such enhancements include labels, instructions, 3D images, shadows, or other objects. Differing from virtual reality in which a user is completely immersed in a computer-generated world, Augmented Reality simply enhances the user’s view of the real world. It is mainly aimed at providing users with additional information about their real-world environments [1].

The Gear

Augmented Reality systems utilize four main components: video cameras, computers, head-mounted displays, and motion-tracking systems. The video cameras capture images of the real world, while computers create graphics, instructions, or other enhancements to superimpose on the real world images. The computers can range from regular desktop computers to wearable computers that strap onto users like backpacks. The computers are then hooked up to video cameras and head-mounted displays so that they can receive and integrate information from these various sources (see Fig. 1).

Head-mounted displays (HMD) are wearable goggle-sets showing users virtual images projected onto physical space. There are two main types of HMDs: optical-see-through and immersive. Optical-see-through HMDs visually superimpose computer-generated graphics onto real-world scenes. People using this type of HMD are really looking at real-world images with virtual images inserted into their field of vision.

Conversely, immersive HMD’s use video cameras to capture images of real items. These pictures are then sent back to the computer and electronically mixed with computer-generated objects. This final combination of virtual and real images is then directed to the user and displayed through the HMD. People using the immersive HMD’s are looking at images that have been generated by the computer. Meanwhile, motion-tracking systems are integral parts of Augmented Reality systems. Sensors and magnets are often used to monitor the movement of the user’s head and body in relation to the virtual objects as well as the real setting (see Fig. 1). In this way, the computer is aware of where the user is looking and it can subsequently supply the appropriate information [2].

In addition to supplementing the home shopping experience, Augmented Reality can be used to improve our quality of life. Most notably, Augmented Reality has been applied to the medical field to enhance visualization for surgical planning. Until recently, doctors would go into surgeries with only the limited knowledge they had gained from viewing limited two-dimensional images such as x-rays and CAT scans prior to performing an operation. These images did not provide spatial information of the organ in relation to the patient’s body. Augmented Reality can be used to superimpose 3D images of an organ onto a patient to give the surgeon an understanding of the spatial relationship of the operation site in relation to the body [1]. Doctors will be able to attack the problem while avoiding damaging healthy organs or fragile structures [2]

Augmented Reality has also been applied to areas of construction, maintenance, and repair by providing additional information such as labels or instructions during tasks. For example, users could look through their head-mounted displays to see the parts of an engine labeled as well as specific directions on how to repair it. Instead of needing to leaf through cumbersome guides with confusing instructions, people would be able to easily and precisely make their own repairs [1]. Likewise, Augmented Reality could provide labels and extra directions in assembly operations to decrease the information overload placed on workers, improve the quality of products, and increase efficiency [2].

Augmented Reality can also provide vital information to people working in emergency situations. Under circumstances in which people need to react to their environment quickly and accurately, Augmented Reality can decrease reaction times and provide them with relevant information to incorporate into their decisions. For example, paramedics could wear head mounted displays during responses to emergencies. Upon reaching a scene, such an emergency technician could verbally input data about the victim such as breathing signs, bleeding, or other trauma. The small computer built into the head-mounted display could then present appropriate procedural information such as what drugs should be administered as well as possible reasons for the sickness. This would eliminate the decision-making time that could be the determining factor between life and death in such situations. It would also decrease the possibility of paramedics administering improper drugs or forming incorrect diagnoses. Augmented Reality and head mounted displays could also present firefighters with important information in their views such as the temperatures of different rooms. Overall, in emergency situations, having additional information visually presented over real world objects in such a quick, understandable way could prove vital [3].

In addition to the medical, manufacturing, and emergency fields, Augmented Reality could also be used to evaluate assembly line sequences and determine the most effective methods of production. When evaluating a process, it is often necessary to consider the effects of existing items such as partially assembled components, existing equipment, or the present layout of the factory on future production. When part of a design is available only in the physical world and the other part is available only in the virtual world, Augmented Reality can be used to combine the two components and predict what the future outcome will be. For example, a user could go into a factory and start assembling virtual parts onto a partly assembled engine.

Although not adding real physical parts, the user would see himself/herself adding components through the head-mounted display. Therefore, to the user, it would seem that he/she was assembling the engine. The head-mounted display would show a certain sequence of steps to follow with detailed directions, labels, and/or instructive video clips. After each step, the computer would quickly compute the cost and time-consumption of the sequence so far and compare this data to that of the current best sequence. Once the computer figures out that a sequence is not efficient, it can alert the user and supply a different set of directions through the head-mounted display.

Since this kind of system provides immediate feedback, the user would save much of the time and effort that he/she would otherwise have used to complete the process. The user would also not have to take off the engine parts incorporated by the previous sequence because those computer-generated images could be cleared instantly, returning the engine to its starting form. The Augmented Reality system effectively integrates real-world factors with virtual information to provide realistic assembly conditions, thereby helping to pinpoint the most efficient approach to a task as accurately and inexpensively as possible [4].

Although Augmented Reality has undergone extensive development and vast improvements within the last five years, there are still many technical problems in this area that must be addressed. Since Augmented Reality centers around the illusion that virtual objects exist in the real world, convincing visual coordination is vital. The human eye is extremely sensitive to visual miscues, so unrealistic positioning of virtual items and time delays destroy the desired effect. To begin with, virtual objects must be realistically inserted into real world scenes. This means that virtual objects viewed through head-mounted displays must be in believable 3D positions and should not appear to float in space. In order to do this, the position of the user’s head must be carefully tracked so that the virtual object can be displayed in a realistic position relative to the user’s viewpoint. In this area, more accurate tracking systems must be developed and better modeling of the scene’s lighting and other environmental attributes should be used.

Another problem is the lag time in image generation and tracking of head-mounted displays. When the user moves his/her head, the virtual object seems to lag behind in movement. This is because sensor information is not relayed quickly enough and the appropriate compensations for movement cannot be made in time. Overall, more accurate and efficient sensing and motion tracking systems must be developed (Tang). There are also efforts to make head-mounted displays smaller and more portable to make headsets less cumbersome. Engineers are concentrating on specific user requirements rather than creating high-power multi-functional goggles. Since most people will be using the goggles for one specific task, head-mounted displays and wearable computers are now being built more function-specific so that they do not possess so many components nor require such large power supplies. This would decrease both the cost and size of the head-mounted displays (Baber).

Augmented Reality is a technology of the future. Engineers are already experimenting with practical applications of Augmented Reality in the areas of medicine, manufacturing, emergency situations, and efficiency evaluation. It also has the potential to be applied to areas like advertising, military training, and entertainment. Although much work has already been done on Augmented Reality, improvements must be made in the areas of tracking and realistic displays of virtual objects. In addition, since head-mounted displays can presently cost up to $60,000, efforts to make them smaller and more function-specific will hopefully cut down costs. Although Mrs. Green must wait a few more years before she can see virtual images projected into her living room, undoubtedly, that day will come.