May 16, 2016
Augmented reality content creation is expected to become easier as evolving capabilities in 3-D tracking and authoring are integrated into a seamless environment.
Audio, still photos, and video have all become content types that almost anyone can create. Most people carry in their pockets the means to capture professional-grade pictures and movies, and the tools to generate these types of content are now simple enough even for children to master. That’s far from the case with augmented reality. Creating AR content often requires specialized software and hardware—and plenty of know-how. The good news is that authoring solutions are emerging that make it easier to develop content for AR.
This article examines the trends in authoring tools that will power AR content development.
The evolution of AR authoring tools
Although these are early days for AR, the use cases have been expanding. What began with simple apps that added context based on GPS data has now entered an era where 3-D images are overlaid on a real-time view of the world. During this time, AR systems and associated authoring tools have evolved in three primary categories. (See Figure 1.)
Figure 1: The three primary categories of AR systems
The earliest of the AR systems are geospatial systems that involve graphical overlays atop a real-time real-world view, driven primarily by GPS data. Such systems are sometimes called GEO augmented reality, and they evolved with the smartphone and the availability of GPS in it.
For example, someone could open the Yelp app on a smartphone and start the Monocle feature. Then the person slowly moves while aiming the phone’s camera around the area. Pop-ups will indicate the location of and ratings for nearby restaurants, bars, or other retail establishments. Similarly, ZipRealty’s StreetScan system uses a person’s camera to overlay the location of nearby homes for sale as the person pans the phone around a neighborhood. Many apps like these are available for smartphone platforms today.
Authoring for this type of GPS-based AR application is now relatively commonplace. Inglobe Technologies, ARLab, Layar (now part of Blippar), Wikitude, and others offer solutions and software development kits (SDKs) for authoring GPS-based AR apps. Using these tools, developers simply define a file that matches GPS coordinates with some form of contextual information. That could be a link to a website, a home’s value, a tourist photograph, or a piece of textual data. When the device is near a certain GPS location, the corresponding data for nearby points of interest will be loaded into the mobile device and can be displayed on the app.
Today, all of these programming tools are designed for app developers to add GPS-based AR features to mobile apps, not for consumers to do their own AR authoring.
2-D augmented reality systems
The next evolution in AR involves the recognition of 2-D objects through the camera. “It predominantly is used to augment posters, books, magazines—anything that is flat and has a 2-D surface,” says Andy Gstoll, chief marketing officer of Wikitude, which offers SDKs for AR authoring.
In 2-D AR, an application detects a certain object in the frame and then reacts in some predefined manner, typically by playing a video or opening a web page or showing an interactive 3-D model.
The makers of many children’s toys are taking advantage of this technology as a way to increase the interactivity of the toy. A gaming figurine might come with a special card (akin to a QR code). When a mobile device running the toy company’s app is aimed at the card and the app detects the symbol on the card, the child gets to see a short video of the toy in motion, or gains access to a virtual version of that character for use in an online version of the game, or gets to interact with a 3-D model of the toy. The concept works through a simple 1-to-1 relationship. Point the camera at item X, and that causes the mobile device to undertake action Y.
In 2-D AR systems, the technology might rely on a marker (such as a distinct symbol) to identify the object or it might identify the image itself through some kind of image recognition technology. Either way, the popular use of these systems today is for marketing purposes to bring 2-D information to life and improve user engagement.
As with geospatial AR, building for 2-D AR is largely a matter of developing the necessary content, normally as an app, by using an SDK from an AR software provider. For example, in addition to its geospatial system, Layar’s SDK enables 2-D image recognition AR. The CraftAR system includes a large image database, which could ease the complexity of teaching an app how to recognize certain objects. Blippar, Metaio (acquired by Apple in 2013; the product is no longer available to the public), and Wikitude are other vendors that offer 2-D authoring SDKs.
3-D augmented reality systems
Geospatial and 2-D image systems can provide useful contextual information, but 3-D AR systems can enable real work through cutting-edge industrial and entertainment applications that likely will be the most important for the future of AR. With 3-D AR, maintenance workers disassembling an engine may see live, real-time directions overlaid on the parts set out in front of them, and 3-D video may show exactly what steps to take next. Similarly, a 3-D AR game may insert virtual enemies into an otherwise real-world environment.
“An enterprise might have a factory with lots of machinery and might need to train a new employee to perform maintenance on the machinery. The easiest way to transfer knowledge would be to give the employee a smartglasses headset that augments reality during the training or that eliminates the training process altogether,” says Gstoll of Wikitude. “We’re working with a car manufacturer to build 3-D use cases for the company’s service technicians. They would work through a procedure while wearing AR glasses that would show them how to test or diagnose the braking system, remove particular components, and carry out some necessary repair tasks, so they know what to do for a specific car model.”
NGRAIN is another vendor creating an authoring solution for 3-D AR. The company has adapted its technology for automated damage assessment and visual inspection. An AR overlay lets technicians more easily locate flaws or other imperfections in an object.
“This capability is particularly useful in manufacturing situations where products require very tight levels of tolerance,” says Barry Po, NGRAIN senior director of product and business development. “Lockheed Martin is one of NGRAIN’s customers, and the company deployed damage assessment technology operationally for use in the F-35 and the F-22 fighter programs. The maintenance folks can perform diagnostics and damage assessment in real time after fighter jets come in from a flight.”
Many AR vendors are integrating 3-D authoring capabilities into their solutions. Some 3-D AR authoring solutions include Wikitude’s Wikitude Studio, Total Immersion’s D’Fusion Studio, Atheer’s AiR Suite, APX Labs’ Skylight, and NGRAIN’s Vergence.
Computer-aided design and 3-D augmented reality
In general, 3-D tools are complex and require serious expertise to use and master. Considerable production effort is required to create high-quality and accurate content for computer-aided design (CAD) models or games. Should AR content authoring require a similar level of effort, it could limit AR adoption.
“Our objective was to create a tool that people other than 3-D graphics authoring experts and 3-D modelers can use,” shares Po. Enterprises typically have many more subject matter and domain experts than 3-D modelers or 3-D specialists. Making AR authoring accessible to these experts will vastly expand the ability to create compelling AR applications, something the industry is aspiring to do.
Most enterprises already have a library of CAD models of their products or assets. AR authoring tools can import these CAD models or other 3-D content sources, including 3-D point clouds (from 3-D scans) and gaming models. (See Figure 2.) Authoring tools provide simple ways to augment the imported 3-D assets with images, charts, videos, instructional content, step-by-step directions, or real-time sensor information.
Similar to video, 3-D content is very data intensive. CAD models of any reasonably complex device or system can be very large, which makes processing and rendering on smartglasses or other mobile devices difficult. “You need to reduce the files and encode them into something we can work with,” explains Gstoll. As part of its authoring environment, Wikitude has a proprietary encoder that optimizes large files to reduce their size for use on smartglasses and mobile devices.
Over time, conventional software tools such as 3-D CAD authoring systems will likely expand into AR authoring as the boundaries between AR authoring and 3-D design begin to blur and merge. In late 2015, established CAD vendor PTC acquired Vuforia, an AR authoring platform. Unity, a widely used 3-D gaming engine, has a plug-in to integrate the functionality of ARToolKit, a computer vision library that provides a tracking capability, to make it easier to create AR applications in Unity.
More developments like these can be expected in the future.
3-D AR tools are really only at the beginning of their development, and a rich array of capabilities should emerge in the years to come. AR authoring systems eventually will work with and display a variety of 3-D content types, including CAD files and wireframes, 3-D video, holographs, 360-degree images, 3-D representations of real environments (based on scans), and 3-D sound.
Despite the many unknowns, what remains clear is that authoring systems will need to satisfy a number of challenging tasks and activities that require significant production effort. “For AR to take off, the ability to rapidly create, manage, edit, and deploy 3-D content is one of the problems that should be addressed,” says Po.
Figure 2: Authoring tools for 3-D AR can import 3-D information from a variety of 3-D content domains and export content for use on different AR devices.
The evolution of the internet was inextricably tied to the quality and availability of the authoring tools for creating the content delivered over the internet. The web was static and dull, mimicking what people had produced for paper-based distribution, until content development software allowed site owners to create dynamic and interactive content, and the digital age of today was born.
Augmented reality can be expected to follow a similar pattern. Today, many AR solutions use content that was already prepared for other purposes—content that includes pictures, documents, checklists, and so on. As the old adage goes, “Content is king.” As AR authoring tools evolve, the nature of content that is best suited for AR will evolve as well and will lead to a future that merges the physical and digital realities.