Most developers exploring the Augmented Reality space become fixated on the first SDK they find.
And that's an easy thing to do, because the first one most people run across will usually be one of the big ones. There was Metaio, which was spectacular - I don't think any other single solution comes close to its feature set, so it was quite an upset when Apple purchased it and took it dark last year. A lot of people flocked to Vuforia, then, and others, but many SDKs for Augmented Reality, both commercial and open source, remain available and aren't as widely known.
What do we mean by Augmented Reality?
While I tend, elsewhere, to use the term "Extended Reality" to refer to the confluence of interrelated terms and disciplines made up by Augmented, Mixed and Virtual Reality, I'm putting that aside for this article. For the sake of discussion, the term "Augmented Reality" in this article is used to broadly refer to the range of experiences wherein graphics, imagery and data is inserted into the real world through various digital means, typically tracked to an object or feature in the world, or more generally inserted spatially. I understand that many prefer a strict definition of "Augmented Reality" with a distinction from "Mixed Reality" but I choose to accept that "AR" has come to thoroughly dominate the choice of terms in common use, even in cases that are technically regarded as "Mixed Reality." It should be noted that none of the SDKs, products or services that I will discuss in this article call themselves "Mixed Reality" SDKs - even though we'd have to admit that they all fit that term.
Why we'll focus on Unity-compatible SDKs
Well, for one, I know of nearly 50 different SDKs for developing Augmented Reality applications. Writing a comprehensive overview of the entire AR software development landscape is simply infeasible. That's a thick technical book, not an article. I'm also ruling out anything that requires very specific hardware. For that reason, we won't be discussing Hololens (except where we mention support for Hololens in another SDK) or Tango, RealSense or Kinect hardware-dependent SDKs.
So for this article, I'm focusing on those SDKs which provide native Unity support and work with commodity hardware. There's no doubt: at the moment, Unity is the winning platform to develop augmented reality applications on. It's pretty big in VR, too, owing to the widespread native support it's received from many VR and AR vendors. When Microsoft began describing how developers would create applications for Hololens, it became apparent that the only way they intended development to happen, at that stage, was in Unity. When Google released their Cardboard SDK, they provided Unity support. When they released the Tango SDK, they provided a Unity package. For Intel's RealSense, they directly supported Unity. There are certainly other options for developers - but nearly all of them require rolling your own support to get these various SDKs onto another interactive engine.
As a result, those Augmented Reality SDKs which directly support Unity are of the greatest interest to most commercial AR developers today - and what I've found is that there are quite a few more of these SDKs than most aspiring AR developers are aware of. (There are also a ton of others which don't provide native Unity support which I'll briefly cover at the end of this article, so if you're more interested in rolling your own support, or plan to use it in Unreal or on the web, I'll provide links and descriptions there.) In most cases these SDKs will have a C# interface. In a couple of cases, JavaScript may be available where mentioned.
Let's get started!
1. The ARToolkit SDK from DAQRI
DAQRI is a sleeping giant of the Augmented Reality world (see them here on LinkedIn). Already famous for the still-unreleased DAQRI Smart Helmet, and their numerous acquisitions of startlingly futuristic tech centered around everything from mind-reading headbands to holographic headset technology, they're almost more of a well-funded thinktank than a traditional tech company. They acquired ARToolworks in 2015 and along with it, the ARToolkit SDK which they promptly re-released as a fully open-source project, releasing all of the features previously available only in the "Pro" version to the open source community.
The DAQRI Smart Helmet
Of special interest is that ARToolkit was originally developed in 1999 - a decade and a half before the first developer releases of commercial AR headsets. ARToolkit was originally developed by Hirokazu Kato at NAIST (Nara Institute of Science and Technology - perhaps the most prestigious Japanese post-graduate research university) and later released by the University of Washington's famous HIT Lab. If you're not familiar with the Human Interface Technology Lab already (and its often-related "sister lab" the Human Photonics Lab), they're one of the more influential academic groups in the past 20 years of Augmented and Mixed Reality, responsible for key research in everything from Virtual Retinal Display technology (rumored to be the underlying tech for Magic Leap's mythical headset) to research into shared virtual spaces. It's almost impossible to review the literature on Extended Reality without encountering notable papers out of this research group.
Thanks to DAQRI's OpenSource release of ARToolkit, all of the features described below are available for free on a variety of platforms.
ARToolkit provides support for three general categories of tracker:
- Natural Feature Tracking (NFT)
NFT is a sort of gold standard for feature tracking - it supports freeform 2d images which may not have a clearly defined and consistent outside edge. Examples include anything from a drink coaster to a roadside billboard. - Traditional template square marker
These markers are generally a fairly simple icon with a mandatory solid black border around the periphery. These are best thought of as "designed" markers rather than images that you may already exist organically. - 2D Barcode Markers
These markers are predefined (in the SDK itself) and are typically highly optimized for rapid detection and solid tracking in variable lighting conditions. Because they're so highly optimized, it's typically possible to have more of them detected and tracked simultaneously, and they'll be rapidly detected under circumstances that other types of tracker might not be detected at all. These look more like fiducial markers or low-resolution QR Codes than other types of marker, so they'll typically be obvious where you might want something more discreet.
There are a number of features that are somewhat unique to ARToolkit.
- Multiple camera support
While the most obvious example of multiple camera support would be for stereo cameras, this does not seem to be the limit for ARToolkit. Multiple cameras are supported in other configurations as well. The presence of overlapping areas, such as found in a stereo camera arrangement, provides extremely robust pose estimation for the camera array. - One of comparatively few SDKs that supports Windows Phone
Windows Phone hasn't gotten as much love from the AR developer community as its competing architectures, but ARToolkit provides support. - Very robust tracking for markers over a range of distances
Training ARToolkit for Natural Feature Tracking gives it a notion of what features will stand out over varying distances - something fewer SDKs take into account. It's not uncommon for an images features to blur together at a distance and what worked close by to become less meaningful at a distance. Likewise, the opposite is true - that features that are trackable at a distance become a distinct array of their own smaller features as you approach. ARToolkit takes this into account. - JavaScript-based version for integration into web frameworks
I'm still surprised when I see this, even though it's fairly common. ARToolkit has a companion JavaScript SDK known as JSARToolKit. Don't get confused - there's also another JavaScript AR SDK that is also called JSARToolKit - so make sure you're looking at this one, not this one! (We'll talk about that second one below).
Extra Super-Special Bonus
Thanks to the Open Source nature of the ARToolkit SDK (it's released under LGPLv3), it's the only AR library I'm aware of that currently has publicly available Unreal Engine support, contributed by one of its users - so if you're dead set on using Unreal for AR, DAQRI's ARToolkit might demand your first look.
