Change Log:
Thu., April 3, 3pm: Exercise announced in class
In this homework exercise, you will install and run examples in three different XR-enabling SDKs, libraries, or tools. It is possible that you will run into installation problems with some of these platforms on your specific development platform. In that case, we are interested in how far you got and where the stumbling blocks are, so you don't need to successfully conclude all of your installations/tests.
The deliverable on this assignment is a questionnaire submission that gathers brief experience reports for your exploits. Depending on your OS and platform, there will be several additional steps that you will likely proactively need to figure out (e.g. version mismatches, or smth like registering a developer account in Xcode on Mac). Be a researcher, and try to get as far as you can, but feel free to limit your time investment on each of these SDKs to a couple of hours each (you don't have to).
You have to test three different SDKs/libraries/tools of your choice and fill in the questionnaire below for each of them. For each, you will have to test (install and deploy a simple example ("Hello World" style), where you actively make a modification/decision of your own, not just run a premade demo)
Here is a non-exhaustive table of SDKs with MR/XR applicability. You are specifically encouraged to look beyond this table for your explorations. Finding additional toolkits/SDKs is encouraged and appreciated!
SDK Name |
Provider/Originator |
Supported Hardware Platforms |
Main URL |
Licensing |
ARKit |
Apple |
iOS devices with A9 chip or later |
Free to use under Apple's Developer Program License Agreement |
|
ARCore |
Android (ARCore-compatible devices), iOS (via ARCore SDK for iOS) |
Free to use under Google's Developer License Agreement |
||
Android XR SDK |
Android-based XR devices, supports Unity, OpenXR, WebXR |
Free to use under Android Software Development Kit License Agreement |
||
MediaPipe Solutions |
Android, iOS, Web, Python, Raspberry Pi |
Apache License 2.0 (Free) |
||
Niantic Lightship (AR DK) |
Niantic |
Android, iOS (via Unity AR Foundation) |
Free tier available; pricing details on official website |
|
Unity XR (XR Interaction Toolkit, + XR Plugins, + OpenXR) |
Unity Technologies |
Meta Quest, HoloLens 2, Windows Mixed Reality, Pico, HTC Vive, Valve Index, Android AR (via AR Foundation), iOS ARKit |
Free tier available under Unity Personal; licensing varies by revenue: Unity Plans |
|
OpenXR |
Khronos Group |
Cross-platform: Meta Quest, HoloLens 2, SteamVR, Windows MR, etc. |
Apache License 2.0 (Free) |
|
WebXR Device API |
W3C (Immersive Web Working Group) |
Compatible web browsers on devices supporting AR/VR hardware; includes desktops, smartphones, and VR headsets. |
w3.org/TR/webxr/ |
|
Oculus SDK |
Meta |
Oculus Rift, Quest, Go |
Free to use under Oculus SDK License Agreement |
|
Meta XR All-in-One SDK |
Meta |
Meta Quest, Quest 2, Quest Pro |
Free to use under Meta's Oculus SDK License Agreement |
|
Mixed Reality Toolkit (MRTK) |
Microsoft |
HoloLens, HoloLens 2, Windows Mixed Reality, Meta Quest (via OpenXR), SteamVR |
MIT License (Free) |
|
Vuforia |
PTC |
Android, iOS, UWP, Unity |
Basic plan is free; Premium and Enterprise plans available. Details |
|
ARToolKitX |
artoolkitX.org |
Windows, macOS, Linux, iOS, Android |
LGPLv3 License (Free) |
|
NVIDIA CloudXR |
NVIDIA |
Cloud streaming to AR/VR devices (OpenVR-compatible), supports major headsets via network |
Free to use under NVIDIA's License Agreement |
|
OpenCV |
OpenCV.org |
Windows, Linux, macOS, FreeBSD, NetBSD, OpenBSD, Android, iOS, Maemo, BlackBerry 10 |
Apache License 2.0 (Free) |
|
Unreal Engine (AR/VR) |
Epic Games |
Oculus, HTC Vive, PS VR, Windows, Linux, macOS, iOS, Android |
Free to use; 5% royalty after first $1 million in gross revenue. Details |
|
CryEngine |
Crytek |
Windows, Linux, PS4, Xbox One, HTC Vive, PS VR |
Free with royalty-based model. Details |
|
Open 3D Engine (O3DE) |
Linux Foundation |
Windows, Linux, major VR HMDs via custom integration |
Apache License 2.0 (Free) |
Note that for webXR, there are many choices of actual libraries / SDKs for you (such as e.g. 8th Wall). Any of those can be one of your choices. Whatever you pick, it only need to be "XR" in the widest sense. For example, Mediapipe above is really a ML solution, but because of the live video overlays, it is clearly also XR. NVidia Omniverse (not mentioned in the table) is a broad 3D simulation platform requiring NVidia hardware, but if you have access to a beefy Nvidia card, it's a perfectly interesting platform.
Please submit this questionnaire, including screen shots as evidence of your efforts).
You have to submit the questionnaire three times, each for a different exploration!
Your first questionnaire (documenting one of your three example installations/tests) must be submitted by Monday, Apr 7, 11:59:59pm (we will discuss some findings in class on Tuesday), all three of them need to be submitted by end of the day Wednesday, April 9 (11:59:59pm) .