An AR game built with Hololens
Microsoft’s Hololens is a truly amazing platform. Although out of the hands of the average consumer and more designed for the enterprise (as recently enforced by MS), it still demonstrates what can be done in the AR space. Also, with MS partnering with Epic, and not just Unity, the next generation of Hololens apps may be more consumer-friendly. That all being said, I have been working with the Hololens for some time and have noticed a lack of detailed help for some common applications.
You may also like: MS Hololens and How to Get Started
The purpose of this series is to help fill in the blanks and put together as many resources available on how to build a simple game. In this series, we will design a very rudimentary game using the main aspects of Hololens v1 to provide a good, in-depth understanding of how to use this piece of hardware. Please note: This is not a game development tutorial, and that is not the purpose of this tutorial series. However, since Hololens development uses Unity and now Epic, it will be necessary at times and I will discuss those relevant concepts.
AR Vs. VR
A common question and point of confusion amongst many technical and non-technical individuals is the difference between AR and VR. So, to keep this document simple and straightforward, and for the sake of clarity, Augmented Reality (AR) is the idea of overlaying existing physical objects with data or other images. Basically, you are bringing computer games or data or whatever into your physical world. This is a state where the user of the AR device does not leave the physical space.
A common AR example that’s often portrayed in movies is the film minority report or any film that presents computers or data in the physical world rather than just a computer. Instead, Virtual Reality (VR) is a completely immersive environment where the only inputs are provided by a computer program. You have generally no interaction with the physical world around you.
A number of terms and references I make throughout this primer refer to Unity terms, and sometimes Microsoft Mixed Reality Academy. Also, although far from necessary, a device or access to a device would be very useful. However, using the emulator is fine.
A bit about the Microsoft Academy. Many people have used these courses, and they are excellent hands-on learning tools to see the amazing benefits of the Hololens device and how you, as a developer, can access them. That being said, and in my opinion, the code that you will use is essentially useless in any other project. Microsoft has engineered many code samples to use specific libraries to provide an easy-to-understand example. I don’t think that you should modify that code for a production-ready product. Many have used it for examples and demos. I would encourage anybody and everybody to run through them, and then move on.
This primer uses a Hololens v1 device and makes use of Visual Studio 2017 and Unity 2017.x and the Hololens Mixed Reality Toolkit. All are now outdated. With the release of the Hololens 2 and the evolution to MixedReality toolkit version 2, many things have changed. However, the concepts and tools have not and are still available. Everything that you read will apply to any third space. It is just in the code where things are a bit different.
So, What Is This Game?
The game is nothing more than a 3D-model chasing you (the user) around a mapped out game space. Maybe a real-life room or two that you map with the device? The enemy will shoot at you and you can shoot back at it. The first one to die loses. Sounds pretty basic – yup. But just wait until you have the enemy chasing you around your house.
Something to say at the start — I have no graphical or modeling ability. So, other than that, hopefully, this will allow you to still understand and build your own project.
What you will need:
Unity (latest version that the author uses) I am currently using 2018.3.7f1 (and the Holotoolkit breaks) so use 2017.x.x if you can.
Hlk-MixedRealityToolkit (17.0.4 Refresh) release
Visual Studio Community (I am using 2017)
*Please note that the technology is constantly evolving and is always changing. Therefore, I have used Unity 2018 and Visual Studio 2017. Newer versions of both are available, and many features probably work in a slightly different way. I don’t know since I haven’t tested them yet. This can be a painful experience, and therefore, for the sake of being able to complete a project, I have not implemented them yet.
One important element in working in the 3D AR space is the concept of spatial understanding of Mixed Reality. Spatial understanding in the simplest of terms is to give your device/application the ability to recognize floors, roofs, walls around a space. To map out a space in detail that would allow you to place or have a program place virtual objects that fit into an area of mapped out space. Of course, what you put in that place is your business and the business of the apps. In the case of this primer, I am making a very basic game that will place an object “enemy” on a pre-mapped out floor space and have that object hunt me by way of using pathfinding.
In order to start any Hololens project, you will need a few development software items. One will be Unity please make sure that you download from the Hololens area. Visual Studio.
The URL for Unity3d: https://unity3d.com/partners/microsoft/mixed-reality
From there, you can navigate to the download page and download the application. This primer will not discuss that.
Microsoft provides this link to help developers get an understanding of the devices and the development world.
Lastly, here is the download link for the tools recommended by Microsoft.
*Please note Microsoft recommends the latest version of Visual Studio. This should work – but the author has not tested it yet. However, upon writing, in regards to some of the material presented around porting, it would appear that the spatial understanding element has not been implemented yet in the new toolkit.
That is all you need to start. Although this primer will try to walk you through all elements to build a game you should use the Microsoft Academy to familiarize yourself with the basic working and building/deploying your project to a Hololens.
So now that we have all the tools set-up and tested. We can now set-up our game project with correct environmental elements.
Setting Up Your Environment
If you haven’t already downloaded the Mixed Reality Toolkit for the Hololens, do so now, and then import it into your project. Set all the settings as per the documentation located on their GitHub page. I would not recommend the non-Hololens toolkit as many elements that are specific to Hololens are not there at the time of writing. Also, unless you have a Hololens 2, it may not work. Microsoft does provide a porting guide.
The name of the Mixed Reality Toolkit specific to the Hololens is located on GitHub and is called:
Once downloaded, start Unity in your project:
- Select Import
- Find the file you downloaded
- Keep the default selections once it is imported
At this point, your project should be set-up and ready to go for Hololens development in Unity.
Stay tuned for part two where we investigate game concepts and assets!