Homebrew VR Flight Simulator

As a kid I was¹ a huge fan of Star Wars. The idea of piloting my own star fighter in battle, narrowly avoiding lazer flak and debris as I landed the killing blow on a huge enemy warship was a constant fantasy of mine.

When I was 8, I got to go to Disneyland and experienced Star Tours for the first time. This stuck with me as the ride was so close to what I felt I wanted. An arcade game where I could pilot a starfighter with complete immersion.

I had tried several times to convince my dad to help me build such a machine. Even though he was an electrician and tinkerer, he obviously had no idea how to go about building a flight simulator like that. Then I tried my gandfather, an engineer, programmer, a former pilot… he loved the idea but also had no idea how to go about building such a thing.

# So the idea laid dormant until 2011…

Sometime later, while living in a tiny apartment with my new wife and working full-time while also attending online school², I thought, “Yeah! Now’s the time to build this thing!”

I was already working as a web developer at this time and had started learning C# and Unity. I figured I could build enough of a prototype in the engine to start building the physical components of the game first.

But I needed a place to build it. My apartment was far too small, so my mom allowed me to begin construction on the flight simulator in the loft area above the family room.

I found the designs on an online forum somewhere. The guy was selling the designs as a PDF, so I bought them. The device relied on a custom joystick in the middle of the seat, which would affect the pitch and yaw of the gimbal. My plan was to disconnect this setup eventually and motorize it through digital signals from the software.

# How do I see the game world?

While creating this PVC monstrosity, I was trying to work out how to further immerse the “pilot” in the game. The plastic piping of the gimbal was already under a lot of stress when someone was sitting in it, so how was I going to mount a monitor to it?

What if I want to look around the cockpit and see enemy ships above or beside me? If I can’t mount one monitor I certainly can’t mount 3 or 4. I also didn’t have the money to spend on a bunch of monitors or a better material to build the chair out of.

I had remembered seeing these VR headsets when I was a kid at a science fair in the 90s. Maybe that could be the answer?

# Creating a VR headset can’t be that hard

There are a few components to a VR headset:

• 2 screens, one for each eye, that can be fed stereoscopic video from 2 in-game cameras (distanced slightly apart).
• Lenses to focus and magnify the screens. These screens are tiny and need to mimic the distance between your eyes and your eye’s field of view.
• Head tracking to move the camera(s) in the game world. Otherwise you’re just looking at an image from a static position.

Hyper-fixated on making this project a reality, I began running through every possible solution in my head.

# “Procuring” the headset base and screens

For starters, I needed something that everything would mount to. There was an old motorcycle helmet in the garage would do the trick and it’d feel like a pilot’s helmet. I had a couple of old iPod knock-offs laying around… maybe I could cannibalize the screens off those?

From some basic reading I had done online, I realized I would need lenses to focus the screens. So I found some small fisheye lenses on eBay that seemed like the right size.

# How do I track a head in 3D space?

Tracking the player’s head was going to be a much more open-ended problem. My initial searches led me to the TrackIR system - but I had almost no money to work with, so that was out. I considered using an accelerometer from my old Android phone but I was already going to have to figure out how to get the video signal from the computer to the screens.

The next logical conclusion was a webcam using some kind of “computer vision” software. There were some early and somewhat promising leads on this front, but I decided to jot it down and keep looking.

Eventually I stumbled upon a YouTube video of a guy who made a head tracking system using a Wii Remote. I already had an extra Wii mote that was collecting dust, so I decided to try that first.

# Attempting to create a single screen from two

How I accomplished anything related to this part is beyond me. Perhaps all the solder and lack of sleep has all but erased the memory of the experience. The only things I do remember are getting a solid color rendering on the screen and having to treat the 2 screens as one³.

I had some previous experience soldering and working with circuit boards thanks to my dad.

# Time for the first test of many… right?

After this minor bit of success, I decided I would try feeding an image to it from Unity. I wasn’t sure what pixels were coming through or even how many, but I was getting something.

After waiting for the glue to dry on the lenses and screens, I put the helmet on and proceeded to be bombbarded with flickering light. It was so bad I had near instant vertigo after mere seconds. Before I knew it, I couldn’t get the helmet off fast enough…

… and I threw up inside the helmet.

# Put aside once again…

After this crushing defeat, I was essentially done. The helmet was ruined, I clearly didn’t know what I was doing nor the effort this project would take. I had been ignoring the other aspects in my life for almost a week with nothing to show for it.

I remember telling my friends, “It’s too early for VR”, when they would ask about the project. Of course, Palmer Luckey would release the first Oculus Rift prototype the next year.

So the project was put aside. The chair sat in my mom’s various garage spaces for a few years as she moved around, until it was eventually thrown out. My dreams being disassembled and trashed along with it.

Years later, ideas I had for the game itself would be revisited in another (unfinished) project.