This was a project at CHIL, showing a before and after vision of EBE for the For All We Call Mizzou Fundraising campaign.
Category: Projects
Hobby projects
Reverse-engineering the WMB Prototcol
I took a graduate networking class and for my project, I decided to reverse-engineer the DS download play protocol (commonly referred to as Wireless Multi-Boot, due to ideological similarities with the GBA Multi-Boot feature). This is the protocol underlying the ability for wirelessly transferring demos from a download station or another DS to your own DS. Many games have the ability to transfer demos of the full game, or multiplayer clients for LAN gaming, and the kiosks are used in stores and at conferences to distribute game demos.
The project title was ‘Inferring a Proprietary Wireless L3 Protocol From Packet Traces’, and I gave a presentation in class, as well as a project report that had the technical details. I’ve since placed all of that information on the NDSTech wiki.
Note: I was not the first person to do this. FireFly did everything that I did first, but he did not publish any information on the protocol or source code for his tools. However, his raw-mode driver for Ralink cards and the already re-assembled WMB captures made my life a whole lot easier. A number of other people on the gbadev.org forums also provided vital insight (to both FireFly and myself).
FPGA 3D Pipeline
All of the information about this project is on the class Wiki : http://wwwx.cs.unc.edu/~joat/fpga3d/index.php
However, a quick demonstration is in order:
aMaze – A game for visually-impaired children
I took the Enabling Technology class and worked in a group on the aMaze project, which provided a maze game to children with visual-impairments. A 2D maze is simulated, and the child can walk around inside the maze using a keyboard, joystick, or Sensable haptic-feedback device. Footsteps, simulated cane taps, and environmental sound sources are modelled in 3D, allowing the child to navigate by sound. There is also a visual display of the maze and current position, which can be used for debugging or disabled if a child has partial vision and wishes to try it by audio alone.
The EVE group at UNC has a bunch of foam cubes that they use to construct interesting virtual environments, and we thought of asking a few children to come in and try walking a real maze and the virtual maze. Some children would do the virtual maze first, and others would do the real maze first, and we could find out if they felt more prepared for the real maze after doing the virtual one.
The experience ended up snowballing (with the help of many) into a Maze Day with 51 visually impaired students and 55 adults visiting the UNC Computer Science department to view all of the class projects and previous research, instead of just aMaze. Maze Day has since become an annual event, and will be held again towards the end of April.
Pictures from preliminary testing
Our aMaze team had a station with 3 computers, one with a keyboard, one with the joystick, and one with the Sensable Phantom. We also made a real maze (complete with wolf and elephant sound sources played back at the bends of the maze) constructed in a room that matched one of the virtual mazes that were being shown. By navigating the maze using the computer beforehand, the general consensus was that it was much easier to navigate the real maze. Maze Day was a lot of fun, and we were proud to be a part of it.
3D Medical Consultation
An introduction to the project is available here.
I am developing a person-portable telepresence device using a PDA. Max Smolens wrote the original PDA client/server code, designed for the HiBall trackers. Since Max graduated, I’ve taken over the code base and converted it to track a fiducial marker with a greyscale camera, eliminating the need for the HiBall infrastructure. The two-handed interaction with patient surrogate and PDA has proven quite useful. I also spent a few months working with inertial measuring units and learning about tracking and filtering.
Future research directions include studying the effectiveness of the user interface and how many tracked degrees of freedom are really necessary in this application, and making the surrogate active in tracking, perhaps via five sequenced IR LEDs or magnetics.
A WMV encoded movie of the PDA system in action is available here.
Update:
Some of the interaction research went on to be published at IEEE VR 2007.
NDS Homebrew SDK
The Nintendo DS is the newest handheld console from Nintendo, released in the U.S. in November, 2004. In general, console manufacturers do not support independant development (homebrew), and the DS is no exception. Everything must be reverse engineered and documented, and free headers, libraries, and tools created
I have been involved in the community since before launch and have made significant contributions to making homebrew on the DS possible. I co-authored the first homebrew development library ndslib with Jason Rogers (dovoto), which provides startup files, link scripts, and a library of functions to utilize the DS hardware. The library was renamed to libnds when it merged repositories with devkitpro, and I continue to maintain and contribute to it there. I also maintain the NDSTech Wiki (posting as Joat), a central repository for all homebrew knowledge about the Nintendo DS.
Below are a few pictures of my early development environment.
Image Analogies
Capstone project (Arcade Game System)
All CS students at the University of Missouri – Columbia are required to do a capstone project in their senior year. My group decided to make a game console for an arcade cabinet and a game to demonstrate it.
The capstone project is broken into two semesters, with the first semester spent on the design process and various state educational requirements they couldn’t fit into any other class like IP law and ethics. The class doesn’t meet during the second semester, with the time instead allocated to group meetings.
Our team (EONGames / Arkanerdz) consisted of:
- Adam Crume
- Dustin Culbertson
- Michael Noland
- Remy Nsengiyumva
- Tim Vette
The console was based around a BlueStreak ARM SOC running at 77 MHz, with a 512×384 framebuffer (visible display region is smaller), and plugged into any standard arcade cabinet with a JAMMA connector.
The game we made for our system was ‘Super Magical Happy Fun Kill Time II’, which is a Leprechaun themed 2D shmup written in C.
I have placed the project presentation from semester 1 online, but the prototype hardware ate itself after the semester 2 final presentation when someone plugged the JAMMA connector in backwards. The polarity key in the JAMMA connector of my old cabinet fell out at some point, and no-one noticed, but thankfully it happened after we had presented for the class! When I get around to it, I was thinking about making an emulator for it to show off the game, or just buying a new SOC board.
Gallery of Art and Technology
The Gallery of Art and Technology is a permanent installation at the University of Missouri – Columbia Museum of Art and Archaeology, created by the Computer Human Interaction Laboratory under the direction of Prof. Ali Hussam.
It features five kiosks running various exhibits, including:
- Lewis and Clark Adventure (Nathan Bleigh and Zach Fischer)
- Harriet Tubman Educational Game (Justin Satterley)
- Virtual Exhibits (Michael Noland)
- 1804 St. Louis (Sunny Chauhan, Eli Kerry, Michael Noland, Remy Nsengiyumva, Tyler Robertson)
- Women in History: Josephine Baker Game (?)
- Interactive Painting: Architectural Capriccio (Zach Fischer)
- Interactive Painting: Dido in Resolve (Nathan Bleigh)
There was also a news story about the exhibits published in the Columbia Missourian.
1804 St. Louis
We constructed a virtual environment modeled on downtown St. Louis in 1804, reconstructed from materials provided by the museum. It includes a number of informative signposts and audio cues throughout downtown St. Louis in 1804, as well as special cue points. At each special point, the user can press a button to view modern St. Louis through a 3D panorama. I did troubleshooting and debugging on the project, as well as taking pictures on-site in St. Louis.
Virutal Exhibits
Many museums have far more artifacts than space to display them in; in this case, more are in storage than on display. To solve this problem, Adel Al-Fayez built a scanning device to digitize the artifacts and I built the virtual display case to show them. There are six screens to display individual artifacts on, and a touch screen to control the exhibit. An object can be selected and rotated or zoomed on the touch screen, or via an attached spin-wheel on the side of the pedestal. Whenever a new object is selected, a description of the object (as would be displayed on a placard in a real display case) appears on the screen. A different ‘gallery’ of six objects can be selected at any time from the list on the touch screen.
Adding new artifacts to the kiosk does require some technical expertise due to the digitizing steps (image capture is fully automatic, but removing outliers is a manual process), but once the object has been produced, it can be added to the display either via a small tool or by copy-pasting a template in the XML database and filling in the name, description, and object URL.
This was my project, from concept to finish, I did all of the specification, programming, and testing.
In addition to work on the individual projects, I was responsible for the specification, ordering, and wiring of all of the kiosk controls. I also created the wrapper program to display the interactive paintings used for the gallery opening, and a VR goggles based panorama viewer (cut from the exhibit as our magnetic tracker died).
Viking Ship Building
I took a very interesting upper level course on Viking History and for my term project, I researched Viking Ship building technology and archaeological ship finds.
Examining the archaeological evidence of ships in Norway and Denmark, and its impact on our understanding of Viking nautical technology.