Overview
In order to stay on top of emerging trends, I looked in to Leap Motion Controller and developed Petpet, a digital pet simulator navigated by the Leap Motion. Within 10 weeks I proposed, researched, and designed the web app using Figma.
Project Context
Background
In recent years, emerging technologies have drastically evolved, leading us into a period of innovation and exploration. As a UX designer, it is important to keep up with the latest technologies and trends to create user experiences that align with the current user’s evolving expectations. This project looks at the Ultraleap Leap Motion Controller, a game-changer in the UI scene transforming the ways users interact with hand-tracking technology. My interest is in the potential to redefine interactive experiences for digital content.
Utilizing the Leap Motion Controller 2, I developed Petpet, a 2D virtual pet simulator web app within 10 weeks. Petpet uses intuitive and meaningful gestures for interacting with virtual pets. ‘Meaningful gestures’ refers to gestures associated with the care and affection used to express love for pets in real life.
Challenge
Create a user interface with the Leap Motion Controller that demonstrates the hand-tracking technology used in a purposeful way.
Goals
- Design a prototype of the Petpet user interface
- Experiment with Leap Motion Controller
- Propose new and unique gesture controls
- Create meaningful interactions to nurture virtual pets
Process & Insights
Process Overview
The project consisted of two parts: research and development. Research included understanding the Leap Motion Controller capabilities, uses and experimenting with it. The development stage included the design of Petpet’s style, wireframes, and gestures using the Leap Motion.
Leap Motion Controller 2
The Leap Motion is a hand tracking device that uses infrared sensors to capture finger movement in 3D space. This allows users to navigate and control of digital content on a computer or as an attachment to VR. For the project I used the second-generation leap motion with a higher res camera, increased field view and more.
Similar Products
As part of my research, I looked into similar products to Petpet that utilized the Leap Motion to interact with animals. Hand Physics Lab and Cat Explorer are two VR-integrated apps that experiment with gesture-launched controls with consideration to natural UI. At some point in both apps, they interact with a cat by reaching out and touching it. The natural gesture UI is designed for how a person would generally interact with the general environment. Petpet’s UI, in contrast, will be designed specifically for interacting with pets.
Spatial UI Design
Designing for the leap motion differs from designing for a website because it utilizes 3D space to control the computer. To learn more about how to design the digital interface of Petpet, I read Ultraleap’s best practices and recommendation for UI components from the developer documentation.
Quick Takeaways:
- Gesture controls and UI should be intuitive to learn and not overly complex
- Because users are interacting into the digital world, it should simulate it by limiting the number of actions simultaneously in the space
- Depth cues are imporant to signal spatial awareness within a 3D environment
Style Guide
As a tribute to classic virtual pet simulators like Tamagotchi, I incorporated the green and gray colors from the old interface into the main color palette. The purpose of this style guide helped to keep all UI components consistent through out the webpages.
Building the prototype
Before the final prototype, I also created low, mid, and high fidelity wireframes. I generated wireframe images of the dog and background using generative fill prompts in Adobe Photoshop. Leveraging AI in this process significantly aided in managing the project scope.
The biggest difference in the initial designs, through feedback, was the incorporation of the health progress bar. This addition aimed to gamify interacting with pets while building motivation to care for the animals.
Final Solution
Petpet is a hybrid-controlled web app that toggles the Leap Motion and cursor. Users start on the home screen cursor only and can toggle once in the pet’s environment. Once in Leap mode, gestures such as petting (stroking with one hand left to right), pouring food (holding a can and tilting), and belly rub (making circular motions with both hands) will trigger a response from the pet that gets displayed through the health bar.
Users are guided through instructions with tips on how to interact with the environment by the help button on the top-right corner. Between the cursor mode and leap motion mode, different instructions will be given.
Results
While the exploration of the Leap Motion in creating Petpet has been enjoyable, there is still a long way to go to fully leverage the device’s capabilities. If I had more time beyond the initial 10 weeks, I would work to improve Petpet by transitioning it into a 3D space, opening more possibilities for diverse interactions. I would also spend more time researching on various pet gestures users use to interact with different animals by engaging with pet owners and expand the collection of animals. This would transform Petpet into a tool for teaching proper interactions with various animals.
I have learned that VR and hand-tracking technologies require an evolving set of knowledge and understanding which makes it important now more than ever to adapt to the evolving landscape.