Programming the Game
The game is built using a plug-in for Unity, Adventure Creator. Adventure Creator, according to the website, is a fully-featured toolkit that has everything needed to make an adventure game. Its interface allows for powerful visual scripting with tools to create NavMeshes for walkable areas, Hotspots for interactivity, and a vast selection of different action types. I chose this plugin for Unity because I wanted to avoid programming the game from scratch considering my two-month time limitation. The visual scripting would allow me to focus more on the idea of clay in a digital space. Because it was designed to create point-and-click games, I was able to use some of the characteristics of point-and-click methodology in my game, but I still had to freedom to make it my own.
Using all of the temporary 2D assets, I began creating a 2D point-and-click game. When I finished
making the game with temporary assets and determined which part of the game was actually feasible, I began making the real clay assets.
Building the Characters
From what I’ve learned during my time studying stop motion, watching behind the scenes videos, and appreciating the craft, I decided to take a unique approach to puppet building. Traditionally, stop motion puppets are made of a non-malleable material like silicone, wood, or resin castings. The material is assembled on top of a posable ball-and-socket armature skeleton.
I wanted to retain the organic look of soft clay, so I covered it in tinfoil and put clay right on top. I used this process for the puppet modeled after myself. When designing myself as a clay character, I decided that my most defining facial features were my bangs and soft nose and eyes. I gave myself an outfit I might actually wear in real life including a pair of Converse Chuck 70s
For Zachary, I did not use a posable armature skeleton because I wouldn’t be doing any complex motion. I shaped tinfoil into an approximate cat shape and then covered it in white clay. I fixed a bendable wire in the rear of Zachary’s body which I then covered in clay. This would allow me to pose the tail for animation frames.
Building the Set
Just like the set designers of The Neverhood, I built the set out of wood then covered it in clay. I referenced my real-life bedroom for this, trying to stay true to the colors, window AC unit, and the artwork on my walls. I also made sure to pay attention to details such as wall trim and wood panel flooring. I created these effects using my fingers and various polymer clay tools. The size of the model is approximately 12” x 12” x 12”. The bed, vanity, and wall shelf were also constructed out of wood then covered in clay. All of the bedroom furniture pieces were moveable in order to allow for separate positioning in Unity.
Raw Image in the Photo Studio
Photography and Set Animations
I took photographs of the bedroom set with bright, clean lighting as far above overhead that I could manage. The camera I used was the Canon Mark IIII was positioned from an angle above to make the game isometric. I first photographed without the furniture that I wanted to be interactive or animated. Then, I inserted the furniture into the room and took more photographs. My idea was that I would remove these furniture items from the room using Photoshop. This way, I would ensure that the furniture angles would match the angles of the entire set.
The final image of the set after Photoshop.
The two items in the room I animated were the drawers and the Kit Kat clock. I used Dragonframe to capture several frames of the drawers opening and closing as well as the iconic Kit Kat clock swinging its tail and moving its eyes. I planned to insert the animations as a result of interaction with the item.
I also took photographs with and without dirty clothes scattered around my room. This would allow me to extract them in photoshop and place them in the room in Unity.
All of the moving parts of my game would be turned into sprites, two-dimensional objects used in computer graphics and video games. Combining frames creates sprite animations. To be able to play these animations in Unity, I had to create sprite sheets. Sprite sheets are documents that have animation frames for a particular sprite. I removed the background and rigging armature, leaving behind the characters with a transparent background and lined up all of the frames for each animation on a transparent background. I exported this as a PNG, brought into Unity where I created a new animation controller for each sprite. Once the animation was created, I was able to control when animation is played with the animation control panel.
The Zachary Sprite Sheet
Animating the Julia puppet was much more challenging. The shot list included eight walk cycle sequences from different directions: walk cycle left, walk cycle right, walk cycle up, walk cycle down, walkcycle diagonal-up-left, walkcycle diagonal-up-right, walkcycle diagonal-down-left, walkcycle diagonal-down-right. It also included a talk sequence and a blink sequence.
I used a reference layer in DragonFrame to rotoscope the animation of my puppet. By using a reference of another walk cycle, I knew how to position the legs and arms.
Because I used non-hardening polymer clay, I had to constantly repair and resculpt the body parts during the animation process. When I would move the arm, a crack would appear in the joints. I think this actually added a layer of noise I wouldn’t have been able to achieve otherwise.
The walk cycles on the animation stage. This video is taken directly from DragonFrame.
The table I animated on is made specifically animating stop motion puppets. The table is made from a perforated sheet metal where the armature can be screwed in place. The metal armature has of the puppet has holes in the feet so that every time a foot needs to make contact with the ground, I can screw it in place.
Here is the final sprite sheet for Julia including walk and talk cycles for each direction as well as an unused animation of the face smooshing away.
Fonts and UI
To really tie everything together, I wanted to make my own fonts and graphics for UI features. These features included the inventory, the menu, the cursor, and speech bubbles. For the custom font, I printed out the alphabet in the font. I sculpted clay over the letters and then scanned them. Using the Photoshop plug-in Fontsself, I was able to turn my clay letters into an .otf file.
I sculpted them onto a piece of paper which I then scanned. For legibility, the font for the text in the game was SyneMono. I chose this monospaced source code inspired font because it reminded me of the early point and click games like Mystery House. The edges were rounded and organic, feeling like it could have been made of clay.
The other UI elements included an inventory box, volume controls, and menus. I sculpted these in the same way I sculpted the font letters on a piece of paper and scanned them in with my printer.
The largest UI element I made was the windows screensaver border. I did this because the edges of the scene weren’t interactive and kind of dull. I thought making it look like it was on a desktop would not only play into the webcore aesthetic, but also make the composition brighter.
And lastly, I made clay pointers.
Music and Sound Effects
Music and sound effects were the last things I worked on for the project. I wanted an ambient, noisy, Aphex Twin inspired soundtrack which I got from Noah Cote. Thanks Noah!
In the opening scene when the user clicks the “Lutum!” banner to enter the space, the sound effect that is triggered is a layering of my voice and Zachary’s meow with an 8-bit texture added to it created by my dear friend, Ethan O’Donovan.
A couple other thinsg make osund in the room. The Kit Kat Clock will make a clock sound when you press it and the air conditioner will also beep and make a fan noise. These sounds are from freesound.org.
When clicking on the interactive objects around the room, Julia will make various comments hinting towards the task to clean the room. For example, she will say, “No time to sit, I have to clean my room!” when the chair is clicked on. The conversation with Zachary alludes to his death and his ressurection. Unfortunately I didn’t get around to adding the dialogue before the presentation, but its been added to the to-do list.
THE FINAL PRODUCT
There are no initial instructions so the player begins by exploring the space. They soon realize that the clothes are clickable. As the continue to explore the space, they discover the inventory. The clothes can be put into the drawers.
Conversation with Zachary can begin at any time. There are three different dialogue options with different responses from Zachary.
Here is the Full Gameplay:
And I also made two devlog style videos which recap this entire write-up:
PART 1 & PART 2
Finally, after all of these steps, I felt my game was ready to be playtested. I sat alongside my playtesters and took note of any confusion they encountered. Some of the adjectives my playtesters used to describe the game were surreal, dreamlike, silly, dark, and uncanny. There were very few issues navigating the space and understanding the UI, but they noted some player control instructions could’ve been helpful. I noticed that people who had experience with point-and-click games had less trouble understanding the player controls.
At a basic level, I achieved my goal of creating an interactive stop motion. I successfully integrated clay into a digital workspace by ocreated everything from fonts and UI elements to intricate walking characters that ould be controlled by your mouse.
Over the past three months, this project almost completely consumed my life. With so many moving parts of the project including stop motion, game design, and narrative, it was a challenge to find a balance between all three. While I did decide to put most of my time into the aesthetics of the stop motion animation, I still constantly thought about how the assets were contributing to my narrative, what the story meant to me, and what I hoped I would gain from it. In the beginning, LUTUM! was an exercise and experiment in interactive stop motion animation. Now, it is a time capsule of my life in this very moment. It’s an archive of my apartment on S 2nd St. It’s the memory of my dead cat. It’s the way I express myself and want to be perceived at this time in my life.
I couldn’t have done it without the help and support of the IMA faculty, friends and family.
IMA Professor: Christina Dacanay Graduate Assistants: Tinrey Wang, Alexandra 3D Scanning: Webb Hunt Unity Knowledge: Sarah Rothberg Music: Noah Cote SFX: Ethan O’Donovan Photography Resources: Noah Pivnik, ITP Documentation Lab, Carter Beardmore Playtesters: Isabelle Rieken, Webb Hunt, Ethan O’Donovan, Bailey Foltz, XY Zhou, Max Chu, Elliot Wright, Aidan Massie, Bailey Foltz, Shiva Viswanathan