The idea behind our final project is inspired by the game ‘Just Dance’ that Omar and I both played when we were younger and still continue to. It is also well-known and universal and almost every individual in our generation has played it or at least knows about it.
Seeing that we currently live in a time with restrictions on the number of people in the same room at the same time and the limit of interaction, the installation we made requires having just one person in the frame at a time.
The first person that goes in has a snapshot taken of their first pose, this image will then be displayed on the screen presented for the people that come after and they would reenact/imitate such image. Once they do it correctly, the screen turns green for a second and plays a ‘success’ sound. They are then given a few seconds to pose in a way that expresses their thoughts, ideas, or a move that they find personal. When the timer gets to 0, the system takes a screenshot of their pose and displays this image for the next person. The next person that goes into the frame will do the same; stop, pose like the previous person, record their individualized pose, and so on.
The Final Interactive Installation
Here are some of the poses that we collected put together in a collage
When working on the implementation, we found that using PoseNet and ml5.neuralNetwork() would be the best way to record poses and check for them later. In order to implement it, we looked into machine learning and watched videos online to learn about it which was challenging but also really exciting. It took us a while to understand key concepts but once we did, it was fairly easy to apply them to our project and plan out how each part of the project would be coded. After setting things up, we took turns doing a standstill pose on camera as the code collected training data. The data was put into an array and saved as a .json file, which we later normalized (the large x & y values for) and used to train the model. At this point, the code runs normally and the users can go in.
Below are the most important sets of codes that are needed to explain how some of the complex functions are actually implemented.