Covid-19 Edition

Course Number: IM-UH 3114
Prof. Aaron Sherwood
Meeting Time: Monday 10:25am – 1:05pm, Wednesday 11:50am – 1:05pm
Email Address:
Office: C3 Rm. 191
Office Hours: Walk-in or by appointment
Credits: 4

Prerequisites (any one or more): IM-UH 1010, IM-UH 1011, IM-UH 2310, IM-UH 2318, IM-UH 3310, CS-UH 1001, or ENGR-UH 1000

This course counts toward the following NYUAD degree requirements:

        • Majors > Interactive Media > Computational Media
        • Majors > Interactive Media > Physical Computing
        • Majors > Theater > Arts Practice
        • Minors > Multidisciplinary Minors > Interactive Media
        • Minors > Multidisciplinary Minors > Interactive Media >Computational Media Electives
        • Minors > Multidisciplinary Minors > Interactive Media > Physical Computing Electives

Course Description:

Using computer vision, machine learning, gesture recognition, wearable technology, projection mapping, and a variety of sensors, students will create interactive art and performances that leverage the full potential of the human body. Directly injecting “people-sensing” into an artwork via these readily accessible open source technologies, generates a unique feedback loop, or dialogue-like relationship, where a person and a computer are continuously reacting to each other’s senses. This course will examine this feedback loop, specifically how a person is directly integrated into the artistic expression of the work. Ultimately, students will create interactive installations and performances where the human body is the central component of the artwork.

Students will create interactive installations and performances where the human body is the central component of the art work. Since the human body is naturally dynamic and unpredictable, students will be challenged to develop algorithms that manage potentially chaotic data as well as design systems that help guide people’s physical interactions and input. Students will study past and present works in the field to help inform and direct their own creative decisions.

From a technical standpoint, topics covered in this course will include physical computing, wearables, computer vision, machine learning, wireless communication systems, projection mapping,  gesture recognition, and sensorial stimulation through sight, sound and touch. Specific sensors covered will include range finders, photocells, contact microphones, tilt switches, capacitive touch sensors, accelerometers, gryoscopes, altimeters, thermal imaging sensors, webcams, 3D depth cameras, and more. Ultimately, all of these technologies will be used to help create both installations and performances where sensing people lies at the conceptual and artistic foundation. Students will also be given technical tools of expression to make meaningful output from the information these input  generate.

Learning Outcomes:

      • Foster an ability to think critically about the human form in the context of art, technology, installation, and performance
      • Develop performances and interactive installations from concept to execution
      • Learn how to use a variety of sensors with a micro controller to track the human body in a multitude of ways
      • Learn how to use various computer vision techniques, communication protocols, platforms, and multi media output technologies in an artistic context
      • Develop an understanding of how to create and integrate wearable technologies.
      • Develop computational algorithms to deal with unpredictable data to achieve reliable and consistent results
      • Create clear, effective experiences for participants and audience members

Teaching Methodologies:

This is a production based course. Students will be learning primarily through hands on experience and iterative experimentation. Classes will move quickly through each technology first providing a theoretical foundation along with technical instruction, and then encouraging students to learn through play, experimentation, collaboration, and exploration. Individual classes will include lectures, discussions, examples, workshops, group work, and presentations. Outside of the class, students will engage in both individual and group work.

      • All synchronous classes wll be recorded. If a student cannot attend the synchornous class they are required to watch the recorded version and write a blog post on what was discussed.
      • Classes will happen both synchronously and asynchronously.
      • There will also be other video recordings required to watch each week with technical lessons.
      • Synchronous classes will *usually* be shorter, so each student is required to have office hours with the prof. at least twice during the semester. Office hours ontop of these are encouraged and welcomed but not required.
      • As part of the participation requirement students will make at least three comments each week on other peoples’ blog posts.
      • Have your camera turned on during class so we can all see each other. This really helps the cohesion of the class while we are doing remote learning. If you have a special reason why you need to have your camera turned off please email me with the details of why and we can discuss.

Course Expectations:

Students can expect to develop an interactive installation/performance during the semester. There will also be weekly production assignments that build upon each other.

Class Participation:

Class participation is vital and takes place during discussions (in class and online), presentations, project critiques, office hours, blog post comments, and the streamed final performance/installation.


Each student will maintain a blog, documenting their work. All assignments are due before class starts on the due date class. Usually production assignments will be assigned on Wednesdays and will be due the following Monday. Every production assignment should be documented on the blog. Each week students will continue to make Tidalcycle sketches (unless otherwise instructed), building a “notebook” of sounds. These will usually be part of the normal weekly assignment. For most reading assignments students are expected to write a short blog post (approx. 100-200 words) with their reactions, prior to the class discussion on the material.

Graded Activities:

  • Participation: 15%
    • 5%: Weekly comments on other’s blog posts (due on Wednesday’s)
    • 5%: Office hours with prof.
    • 5%: Class discussions, presentations, project critiques, office hours, and all other forms of participation
  • Attendance: 20% (if unable to attend sync. class then blog entry about the recording from that class will count as attendance)
  • Weekly Assignments & Blog Posts: 35%
    • Exercises will usually be graded this way (unless otherwise indicated):
      • 50% – Technical Implementation: Demonstrate an understanding of the material, and ability to put together working ideas technically
      • 50% – Creativity: Demonstration of personal expression, and uniqueness
    • Blog Posts will usually be graded this way (unless otherwise indicated):
      • Reading response: 100% – Critical thought & creative analysis
      • Documentation: 100% – Ample pictures, videos, audio recordings, text descriptions, code snippets, drawings, etc.
  • Final Project: 30%
    • 25% Use of Body: demonstrates critical thought about the human form in the context of art and technology; centrally integrates body in art work
    • 25% Clarity & Consistency: has clear, effective experiences for performers and audience members; works every time as planned; effectively uses computational algorithms to deal with unpredictable data to achieve reliable and consistent results
    • 25% Creativity: expressively connects one’s own experience of life to others; includes tasteful and appropriate use of sounds and visuals; conceptually well thought out and relevant
    • 25% Technology use: competently uses sensors with a micro controller and/or computer vision techniques to track the human body in various ways, makes successful use of visual and audio generation techniques, streams online


Students should have a laptop with a good text editor, Tidalcycles, and Arduino installed on it. All hardware will be provided excluding major equipment such as a laptop.


Attendance is mandatory. Unexcused absences or habitual lateness will negatively impact your final grade for the class. If you’re going to be late or absent, please email me in advance. If you have an emergency, please let me know as soon as you can.

Please turn in all assignments on time.

Showing up on time, engaging in the class discussion, and offering advice and input in the class is a major part of your grade. Participating in class discussions is helpful for me to get to know you as an individual and keep track of your progress, but most importantly, it provides you and your classmates with the opportunity to share failures, successes, and insights on the work you are doing.

You are expected to show work in class. This includes working prototypes, failed assignments, things that don’t work the way you expect, and so forth. The shorter of the two classes in a week is given over to your work, expect to be called on and show something. Don’t be afraid to volunteer to show what you did, or failed to do.

If you do not ask questions, I can only assume you understand the material completely. Asking questions about concepts you do not understand and showing work that did not function as expected is not a sign of failure, it is an opportunity to learn.


Laptop use is fine if you are using your laptop to present in class, or if we’re in the middle of an exercise that makes use of it. Whenever classmates are presenting or we’re in the midst of a class discussion, please keep your laptop closed. The quality of the class depends in large part on your attention and active participation, so please respect that and close your lid.

Mobile Phones:

Please put them on vibrate or turn them off before you come to class. If you have an emergency that requires you to answer your phone during class, please tell me ahead of time.

Academic Integrity:

As set forth in NYU Abu Dhabi’s Academic Integrity Policy, the relationship between students and faculty at NYU Abu Dhabi is defined by a shared commitment to academic excellence and is grounded in an expectation of fairness, honesty, and respect, which are essential to maintaining the integrity of the community. Every student who enrolls and everyone who accepts an appointment as a member of the faculty or staff at NYU Abu Dhabi agrees to abide by the expectation of academic honesty. The full policies and procedures relating to Academic Integrity may be found on the NYUAD Student Portal:

Major Project:

  • Final: 3 min Interactive Performance/Installation