Real-Time Media

NYU IMA, Fall 2023 // Instructor: Carrie Sijia Wang

Quick Links:

Homework Submissions Doc

Final Project Class Folder

Max/MSP/Jitter Example Patchers

Week 1: 09/06

Week 2: 09/11 & 09/13

Week 3: 09/18 & 09/20

Week 4: 09/25 & 09/27

Week 5:  10/02 & 10/04

Week 6: 10/10 & 10/11

Week 7: 10/16 & 10/18

Week 8: 10/23 & 10/25

Week 9: 10/30 & 11/01

Week 10: 11/06 & 11/08

Week 11: 11/13 & 11/15

Week 12: 11/20

Week 13: 11/27 & 11/29

Week 14: 12/04 & 12/06

Week 15: 12/11 & 12/13

About

Class Information

Time: Mondays & Wednesdays 5:20pm – 6:50pm

Location: 410

Instructor: Carrie Wang / [email protected] / Office Hour Signup

Coding Lab (search for “Max”)

Doc Lab (see if they can help you document your performance)

Resident Office Hours (scroll to research residents)

Description

This course focuses on designing, developing and delivering real-time, performative work using audio and video elements. The class will have an emphasis on using MaxMSPJitter and other tools to create performative experiences that dynamically combine interactive elements such as video, sound, and code, allow for the unfolding of engaging narratives, and generate compelling visuals in real time.

We will look at various examples of both multimedia performances and installations, explore how we can apply the technologies we have learned to design real-time systems, and discuss methods we can use to make our work more engaging.

The class is three-fold and divided into tech tutorials, discussions of existing examples, and in-class performances.

Evaluation

Grades will be determined according to this criteria:

  • Participation and Attendance 40%
  • Performance Projects:
    • Audio Performance 12%
    • Video Performance 12%
    • Final Project Development and Execution 16%
  • Other Homework Assignments 20%

The performance projects will be evaluated based on:

  1. Concept – clear intention to tell a story, express emotions, or convey messages
  2. Process – active efforts to develop the piece, meeting in-progress milestones, 
  3. Execution – level of completion + how well the performance reflects the concept 

Attendance

Attendance is mandatory. Please email your instructor if you are going to miss a class. Two unexcused absences is cause for failing the class. An unexcused lateness of 10 minutes or more is equivalent to 1/2 of an absence.

EQUITABLE IMA/ITP

NYU Bias Response (Referral) Line

More Resources on Equity & Inclusion

STATEMENT OF ACADEMIC INTEGRITY

Plagiarism is presenting someone else’s work as though it were your own. More specifically, plagiarism is to present as your own: A sequence of words quoted without quotation marks from another writer or a paraphrased passage from another writer’s work or facts, ideas or images composed by someone else.

STATEMENT OF PRINCIPLE

The core of the educational experience at the Tisch School of the Arts is the creation of original academic and artistic work by students for the critical review of faculty members.  It is therefore of the utmost importance that students at all times provide their instructors with an accurate sense of their current abilities and knowledge in order to receive appropriate constructive criticism and advice.  Any attempt to evade that essential, transparent transaction between instructor and student through plagiarism or cheating is educationally self-defeating and a grave violation of Tisch School of the Arts community standards.  For all the details on plagiarism, please refer to page 10 of the Tisch School of the Arts, Policies and Procedures Handbook, which can be found online at https://tisch.nyu.edu/student-affairs/important-resources/tisch-policies-and-handbooks

STATEMENT ON ACCESSIBILITY

Please feel free to make suggestions to your instructor about ways in which this class could become more accessible to you.  Academic accommodations are available for students with documented disabilities. Please contact the Moses Center for Students with Disabilities at 212 998-4980 for further information.

STATEMENT ON COUNSELING AND WELLNESS

Your health and safety are a priority at NYU. If you experience any health or mental health issues during this course, we encourage you to utilize the support services of the 24/7 NYU Wellness Exchange at 212-443-9999. Also, all students who may require academic accommodation due to a qualified disability, physical or mental, please register with the Moses Center at 212-998-4980. Please let your instructor know if you need help connecting to these resources.

STATEMENT ON USE OF ELECTRONIC DEVICES

Laptops will be an essential part of the course and may be used in class during workshops and for taking notes in lectures. Laptops must be closed during class discussions and student presentations.  Phone use in class is strictly prohibited unless directly related to a presentation of your own work or if you are asked to do so as part of the curriculum.

STATEMENT ON TITLE IX

Tisch School of the Arts to dedicated to providing its students with a learning environment that is rigorous, respectful, supportive and nurturing so that they can engage in the free exchange of ideas and commit themselves fully to the study of their discipline. To that end Tisch is committed to enforcing University policies prohibiting all forms of sexual misconduct as well as discrimination on the basis of sex and gender.  Detailed information regarding these policies and the resources that are available to students through the Title IX office can be found by using the following link: Title IX at NYU.

CODE OF CONDUCT

Our signature is collaboration – Not competition.
– Red Burns

We live in a fractured world where communities are burdened by differences – so what we ask of you is nothing short of miraculous.
— Red Burns

ITP/IMA is a community whose mission is to explore the imaginative uses of emerging technologies — to make people’s lives safer, more just, more beautiful, more meaningful, and more fun. We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.

The scope of this Code of Conduct applies to all ITP/IMA community members (faculty, staff, students, residents, fellows, and guests), ITP/IMA physical and digital spaces (e.g. Email listservs, Yorb, and ITP/IMA Discord server), and all courses, events, and projects created at or in service to ITP/IMA.

SOLIDARITY

We are a community of and in solidarity with, people from every gender identity and expression, sexual orientation, race, ethnicity, language, neuro-type, all visible or invisible disability, body size, ability, class, religion, culture, subculture, age, skill level, occupation and background. As a result, we continually work to make participation in our community a harassment-free experience for identities above as well as all levels of experience, education and socio-economic status.

INTERNATIONAL

Our student body typically has a large International student presence. We understand that, by the nature of being established in New York, our curriculum and social environment naturally centers events and histories specific to the United States of America. We know that a large percentage of our student body are international and we will continue to incorporate, celebrate and acknowledge events and histories from those countries with respect and an open mind.

We align ourselves with any values in pursuit of equity and freedom for all identities and, by that nature, are in alignment with similar views in International countries as well.

LAND ACKNOWLEDGEMENT

We acknowledge that we are gathered on the unceded land of the Lenape and Canarsie peoples. We acknowledge the Lenape and Canarsie communities, their elders both past and present, as well as future generations.

We also acknowledge that NYU, Tisch School of the Arts, and ITP were founded upon exclusions and erasures of many Indigenous peoples, including those on whose land this institution is located. This acknowledgement demonstrates a commitment to beginning the process of working to dismantle the ongoing legacies of settler colonialism.

PRIVILEGE

We recognize that simply being part of the ITP/IMA community places us in a privileged position as we have the opportunity to receive an education from a private, expensive and prestigious university. We know that there are many equally talented students around the world who would like to partake of this community and, due to academic inaccessibility, it is not possible. We know and will continue to acknowledge that our privilege does not mean that we have more value than other people. We acknowledge this inequality and hold personal responsibility to share what we do in the spirit of inclusivity and create access where we can, as often as we can. We will use this privilege as a tool to help those most in need.

TECHNOLOGY

We pledge to center creative and ethical uses of technology in our research, teaching, and making. We accept the claim that technology is a reflection of society, its histories, and its politics. We reject the claim that technology is neutral and acknowledge that every technology has the potential to do as much harm as good. We acknowledge that when technologies cause harm, the harm disproportionately affects Black, Brown, Indigenous, People of Color (BIPOC), queer, trans, disabled, femme, low-income, survivors, and all other marginalized bodies and communities worldwide.

Schedule

Week 1: Intro

Wednesday 09/06: SLIDES

In Class:

  • Introductions
  • What is this Class?
  • What are we doing in this class?
  • Introduction to OBS
  • Download & Install Max

Homework:

Week 2: Practice Performance I

Monday 09/11: SLIDES

In Class:

  • Examples: Desktop Performance
  • Performance Critique
  • Practice Performance I: Prompt & Groups
  • Review OBS Studio
  • Ideation & Composition

Homework:

Wednesday 09/13

In Class:

  • Show Desktop Performances
  • Feedback and Reflections

Homework:

Week 3: Practice Performance II

Monday 09/18: SLIDES

In Class:

  • Examples: Connection
  • Networked Performance Tools
  • Practice Performance II: Prompt
  • Work in Groups

Homework: In Class Performance (Due 09/15)

Wednesday 09/20

In Class:

  • Show Connection Performances
  • Reflections

Homework:

Week 4: Audio I

Monday 09/25: SLIDES

In Class:

  • Project Example
  • Basics of MIDI
  • Simple Piano
  • Player Piano
  • Make Your Own Player Piano

Wednesday 09/27: SLIDES

In Class:

  • Project Example
  • Look at Timer Patchers
  • Turn the Computer Keyboard into a MIDI Controller
  • Control Note On/Off with Computer Keys
  • Activity: Make your piano patcher better

Homework:

Week 5: Audio II

Monday 10/02: SLIDES

In Class:

  • Project Examples
  • Analog Sound Waves
  • Sample and Manipulate Sound with Buffer and Groove
  • Activity: Make Your Own Audio Performance System

Wednesday 10/04: SLIDES

In Class:

  • Project Critique
  • Look at Piano Patchers
  • Sampling Sound and Using It as Musical Instrument
  • Activity: Make the Sampler Play Multiple Notes OR Build Your Own Audio System

Homework:

Week 6: Audio III

Tuesday 10/10: SLIDES

In Class:

  • Connecting to Arduino
  • Ideas for Audio Performance
  • Getting Ready for Tech Check & Performance
  • Playing with the Max Interface

Wednesday 10/11

In Class:

  • Lighting

Homework:

  • Audio Performance (SLIDE 17 & 18):
    • Tech Check/Practice Setting It Up – Monday 10/16 (Classroom + Room 404)
    • Record One of Your Rehearsals – Monday 10/16
    • Performance – Wednesday 10/18 (Classroom + Room 404)

Week 7: Audio Performance

Monday 10/16

In Class:

  • Notes on Wednesday’s Performances
  • Tech Check
  • Studio Time

Wednesday 10/18:

In Class:

  • Share Audio Performances:
  • Meet in the classroom and follow the order and locations to see all the performances

Homework:

Week 8: Video I

Monday 10/23: SLIDES

In Class:

  • Survey Results
  • Project Example
  • Jitter Matrix
  • Generate & Animate Patterns with jit.matrix
  • OBS + Max?

Wednesday 10/25: SLIDES

In Class:

  • Project Example
  • Video Basics – Webcam Feed, Resolution, & Resequencing Frames
  • Video Basics – Playing & Controlling Videos

Homework:

Week 9: Video II

Monday 10/30: SLIDES

In Class:

  • Project Example
  • Video Effects
  • Activity: Make a Video Effects System

Wednesday 11/01: SLIDES

In Class:

  • Project Example
  • Look at Video Playback System Patchers
  • Audio Visualization
  • Intro to OpenGL (3D)

Homework:

Week 10: Video III

Monday 11/06: SLIDES

In Class:

  • Project Critique
  • More OpenGL
  • Activity: Combining Video Effects with OpenGL

Wednesday 11/08: SLIDES

In Class:

  • Even More OpenGL?
  • Share Video Performance Ideas
  • Free Writing Exercise

Homework:

  • Video Performance (SLIDE 13 & 14):
    • Tech Check/Practice Setting It Up – Monday 11/13 (Classroom + Room 404)

Week 11: Video Performance

Monday 11/13

In Class:

  • Notes on Wednesday’s Performances
  • Tech Check
  • Studio Time

Wednesday 11/15

In Class:

  • Share Video Performances
  • Meet in the classroom and follow the order and locations to see all the performances

Homework:

Week 12: Final Project Development

Monday 11/20: SLIDES

In Class:

  • Object Theater Examples
  • Elements of Object Theater
  • Final Project Assignment
  • Form Groups

Homework:

Week 13: Final Project Proposal

Monday 11/27

In Class:

  • Share Final Project Proposals

Wednesday 11/29

In Class:

  • Guest Speaker

Homework:

  • Work on your final. Get the project to work in the most basic sense (with all the technical aspects figured out, the structure of the performance/installation in place).
  • Edit your part in the Equipment List.

Week 14: Final Project Work-In-Progress

Monday 12/04

In Class:

  • Share Work-In-Progress

Wednesday 12/06

In Class:

  • First Rehearsal.

Homework:

  • Get your final project to 90%.
  • Finalize your Equipment List.
  • Rehearse as much as possible. Rehearse both the performance and the setup (it’s essential to streamline your setup process).
  • Invite your friends! All are welcome to the 12/13 show.

Week 15: Final Project Rehearsal + Show

Monday 12/11

  • Tech Check & Rehearsal.

Wednesday 12/13

  • Final Show