Chatbots for Art’s Sake
NYU IMA, Spring 2023 // Instructor: Carrie Sijia Wang
Quick Links:
About
Class Information
Time: Mondays & Wednesdays 10:40am – 12:10pm
Location: 370 Jay St., 407
Instructor: Carrie Wang / sw3923@nyu.edu / Office Hour Signup
Zoom Link (Only use when you have COVID related concerns such as having to quarantine.)
Support:
ITP Residents Office Hours
Scroll down to “Research Residents.”
Description
This class aims to repurpose existing chatbot technologies and use them for the sake of art. The class is twofold, students will engage in labs and workshops to learn and practice different techniques— such as p5.js, p5.speech, RiveScript, RiTa, and Alexa Skills—to create functional chatbots. They will also participate in lectures and discussions that look at the different roles Artificial Intelligence plays in human society, including but not limited to authority, companions, or simply reflections of the humans it interacts with. Chatbots can create experiences that other forms of art cannot. They are interactive by nature—an art piece in the form of a chatbot is only complete with audience participation. They also invite introspection more than any other medium—a user talking to a chatbot is essentially talking to themselves with the assistance of a computer program.
Learning Objectives
At the completion of this course, the students will:
- Creatively adopt chatbot technologies to complete artistic projects.
- Have a better understanding of the process of project development from ideation to completion.
- Develop a set of skills to build functional chatbots.
- Learn to think critically about how technology is shaping the world we live in.
Teaching Methodology
This is a 14-week course that meets twice a week. Class time is divided into lectures, discussions, labs, and student presentations.
This course values critical thinking about technology and society as much as learning programming skills.
Monday classes are usually dedicated to tech labs in which students work together with the instructor, and with each other, to learn new skills in a workshop setting.
Wednesday classes are dedicated to exploring topics related to chatbots and artificial intelligence. These classes alternate between two forms:
01 Lecture, discussion, and student presentations about reading materials.
02 Student project presentations.
Weekly Assignments
- Weekly assignments alternate between creative assignments and reading assignments.
- All assignments are due in the form of google slideshow presentations on Wednesdays.
- For reading assignments, students are required to do research, come up with their own unique interpretation of the assigned materials, and present their thoughts in class.
- For creative assignments, students are required to complete functional projects, present their process and results in class.
- Students can choose to work on their own or with ONE other classmate for creative assignments.
Final Project
- The course will culminate with the completion of a final project to be shared in class.
- We are expected to push our abilities to produce something that builds off or is inspired by the concepts we cover during the semester.
Evaluation
Grades will be determined according to this criteria:
- Participation (including reading presentations) and Attendance 40%
- Creative Assignments 40%
- Final Project 20%
To do well in this class, you should make efforts to learn and explore the lecture/discussion topics and technical tools. You are expected to complete all the assignments on time, have things to show for the in class presentations, and actively participate in discussions.
Attendance
Attendance is mandatory. Please email your instructor if you are going to miss a class. Two unexcused absences is cause for failing the class. An unexcused lateness of 10 minutes or more is equivalent to 1/2 of an absence.
Equitable IMA/ITP
More Resources on Equity & Inclusion
Statement of Academic Integrity
Plagiarism is presenting someone else’s work as though it were your own. More specifically, plagiarism is to present as your own: A sequence of words quoted without quotation marks from another writer or a paraphrased passage from another writer’s work or facts, ideas or images composed by someone else.
Statement of Principle
The core of the educational experience at the Tisch School of the Arts is the creation of original academic and artistic work by students for the critical review of faculty members. It is therefore of the utmost importance that students at all times provide their instructors with an accurate sense of their current abilities and knowledge in order to receive appropriate constructive criticism and advice. Any attempt to evade that essential, transparent transaction between instructor and student through plagiarism or cheating is educationally self-defeating and a grave violation of Tisch School of the Arts community standards. For all the details on plagiarism, please refer to page 10 of the Tisch School of the Arts, Policies and Procedures Handbook.
Statement on the Use of Free and Open Source Materials (including code)
This statement is written by Ellen Nickles, adapted from Dan Shiffman’s Code! Course (Spring 2020) at New York University and Golan Levin’s Interactivity and Computation Course (Fall 2018) at Carnegie Mellon University.
You must cite the source of any material / code you use with the exception of examples specifically provided by the instructors or demonstrated in media assets for the program courses. Please note the following additional expectations and guidelines:
Check the License. When using others’ code, pay attention to the license under which it has been released, and be certain to fulfill the terms and requirements of those licenses. Descriptions of common licenses, and their requirements, can be found at choosealicense.com. Some licenses may require permission. If you are confused or aren’t sure how to credit code, ask one of the course instructors and make your best good faith effort.
Use Libraries. The use of general, repurposable libraries is strongly encouraged. The people who developed and contributed these components to the community worked hard, often for no pay; acknowledge them by citing their name and linking to their repository.
Be Careful. It sometimes happens that an artist places the entire source code for their sketch or artwork online, as a resource from which others can learn. Assignments professors give in new-media arts courses are often similar; you may also discover the work of a student in some other class or school, who has posted code for a project which responds to a similar assignment. You should probably avoid this code. At the very least, you should be careful about approaching such code for possible re-use. If it is necessary to do so, it is best to extract components that solve a specific technical problem, rather than those parts which operate to create a poetic experience. Your challenge, if and/or when you work with others’ code, is to make it your own. It should be clear that downloading an artwork from someone’s GitHub and simply changing the colors would be disgracefully lazy. And doing so without proper citation would be outright plagiarism.
Statement on Accessibility
Please feel free to make suggestions to your instructor about ways in which this class could become more accessible to you. Academic accommodations are available for students with documented disabilities. Please contact the Moses Center for Students with Disabilities at 212 998-4980 for further information.
Statement on Counseling and Wellness
Your health and safety are a priority at NYU. If you experience any health or mental health issues during this program, we encourage you to utilize the support services of the 24/7 NYU Wellness Exchange (US +1 212-443-9999). Also, all students who may require an academic accommodation due to a qualified disability, physical or mental, please register with the NYU Moses Center (US + 1 212-998-4980). Please let your instructor know if you need help connecting to these resources.
Statement on Use of Electronic Devices
Laptops will be an essential part of the course and may be used in class during workshops and for taking notes in lecture. Laptops must be closed during student presentations. Phone use in class is strictly prohibited unless directly related to a presentation of your own work or if you are asked to do so as part of the curriculum.
Statement on Title IX
Tisch School of the Arts to dedicated to providing its students with a learning environment that is rigorous, respectful, supportive and nurturing so that they can engage in the free exchange of ideas and commit themselves fully to the study of their discipline. To that end Tisch is committed to enforcing University policies prohibiting all forms of sexual misconduct as well as discrimination on the basis of sex and gender. Detailed information regarding these policies and the resources that are available to students through the Title IX office can be found by using the following link: Title IX at NYU.
Schedule
Week 1: Chatbots in Art
Session 1: Introduction // Monday 01/23
In Class:
- Self Introductions
- Course Overview
- Group Activity: Turn Yourself into a Chatbot
Homework – Mini Research: LINK TO REQUIREMENTS
Present your findings THIS Wednesday.
Session 2: Lecture & Discussion // Wednesday 01/25
In Class:
- Lecture and Discussion: Chatbots in Art
- Share Mini Research
Homework – Read/Watch/Play:
1. Read: A one-word Turing Test suggests “poop” is what sets us apart from the machines
2. Watch: How the “Most Human Human” passed the Turing Test
3. Play: Agent Ruby, 1999, Lynn Hershman Leeson
Be patient with the chatbot, observe how it behaves, pay attention to how your feelings change in the process of interacting with it.
You don’t have to prepare a presentation this time. But get ready to share your thoughts in class next Wednesday.
Week 2: The Human-Machine Divide
Session 1: Lab – p5.js + JSON // Monday 01/30
In Class:
- Tech Demo
- Coding Exercise
- Share Sketches from Exercise
Homework – Creative Assignment #1: A Chatbot Version of You
Present your project in class next Wednesday.
Create a chatbot based on how you talk or write in text within a specific context. Define your bot’s purpose, and then think about how you can add your personality to a simple bot with limited capabilities.
Session 2: Lecture & Discussion // Wednesday 02/01
In Class:
- Lecture & Discussion: The Human-Machine Divide
- Hosting Your Chatbot on Glitch
- Share Thoughts About Read/Watch/Play Homework or Remaining Mini Research Presentations
Week 3: Lab & Project Feedback
Session 1: Lab – p5.speech I // Monday 02/06
In Class:
- Introduce/Review p5.speech
- Tech Demo: Add Sound to a JSON Chatbot
- Coding Exercise: Add Sound to Your Chatbot
- Share Your Sketches
Homework – Read/Watch/Play & Respond: LINK TO REQUIREMENTS
Present your response in class next Wednesday.
1. Read: Everyone will be able to clone their voice in the future
2. Watch: Google Duplex: A.I. Assistant Calls Local Businesses To Make Appointments & Moniker – Repeat After Me
3. Play: Click Click Click, Moniker, 2016
Session 2: Presentations & Feedback // Wednesday 02/08
In Class:
- Present Creative Assignment #1: A Chatbot Version of You
Week 4: Homage to the Human Voice
Session 1: Lab – p5.speech II // Monday 02/13
In Class:
- Tech Demo: Voice Controller
- Tech Demo: Google Translate
- Coding Exercise: Voice-Centered Experience
- Share Your Sketches
Homework – Creative Assignment #2: The Computer is Listening
Present your project in class next Wednesday.
Create a voice/sound only experience. When the user talks, the computer is listening. What will be the computer’s response? Will it repeat what the user has said, try to strike a conversation, play a piece of music that fits the user’s mood, or translate the user’s sentence into another language?
Session 2: Lecture & Discussion // Wednesday 02/15
In Class:
- Present Read/Watch/Play Response Homework
- Lecture & Discussion: Homage to the Human Voice
Week 5: Project Feedback
Session 1: Lab – Presentations & Feedback // Wednesday 02/22
In Class:
- Present Creative Assignment #2: The Computer is Listening
Homework – Read/Watch/Play & Respond: LINK TO REQUIREMENTS
Present your response in class next Wednesday.
1. Read: How algorithms keep workers in the dark
2. Read: How automation can solve recruitment bias
3. Play: Survival of the Best Fit, Gabor Csapo, Jihyun Kim, Miha Klasinc, and Alia ElKattan, 2019
Week 6: My Manager Is a Bot
Session 1: Lab – Rivescript // Monday 02/27
In Class:
- Introduce RiveScript
- Tech Demo
- Coding Exercise
Homework – Creative Assignment #3: A Chatbot That Judges
Present your project in class next Wednesday.
So far we have made a bot that mimics us, and another one that listens to us. Now it’s time to create a bot that evaluates and judges us or the work we do. What is the bot judging? What are the standards? Are the standards fair? Do they carry biases embedded in existing systems? Perhaps the chatbot being biased is the point you want to make. Try to make this a fun and/or reflective experience for your user rather than a practical tool.
Session 2: Lecture & Discussion // Wednesday 03/01
In Class:
- Present Read/Watch/Play Response Homework
- Lecture & Discussion: My Manager Is a Bot
Week 7: Lab + Project Feedback
Session 1: Lab – RiTa // Monday 03/06
In Class:
- Introduce RiTa
- Generative Text: Markov Chain
- Tech Demo: Movie Character Bot
- Coding Exercise
Homework – Read/Watch/Play & Respond
1. Read: Inside the mechanical brain of the world’s first robot citizen
2. Read: AI Is Learning From Humans. Many Humans. – The New York Times
3. Listen: Internet Trolls Turn A Computer Into A Nazi : NPR
4. Play: Baby Faith, Ryan Kuo, 2020 (Use a non-NYU network to access this one.)
You don’t have to prepare a presentation this time. But get ready to share your thoughts in class next Wednesday.
Session 2: Presentations & Feedback // Wednesday 03/08
In Class:
- Present Creative Assignment #3: A Chatbot That Judges
Homework – Log in to Firebase Console with a NON-NYU google account.
Week 8: Who Is the Machine Learning from?
Session 1: Lab – Firebase // Monday 03/20
In Class:
- Firebase – database as a service
- Tech Demo: Impressionable Chatbot (using Firebase)
- More Coding Time
Homework – Creative Assignment #4: Frankenstein Chatbot
Present your project in class next Wednesday.
Create a “hybrid” bot whose “brain” is based on a combination of data sources. Think about where you are getting your data and what it means to make a bot from these different sources. Is the bot a mixture of movie characters? Politicians? Pop icons? Politicians + pop stars? Does the bot change how it talks based on user’s input?
Session 2: Lecture & Discussion // Wednesday 03/22
In Class:
- Guest Speaker: Mario Guzman
- Share thoughts about Read/Watch/Play Response Homework
Week 9: Lab + Project Feedback
Session 1: Lab – Alexa Skills I // Monday 03/27
In Class:
- Tech Demo
- Coding Exercise
- Share Sketches from Exercise
Homework – Read/Watch/Play & Respond: LINK TO REQUIREMENTS
Present your response in class next Wednesday.
Topic 01:
1. Read: Can emotion-regulating tech translate across cultures? | Aeon Essays
2. Read: Men Are Creating AI Girlfriends and Then Verbally Abusing Them
3. Watch: Why China is putting robots in nursing homes
Topic 02:
In Sudden Alarm, Tech Doyens Call for a Pause on ChatGPT
Pause Giant AI Experiments: An Open Letter
Session 2: Presentations & Feedback // Wednesday 03/29
In Class:
- Present Creative Assignment #4: Frankenstein Chatbot
Week 10: My Imaginary Friend
Session 1: Lab – Alexa Skills II + Assign Final // Monday 04/03
In Class:
- Assign Final Project
- Tech Demo
- Coding Exercise
Homework – Final Project Proposal (Due 4/12)
Present your project pitch in class next Wednesday.
Work on your own or in pairs, design, develop, and present a project inspired by the concepts, ideas, and technical tools introduced in this class. The project should creatively utilize the form and technology of chatbots to tell stories, express feelings, and/or convey a certain message to the audience.
Session 2: Discussion & Emerging Media Projects Share or Brainstorm Final Project Ideas // Wednesday 04/05
In Class:
- Present Read/Watch/Play Response Homework
- Share and Discuss Relevant Emerging Media Projects
- Or: Brainstorm Final Project Ideas
Homework – Bring a Project (Due 4/10)
Bring one project in the emerging media field that inspires you and share it with the class on Monday.
It can be any emerging media project (not necessarily related to chatbots) done by anybody other than yourself. Be prepared to tell us what it is, play a short clip of its documentation if it’s available, and share with us why and how it inspires you.
Add a link to the project in our inspirations doc.
Week 11: Inspiration Share + Final Project Pitch
Session 1: Inspiration Share // Monday 04/10
In Class:
- Share projects that inspire you (one project per person)
- Studio time
Session 2: Final Project Pitch // Wednesday 04/12
In Class:
- Present Final Project Pitch
Homework – Final Project Proof of Concept
- Provide feedback for your classmates here.
- Sign up to meet with instructor next week (only sign up for one slot per person/group).
- Complete a minimal viable “product” ready to share in the one-on-one meeting.
Week 12: Work-In-Progress + One-on-One Meetings
Session 1: One-On-One Meetings I // Monday 04/17
In Class:
- Meet with the instructor one-on-one
- The rest of the class can use the time to work
Homework – Final Project
- Read everything on the User Testing Slides.
- Prepare your project for user testing next week.
Session 2: One-On-One Meetings II // Wednesday 04/19
In Class:
- Meet with the instructor one-on-one
- The rest of the class can use the time to work
Week 13: Final Project User Testing + Work-In-Progress
Session 1: User Testing I // Monday 04/24
In Class:
- Assign Final Project Presentation
- User Testing I
Homework – Final Project Documentation & Presentation
Complete your final project. Document it, and get ready to present your process and result to the class. Add your slides to the slideshows below:
Session 2: User Testing II // Wednesday 04/26
In Class:
- Presentation Tips
- User Testing II