Chatbots for Art’s Sake
NYU IMA, Spring 2024 // Instructor: Carrie Sijia Wang
Quick Links:
About
Class Information
Time: Mondays & Wednesdays 2:00 – 3:30pm
Location: 370 Jay St., 410
Instructor: Carrie Wang / [email protected] / Office Hour Signup
Support:
ITP Residents Office Hours
Scroll down to “Research Residents.”
Description
This course is designed to repurpose existing chatbot technologies and use them for the sake of art. Comprising technical labs, design workshops, thematic seminars, and creative project development, it offers an exploration of the historical, present, and future dimensions of conversational AI; and the various roles AI has played and could play in human society. Students will delve into the design elements of conversational AI, and learn to use different techniques— such as RiveScript, p5.speech, APIs, Markov Chains, and Language Models—to create functional and artistic chatbots. The course expects students to conduct research and complete creative assignments, encouraging them to express their unique artistic visions.
Learning Objectives
At the completion of this course, the students will:
- Creatively adopt chatbot technologies to complete artistic projects.
- Have a better understanding of the process of project development from ideation to completion.
- Develop a set of skills to build functional chatbots.
- Learn to think critically about how technology is shaping the world we live in.
Teaching Methodology
This is a 14-week course that meets twice a week. Class time is divided into topic-based lectures and discussions, design and content development workshops, technical labs and experiments, creative project development and presentations.
This course values critical thinking about technology and society as much as learning programming skills.
Creative Assignments
- All creative assignments are due in the form of in-class presentations.
- Students are required to complete functional projects following specific requirements, document their processes, and present in class.
Independent Research Assignments
- Throughout the semester, every student will be assigned a selection of materials as a basis for further research.
- Students are required to conduct research, come up with their own unique interpretation of the assigned materials, and present their reflections in class.
Final Project
- The course will culminate with the completion of a final project to be shared in class.
- We are expected to push our abilities to produce something that builds off or is inspired by the concepts we cover during the semester.
Evaluation
Grades will be determined according to this criteria:
- Participation (including research presentations) and Attendance 40%
- Creative Assignments 40%
- Final Project 20%
To do well in this class, you should make efforts to learn and explore the lecture/research topics, design concepts, and technical tools. You are expected to complete all the assignments on time, have things to show for the in class presentations, and actively participate in discussions.
Attendance
Attendance is mandatory. Please email your instructor if you are going to miss a class. Two unexcused absences is cause for failing the class. An unexcused lateness of 10 minutes or more is equivalent to 1/2 of an absence.
ITP/IMA Code of Conduct
ITP/IMA is a community whose mission is to explore the imaginative uses of emerging technologies — to make people’s lives safer, more just, more beautiful, more meaningful, and more fun. We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community (excerpt from ITP/IMA Code of Conduct).
This course is committed to providing an inclusive, welcoming, and harassment-free space for everyone in our community. Harassment or discrimination in any form will not be tolerated, and this applies to any interactions and content.
Statement on Title IX
Tisch School of the Arts is dedicated to providing its students with a learning environment that is rigorous, respectful, supportive and nurturing so that they can engage in the free exchange of ideas and commit themselves fully to the study of their discipline. To that end Tisch is committed to enforcing University policies prohibiting all forms of sexual misconduct as well as discrimination on the basis of sex and gender. Detailed information regarding these policies and the resources that are available to students through the Title IX office can be found by using the following link: Title IX at NYU.
Accessibility & Wellness
Academic accommodations are available for students with documented disabilities. Please contact the Moses Center for Students with Disabilities at 212-998-4980 for further information.
24/7 Mental / Physical Health, Wellness, Counseling, and Crisis Response resources can be accessed via Wellness Exchange online or by phone at the 24/7 hotline at (212) 443-9999
More detailed for medical, wellness, counseling and crisis response resources see here:
https://www.nyu.edu/students/health-and-wellness.html
For emergency response:
dial 911 for NYC Medical, Fire, or Police
NYU Campus Safety at (212) 998-2222 to report the emergency
For more emergency and urgent NYU resources:
https://www.nyu.edu/students/health-and-wellness/wellness-exchange/emergencies.html
Academic Integrity & Statement Principle
Excerpt from the NYU Tisch School of the Arts Policies and Procedures Handbook, please read the full text for additional information:
The core of the educational experience at the Tisch School of the Arts is the creation of original academic and artistic work by students for the critical review of faculty members. It is therefore of the utmost importance that students at all times provide their instructors with an accurate sense of their current abilities and knowledge in order to receive appropriate construc- tive criticism and advice. Any attempt to evade that essential, transparent transaction between instructor and student through plagiarism or cheating is educationally self-defeating and a grave violation of Tisch School of the Arts community standards.
Plagiarism is presenting someone else’s original work as if it were your own. More specifically, plagiarism is to present as your own: a sequence of words quoted without quotation marks, a paraphrased passage from another writer’s work, ideas, sound recordings, computer data, or images composed or created by someone else.
Students are expected to build their own work on that of other people, just as professional artists, scholars, and writers do. Giving credit to the creator of the work you are incorporating into your own work is an act of integrity; plagiarism, on the other hand, is a form of fraud.
Use of Free and Open Source Materials Including Code
You must cite the source (link to) of any material/code you use with the exception of examples specifically provided by the instructor or demonstrated for the course. Please note the following additional expectations and guidelines:
- Check the license. When using others’ code, pay attention to the license under which it has been released, and be certain to fulfill the terms and requirements of those licenses. Descriptions of common licenses, and their requirements, can be found at choosealicense.com. Some licenses may require permission. If you are confused or aren’t sure how to credit code, ask one of the course instructors and make your best good faith effort. Not properly citing code sources is grounds for receiving a 0 on an assignment.
- Use of code libraries. The use of general, repurposable libraries is strongly encouraged. The people who developed and contributed these components to the community worked hard, often for no pay; acknowledge them by citing their name and linking to their repository.
- Be careful. It sometimes happens that an artist places the entire source code for their sketch or artwork online, as a resource from which others can learn. Assignments professors give in new media arts courses are often similar; you may also discover the work of a student in some other class or school, who has posted code for a project which responds to a similar assignment. You should probably avoid this code. At the very least, you should be careful about approaching such code for possible re-use. If it is necessary to do so, it is best to extract components that solve a specific technical problem, rather than those parts which operate to create a poetic experience. Your challenge, if and/or when you work with others’ code, is to make it your own. It should be clear that downloading an artwork from someone’s p5 account or GitHub and simply changing the colors would be disgracefully lazy. And doing so without proper citation would be outright plagiarism.
- This statement is adapted from Dan Shiffman’s Code! Course (Spring 2020) at New York University and Golan Levin’s Interactivity and Computation Course (Fall 2018) at Carnegie Mellon University.
Schedule
Week 1: Past and Present
Session 1: History // Monday 01/22
In Class:
- Introductions
- Course Overview
- A Brief History
- Group Activity: A Turing Test!
Session 2: Landscape // Wednesday 01/24
In Class:
- Chatbot + Art?
- How to Make a Chatbot
- Large Language Models and Generative AI
- Group Activity: A Chatbot Version of You
Homework
Mini Research: Due next Monday before class:
Week 2: Scenarios of the Future
Session 1: Share Mini Research // Monday 01/29
In Class:
- Share Mini Research
Session 2: Conversation Writing + RiveScript Intro// Wednesday 01/31
In Class:
- Assign Creative Assignment #1
- Versions of the Future
- Group Activity: Write a Chatbot Conversation
- Introduce RiveScript
- Coding Time: Turn the Conversation into an Interactive Program
Homework
Creative Assignment #1: Can You Predict the Future? — A Performance: Due in class on Monday, Feb 12
For this week, focus on writing the conversation and getting the RiveScript part to work. We will move on to voice design next week.
Independent Research 01: Due Feb 05
Independent Research 02: Due Feb 07
Week 3: Homage to the Human Voice
Session 1: Voice Design // Monday 02/05
In Class:
- Present Independent Research
- Voice Interface
- Characteristics of the Human Voice
- A Brief History of Voice Synthesis
Session 2: p5.speech + Other Tools // Wednesday 02/07
In Class:
- Present Independent Research
- p5.speech Basics
- Combine p5.Speech with RiveScript Chatbot
- Other Voice Design Tools
- Voice Design Exercise
Homework
Creative Assignment #1: Can You Predict the Future? — A Performance: Due in class on Monday, Feb 12
For this week, complete the project. Design and add a voice for the AI.
Independent Research 03: Due Feb 14
Week 4: Creative Assignment #1 Presentations
Session 1: Creative Assignment #1 Presentations // Monday 02/12
Session 2: Roles AI Plays // Wednesday 02/14
In Class:
- Present Independent Research
- AI in Fiction
- AI in Reality
- Relevant Art Projects
- Character Design
Homework
Design and Enrich an AI Character: Due next Wednesday
Design an AI character that reflects a part of your personal reality, and/or a social issue you care about in a meaningful way.
Refer to the Character Design Exercise introduced in class.
You will make this character into an interactive chatbot in Creative Assignment #2 (details to follow).
Week 5: Building a Character
Session 1: Character Building Tools // Wednesday 02/21
In Class:
- Assign Creative Assignment #2
- A Guided Experience with RiveScript
- An Open Conversation with OpenAI API
- Coding Exercise
Homework
Creative Assignment #2: A Chatbot with a Personality
For this week, focus on building the character (appearance, voice, and more), exploring the tools introduced, and do some conversation/instruction writing.
Independent Research 04 – Due Feb 26
Week 6: Interface and Experience Design
Session 1: AI Interface and User Experience // Monday 02/26
In Class:
- Present Independent Research
- User Experience and User Interface Design
- Real-World AI Interfaces
- Sci-Fi AI Interfaces
- User Interface Design Exercise
Session 2: Studio and 1:1 Meetings // Wednesday 02/28
Studio and Open Office Hours
Homework
Creative Assignment #2: A Chatbot with a Personality
Refine the interface and experience design of your AI character. Do some user testing with friends and classmates. Complete the assignment and get ready to present in class.
Week 7: Creative Assignment #2 Presentations
Session 1: Creative Assignment #2 Presentations Part 01 // Monday 03/04
Session 2: Creative Assignment #2 Presentations Part 02 // Wednesday 03/06
Homework
Independent Research 05 – Due Mar 11
Midterm Survey – Due Mar 11
Week 8: Who Is the Machine Learning From?
Session 1: Data in AI and Markov Chain // Monday 03/11
In Class:
- Data in AI
- Present Independent Research
- Markov Chain
Session 2: Markov Chain Experiment // Wednesday 03/13
In Class:
- Data Collection
- Data Processing
- Coding a Bot
- Reflections
- Assign Creative Assignment #3
Homework
Creative Assignment #3: An Experiment on Data
For this week, come up with an initial concept for the project, do research, and start collecting and processing data.
Week 9: Data Data Data
Session 1: CharRNN & ml5.js // Monday 03/25
In Class:
- Markov Chain, CharRNN, & GPT Models
- Using a Desktop Editor
- CharRNN & ml5.js Demo
- More Examples
- Hosting on Glitch
Session 2: Studio and 1:1 Meetings // Wednesday 03/27
Studio and Open Office Hours
Homework
Creative Assignment #3: An Experiment on Data
For this week, experiment with different methods to process your collected data. Complete the project by refining design elements such as interface and voice.
Week 10: Creative Assignment #3 Presentations
Session 1: Creative Assignment #3 Presentations Part 01 // Monday 04/01
Session 2: Creative Assignment #3 Presentations Part 02 // Wednesday 04/03
Homework
Bring a Project That Inspires You – Due April 08
Post a link to a project that inspires you here, and share it with the class on Monday. It can be any emerging media project (not necessarily related to chatbots) done by anybody other than yourself. Be prepared to tell us what it is, play a short clip of its documentation if it’s available, and share with us why and how it inspires you.
Independent Research 06 – Due April 08
Week 11: Ethical AI
Session 1: Assign Final & Share Inspirations // Monday 04/08
In Class:
Session 2: Ethical AI Manifesto // Wednesday 04/10
In Class:
- Speculative AI Discussion
- Propose an “Ideal AI”
- Contribute to an Ethical Manifesto for Chatbots
- Train a Collective AI Voice to Read the Manifesto
Homework: Final Project Pitch Presentation – Due Monday 04/15
Consolidate your ideas and experiments you’ve done so far into a final project pitch presentation. Add your slides to this folder.
Week 12: Final Project – Pitch
Session 1: Pitch Presentations 01 // Monday 04/15
Session 2: Pitch Presentations 01 // Wednesday 04/17
Homework: Final Project Proof of Concept – Due Monday 04/22
Get your project to work in the most basic way. Have something at least partially functional for your classmates to test out!
Week 13: Final Project – User Testing
Session 1: User Testing Part 01 // Monday 04/22
Session 2: User Testing Part 02 // Wednesday 04/24
Homework: Final Project Presentations – Due Monday 04/29
Add your slides to this folder.