Immersive Storytelling and Emerging Technologies

This introductory course will provide students with the tools and the mind set for making compelling VR/AR experiences. While the industry is nascent, the technological and storytelling innovations move forward at breakneck speed. Students will also, each class, dissect to understand the approaches to the current catalog of immersive experiences, ranging from 360 film, to animation and room scale installation experiences, often with creators who made them to understand challenges and lessons learned. Subsequently, after this overview, students will have the option to build their own prototypes and, also, to support a VR/AR project housed within the program with a leading artist.

An introductory course that provides students with an overview of the process to create innovative and meaningful cinematic stories in the evolving field of interactive games. From concept to completion, the class will explore the creative architecture, production process and technical considerations necessary for developing for the new wave of interactive entertainment across platforms. Drawing from theoretical and production frameworks in game design, narrative and documentary filmmaking, art, immersive theatre, and motion capture––critical attention will be given to intuitive and engaging design. The hands on portion of the class will culminate with students developing a prototype for their own original interactive cinematic project.

Lush and realistic virtual worlds that were recently impossible are suddenly commonplace. These synthetic scenes surround us, hidden in sweeping virtual backgrounds on film sets or featured prominently in seemingly endless videogame landscapes. While breakthroughs in computer graphics and game engines make this possible, new tools make it common. The hidden development conjuring up seemingly anything is a new class of reality capture and filmmaking tools that creators to quickly bring the real world into virtual spaces. These are the tools of virtual production and volumetric filmmaking.

This course starts with personal stories, transforming a vivid memory into a sharable immersive experience with tools and insights from volumetric filmmaking, virtual production and game design. Students learn how to adapt a script for immersion, 3D modeling, photogrammetry scanning, and applying techniques like virtual lighting, virtual cinematography, and the basics of motion capture and volumetric capture to bring their scene to life. The course invites students to create lush and imaginative worlds immediately with common hardware – mobile phones and computers.

Discover the intersection of artificial intelligence and the creative process. Recent developments in generative AI tools have made it possible for amateurs to create visual art in seconds that previously would have required hours or days of expert effort. This has raised fascinating legal and philosophical questions about authorship, craft, and even consciousness. Through collaborative activities, students will develop new norms and ethics for our age of generative AI and test them while learning to use the major AI tools available. Explore the ethical implications and best methods for working with AI-generated media, while engaging in discussions about authorship and creativity in the context of artificial collaboration. Students will gain practical experience using AI tools, learn to balance technical proficiency with artistic vision, and develop a deeper understanding of the creative potential of AI as well as the ethical considerations involved in using AI tools in the creative process. The course is designed to equip students with the skills and knowledge needed to become ethical, informed, and creative practitioners in the age of generative AI.

This enhanced 12-week course takes a deeper dive into the tools and techniques learned in the Foundations of Immersive Storytelling: Theory and Practice course. The curriculum is designed to equip participants with enhanced skills in 360 video production, WebXR content creation, 3D scanning and the utilization of Unreal Engine for scene creation with 3D assets.