After The Weeknd’s tour for his new album, After Hours, was cancelled due to COVID, Spotify was looking to create a personalized experience of The Weeknd for his fans. Jam3 created a fully-virtual AI version of The Weeknd, utilizing deepfake technology and audio AI (voice synthesis). This is the first-ever personalized deepfake experience, and Jam3 used this work to show how technology can create unique, positive, and human-like connections.
The Weeknd that you see on the experience is all deepfaked onto a body double. Jam3 cast a look-a-like actor styled like The Weeknd and set up a remote green-screen shoot. The agency recorded the actor miming to the baseline script to capture proper lip movement as a framework for their deepfake video. Jam3 then used baseline audio from The Weeknd to train an AI that generated deepfake audio for dynamic data points unique to each user. Each fan can get a personal moment with the artist, where he talks to you, knows you, and plays you some of his latest tracks off of the After Hours album.
Jam3 also added a layer of interactive webGL on top of their deepfake, to let users discover and provide moments of surprise and delight. The WebGL layer also provided a feeling/insight into revealing The Weeknd’s alter ego during his After Hours music videos.
Thanks to COVID, Jam3 had to get creative on how to execute the project. Their team worked collaboratively globally to pull it off; everything from wigs and wardrobe to directing a fully remote shoot and creative reviews and work sessions all took place virtually.
Chat with The Weeknd: https://theweeknd.withspotify.com/
Credits: Serene Wong, Senior Art Director, Jam3
Branko Jass, Lead Motion Designer, Jam3
William Chen, Technical Director, Jam3
Greg Meltzer, Executive Producer, Jam3
Email
LinkedIn
X
Threads
WhatsApp
Telegram
More
After The Weeknd’s tour for his new album, After Hours, was cancelled due to COVID, Spotify was looking to create a personalized experience of The Weeknd for his fans. Jam3 created a fully-virtual AI version of The Weeknd, utilizing deepfake technology and audio AI (voice synthesis). This is the first-ever personalized deepfake experience, and Jam3 used this work to show how technology can create unique, positive, and human-like connections.
The Weeknd that you see on the experience is all deepfaked onto a body double. Jam3 cast a look-a-like actor styled like The Weeknd and set up a remote green-screen shoot. The agency recorded the actor miming to the baseline script to capture proper lip movement as a framework for their deepfake video. Jam3 then used baseline audio from The Weeknd to train an AI that generated deepfake audio for dynamic data points unique to each user. Each fan can get a personal moment with the artist, where he talks to you, knows you, and plays you some of his latest tracks off of the After Hours album.
Jam3 also added a layer of interactive webGL on top of their deepfake, to let users discover and provide moments of surprise and delight. The WebGL layer also provided a feeling/insight into revealing The Weeknd’s alter ego during his After Hours music videos.
Thanks to COVID, Jam3 had to get creative on how to execute the project. Their team worked collaboratively globally to pull it off; everything from wigs and wardrobe to directing a fully remote shoot and creative reviews and work sessions all took place virtually.
Chat with The Weeknd: https://theweeknd.withspotify.com/
Credits: Serene Wong, Senior Art Director, Jam3
Branko Jass, Lead Motion Designer, Jam3
William Chen, Technical Director, Jam3
Greg Meltzer, Executive Producer, Jam3