How Synthetic Media will change Hollywood? with AI, Digital Humans, Voice Cloning + Synthesia Demo

Transcript

0:00 Speaker A: Hey, I'm Sander, and today I'm really excited to explore with you the possibilities of synthetic. Speaker B: No, you're not, Sander. Speaker B: I don't think you're going to be exploring any opportunities here. Speaker A: Yeah, well, I look like you, I can speak like you, and I can probably even write better than you. Speaker A: So maybe just let me take this one. Speaker B: No, you're not going to be taking this one. Speaker B: I'm going to be taking this one. Speaker B: But maybe next time. Speaker A: Okay, but don't forget to like this video and subscribe. Speaker B: Yeah, all of this that you just saw was fully, completely computer generated.

0:29 Speaker B: The visual and the audio and the voice as well. Speaker B: I'm really excited to explore with you synthetic media. Speaker B: And when I say synthetic media, when people first come to think about, is actually deep fakes. Speaker B: And those are those videos that we've seen on social media, like this one,. Speaker C: When there's so many haters, I really don't care because their data has made me rich beyond my wildest dreams. Speaker B: Or this one that was of President Obama. Speaker D: We're entering an era in which our enemies can make it look like anyone is saying anything at any point in time, even if they would never say those things.

1:03 Speaker B: But actually, synthetic media is any media that's created by computers or modified by computers. Speaker B: So actually the media that we see around us, for example, playing around with Instagram filters or TikTok filters, all of those things are synthetic media because they're manipulated or they're creating or modified using the help of computers. Speaker B: So when you turn into a cat or you turn your mask on, that's actually synthetic media as well. Speaker B: And then there's also this high end synthetic media that the big movie producers use, you know, for making those masks, tracking people's faces and those very complex models in mapping those out and making them sound and look real in movies.

1:40 Speaker B: And there's a great development that's happening there. Speaker B: This is also synthetic media through this digital domain example here. Speaker B: So there's these two categories of the consumer side where, you know, we play around with the filters. Speaker B: You can use the Time Machine filter or Snapchat that I've used here when I turn myself older or younger, or they're those big high end movie productions where these professionally made used hours and months of work and millions of dollars to make happen. Speaker B: But now using those tools, those tools are getting more and more accessible.

2:08 Speaker B: We see the consumer ease of use coming to professional tools and professional tools becoming more and more available to consumers. Speaker B: And we see this middle persumer class coming up from companies like Synthesia, Rephrase or Windsor that we're going to see later on. Speaker B: And we're moving this general phase from digital to everything driven by AI. Speaker B: We already did that in music, where we moved from analog instruments to digital instruments. Speaker B: So now most of the music being produced, beats, voices, pitch corrections, all done by AI using computers.

2:39 Speaker B: Same thing happened in written press, from writing, sending fax to sending emails to now most of the emails being generated by computers using the titles and the content. Speaker B: And they're mostly automatically even sent out. Speaker B: And the same thing is happening now in the video world. Speaker B: We're no need for cameras, no need for microphones, lights or crews, and we can just generate all of that content. Speaker B: We can produce content on our computers. Speaker B: And I'm going to show you how I did the intro bit as well. Speaker B: But why now and why video?

3:05 Speaker B: First of all, why video? Speaker B: 68% Of consumers prefer watching videos instead of reading articles or looking infographics or ebooks. Speaker B: That's why YouTube is the second largest search engine. Speaker B: And the top three most popular apps for Gen Z are all video apps. Speaker B: Snapchat, Twitch and TikTok. Speaker B: And the consumer traffic by the end of this year will be 82% video. Speaker B: It's also important for business because it increases the click through rate, increases the exposure in Google search As they include 62% of videos in Google search results.

3:34 Speaker B: And people who have video on the website, they spend actually longer on the website, two minutes longer the next day. Speaker B: So here's an example. Speaker E: Everyone's got a personal video just for them. Speaker E: Hey Sebastian, how are you doing? Speaker E: Hey David, how are you doing? Speaker E: Hey Michael, how are you doing? Speaker E: And they just had to reply Using. Speaker B: Computer generated personalized email is actually having business impact as you saw before.

4:00 Speaker B: Peel, a company who's making phone cases used this as an example with 10,000 of their customers. Speaker B: The ones who got the personalized video from their CEO, thanking them from their purchase and the ones who didn't, and the ones who did get that personalized video, calling out their name and thanking them for the purchase were 87% more likely to buy from them again. Speaker B: So it's a significant business driver in getting more revenue and having a more personalized connection.

4:25 Speaker B: Synthesia themselves also reports that they're getting higher engagement when they're using video one of their clients and their massive cost savings in video production by using synthetic media production platforms. Speaker B: And what why now? Speaker B: Why we're just now getting those tools accessible to everybody? Speaker B: Number one thing is that Google machine learning for Word Accuracy surpassed human level accuracy in 2017, just four years ago. Speaker B: The same thing is now happening with.

4:50 Speaker A: The production of voices. Speaker B: So Google WaveNet is 92% human speech quality and being able to do that in 80 plus different languages and variants. Speaker B: So you can produce 20 seconds of audio in just one second of production. Speaker B: And this truly shows that we're passing the Turing test, which means that whether computer processes the audio or human processes the audio, we don't hear the difference. Speaker B: And I think we're hitting that inflection point now with some of the demos that we've seen from Google.

5:19 Speaker B: And also what else is happening is that image classification themselves. Speaker B: And the visual side is getting better. Speaker B: We passed human accuracy in 2015 and it is just getting better. Speaker B: We can use the machine learning to see people's emotions, recognize their faces and all the other attributes to then create engines to create people's faces. Speaker B: This is an Unreal Engine metahumans example to see how you can use their platform to generate all kinds of different faces. Speaker B: Or you can use platforms like thispersondoesonexist.com to generate as many fake faces as you want.

5:49 Speaker B: But what are the benefits? Speaker B: What is the reason of using these kinds of technologies? Speaker B: Number one, it's democratization. Speaker B: You're giving that ability to create videos to everybody. Speaker B: So you don't need a camera, it's accessible to everybody. Speaker B: It's much lower cost because you don't need to invest in that equipment. Speaker B: It's much more simpler. Speaker B: You don't know how to operate all of these platforms. Speaker B: So let's go into each one of them. Speaker B: How the content creation now drives scale, how it's much more personal, how it helps you reach more people by knowing or being able to share your content in more languages.

6:20 Speaker B: Number one is scale through automation. Speaker B: You know, if somebody purchases something on your website, you can then trigger it automatically through Zapier to go through, for example, someone like Synthesia or Windsor to trigger that personalized email to go to the customers to drive the customer love that we already saw for them coming back to you. Speaker B: The second thing is speed. Speaker B: You know, for me to record a 10 minute video takes 10 minutes. Speaker B: For AI to generate 10 minutes video,. Speaker A: Just 30 seconds, it's very, very fast.

6:45 Speaker B: It accelerates your production pipeline. Speaker B: And in order to create a digital version of my voice, for example, I descript in the first iteration, the first video that you saw, so I had to read 30 minutes of audio to the computer to then generate my voice. Speaker B: Custom voice model in Synthesia it cost thousand dollars to Create a custom avatar of yourself. Speaker B: And for that I needed to do five takes of three minute video and then I'll have an avatar that I can use indefinitely across all of my videos.

7:15 Speaker B: So how did it actually look like? Speaker B: Or how do those tools work? Speaker B: Because we talk about simplicity, right? Speaker B: This is an example of a descript where you can just type in the words or you can take existing script or existing video and then it automatically generates your voice, which you can then just export to be used across all of your distribution channels. Speaker B: Or you can use that voice recording to then put against your video. Speaker B: So here's what I'm going to do.

7:40 Speaker B: Within Synthesia and their interface, I'm showing how I created the intro video. Speaker B: First off, you go to their platform. Speaker B: There are so many options out there. Speaker B: It's like a true creation platform. Speaker B: So you can use their existing templates or you can import your own power PowerPoints to build your own templates. Speaker B: You can then also choose background, for example. Speaker B: In this case, I'm using the same background that I'm using for these videos to make it look very similar. Speaker B: You can size the avatar, you can add your text, you can add your graphics, you can add music, you can add different elements into the video and you can also change the person in the video.

8:11 Speaker B: Then in order to generate the voice, you can use their own voice generation platform, which is really good, actually even better than my voice model. Speaker B: And you can choose that in 55 different languages. Speaker B: In this case, I'm using the audio file that I generated already in descript as I want it to sound like me using my own voice, custom voice, and that's it. Speaker B: Once you hit generate, you just wait a couple of minutes and the video is going to be ready what you already saw in the intro bit. Speaker B: So we're moving from this investing 20 or hundreds of thousands of dollars to analog equipment to now being recently just able to couple of thousand to start being your filmmaking career.

8:45 Speaker B: By now just, you know, paying $30 in Synthesia platform and being able to create videos just using your computer with no need for cameras, lights or microphones. Speaker B: Here's an example of how it also makes it very personal. Speaker B: Messi, a famous football player. Speaker B: Here you can use your. Speaker B: You can just by entering some triggers like your name, your friend's name and where you want to see the game together with your friend, you can automatically generate videos that sound completely as it's coming from Messi and your friends won't notice the difference.

9:13 Speaker B: This is unbelievable, how good it is. Speaker F: Hey Stefan, what's up. Speaker F: My friend Sander has invited us to watch the game online. Speaker F: I hope I can make it if I can't be there. Speaker F: Enjoy the game. Speaker F: Ah, and don't forget to bring the snacks. Speaker F: Ciao. Speaker B: It's unreal how good it is, and I'm always amazed when I see examples like that. Speaker B: It can also help you reach a lot more people. Speaker B: In addition to being personal, you can also reach more people by having it available in many more languages.

9:41 Speaker B: Synthesia, for example, supports more than 50 languages. Speaker B: Here's an example of how it can be very powerful. Speaker B: In one of the campaigns that Synthesia did. Speaker B: Malaria isn't just any disease. Speaker B: It's the deadliest disease there's ever been. Speaker B: So imagine the reach that you can drive by being able to speak everybody's language around the world.

10:11 Speaker B: If you want to check out more examples and how all the other big companies that you see here are using their tools and go to Synthesia's website, which I've linked down below in this video. Speaker B: But what are the other use cases that synthetic media allows us to do, which we were not able to do before? Speaker B: A good example of that is the movement of VTubers, you know, where you can actually, rather than you being in the video, you can have a virtual character or your avatar being in the video. Speaker B: And here's example, one setup from Codemico, whose channel is also linked below.

10:41 Speaker C: My facial cam goes right here, okay. Speaker C: And it's an iPhone X. Speaker C: And so this is basically it. Speaker C: I have new fingers on. Speaker C: These are the gloves. Speaker C: See that? Speaker C: Thumbs up, whatever this is. Speaker C: Peace sign, 3, 4, 5. Speaker B: And this is incredible how much you can do with that.

11:07 Speaker B: You can create the whole virtual space, not just the character within that space. Speaker B: And of course, it's much more accessible. Speaker B: While this setup still cost like $10,000 for her, you can use Memojis and Animojis to animate yourself by just using your phone these days, by using the avatars. Speaker B: This has also sparked the start of virtual influencers. Speaker B: A company called Brute, which is in LA, has created Lil Miquella, who has got more than 3 million followers on Instagram.

11:34 Speaker B: By actually being completely virtual, it's also allowed AI companions to come around us. Speaker B: And it's not just, you know, being able to chat with them or talk in a chat environment and then reacting it on the screen, but also you can pick them out and actually place them within your space as a completely augmented reality experience where you can have a virtual friend, a companion that you can talk to anytime, that always Listens to you. Speaker B: It also sparked the start of digital humans by Unique, for example, where they create digital humans, but they also have created some that you can already interact with so you can go on their website and have a conversation with Einstein.

12:11 Speaker B: It's also used widely much more like closer example in learning and development in different companies, you know, to drive down the cost and make their content much more accessible and much more engaging. Speaker B: It's also used much in corporate communications, you know, where you just need to get a message across the whole company or things that move or change constantly. Speaker B: You can have those videos automatically generated or you can just use it, you know, for your own fun. Speaker B: We all know Reface app where you have the ton of templates where you can replace your face in any of the videos just to make them look, you know.

12:53 Speaker B: You know, those are just fun examples. Speaker B: But what are the risks by using those videos or those technologies across all of your video outputs and the use cases that we talked about? Speaker B: Number one certainly is ethics. Speaker B: While the tools are very powerful, it's important to keep people first always. Speaker B: And Synthesia is part of this content authenticity initiative. Speaker B: And I think whenever you're talking about these tools, you make sure that those companies have those ethics and principles in place.

13:19 Speaker B: You can only use the avatars with the person's permission, and they're not going to be shared publicly unless obviously people choose to do that. Speaker B: So ethics are key for those companies who have access to those technologies. Speaker B: But I think what is even more important than ethics for these companies is education and public knowledge. Speaker B: You know, we all got used to seeing costumes and then they're centuries old, from the sixth century before Christ, when we know that the person behind the costume is not the same person that they're acting out to be.

13:48 Speaker B: We're now used to that on TikTok and Instagram filters. Speaker B: You know, we know that's not real. Speaker B: We know that's actually generated by computer or lenses when somebody makes them look younger, older, or has different effects in them. Speaker B: You know, we got used to that in emails, you know, when 1970s when emails started, the first emails were sent, 99% of them were written by humans. Speaker B: While in 2000s, we're used to that receiving. Speaker B: The 99% of emails that we get are actually generated by computers using the name and the personalization and the titles and everything else that goes with an email.

14:18 Speaker B: So I think education is key. Speaker B: And while, you know, those cheaper tools are very accessible to everybody, they still don't look as good. Speaker B: Even the Pro Zoomer tools that don't look as good as something that we go and see in the movies because they take much longer time and much more effort. Speaker B: So there's really good defects that you see out there. Speaker B: Actually, somebody has to pay their time and energy to make those happen. Speaker B: There's usually somebody. Speaker B: Somebody's interest behind that.

14:43 Speaker B: But I think public knowledge is really the key. Speaker B: While public knowledge is key, I think it's going to be less and less easy to distinguish them, even with a higher awareness that those kinds of videos are out there. Speaker B: And I think fingerprinting is such a key area that needs to happen, whether on a device level or an application level where those videos are produced. Speaker B: And then with that, while we have the fingerprint, who's the original author of that video that's always attached and embedded into the file. Speaker B: We should have tools for consumers to then find out who is the owner.

15:10 Speaker B: You know, like in music, I hear a good song, I want to know who was the singer, who was the writer. Speaker B: Same way in YouTube, when you see a video that's using music, you could go out and validate, you know, who is the actual owner and who should claim the revenue from that video. Speaker B: So those tools already exist in music. Speaker B: They need to happen also for video. Speaker B: What does the future look like for video? Speaker B: I'm really excited about this part. Speaker B: We're moving from this idea that somebody looks like you, can act like you, but in the future can also think like you.

15:39 Speaker B: And I think this is a super powerful development. Speaker B: When somebody looks like you, we can make anybody say anything to use their likeness. Speaker B: And here's an example of digital domain Charlatan in the DHG group, we've been. Speaker G: Doing a lot of research, a lot of research on how to create digital humans, digital creatures, digital characters. Speaker B: There really is. Speaker G: There isn't any real way just to turn one person into somebody else. Speaker G: That technology just doesn't exist. Speaker G: We're not able to sort of take somebody's face and immediately just suddenly transform it into my face.

16:09 Speaker B: And there truly that technology does not exist. Speaker B: Never seen that. Speaker B: Of course it exists and it's actually live now. Speaker B: The second thing, to make things act like you. Speaker B: This gives us an ability to make things come alive. Speaker B: And a great example is the Google Lambda where they made paper airplanes and planet Pluto so you can converse with them. Speaker B: They have the knowledge of a Pluto and paper airplane from the world, from the web and so that you can now start having conversation with them.

16:35 Speaker B: Listen to a conversation the team had with Pluto a few days ago. Speaker H: I'm so curious about you, I sense your excitement. Speaker I: Ask me anything. Speaker H: Tell me what I would see if I visited. Speaker I: You would get to see a massive canyon, some frozen icebergs, geysers and some craters. Speaker H: It sounds beautiful. Speaker I: I assure you it is worth the trip.

17:00 Speaker I: However, you need to bring your coat because it gets really cold. Speaker H: I'll keep that in mind. Speaker H: Hey, I was wondering, have you ever had any visitors? Speaker B: And I think this is powerful. Speaker B: If you now put the likeness together with act like. Speaker B: So if you take the knowledge someone's likeness, how they look and how they sound, and then put that together with the knowledge, you get to autonomous digital humans. Speaker B: And this is another example of what Digital Domain is working at, where they created Duck as somebody you can converse with without necessarily knowing what they're going to say, because they're fully driven by their own AI in their looks, in their sound, in their knowledge.

17:35 Speaker B: Would you like to introduce yourself? Speaker J: Hello everyone. Speaker J: I'm an autonomous digital human digital replica of Doug Roble. Speaker J: Digital Domain has been on the forefront of visual effects for over two decades. Speaker J: These effects take thousands of hours and hundreds of skilled artists. Speaker B: But things have changed and things have truly changed. Speaker B: So imagine even in my case like creating a YouTube video without writing or filming.

18:02 Speaker B: Or you could just ask me, hey, can you tell me more about synthetic media? Speaker B: Then it automatically generates a video for you. Speaker B: So even today I can use a writer such as OpenAI GPT3 to write the script for sound. Speaker B: I can use Descript to then make it sound like me or someone like re speech or tool that was used in the recent Mandalorian series from Star Wars. Speaker B: And for camera and editing, you could use Synthesia, you know, a platform that brings it all together where you can add effects and text and then make it actually come out and sound really good.

18:32 Speaker B: So just maybe we are very soon. Speaker B: Of course it's going to take time getting to a place where you can start making Hollywood films with a laptop, which is the vision of for Synthesia company and the CEO as well. Speaker B: So thank you very much for watching. Speaker A: I hope you thank you from me as well. Speaker A: And just one more thing, please let me know down in the comments if you'd like me to create a video that is entirely not created by Sander, but AI. Speaker A: I'll ask GPT3 to write it, Descript to do the audio and Synthesia to film it.

19:02 Speaker A: Thanks and hope to see you next time.