Skip to main content

Review and summarization of Deepfake videos course on LinkedIn

Review of Deep Fake videos 

Introduction:

Deepfake videos mean fake videos created with deep learning videos. With the recent wake of deep learning frameworks and researches, it has become way too easy to create very reliable results in generating fake videos using deep learning techniques. In this post, we will go through the content of the LinkedIn course "understanding the impact of deep fake videos" and thereby discuss deep fake technologies.

What are the elements of deep fake?

Deep fake includes two things: deep learning and fake content. A deep fake video is created often using fake voice and original video, or original voice and fake video; or even both fake voice and fake videos. The thing which we are referring as fake; is basically synthetic content, used by deep learning technologies. In deep fake, image to image, speech to speech, text to speech techniques are used to create these synthetic contents. For classic, check this amazing Nixon's speech about moon landing disaster; which Nixon never gave!!

Different categories of deep-fake:

As we already have started digging, there are these categories of deep fake:

(1) fake video with original video and a voice, synthetically generated or morphed by AI using a base of text script or a good impersonation of tones and emotions by a voice artist.

(2) fake video with morphed video and original voices. This is mainly where faces of one person is inserted in someone else's face, and then the resultant videos shows these people doing something they never did. This is where 96% of currently generated deep fake videos are, and these are mainly used in adult industries to create defamatory contents using famous people's face.

(3) fully fake content; using both fake videos and fake voices. The famous Nixon video is an example of that. The video is morphed to provide correct lip sync; and fake voice is generated using Nixon's original speech for tone and voice modulation to morph a specially recorded voice from a voice artist.

What are good things with deep-fake?

With all that hue and cry, probably you have already put deep fake in a bucket of bad stuffs. But no, deep-fake has pretty good achievements and applications also. One of the best use of deep fake is in movie industries. Deep fake can be used to replace high-end CGI work, by using huge amount of other video data or movie data. Some of the possible use cases here are:

(1) changing scripts of actors/actresses to generate new or corrected lines

(2) putting animations and events in a movie; without even have to spend time creating it but by using alternative ample amount of data for the same. 

(3) reanimating famous actors in movies where they are not present/ even reanimation of actors who have passed away

and many others.

One great example of creating reanimation is the deepfake reanimation of dali in dali museum. Check the video about dali museum here.

The technology behind deep-fake:

The basic of deep-fake is basically generating new images/audios from previous present images/audios, based on different context and inputs. Image deep fakes are generated using model architectures like encoder decoder or GANs. In such architecture, there are two sides of the network. One side tries to come up with fake examples from the dataset; while the other side keeps on improving to detect fake ones. The training ends, when the network can't detect the fake images generated by the first half using the second half. So this is one of the reason why deepfakes are so good in quality; as they are built in some way by "fake it till you make it" philosophy.

In the course, we see a mention of two free softwares, named deepface and fakelab and if possible we will try to explore them and how things are done using them in the later part of this post.

Advancement in deepfake detection:

So the course discusses in details of what are the detection techniques being used currently to flag and remove any malicious deepfakes. In 2017-18, deepfakes used to be more obvious and human detectable. You would see some of the following issues with deepfake videos:

But with more recent technologies, much of these are gone; and videos generated are mostly eerily realistic. Yet, there are slight movements between different objects as well as intricate sound features which are not possible for current deepfake techniques to correct.

These and other features are generally used to detect deepfake images and videos nowadays using ai programs. Facebook and other tech giants also have taken significant steps in detecting and removing deepfake videos which are malicious. 

There will be definitely a time when advancements in technology will eliminate such small errors. In such cases, techniques of water-marking are being developed. A unique water mark for an  institute which produces the original videos will not be possible for ai to keep intact while morphing the video; therefore losing the authenticity. Much in this is still developing, but these are some from the more promising side of deepfake detection research. Here is a kaggle challenge of 1 million dollars which took space 8 months ago to detect deepfake. Check it if you are interested.

Conclusion:

The course also involves an interview with an expert in deepfake; which I am not including as it is more of an experience to go through. In short, the course is a good starter if you are interested in deepfake or deepfake detection. I have highlighted some of the key points in the course. Now I recommend you to go through this course if you are interested in the complete content.

Thanks for reading!

Comments

Popular posts from this blog

Mastering SQL for Data Science: Top SQL Interview Questions by Experience Level

Introduction: SQL (Structured Query Language) is a cornerstone of data manipulation and querying in data science. SQL technical rounds are designed to assess a candidate’s ability to work with databases, retrieve, and manipulate data efficiently. This guide provides a comprehensive list of SQL interview questions segmented by experience level—beginner, intermediate, and experienced. For each level, you'll find key questions designed to evaluate the candidate’s proficiency in SQL and their ability to solve data-related problems. The difficulty increases as the experience level rises, and the final section will guide you on how to prepare effectively for these rounds. Beginner (0-2 Years of Experience) At this stage, candidates are expected to know the basics of SQL, common commands, and elementary data manipulation. What is SQL? Explain its importance in data science. Hint: Think about querying, relational databases, and data manipulation. What is the difference between WHERE

What is Bort?

 Introduction: Bort, is the new and more optimized version of BERT; which came out this october from amazon science. I came to know about it today while parsing amazon science's news on facebook about bort. So Bort is the newest addition to the long list of great LM models with extra-ordinary achievements.  Why is Bort important? Bort, is a model of 5.5% effective and 16% total size of the original BERT model; and is 20x faster than BERT, while being able to surpass the BERT model in 20 out of 23 tasks; to quote the abstract of the paper,  ' it obtains performance improvements of between 0 . 3% and 31%, absolute, with respect to BERT-large, on multiple public natural language understanding (NLU) benchmarks. ' So what made this achievement possible? The main idea behind creation of Bort is to go beyond the shallow depth of weight pruning, connection deletion or merely factoring the NN into different matrix factorizations and thus distilling it. While methods like knowle

Spacy errors and their solutions

 Introduction: There are a bunch of errors in spacy, which never makes sense until you get to the depth of it. In this post, we will analyze the attribute error E046 and why it occurs. (1) AttributeError: [E046] Can't retrieve unregistered extension attribute 'tag_name'. Did you forget to call the set_extension method? Let's first understand what the error means on superficial level. There is a tag_name extension in your code. i.e. from a doc object, probably you are calling doc._.tag_name. But spacy suggests to you that probably you forgot to call the set_extension method. So what to do from here? The problem in hand is that your extension is not created where it should have been created. Now in general this means that your pipeline is incorrect at some level.  So how should you solve it? Look into the pipeline of your spacy language object. Chances are that the pipeline component which creates the extension is not included in the pipeline. To check the pipe eleme