{"id":238,"date":"2019-06-16T08:50:34","date_gmt":"2019-06-16T15:50:34","guid":{"rendered":"http:\/\/codingrestart.com\/home\/?p=238"},"modified":"2019-06-17T15:44:50","modified_gmt":"2019-06-17T22:44:50","slug":"mona-lisas-video","status":"publish","type":"post","link":"https:\/\/codingrestart.com\/home\/mona-lisas-video\/","title":{"rendered":"Mona Lisa’s video"},"content":{"rendered":"\n

For centuries, people have wondered about Mona Lisa’s smile. Now they can stop wondering and just watch her videos.<\/p>\n\n\n\n

A group of AI researchers published a paper titled “Few-Shot Adversarial Learning of Realistic Neural Talking Head Models<\/a>“, where they describe a new algorithm to generate videos of peoples’ heads (talking heads models). Methods to produce talking heads models using generative adversarial networks (GAN) were already published previously. GANs are essentially two neural networks combined into one system, where one NN is trained to produce samples and the second NN is trained to identify good examples. <\/p>\n\n\n\n

However, the existing methods using GANs required long videos or large sets of photographs of each talking head to train GANs. The existing methods used various warping techniques; for an overview read the introduction in the “Facial Animation System Based on Image Warping Algorithm<\/a>” study. <\/p>\n\n\n\n

The above paper describes a new way of producing the talking heads using just a few training examples, possibly only a single photograph. Instead of warping, a direct synthesizing method is used. This method is called few-shot learning and relies on pre-trained models that were trained using a large number of videos of various people in different situations. In those models, a critical part of the training relies on the identification of face landmarks, like eyes, nose, mouth, and chin.<\/p>\n\n\n\n

The results of the new research are summarized in a 5 minutes video<\/a> that shows how the properly trained GAN can produce short talking head videos from still images. A talking head created from the Mona Lisa painting was particularly impressive because it was trained on several human models and the differences in three facial expressions are easily recognizable. The process of video synthesis of a certain person based on the face landmarks of a different person is called puppeteering. <\/p>\n\n\n\n

\"\"\/
Mona Lisa trained using three different training videos to give her three distinct personalities. Courtesy of Zakharov at al.<\/figcaption><\/figure>\n\n\n\n
\n\n\n\n

Talking head videos could be combined with the latest NLP improvements that I described in an earlier post<\/a>. This would create highly realistic fake videos and text. If you were concerned about the proliferation of deepfakes<\/a> before reading this post, this will only heighten your fears. And how are the authors of the above described few-shot adversarial learning algorithm responding to those concerns? The statement below their YouTube video that I linked above states: “Shifting a part of human life-like communication to the virtual and augmented worlds will have several positive effects. It will lead to a reduction in long-distance travel and short-distance commute. It will democratize education, and improve the quality of life for people with disabilities.” Noble enough. But considering that researchers are from Russia and Russia’s proven track of meddling in recent US\/EU elections, it is not far-fetched to assume that high-quality deepfakes will be common soon. <\/p>\n\n\n\n

How soon? Let’s look at the progress of images<\/a> generated by GANs over the past five years. <\/p>\n\n\n\n

\"\"
GANs results throughout the years. Please note that none of the above images is a real person. Courtesy of Gidi Shperber.<\/figcaption><\/figure>\n\n\n\n

I’ll let you extrapolate the progress into the future.<\/p>\n","protected":false},"excerpt":{"rendered":"

For centuries, people have wondered about Mona Lisa’s smile. Now they can stop wondering and just watch her videos. A group of AI researchers published a paper titled “Few-Shot Adversarial Learning of Realistic Neural Talking Head Models“, where they describe a new algorithm to generate videos of peoples’ heads (talking heads models). Methods to produce … Continue reading “Mona Lisa’s video”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[4],"tags":[],"class_list":["post-238","post","type-post","status-publish","format-standard","hentry","category-machine-learning"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/paAAlH-3Q","jetpack-related-posts":[{"id":69,"url":"https:\/\/codingrestart.com\/home\/ml-msdn-and-xavier\/","url_meta":{"origin":238,"position":0},"title":"ML: MSDN and Xavier","author":"Viktor Sanek","date":"January 19, 2019","format":false,"excerpt":"Microsoft\u2019s MSDN magazine for its development community has been publishing quite a few introductory articles to Machine Learning (ML) over the past few months. January's issue emphasizes ML with another series of articles, albeit with differing qualities. I liked this quote from the editorial \"Advancing AI\": \u201dML is a huge\u2026","rel":"","context":"In "Machine Learning"","block_context":{"text":"Machine Learning","link":"https:\/\/codingrestart.com\/home\/category\/machine-learning\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":233,"url":"https:\/\/codingrestart.com\/home\/dangers-of-nlp\/","url_meta":{"origin":238,"position":1},"title":"Dangers of NLP","author":"Viktor Sanek","date":"June 9, 2019","format":false,"excerpt":"Natural language processing (NLP) continues its rapid advance, leading some people to fear its latest results. The research organization OpenAI published a blog post titled \"Better Language Models and Their Implications\" summarizing its progress on \"predicting the next word, given all of the previous words within some text\". OpenAI calls\u2026","rel":"","context":"In "Machine Learning"","block_context":{"text":"Machine Learning","link":"https:\/\/codingrestart.com\/home\/category\/machine-learning\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/codingrestart.com\/wp-content\/uploads\/2019\/06\/GPT-2.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/codingrestart.com\/wp-content\/uploads\/2019\/06\/GPT-2.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/codingrestart.com\/wp-content\/uploads\/2019\/06\/GPT-2.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/codingrestart.com\/wp-content\/uploads\/2019\/06\/GPT-2.png?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":88,"url":"https:\/\/codingrestart.com\/home\/webassembly\/","url_meta":{"origin":238,"position":2},"title":"WebAssembly","author":"Viktor Sanek","date":"February 3, 2019","format":false,"excerpt":"Until recently, JavaScript was the only language supported by all web browsers. But it was difficult to compile it efficiently and the JavaScript applications typically run much slower than native applications. Then, programmers from the four main browser vendors designed a new language, sort of a machine code for the\u2026","rel":"","context":"In "Web"","block_context":{"text":"Web","link":"https:\/\/codingrestart.com\/home\/category\/web\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/codingrestart.com\/wp-content\/uploads\/2019\/01\/WASM.png?resize=350%2C200&ssl=1","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/codingrestart.com\/wp-content\/uploads\/2019\/01\/WASM.png?resize=350%2C200&ssl=1 1x, https:\/\/i0.wp.com\/codingrestart.com\/wp-content\/uploads\/2019\/01\/WASM.png?resize=525%2C300&ssl=1 1.5x, https:\/\/i0.wp.com\/codingrestart.com\/wp-content\/uploads\/2019\/01\/WASM.png?resize=700%2C400&ssl=1 2x"},"classes":[]},{"id":51,"url":"https:\/\/codingrestart.com\/home\/really-old-geek\/","url_meta":{"origin":238,"position":3},"title":"Really Old Geek","author":"Viktor Sanek","date":"December 28, 2018","format":false,"excerpt":"The New York Times published an article about Donald Knuth \u201cThe Yoda of Silicon Valley\u201d. The article might idolize his work and impact, but provides a great overview of his work. Mr. Knuth is a living legend in the field of computer science, known mainly for his seminal book on\u2026","rel":"","context":"In "C\/C++"","block_context":{"text":"C\/C++","link":"https:\/\/codingrestart.com\/home\/category\/uncategorized\/cc\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":171,"url":"https:\/\/codingrestart.com\/home\/theory-behind-anki\/","url_meta":{"origin":238,"position":4},"title":"Theory Behind Anki","author":"Viktor Sanek","date":"April 23, 2019","format":false,"excerpt":"This post covers the theory behind Anki and follows the previous post introducing Anki. The last post in this series describes best practices for using Anki. One of the best meta-analyses reviewing best approaches to learning is Improving Students\u2019 Learning With Effective Learning Techniques by Dunlosky et al. If I\u2026","rel":"","context":"In "Anki"","block_context":{"text":"Anki","link":"https:\/\/codingrestart.com\/home\/category\/anki\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":180,"url":"https:\/\/codingrestart.com\/home\/anki-best-practices\/","url_meta":{"origin":238,"position":5},"title":"Anki: Best Practices","author":"Viktor Sanek","date":"April 28, 2019","format":false,"excerpt":"This post concludes the series of posts on Anki and follows the post introducing Anki and another one about the theory behind Anki. I have been using Anki for five years and achieved high consistency in completing daily reviews. Here are my best practices: Be selective about cards you create\u2026","rel":"","context":"In "Anki"","block_context":{"text":"Anki","link":"https:\/\/codingrestart.com\/home\/category\/anki\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/codingrestart.com\/home\/wp-json\/wp\/v2\/posts\/238","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/codingrestart.com\/home\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/codingrestart.com\/home\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/codingrestart.com\/home\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/codingrestart.com\/home\/wp-json\/wp\/v2\/comments?post=238"}],"version-history":[{"count":11,"href":"https:\/\/codingrestart.com\/home\/wp-json\/wp\/v2\/posts\/238\/revisions"}],"predecessor-version":[{"id":263,"href":"https:\/\/codingrestart.com\/home\/wp-json\/wp\/v2\/posts\/238\/revisions\/263"}],"wp:attachment":[{"href":"https:\/\/codingrestart.com\/home\/wp-json\/wp\/v2\/media?parent=238"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/codingrestart.com\/home\/wp-json\/wp\/v2\/categories?post=238"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/codingrestart.com\/home\/wp-json\/wp\/v2\/tags?post=238"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}