| Colorization Of Early Films Good or Bad? |
| Written by David Conrad | |||
| Sunday, 17 January 2021 | |||
|
Here's another example of the use of AI for colorizing and breathing new life into photos, or rather in this case early film footage. This one goes an extra step - adding sound effects. While the results are impressive they are also controversial. Does this treatment bring the past alive for us in an authentic way or introduce a false reality? Denis Shiryaev is the founder of Gdansk-based Neural Love, a company that offers a variety of AI-based services for films and photographs, including colorization encompassing noise removal, stabilization, damage inpainting, etc, together with motion smooth and resolution upscaling. His latest project, shared on his YouTube channel, applies the techniques to 21 snippets of footage filmed by the Lumière brothers at the turn of the 20th century. Before revealing the results, however, Shiryaev explains all the processes used and tries to allay the criticisms he has come to expect from the reception given to his previous efforts: The warning he gives at the very start is that his use of AI technique is not, nor is it intended to be, historically accurate, explaining: Neural networks can only guess colors when we’re doing colorization, or new pixels when we’re doing upscale, and faces when working with some facial restoration. It took a few months to process the footage and add ambient sound and Shiryaev outlines the following steps.
By upscaling every frame to 4k and then generating four times more frames per film, Neural Love achieves 60 frames per second playback. Films are colorized using DeOldify, an open source project which we covered back in 2018 and for facial restoration Shiryaev says he incorporates: "gentle" facial enhancements using a GAN algorithm that "slightly" enhances all the faces and makes them more "detailed" by synthesizing facial details. In the video Shiryaev says: After the images were perfect, I spent almost two weeks adding sound to each film, because.... I can so why not. The sounds are not real, and sometimes I did a terrible job. But I really like this ambient effect when, for example, you can see and hear a dog starting to bark on screen and experience the moment as if you were there. So will the historians who have previously criticized Shiryaev for colorizing black and white photographs be appeased? Somehow I doubt it. If you accept the argument made by Emily Mark-FitzGerald, Associate Professor at University College Dublin's School of Art History and Cultural Policy that: "The problem with colourization is it leads people to just think about photographs as a kind of uncomplicated window onto the past, and that's not what photographs are." then you will probably see adding sound as an even more heinous sin. Another historian, Luke McKernan, lead curator of news and moving images at the British Library, writing with reference to Peter Jackson’s 2018 World War One documentary They Shall Not Grow Old, which upscaled and colourized footage from the Western Front, stated: “Colourization does not bring us closer to the past; it increases the gap between now and then. It does not enable immediacy; it creates difference.” However he takes a different attitude to the work done by Neural Love commenting: I rather admire this work, which is reasonably honest about how it has treated the originals, and is fun. Whether this extends to the latest work, with its sound effects remains to be be seen.
More InformationRelated ArticlesDeep Learning Restores Time-Ravaged Photos Deep Angel-The AI of Future Media Manipulation To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin. 
Comments
or email your comment to: comments@i-programmer.info |
|||
| Last Updated ( Sunday, 17 January 2021 ) |
