How We Made Your AI-Generated Lover With A GAN
And you can find your AI-Generated Lover too!
BuzzFeed is all about exploring new ideas for content, no matter how outside the box. Last December, BuzzFeed CEO Jonah Peretti posted an ambitious idea into the company’s Slack (paraphrased):
It could be fun to create a quiz called ‘Who’s Your Future Partner?’ where the BuzzFeed user inputs what they want in a partner, and the result is the image of a fake person with the caption ‘Congrats, hope you have a great life together!’
Always up for a challenge, a number of BuzzFeed writers, engineers, and myself (as a data scientist) decided to experiment if this more advanced quiz format was feasible. It felt like an opportunity to learn something new and create an even more engaging experience for our audience.
Besides, I knew how to best generate fake images. And I couldn’t pass up the opportunity to play with GANs!
What are GANs?
A GAN, a generative adversarial network, is nowadays the most popular way to generate believable fake images. I won’t go into detail on how GANs work as there are many, many other articles that do it better, but the tl;dr is that they are a neural network architecture that simultaneously trains both a fake image generator and a discriminator to tell real input images from fakes ones. As the GAN trains, fake images become progressively more “real” over time. We can then use the generator to create fake images for fun!
The process isn’t perfect though. GANs can occasionally go off the rails and include hilarious artifacts. The Twitter account Normal Cat Pics is a good novelty account illustrating generated cats from a GAN trained on cats:
However, good GANs are very computationally expensive to train, and are still very much an evolving area of research. The most notable breakthrough was StyleGAN by NVIDIA. The pretrained weights for that model have been released for anyone to use. The most notable demo of StyleGAN is thispersondoesnotexist.com, which has a self-explanatory URL and simply shows a StyleGAN-generated person, no frills. But it’s effective: how would you tell that is not a real person?
Generating Images Using a GAN
The hard part about using a GAN is setting up a system capable of running it. Larger GANs like StyleGAN require an NVIDIA GPU to generate images in a reasonable timeframe, CUDA to interface with the GPU, and a deep learning framework like TensorFlow or PyTorch to construct the model. It can be time-consuming and expensive to set up by hand.
Fortunately, Google released Google Colab, which allows anyone to instantly set up a free virtual machine with a compatible GPU + deep learning framework with just a click, then share those notebooks with others who can tweak it if necessary.
Recently, NVIDIA released StyleGAN2 ADA, which further improves StyleGAN architecture and solves some artifact issues from the generated images using adaptive discriminator augmentation. This network is trained on a dataset of high-quality faces from Flickr. This Colab Notebook by Jeff Heaton at Washington University includes the code and boilerplate necessary to run StyleGAN2 ADA, and you can run it too! The notebook also contains more detailed information about how GANs work.
The latent vector is a 512-element vector of random noise that is passed to StyleGAN2, with a specified seed used to make the random noise reproducible. For example, here’s what you pass a latent vector generated with a seed of 1234:
Pretty humanlike, isn’t it? The StyleGAN2 images are so high quality I actually had to shrink and compress them before saving to avoid using too much storage. (You can do that in Colab by installing and using imagemagick and pngquant respectively.)
After generating and presenting proof-of-concept AI-generated images to the team, I got the go-ahead to run some more experiments with StyleGAN2.
Iterating on the GAN Results
Because GANs are computationally expensive, most websites that use large GANs like This Person Does Not Exist precompute thousands of images instead of generating them in real-time. We ended up doing that as well, but for other reasons I hadn’t expected when starting this project.
There were two issues immediately apparent: the StyleGAN had a tendency to output images of caucasian faces with relatively few images corresponding to racial minorities (this is, unfortunately, a consequence of the inherent bias of the input Flickr data), and StyleGAN2 had the possibility to generate images of children, which is a massive problem when you are creating a dating quiz.
A team member created a mini-webapp to allow a panel of diverse human BuzzFeeders to classify a batch of generated images with their presented gender, age bucket, whether the image is problematic and should not be used, and whether the image is good or an artifacted trainwreck.
Another problem is that StyleGAN2 is too good and didn’t create enough trainwrecks. Trainwreck images of half-rendered or fake-looking faces are funny, especially when the quiz is supposed to generate an ideal romantic partner.
In the end, several thousand GAN-generated images were classified. The logic was that if a given user takes a quiz more than once (which we expected), the odds of them getting the same random AI-generated lover from the pre-generated set had to be very low.
AI-Generated Flirting
Another aspect of the AI-generated lover is how they would introduce themselves (a la a dating app) and how they describe themselves. For those who know me outside my work at BuzzFeed, I specialize in AI text generation and have written open-source tools to help make it easier for everyone.
Unfortunately, my attempts at training an AI to do this went poorly, plus we would have similar real-time generation issues as noted above with StyleGAN.
We decided to compromise, and BuzzFeed staff created templates of flirty text to capture the tone and sentiment of many, many different types of theoretical people. The templates were populated randomly from a spreadsheet of prewritten possible answers and became part of the user’s quiz results. This more controlled approach also avoided risking the AI-generated model outputting incoherent or grossly offensive, uncurated output.
Although some argue that powerful AI text generation models such as GPT-3 could replace the writers on a site like BuzzFeed, we’re confident that won’t happen. Our editorial team’s unique voice and perspective is essential, and will only continue to be more effective and powerful when layered with AI to yield new innovative formats. Note: Even GPT-3 has issues with pickup lines!
Results
After the quiz went up on a Friday morning, we promoted it via our usual social media channels and it became a featured Twitter Moment. The main BuzzFeed Twitter account tweeted a link to the quiz, asking for others to reply with their own AI-generated lovers:
…and indeed they did, by posting screenshots along with their reactions to their new life partner.
In the end, we received over 1 million views of the quiz.
User reaction was overwhelmingly positive, both in response to the generated text and generated images. The tweaks and constraints for both images and texts worked out and created a super engaging experience for our audience!
What’s Next For AI and BuzzFeed
In the few months since the original quiz was published, research into GANs has yielded exciting projects, including being able to generate an image from text alone, and even editing existing images to style them according to a text prompt!
This experiment was an effective illustration of BuzzFeed’s scrappy creative culture that enabled us to move quickly, think critically, and collaborate across multiple teams to innovate. We took a signature BuzzFeed format and powered it with new technology to create an enhanced experience for our audience.
Given the very positive feedback to the quiz, we’re eager to expand new ideas using AI in the creative process. We’d also like to experiment with building our own models or work with specialized partners. We’re always testing and experimenting with new content formats at BuzzFeed, so keep an eye on this space for what we’ll pioneer next!
This work has been a collective effort across BuzzFeed Tech and enables us to explore many new and exciting data-driven initiatives! If you’d like to join us, BuzzFeed Tech is hiring! To browse openings, check out buzzfeed.com/jobs.
You can also follow us on Twitter @buzzfeedexp!
