Clip knows artist names

An interesting thing I learned today about CLIP is that it knows the styles of different artists! Here are three studying wizards, all from the same random seed, but guided with the names of the artists James Gurney, Paul Kidby, and Jean “Moebius” Giraud (respectively):

It seems like images that imitate painters are much more coherent than images that try to imitate photographs or renders, because people photograph things from every angle but only paint them from the side, so it knows what perspective to use, and doesn't end up in conflict.
— Ryan Moulton (@moultano) July 21, 2021

A lone figure in a dark hat and coat sits at a desk in a dimly lit, cluttered room filled with stacks of books and papers, under a glowing blue and gold emblem on the wall.

In a grand, dimly lit library with towering shelves and glowing candles, a figure in a blue hooded cloak sits on a chair reading, while another person in a brown cloak is seated further back, also engrossed in a book.

A young wizard in a blue robe and pointed hat sits in a grand, dimly lit library, surrounded by towering bookshelves and stone pillars, with a glowing lantern nearby.

This is a great example of the compositionality of language. Style transfer has been done before, but you would need an example image for the style. Here CLIP has conflated knowledge of multiple works by each artist into a general knowledge of their aesthetic. I changed 3 words!!

View original