Neural Style Transfer
Upload a photo and a style image — stylize in the browser using fast neural style transfer
Before large language models took over the conversation, computer vision had its own viral moment. Neural style transfer — the ability to repaint a photograph in the visual style of a painting — was one of those results that genuinely surprised people when it came out. The outputs looked like something a human would spend hours doing, and the model did it algorithmically.
How It Works
The original paper by Gatys, Ecker, and Bethge (2015) framed style transfer as an optimization problem. Given a content image and a style image, you iteratively update a third image — starting from noise — until it matches the content structure of the first and the texture statistics of the second. The mechanism is a pretrained convolutional network (VGG-19). Early layers capture local textures; deeper layers capture semantic structure. Style is captured by the Gram matrix of feature activations — a measure of which features co-activate together, stripped of spatial information. Content is captured by the feature activations themselves at a deeper layer.
It worked beautifully. It was also slow — several minutes per image on a GPU, since every stylization required hundreds of gradient steps.
Johnson, Alahi, and Fei-Fei (2016) fixed the speed problem by training a feedforward network to perform the optimization in a single forward pass. The network learns to approximate the output of the iterative process, but at inference it runs in milliseconds. The catch: you train one network per style. Changing styles means a different model.
The final step was arbitrary style transfer (Huang & Belongie, 2017). The key insight was Adaptive Instance Normalization (AdaIN): shift and scale the content image’s feature statistics to match those of the style image, then decode. This made it possible to use any style image at inference time without retraining — which is what this demo does.
This demo runs entirely in your browser using TensorFlow.js and the pretrained Magenta arbitrary style transfer model. No data leaves your device.
Try a painting or texture