Generative Adversarial Networks

Overview

A GAN (Goodfellow et al. 2014) trains two networks against each other: a generator maps a random latent code to a sample, and a discriminator learns to tell real samples from generated ones. The generator wins when its samples become indistinguishable from the data. Rasmah’s gan_fit alternates the two with Adam (minimax training), and gan_sample draws new samples from the trained generator.

using Rasmah
using Random
using LinearAlgebra

Learn to draw a circle

Train a GAN on 2-D points on the unit circle, then check that its samples land near radius 1.

θt = range(0, 2π; length = 200)
X = vcat(cos.(θt)', sin.(θt)')                    # data: the unit circle

g = GAN(MLP([2, 16, 2]), MLP([2, 16, 1]), 2)
θ0 = vcat(he_init(MersenneTwister(0), [2, 16, 2]), he_init(MersenneTwister(1), [2, 16, 1]))
θ = gan_fit(g, θ0, X; epochs = 1500, lr = 1e-3)

147-element Vector{Float64}:
 -0.6696770829458729
 -0.13676611714363335
 -0.08584032385668486
  0.16976210266679
 -0.6938372206461297
  0.20742852755584656
 -1.1015463042058837
 -0.9019810563747482
  0.5907847564189395
 -1.31394179445029
  ⋮
  0.2709306871987002
 -0.7487607324931014
 -0.6133652616425497
  0.21056276485051584
 -0.5199406115356021
  0.39778401162843263
 -0.18186273841032857
 -0.13473740178644217
  0.12007636414494036

Verification — the samples match the data

Draw fresh samples and check their radius concentrates around 1:

samples = [gan_sample(g, θ, MersenneTwister(k)) for k in 1:200]
r = [norm(s) for s in samples]
println("mean sample radius = ", round(sum(r) / length(r); digits = 3), "  (target 1.0)")
println("all samples finite: ", all(isfinite, r))

mean sample radius = 1.1  (target 1.0)
all samples finite: true

Next steps

• Generative design — VAEs and diffusion models.
• Neural networks — the MLP core underneath.