2307.15860 What can Discriminator do? Towards Box-free Ownership Verification of Generative Adversarial Network

What can Discriminator do? Towards Box-free Ownership Verification of Generative Adversarial Network §

Authors: Ziheng Huang, Boheng Li, Yan Cai, Run Wang, Shangwei Guo, Liming Fang, Jing Chen, Lina Wang

Abstract: In recent decades, Generative Adversarial Network (GAN) and its variants have achieved unprecedented success in image synthesis. However, well-trained GANs are under the threat of illegal steal or leakage. The prior studies on remote ownership verification assume a black-box setting where the defender can query the suspicious model with specific inputs, which we identify is not enough for generation tasks. To this end, in this paper, we propose a novel IP protection scheme for GANs where ownership verification can be done by checking outputs only, without choosing the inputs (i.e., box-free setting). Specifically, we make use of the unexploited potential of the discriminator to learn a hypersphere that captures the unique distribution learned by the paired generator. Extensive evaluations on two popular GAN tasks and more than 10 GAN architectures demonstrate our proposed scheme to effectively verify the ownership. Our proposed scheme shown to be immune to popular input-based removal attacks and robust against other existing attacks. The source code and models are available at gan_ownership_verification

What, Why and How §

Here is a summary of the key points in this paper:

What:

• The paper proposes a new method for verifying ownership of GAN models in a “box-free” setting, where the defender cannot choose specific inputs.
• The key idea is to leverage the discriminator’s ability to capture the unique distribution learned by the paired generator. This allows verifying ownership without relying on special inputs.

Why:

• Existing black-box verification methods rely on carefully chosen inputs to trigger hidden behaviors. This is not possible in unconditional generation tasks like image synthesis.
• Input-based methods are also vulnerable to removal attacks that transform inputs to avoid the hidden behaviors.

How:

• Add a Pearson correlation loss to preserve useful representations in the discriminator and prevent it from converging trivially.
• Use the discriminator to learn a hypersphere enclosing the distribution of the paired generator.
• At test time, measure proximity of outputs to the hypersphere center to verify ownership, without needing special inputs.

In summary, the paper proposes a novel box-free verification method for GANs by exploiting the discriminator’s distribution capturing ability. This removes the need for special inputs, making it robust against input manipulation attacks. Ownership is verified by measuring proximity to a hypersphere learned using the discriminator.

Main Contributions §

Here are the main contributions of this paper:

• They identify a limitation of existing black-box verification methods for GANs - the inability to specify inputs in unconditional generation tasks like image synthesis.

• They reveal the potential of leveraging the discriminator’s learned representations to capture the unique distribution of the paired generator. This insight enables box-free verification without relying on extra models.

• They propose a training method that adds Pearson correlation loss to the discriminator to preserve useful representations and prevent trivial convergence.

• They introduce a compactness loss to learn a hypersphere in the discriminator’s embedding space that encloses the generator’s distribution.

• They demonstrate the effectiveness of their proposed verification scheme on two GAN tasks and over 10 GAN architectures. It can accurately verify ownership while being robust to various attacks.

• They show qualitatively and quantitatively that their method is immune to input manipulation attacks like image transformations, since no special inputs are required.

• They demonstrate immunity against ambiguity attacks where the attacker tries to replicate the verification system, due to the uniqueness of the discriminator’s learned representations.

In summary, the key contributions are proposing a novel box-free verification framework for GANs by exploiting the discriminator, demonstrating its effectiveness and robustness against various attacks, and identifying limitations of existing input-based verification methods.

Method Section §

Here is a summary of the method section of the paper:

• They first formalize the threat model with a defender (model owner) and an adversary (who steals the model). The defender can query the model but not specify inputs (box-free setting).

• They propose a compactness loss to learn a hypersphere enclosing the generator’s distribution in the discriminator’s embedding space. This captures the unique distribution.

• They identify the issue of discriminator convergence and propose a Pearson correlation loss to preserve useful representations by reconstructing the latent code.

• The training pipeline first trains the GAN normally plus the Pearson loss. Then the compactness loss is used to learn the hypersphere using the discriminator and generated samples.

• At test time, they measure proximity of outputs to hypersphere center to verify ownership. No special inputs needed.

• For supervised GANs like image-to-image translation, the additional Pearson loss is not needed since the discriminator has other tasks.

• The discriminator plays a similar role to private keys and should be kept secret. Attacks requiring the discriminator are not considered.

In summary, they identify the problem of trivial convergence of the discriminator, propose an additional Pearson loss to maintain useful representations, then use the discriminator to learn a hypersphere capturing the unique distribution for box-free verification at test time. The method is tailored for unsupervised and supervised GAN settings.

High-Level Pseudo Code §

Here is the high-level pseudo code for the method proposed in this paper:

# Step 1: Redefine training objective
# Add Pearson correlation loss to GAN objective
 
loss_G = GAN_loss(G,D) - lambda * pearson_corr(z, D(G(z)))  
loss_D = GAN_loss(D) - lambda * pearson_corr(z, D(G(z)))
 
# Step 2: Train GAN normally plus Pearson loss
for i in range(epochs):
    train_step(G, D, loss_G, loss_D) 
 
# Step 3: Learn hypersphere with discriminator
samples = [G(z) for z in latent_samples] 
 
# Initialize hypersphere center c and radius R
loss_c = R**2 + 1/n sum_i max(0, ||D(x_i) - c||^2 - R^2)  
 
# Alternatingly optimize network and radius
for i in iterations:
    if i % k == 0: 
        R = line_search(loss_c)  
    else:
        update_D(loss_c) # SGD step
        
# Result: G, D, c, R        

In summary, the key steps are:

1. Add Pearson loss to GAN objectives to maintain discriminator representations
2. Train GAN normally
3. Use discriminator and compactness loss to learn hypersphere
4. Alternate between optimizing network and radius

The result is the generator, discriminator, hypersphere center and radius that can be used for box-free verification.

Detailed Pseudo Code §

Here is a more detailed pseudo code to implement the method proposed in this paper:

# Hyperparameters
lambda, nu, k, lr, lr' 
 
# Step 1: Redefine objectives
def pearson_corr(z, z_hat):
   # Pearson correlation calculation
   ...
 
def G_loss(G,D):
   return BCE(D(G(z)), 1) - lambda * pearson_corr(z, D(G(z)))
 
def D_loss(G,D):
   return BCE(D(x), 1) - lambda * pearson_corr(z, D(G(z))) 
 
# Step 2: Train GAN 
G, D = initialize_GAN() 
 
for i in range(epochs):
   # Train discriminator
   x, z = get_batch()
   l_D = D_loss(G, D)  
   update_D(l_D)
 
   # Train generator  
   z = sample_latent()
   l_G = G_loss(G, D)
   update_G(l_G)
 
# Step 3: Learn hypersphere
samples = [G(z) for z in sample_latent(n)]
c = mean(D(samples)) # Center
R = initialize_radius() 
 
for i in iterations:
   x = sample(samples)
   l_c = R**2 + 1/nu*n * sum_i max(0, ||D(x_i)-c||^2 - R^2)
   
   if i % k == 0:
      R = line_search(l_c) 
   else:
      update_D(l_c, lr')
 
return G, D, c, R

This shows more implementation details like hyperparameter settings, loss calculations, optimizations, etc. The overall pipeline is similar to the high-level version.