Distributional Robustness with IPMs and links to Regularization and GANs
Authors: Hisham Husain
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | Our contributions come in three Theorems, where the first two concern DRO with IPMs (Section 3) and the third is an extension to understanding GANs (Section 4): (Theorem 1) An identity for distributional robustness using uncertainty sets induced by any IPM. Our result tells us that this is exactly equal to regularization with a penalty ΛF. We show that this penalty can be upper bounded by another penalty ΘF which recovers existing work when the IPM is set to the MMD and Wasserstein distance, tightening these results. Since our result holds in much more generality, we derive penalties for other IPMs such as the Total Variation, Fisher IPM, and Sobelov IPM, and draw connections to existing methods. (Theorem 2) A necessary and sufficient condition under which the penalties ΛF and ΘF coincide. It turns out this condition is linked to regularized binary classification and is related to critic losses appearing in penalty-based GANs. This allows us to give positive results for work in this direction, along with drawing a link between regularized binary classification and distributional robustness. (Theorem 3) A result that characterizes the distributional robustness of the f-GAN objective showing that the discriminator set plays an important part for the robustness of a GAN. This is, to the best of our knowledge, the first result on divergence-based distributional robustness of f-GANs. Our result allows us to provide a novel perspective for several existing penalty-based GAN methods such as Wasserstein-, MMD-, and Sobelov-GANs. |
| Researcher Affiliation | Collaboration | Hisham Husain The Australian National University & Data61 hisham.husain@anu.edu.au |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described. |
| Open Datasets | No | The paper is theoretical and does not use or reference specific datasets for training. |
| Dataset Splits | No | The paper is theoretical and does not provide specific dataset split information. |
| Hardware Specification | No | The paper is theoretical and does not provide specific hardware details used for running experiments. |
| Software Dependencies | No | The paper is theoretical and does not provide specific ancillary software details with version numbers. |
| Experiment Setup | No | The paper is theoretical and does not contain specific experimental setup details such as hyperparameter values or training configurations. |