LipSim: A Provably Robust Perceptual Similarity Metric
Authors: Sara Ghazanfari, Alexandre Araujo, Prashanth Krishnamurthy, Farshad Khorrami, Siddharth Garg
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, a comprehensive set of experiments shows the performance of Lip Sim in terms of natural and certified scores and on the image retrieval application. |
| Researcher Affiliation | Academia | Sara Ghazanfari , Alexandre Araujo Prashanth Krishnamurthy, Farshad Khorrami, Siddharth Garg Department of Electronic and Computer Engineering New York University sg7457@nyu.edu |
| Pseudocode | No | The paper describes methods textually and with mathematical equations but does not include formal pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide an explicit statement or link for the open-sourcing of its code. |
| Open Datasets | Yes | To fine-tune the Dream Sim distance metric, the NIGHT dataset is used which provides two variations, x0 and x1 for a reference image x, and a label y that is based on human judgments about which variation is more similar to the reference image x (some instances and more explanation of the NIGHT dataset are deferred to Appendix B). By combining the capabilities of Dream Sim with the provable guarantees of a Lipschitz network, our approach paves the way for a certifiably robust perceptual similarity metric. Finally, we demonstrate good natural accuracy and state-of-the-art certified robustness on the NIGHT dataset. |
| Dataset Splits | No | The paper refers to using specific datasets for training and fine-tuning but does not specify the exact training, validation, and test splits (e.g., percentages or sample counts) used for its experiments. |
| Hardware Specification | No | The paper does not specify the exact hardware (e.g., GPU models, CPU types) used for running its experiments. |
| Software Dependencies | No | The paper does not provide specific version numbers for software dependencies or libraries used in the experiments. |
| Experiment Setup | Yes | However, while Dream Sim has used a margin parameter of 0.05, we used a margin parameter of 0.5 for fine-tuning Lip Sim in order to boost the robustness of the metric. Remarkably, Lip Sim achieves strong robustness using a 1-Lipschitz pipeline composed of a 1-Lipschitz feature extractor and a projection to the unit ℓ2 ball that guarantees the 1-Lipschitzness of cosine distance. To evaluate the performance of Lip Sim and compare its performance against other perceptual metrics, we report empirical and certified results of Lip Sim for different settings. |