Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Unveiling Transformer Perception by Exploring Input Manifolds
Authors: Alessandro Benfenati, Alfio Ferrara, Alessio Marta, Davide Riva, Elisabetta Rocchetti
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In Section 4, we empirically investigate the effectiveness and applicability on Transformer models on textual and visual data. Experiments are conducted on textual and visual data and are aimed at two objectives: (i) obtaining an empirical verification of the behavior of Si MEC and Si MExp under diverse settings, and (ii) verifying the consistency of interpretation outputs with the ones from exploration only, in order to test their usability as alternative input data. |
| Researcher Affiliation | Academia | Alessandro Benfenati1a Alfio Ferrara1b Alessio Marta1c Davide Riva2d Elisabetta Rocchetti1b a Department of Environmental Science and Policy, b Department of Computer Science, c Department of Mathematics, d Department of Control and Computer Engineering 1Universita degli Studi di Milano, 2Politecnico di Torino EMAIL EMAIL {davide.riva}@polito.it Corresponding author |
| Pseudocode | Yes | Algorithm 1 The Singular Metric Equivalence Class (Si MEC) algorithm. 1: Set the network N; choose the number of iterations K. Choose the input x(0). 2: for k = 0, 1, . . . , K 1 do 3: Compute gn N (x(k)) 4: Compute the pullback metric g0 x(k) 5: Diagonalize g0 x(k) and find the eigenvectors {vl}l L0 associated to the zero eigenvalue ̸0 6: Randomly select v {vl}l L0 7: ̣(k) = η q minj:̸j =0 |̸j|/ maxj |̸j| 8: x(k+1) x(k) + ̣(k) v 9: end for 10: Project x(k+1) to the feasible region X Algorithm 2 The Singular Metric Exploration (Si MExp) algorithm. 1: Set the network N; choose the number of iterations K. Choose the input x(0). 2: for k = 0, 1, . . . , K 1 do 3: Compute gn N (x(k)) 4: Compute the pullback metric g0 x(k) 5: Diagonalize g0 x(k) and find the eigenvectors {wl}l L+ associated to eigenvalues ̸l = 0 6: Randomly select w {wl}l L+ 7: ̣(k) = η q minj:̸j =0 |̸j|/ maxj |̸j| 8: x(k+1) x(k) + ̣(k) w 9: end for 10: Project x(k+1) to the feasible region X Algorithm 3 Interpretation for Exploration results for Vi T and BERT models. |
| Open Source Code | Yes | The code to reproduce our experiments can be found here:https://github.com/alessiomarta/ transformers_equivalence_classes. |
| Open Datasets | Yes | In each data modality, we experiment with two datasets presenting different features: (i) MNIST [13], a grayscale digit image dataset; (ii) CIFAR10 [11], a RGB object image dataset; (iii) Wino Bias [25], a textual dataset for MLM, especially focused on gender bias; (iv) Measuring-Hate-Speech (MHS) [16], a textual dataset for Text Classification, especially focused on hate speech detection. |
| Dataset Splits | No | More details about adopted models, experimental results in further configurations, and full experimental details are provided in the Supplementary Materials. The paper does not explicitly mention the training/validation/test splits used for the datasets in the main text. |
| Hardware Specification | Yes | All experiments are based on the current Py Torch implementation of the algorithms and run on a Ubuntu 22.04 machine endowed with one NVIDIA H100 GPU and CUDA 12.4. |
| Software Dependencies | Yes | All experiments are based on the current Py Torch implementation of the algorithms and run on a Ubuntu 22.04 machine endowed with one NVIDIA H100 GPU and CUDA 12.4. |
| Experiment Setup | Yes | Algorithms are run for K = 1000 iterations, which we prove sufficient to capture their behavior, with delta multiplier η {1; 10}, the latter used with the aim to verify whether it is possible to speed up the process pace without compromising its stability. |