Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Revisiting Non-separable Binary Classification and its Applications in Anomaly Detection
Authors: Matthew Lau, ISMAILA SECK, Athanasios P Meliopoulos, Wenke Lee, Eugene Ndiaye
TMLR 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Springboarding from this theoretical connection between binary classification and anomaly detection, we test our hypothesis on supervised anomaly detection experiments, showing that equality separation can detect both seen and unseen anomalies. ... We provide quantitative (Table 2) and qualitative (Figure 3) analyses that suggest either accurate one-class classification or overfitting to seen anomalies. |
| Researcher Affiliation | Collaboration | Matthew Lau EMAIL School of Cybersecurity and Privacy, Georgia Institute of Technology; Ismaila Seck EMAIL Lengo AI; Athanasios P Meliopoulos EMAIL School of Electrical and Computer Engineering, Georgia Institute of Technology; Wenke Lee EMAIL School of Cybersecurity and Privacy, Georgia Institute of Technology; Eugene Ndiaye EMAIL Apple |
| Pseudocode | No | The paper describes methods and algorithms verbally and mathematically, but no structured pseudocode or algorithm blocks are explicitly presented or labeled. |
| Open Source Code | Yes | Code used can be accessed at https://github.com/mattlaued/XOR-is-Linearly-Classifiable. |
| Open Datasets | Yes | We use the NSL-KDD network intrusion dataset (Tavallaee et al., 2009) for real-world cyber-attack experiments. ... We use the thyroid dataset (Quinlan, 1987) for medical anomalies. ... MVTec dataset (Bergmann et al., 2019) has images of normal and defective objects. |
| Dataset Splits | Yes | To demonstrate AD with seen and unseen anomalies, we generate synthetic 2D data... The normal (positive5) class is uniformly sampled from a circle of radius 1, and the anomaly (negative) class during training and testing is uniformly sampled from circles of radius 2 and 0.5 respectively, with all centred at the origin. ... For supervised AD, we explore 2 settings. The first is with synthetic data... The second is with real-world data from cyber-security, medical and manufacturing. ... training data only has normal data and denial of service attacks (DoS), and test data has DoS and unseen attacks, probe, privilege escalation and remote access attacks. ... Using a validation split of 0.15, we confirm that we do not overfit to the seen data during training. |
| Hardware Specification | No | The paper does not explicitly mention any specific hardware (e.g., GPU/CPU models, TPU, or cloud instance types) used for running the experiments. |
| Software Dependencies | No | To implement the SVM, we use the sklearn.SVM.SVC support vector classifier class in the scikit-learn package. ... We use the minimize function in the scipy.optimize package ... NNs are trained using Tensor Flow. ... boosted trees from the XGBoost library. The paper mentions various software packages and libraries but does not provide specific version numbers for any of them. |
| Experiment Setup | Yes | Unless otherwise stated, σ is assumed to be a fixed hyperparameter at 10. In equality separator NNs, we fix σ = 0.5 in the output layer for stability during training. ... The gradient-based optimizer used is Broyden-Fletcher Goldfarb-Shannon (BFGS), the loss function used in Section F.2 is MSE, and the parameters are initialized to a standard multivariate normal distribution. ... We use 5 neurons per hidden layer, limiting the chance of overfitting in this low data regime. ... NNs with leaky Re LU activations in their hidden layers have the leaky Re LU with parameter 0.01. ... NNs with bump activations in their hidden layers are initialized with σ = 0.5, which is not trainable by default. ... Weights are initialized with the default Glorot uniform initializer. ... NNs are trained with the Adam optimizer under a constant learning rate of 0.001. ... Training is done for 1000 epochs with early stopping. Early stopping monitors validation loss with a patience of 10 and best weights are restored. Validation split is 0.1. |