Safe Partial Diagnosis from Normal Observations
Authors: Roni Stern, Brendan Juba3084-3091
AAAI 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our second contribution is a statistical analysis of the number of normal observations needed to learn an effective model. This novel form of statistical analysis draws from the well-known probably approximately correct (PAC) framework for analyzing learning algorithms (Vapnik and Chervonenkis 1971; Valiant 1984; Haussler 1992). Our main theoretical result is that under several assumptions, the number of observations needed to learn, with high probability, a model that will identify faults in most cases is only linear in the number of components. Our third contribution is a method for using knowledge about the system topology, in addition to the set of normal observations, to learn a useful and more informed par-tial model of the system. Here too, we provide theoretical bounds relating the number of observations and properties of the system topology to the probability of learning a model that can identify faulty components in abnormal observations with high probability. We demonstrate our learning approach on a Boolean circuit from the standard ISCAS 85 benchmark (Brglez, Bryan, and Kozminski 1989), showing that with a single probe and only 256 observations, we are able to find a faulty component in 28% of the cases, and this percentage increases rapidly. |
| Researcher Affiliation | Academia | Roni Stern Software and Information Systems Engineering Ben Gurion University Be er Sheba, Israel sternron@post.bgu.ac.il Brendan Juba Dept. of Computer Science & Engineering Washington University in St. Louis 1 Brookings Dr., St. Louis, MO, 63130 USA bjuba@wustl.edu |
| Pseudocode | No | The paper describes methods and concepts using text and mathematical formulations (equations, theorems, propositions), but it does not include any explicit pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any information about open-sourcing the code for the described methodology. It mentions implementing the algorithm but not its availability. |
| Open Datasets | Yes | As a proof-of-concept, we implemented our diagnosis algorithm and ran it on system 74181 from the ISCAS 85 Boolean circuit benchmark (Brglez, Bryan, and Kozminski 1989). |
| Dataset Splits | No | The paper mentions varying the 'size of the training set' but does not specify a training/validation/test split or percentages. It describes creating training sets and then injecting faults for evaluation, but without explicit partitioning of a single dataset. |
| Hardware Specification | No | The paper does not specify any details about the hardware (e.g., CPU, GPU, memory) used for conducting the experiments. |
| Software Dependencies | No | The paper does not provide any specific software dependencies or version numbers (e.g., programming languages, libraries, frameworks, or solvers used in implementation). |
| Experiment Setup | Yes | We created training sets by setting uniform-random inputs to the system. In each trial, we injected faults randomly to the system, ran our diagnosis algorithm, and checked if the injected fault was identiļ¬ed as part of a surely faulty subsystem. We varied the size of the training set (4, 16, 64, 256, 1024, 2048, and 4096) and the number of probes (1, 4, 8, 16, and 57, where 57 means full probing for this system). |