Representation Learning for Event-based Visuomotor Policies
Authors: Sai Vemprala, Sami Mian, Ashish Kapoor
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our first set of experiments aims to validate the learning of compressed representations encoded from the event sequences, and analyze the context-capturing ability of the e VAE. ... We validate this framework of learning event-based visuomotor policies by applying it to an obstacle avoidance scenario in simulation. Compared to techniques that treat event data as images, we show that representations learnt from event streams result in faster policy training, adapt to different control capacities, and demonstrate a higher degree of robustness. |
| Researcher Affiliation | Collaboration | Sai Vemprala Microsoft Research Redmond, WA 98052 savempra@microsoft.com Sami Mian University of Pittsburgh Pittsburgh, PA 15260 sami.mian@pitt.edu Ashish Kapoor Microsoft Research Redmond, WA 98052 akapoor@microsoft.com |
| Pseudocode | No | The paper includes architectural diagrams (e.g., Figure 2) but does not provide structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | In the interest of furthering research, we open source our representation learning and reinforcement learning framework along with the environments 2. 2Our code and environments can be found at https://github.com/microsoft/event-vae-rl |
| Open Datasets | No | To train the e VAE, we simulate event data through Air Sim s event simulator in three environments named poles, cones, and gates (drone racing gates), each indicative of the object of interest in it. More details about these environments can be found in Appendix B. (The paper uses simulated data in custom environments but does not provide concrete access information or a citation for a publicly available or open dataset). |
| Dataset Splits | No | The paper mentions environments used for training and testing policies (Figure 3) and discusses success percentages over trials, but it does not provide specific dataset split information (exact percentages, sample counts, or citations to predefined splits) needed for data partitioning. |
| Hardware Specification | No | The paper states that experiments were conducted in a "high fidelity quadrotor simulator Air Sim" and involved "simulated event data," but it does not provide specific hardware details (e.g., exact GPU/CPU models, memory amounts, or cloud instance types) used for running these simulations or experiments. |
| Software Dependencies | No | We create an obstacle avoidance scenario within the high fidelity quadrotor simulator Air Sim [44]... We train our policies using the Proximal Policy Optimization (PPO) [43] algorithm. (The paper mentions software like Air Sim and algorithms like PPO, but does not provide specific version numbers for these or other key software components used in the experiments.) |
| Experiment Setup | Yes | We create an obstacle avoidance scenario within the high fidelity quadrotor simulator Air Sim [44]... To emulate different control frequencies, we assume that the drone is moving at a constant predefined velocity and vary the step size of the actions dependent on the desired frequency. We assume the drone to be a simplistic model capable of moving at a speed of 20 m/s; thus, for example, the step size for a 200 Hz control would be 0.1 m. ... The simulated event camera is assumed to be of 64 64 resolution and the data is collected by navigating in 2D around the objects. Further details about the RL training procedure and the environment can be found in Appendix D. |