Adversarial Resilience Learning

As a researcher at OFFIS, I co-initiated the research area of Adversarial Resilience Learning (ARL). In ARL, two classes of agents act against each other in a shared environment. The first agent class, called attacker agents, work to de-stabilize the cyber-physical system. The class serves to represent actual (cyber-) attackers or other extreme actors as well as to model market actors trying to exploit loopholes in current regulations, as well as external factors such as deviations of forecasts of renewable energy sources, or simply accidents. The goal of ARL is the training of the second class of agents, the defender agents that aim to provide a resilient, fully AI-based grid operation. Both, attacker and defender, are not provided with any domain knowledge—sensors and actuators are only described in an abstract, mathematical sense—and do not even have explicit knowledge of each other. The premiss of this approach is the complexity of modern cyber-physical system, where the inclusion of extensive ICT infrastructure, market, and AI technologies is necessary, but makes a descriptive modelling or simulation approach largely infeasible. Instead, the agents explore the cyber-physical systems, building their own system model, using, at their strategy core, Deep Learning approaches that approximate algorithms.

Model Creation and Description using Software Agents

In addition, and as a general extension to, ARL, I research Deep Learning-based methodologies for modeldeviation and development. These models are formed as algorithms explore simulated environments. This independent exploration and model creation will find hitherto ignored or unknown interdependencies and characteristics for power grids and ICT, as this has happend in a similar manner with AlphaGo Zero. A practical and current example is the reactive power management in distribution grids, which does not just include inverter-based generators as economic optimization problem, but also, as a oscillating system, needs to be managed by grid operators.

Furthermore, my research considers the augmentation of Deep Learning models using traceable modelling methods stemming from other domains, such as the Boolean algebra and the Boolean differential calculus. While applied AI serves to cope with complex dynamics, allows the creation of hybrid, augmented models with deterministic properties to at least substantiate the resiliency of AI methodologies in critical infrastructures, specifically Deep Reinforcement Learning and Multi-Agent System approaches.