Inverse Reinforcement Learning

Problems in sensor-level target tracking using filtering algorithms have been exhaustively studied in the literature. We are interested in two deeper ideas that go beyond sensor-level tracking to the meta-level:

  1. Inverse Filtering and Inference: Given the response of a radar to our probe signal, how can we decide if the radar is cognitive? How can we estimate the radar’s accuracy? How can we reconstruct the tracked estimate of the enemy? How do we optimally probe the enemy’s radar to estimate its gain with minimum covariance? Such problems have significant relevance in electronic warfare.
  2.  Meta-level tracking: Given track estimates, how do devise automated signal interpretation algorithms that assist a human operator to interpret the trajectory of a target? For example: How can one infer if a target is circling a building and therefore behaving suspiciously? A closed trajectory has long range dependencies – the beginning and end points coincide. Standard Markovian state space models cannot capture the long-range dependencies and spatial complexities of such trajectories. We have used natural language processing models such as stochastic context free grammars and reciprocal stochastic processes to model the trajectories of targets. Our meta-level target tracking goes beyond sensor-level tracking. We aim to infer the intent of the target – this constitutes the human-sensor interface (middleware).

Relevant Presentations:

Rationalizing Youtube Commenting Behavior via Inverse Reinforcement Learning

Inverse Reinforcement Learning

Meta-Cognition: Inverse-Inverse Reinforcement Learning for Cognitive Radars

Relevant Publications