| Summary: | Scientific analysis of motion and social interaction can identify animal models of human disease by relating genetics or neural activity to behavior. However, experiments are often limited in scope because they require vast quantities of expert annotation on private data. Attempts to automate aspects of behavior science typically have limited interpretability and lack uncertainty representation. Errors will go unrecognized without manual inspection and propagate to hypothesis tests, corrupting conclusions. In response, this dissertation develops principled Bayesian approaches to low-level behavior analysis that discover the articulated part structure of a moving object and quantify uncertainty in the motion of multiple objects. Uncertainty is used to identify possible errors and automatically schedule sparse annotations. We apply parts modeling and tracking to primate behavior data in experimental and observational settings, in one case contributing to the first evidence supporting the use of primate animal models in autism research. We additionally develop Marmoset100, a 100-hour RGB-Depth dataset of pairwise primate social interactions labeled with 25 high-level behaviors, and show that uncertainty representation in tracking estimates improves behavior classification.
The Nonparametric Parts Model (NPP) discovers structure by learning articulated parts decompositions in an unsupervised fashion by briefly observing objects moving in an image, depth, point cloud, or mesh sequences. NPP combines distributions on Lie groups with a Bayesian nonparametric prior to perform joint reasoning over an interpretable state-space model with nonlinear dynamics and state-dependent observation noise. In developing sampling-based inference for NPP, we discover a novel and efficient Gibbs decomposition for prior distributions on SE(D), the manifold of rigid transformations. We show that NPP learns intuitive part segmentations for diverse objects and enables both analysis and synthesis of relative part motion in the body frame.
The Joint Posterior Tracker (JPT) is a comprehensive Bayesian treatment of the general multiobject tracking problem that quantities uncertainty in the motion of multiple objects. JPT uniquely performs asymptotically exact inference without gating heuristics or the combinatorial costs of exponential and factorial complexity. We develop novel Metropolis-Hastings proposals that reason over permutations of the latent space and enable efficient posterior mode hopping that correspond to possible confusion events. We show that JPT yields accurate uncertainty representation of data associations with high performance on standard metrics. Finally, we use posterior uncertainty to identify ambiguities in observed data and automatically schedule sparse human annotations that rapidly improve posterior estimates and reduce uncertainty.
|
|---|