2/25/2022: Deep learning in real-world wildlife recognition, Zhongqi Miao, University of California, Berkeley

Speaker: Zhongqi Miao

Abstract:

Deep learning has attracted much attention from the ecological community for its capability of extracting and generalizing patterns from datasets with highly complicated structures, such as images, audios, and motion signals. However, despite the promising cases, deep learning may have shortcomings when applied to real-world ecological datasets.

In this presentation, I will summarize three challenges in real-world wildlife recognition: 1) long-tailed distribution; 2) multi-domain; and 3) inevitable humans in the loop. I will also discuss three projects that deal with these challenges: 1) Open Long-Tailed Recognition (OLTR), 2) Open Compound Domain Adaptation (OCDA), and 3) Iterative Human and Automated Identification of Wildlife Images.

In OLTR, we address open long-tailed recognition with an integrated algorithm that handles imbalanced classification, few-shot learning, and open-set recognition simultaneously. Our OLTR algorithm maps an image to a feature space such that visual concepts can relate to each other based on a learned metric that respects the closed-world classification while acknowledging the novelty of the open world.

In OCDA, we consider the open compound domain adaptation problem. The compound target domain combines multiple traditional target domains without domain labels, reflecting realistic data collection in various mixed and novel conditions. Contrary to existing single- or multi-target domain adaptation works, where known clear distinctions between domains are often assumed, OCDA does not rely on domain boundaries and continuously adapts the model within a learned domain space. Our model consists of two technical insights into OCDA: 1) a curriculum domain adaptation strategy to bootstrap generalization across domain distinction in a data-driven, self-organizing, and continuous fashion and 2) a memory module to increase the model’s agility towards novel domains.

In Iterative Human and Automated Identification of Wildlife Images, we present a self-updating framework with humans in the loop to capture data dynamics of rapidly changing natural systems and address long-tailed distribution. With an example application to wildlife recognition project, we show that a synergistic collaboration of humans and machines transforms deep learning from a relatively inefficient post-annotation tool to a collaborative, ongoing annotation tool that vastly relieves the burden of human annotation and enables efficient and constant model updates.

The recording of the presentation can be found here: https://drive.google.com/file/d/1vg3CYeFy_fFpX_bH_bUgUYh76tdg1mAG/view