Understanding People in Long, Real-World Video

In long, multi-camera, real-world video, can we persistently understand who a person is, what they are doing, how they change over time, and which moments deserve attention?

Real deployments produce far more video than anyone can watch. We build systems that follow people through long, unconstrained, multi-camera and aerial-ground footage: recognizing identity across time, clothing, and viewpoint, interpreting activities and motion, and deciding which evidence is reliable enough to act on and which needs a human's judgment.

What we work on

  • Persistent person understanding across long videos
  • Multimodal person recognition combining appearance, 3D body shape, and gait
  • Multi-camera and aerial-ground video understanding
  • Open-set recognition and uncertainty-aware matching
  • Person-centric video retrieval and ranking
  • Cost-aware evidence selection for video analysis
  • Human motion as interpretable, language-grounded evidence

Emerging directions

We are extending this line toward video analysis that non-expert users can steer. Rather than requiring labeled data or precisely specified queries, we want systems that infer what a user is looking for from a few example clips, short corrections, and ordinary review behavior, then reorganize large unwatched collections accordingly: surfacing likely-relevant moments, flagging the unexpected, and knowing when to defer to human judgment. Our recent work on cost-aware model selection (IDSelect) is a first step — a system that decides, per video, which evidence is worth computing and which results are worth a person's attention.

Selected work

Where it applies

Public safety Defense and national security Large-scale video analytics Forensics Smart infrastructure
← All research