Max Bain

whisperX. WhisperX: Automatic Speech Recognition with Word-level Timestamps (& Diarization)

webvid. Large-scale text-video dataset. 10 million captioned short videos.

frozen-in-time. Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrieval [ICCV'21]

CondensedMovies. Story-Based Retrieval with Contextual Embeddings. Largest freely available movie video dataset. [ACCV'20]

video-transformers. Implementations of Transformers for Video

CondensedMovies-chall. Condensed Movies Challenge 2021

clip-hitchhiker. A Clip-Hitchiker's Guide to Long Video Retrieval [Arxiv 2022]

pytorch-multi-label-classifier. A pytorch implemented classifier for Multiple-Label classification

LAVIS. LAVIS - A One-stop Library for Language-Vision Intelligence

PaddleOCR. Awesome multilingual OCR toolkits based on PaddlePaddle (practical ultra lightweight OCR system, support 80+ languages recognition, provide data annotation and synthesis tools, support training and deployment among server, mobile, embedded and IoT devices)

SimpleDiarization. Simple Diarization model

whisper-asr-webservice. OpenAI Whisper ASR Webservice API

chimp-detector. Python

reka-vibe-eval. Multimodal language model benchmark, featuring challenging examples

primate-behaviour-recognition. Automated Audiovisual Behaviour Recognition in Wild Primates

CLIP. CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image

video2dataset. Easily create large video dataset from video urls

web-retrieval-demo. Python

pyannote-audio. Neural building blocks for speaker diarization: speech activity detection, speaker change detection, overlapped speech detection, speaker embedding