Home on Keyne Oei

VideoAlign: A Toolkit to Make Video Analysis Accessible to HCI Practitioners

Mon, 01 Jun 2026 00:00:00 +0000

Despite the potential of self-supervised video alignment algorithms for advancing human-computer interaction, they remain largely inaccessible to practitioners without machine learning expertise. To bridge this gap, we introduce VideoAlign, an open-source toolkit designed to facilitate training and integration of video alignment approaches in interactive applications. VideoAlign offers guidance for training models to align videos, detecting and tagging specific events, and performing anomaly detection through an interactive system without requiring machine learning experience. In addition to implementing state-of-the-art alignment techniques, our toolkit introduces a novel Local-Alignment Contrastive (LAC) loss. Unlike global methods that compare entire video sequences, LAC aligns localized segments independently. This capability enables robust matching when video structure or timing varies, which is crucial for real-world interactive applications. We demonstrate how VideoAlign facilitates the creation of a wide variety of interactive applications through three application scenarios: a teacher-support tool for providing efficient video feedback, a mixed reality application that tracks activity progress in real time, and an anomaly detection tool to monitor cooking activities.

Self-Supervised Contrastive Learning for Videos using Differentiable Local Alignment

Mon, 09 Sep 2024 00:00:00 +0000

Robust frame-wise embeddings are essential to perform video analysis and understanding tasks. We present a self-supervised method for representation learning based on aligning temporal video sequences. Our framework uses a transformer-based encoder to extract frame-level features and leverages them to find the optimal alignment path between video sequences. We introduce the novel Local-Alignment Contrastive (LAC) loss, which combines a differentiable local alignment loss to capture local temporal dependencies with a contrastive loss to enhance discriminative learning. Prior works on video alignment have focused on using global temporal ordering across sequence pairs, whereas our loss encourages identifying the best-scoring subsequence alignment. LAC uses the differentiable Smith-Waterman (SW) affine method, which features a flexible parameterization learned through the training phase, enabling the model to adjust the temporal gap penalty length dynamically. Evaluations show that our learned representations outperform existing state-of-the-art approaches on action recognition tasks.

Real-time Machine Learning Auslan Recognition to Promote Social Robotic Interaction

Tue, 01 Jan 2019 00:00:00 +0000

Indonesia's Future in Tech, from a STEM PhD Fellow Perspective (2026)

Thu, 02 Jul 2026 00:00:00 +0000

Is there still hope for Indonesia in technology sector?

Notes from DL2026

Mon, 29 Jun 2026 00:00:00 +0000

Notes from DL2026, focusing on the EHR talks and the transformer.

Welcome to my blog

Thu, 25 Jun 2026 00:00:00 +0000

A first post — what this blog is for, and what I plan to write about.