1 LIVIA, Dept. of Systems Engineering, ÉTS Montréal · 2 LIVIA, Dept. of Software & IT Engineering, ÉTS Montréal · 3 Dept. of Health, Kinesiology & Applied Physiology, Concordia University · 4 Montreal Behavioural Medicine Centre, CIUSSS-NIM
Ambivalence and hesitancy are the number-one reasons people delay, avoid, or abandon a change to their health behaviour. They are subtle, conflicting emotions, a person caught between acceptance and refusal, and they leak out as a discord between the face, the voice and the body. BAH is the first dataset built to recognize that signal automatically, in video.
01 · MotivationWhy ambivalence & hesitancy matter
When a clinician sits across from a patient, they can read hesitation in a half-second pause or a flicker of doubt. Digital behaviour-change interventions, the apps and online programs that increasingly deliver health support, are blind to it. Training human experts to spot ambivalence and hesitancy (A/H) works, but embedding that expertise into software at scale is costly and far less effective.
Automatic A/H recognition is therefore the missing piece for personalized, cost-effective digital health: a model that notices when a user is wavering can adapt its message in the moment. The obstacle has been simple, until now, no dataset existed to train such a model.
A/H sets a person in a state between positive and negative orientations, between acceptance and refusal, and shows up as a discord in affect across face, voice and body.
02 · The datasetWhat's inside BAH
BAH was collected to mirror real-world digital interventions delivered online. Through a web platform, 300 participants from across Canada recorded themselves on webcam answering a set of predefined questions, several of them deliberately designed to elicit ambivalence and hesitancy. The result is a corpus of natural, low-production, in-the-wild video that looks exactly like what an intervention would actually receive.
Three experts annotated every recording, marking the timestamps where A/H occurs and providing both frame-level and video-level labels along with the underlying A/H cues. Because ambivalence and hesitancy manifest so similarly in practice, the released labels are binary, A/H present, or absent. Each video also ships with its transcript, cropped and aligned face crops for every frame, and per-participant metadata.
03 · CaptureHow the recordings were made
Participants completed the study entirely online. Each answered up to seven predefined questions while recording themselves via webcam and microphone, a setup chosen specifically so the data reflects the noise, lighting and framing of genuine at-home interventions rather than a controlled lab. The questions were scripted to gently surface moments of doubt and conflict.
04 · DiversityWho's in the dataset
A model that only works for one kind of person is no use in a public-health setting, so BAH was built for breadth. Participants span multiple Canadian provinces, a wide age range, both sexes, and diverse ethnic backgrounds, and, importantly, a majority who are not students, which sidesteps a common recruitment bias in affective-computing datasets. Crucially, A/H episodes themselves are brief, often only a few seconds, and a single video may contain several, which makes precise temporal localization genuinely hard.
05 · ProtocolSplits, imbalance & metrics
BAH is split by participant, every video from a person lives in exactly one of the train, validation or test sets, so models are always evaluated on people they have never seen. This subject-based design makes the benchmark honest, and the released split files are ready to use at both video and frame level.
The data is also deliberately, realistically imbalanced: A/H is the exception, not the rule, so the positive class is rare at frame level. Evaluation therefore leans on metrics that respect imbalance, the F1 score of the positive class, a weighted F1, and average precision, all with evaluation code provided.
06 · BenchmarksHow far do baselines get?
To map the difficulty, the paper benchmarks baseline models across several setups, frame- and video-level recognition, mono- and multi-modal inputs, zero-shot prediction, and personalization through unsupervised domain adaptation. The headline result is sobering and useful: baselines struggle. That gap is exactly the point, it shows that recognizing A/H in real video needs purpose-built multimodal and spatio-temporal models, not off-the-shelf classifiers.
Frame-level, multimodal
Video-level recognition
Zero-shot & personalization
Beyond supervised baselines, BAH supports zero-shot prediction and per-user adaptation, because the most useful intervention is one that learns an individual's particular way of hesitating.
07 · TakeawaysWhy this is hard, and worth it
BAH is unique: a multimodal, subject-based video dataset for a complex affective signal that has never had a benchmark before. The limited baseline performance isn't a disappointment, it's a map of the open problems. Leveraging context, fusing modalities, and modelling time are the directions the data points toward, and every piece needed to pursue them, data, code and pretrained weights, is public.
08 · AccessGet the data
BAH is released under a proprietary license for research only. To request access, follow the instructions in the repository download section. Code and pretrained weights are on GitHub and Hugging Face. Building a model for the challenge? Start with the ABAW 11th challenge page.
09 · CiteCitation
@article{gonzalez-25-bah,
title={BAH Dataset for Ambivalence/Hesitancy Recognition in
Videos for Digital Behavioural Change},
author={González-González, M. and Belharbi, S. and Zeeshan, M. O.
and Sharafi, M. and Aslam, M. H. and Pedersoli, M. and
Koerich, A. L. and Bacon, S. L. and Granger, E.},
journal={arXiv preprint arXiv:2505.19328},
year={2025}
}Acknowledgments, supported in part by the Fonds de recherche du Québec-Santé, NSERC, the Canada Foundation for Innovation, and the Digital Research Alliance of Canada. Thanks to annotation interns Jessica Almeida (Concordia, UQÀM) and Laura Lucia Ortiz (MBMC).