Home
The Musical Gestures Toolbox for Python is a collection of high-level modules targeted at researchers working with video recordings. It includes visualisation techniques such as motion videos, motion history images, and motiongrams; techniques that, in different ways, allow for looking at video recordings from different temporal and spatial perspectives. It also includes basic computer vision analysis, such as extracting the quantity and centroid of motion, and using such features in analysis.
The toolbox targets music-related body motion (of musicians, dancers, and perceivers) but is equally helpful for other disciplines working with video recordings of humans, such as linguistics, pedagogy, psychology, and medicine.
The Musical Gestures Toolbox contains functions to analyse and visualise video, audio, and motion capture data. There are three categories of functions:
- Preprocessing (trimming, cropping, colour adjustments, etc.)
- Visualisation (video playback, image display, plotting)
- Processing (videograms, average images, motion images, etc.)
- Quick Start Tutorial
- Jupyter Notebook–can also be run in Colab
- Examples Overview
- Complete Documentation Site
What changed in each release is kept in one place, the release notes and the changelog, rather than repeated here.
This wiki provides detailed documentation for individual MGT functions:
- Installation - Setup instructions for all platforms
- Video Basics - Understanding video processing concepts
- Loading Videos - How to load and display videos
- Preprocessing - Video preprocessing techniques
- Video Analysis - Motion analysis and visualisation
- Audio Analysis - Audio processing functions
- Output Management - Working with results
- Filtering Effects - Understanding filter parameters
- Function Chaining - Combining multiple operations
- File Naming - Output file conventions
Signal-level numpy/scipy functions (mostly not MgVideo/MgAudio methods) for the ro, still standing, Westney-comparisons and cymbal-comparison studies (Jensenius):
- Pulse, Cycles and Alignment - Canonical peak-picker; stroke/cycle segmentation and accelerando fitting; lead/lag estimation, event matching, cross-source agreement
- Motion, Audio and Posturography Toolkit - Band-limited QoM cores for position/pose/accelerometer data; scipy-only audio RMS/flux onsets, T60 decay, attack spectral centroid; sway metrics, complexity/dynamics measures, respiration rate
-
Pose and Motion Capture -
MgVideo.pose()skeleton rendering; MediaPipe landmark-trajectory extraction; pose-derived limb speed/impacts; QTM mocap I/O
- Working with ambiscape - Joint audio-visual sessions: shared clock, device sync, 360-camera workflow
- 360 Video Anglegrams and AEM - Directional motion analysis of 360 video, audio-energy overlays, per-direction views
The speed and efficiency of the MGT are made possible by the excellent FFmpeg project. Many of the toolbox functions are Python wrappers/bindings on FFmpeg commands called in a subprocess.
Please help improve the toolbox by adding bugs and feature requests in the issues section.
Separate packages with separate release cycles, built to be used together and sharing several implementations, so the same measure computed in two of them agrees.
| Toolbox | What it is for | Docs |
|---|---|---|
| ambiscape | soundscapes: the sonic ambience of a place | wiki |
| musiscape | music collections: comparing many tracks and albums | wiki |
| micromotion | human micromotion: quantity of motion from markers, accelerometers, belts and force plates | wiki |