Skip to content

Commit de9ca64

Browse files
alexarjeclaude
andcommitted
docs: add motiondescriptors to README + build examples gallery (#210)
- README: list motiondescriptors() in Video Analysis features and Basic Usage. - examples.md: add a motion-descriptors example and a visual Gallery section grouping all example figures and the new video GIFs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Crffzo4V8sonhgrAABbacF
1 parent 06c3cb5 commit de9ca64

2 files changed

Lines changed: 271 additions & 1 deletion

File tree

README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@ v.heatmap().show() # where the video changes most
3939
# Motion analysis
4040
v.motion().show()
4141
v.motiontempo().show() # dominant movement tempo (Hz/BPM)
42+
v.motiondescriptors().show() # motion energy, smoothness, entropy, spectral descriptors
4243
v.eulerian(mode='motion').show() # amplify subtle motion (EVM)
4344

4445
# Audio analysis
@@ -79,7 +80,7 @@ v.pose(model='mediapipe').show()
7980

8081
## Features
8182

82-
- **Video Analysis**: Motion detection, optical flow, motion vectors, movement tempo, Eulerian Video Magnification, frame-rate/speed resampling (`resample()`)
83+
- **Video Analysis**: Motion detection, optical flow, motion vectors, movement tempo, Eulerian Video Magnification, frame-rate/speed resampling (`resample()`), motion descriptors (energy/smoothness/entropy/spectral via `motiondescriptors()`)
8384
- **Pose Estimation**: MediaPipe (default; fast on plain CPU, GPU-capable) and OpenPose (multi-person) backends, with average-pose and trajectory summaries (per-marker quantity of motion + dominant frequency), optional marker motion trails, a 3D pose waterfall (`pose_waterfall()`), per-segment circular statistics (`pose_segments()`), centroid centring (`pose_center()`), and per-marker distance travelled (`pose_distance()`)
8485
- **Audio–movement analysis**: Compare a single performer's sound and motion — tempo similarity, phase synchrony, structural similarity, per-body-part audio coupling, and loudness/dynamics coupling
8586
- **Audio Processing**: Waveforms, spectrograms, MFCC, chromagrams, tempo/beat tracking, spectral descriptors

docs/examples.md

Lines changed: 269 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -198,6 +198,19 @@ dec = mv.resample(skip=2) # discard 2 frames for every one kept
198198
mv25.show()
199199
```
200200

201+
### Example: Motion descriptors
202+
203+
```python
204+
import musicalgestures as mg
205+
mv = mg.MgVideo(mg.examples.dance)
206+
md = mv.motiondescriptors() # returns MgFigure
207+
print(md.data['motion_energy'], md.data['motion_smoothness'])
208+
print(md.data['motion_entropy'], md.data['dominant_freq'])
209+
md.show() # QoM time series + power spectrum
210+
```
211+
212+
![Motion descriptors of dance.avi](images/examples/motiondescriptors.png)
213+
201214
### Example 8c: Pose waterfall and segment statistics
202215

203216
```python
@@ -343,6 +356,262 @@ mg.MgVideo(mg.examples.dance, skip=3).motionvideo().history(normalize=True).show
343356
mg.MgVideo(mg.examples.dance, skip=15).motionvideo().average().show()
344357
```
345358

359+
## Gallery
360+
361+
A visual tour of the outputs produced by the methods above. Each image was rendered from one of the bundled example clips (`mg.examples.dance` or `mg.examples.pianist`).
362+
363+
### Motion
364+
365+
**Motion video**
366+
367+
![Motion video](images/examples/motion.gif)
368+
369+
*`mv.motion()` — frame-difference motion video.*
370+
371+
**Horizontal motiongram**
372+
373+
![Horizontal motiongram](images/examples/motiongram_h.png)
374+
375+
*`mv.motiongrams()[0]` — motion collapsed over rows, time on the x-axis.*
376+
377+
**Vertical motiongram**
378+
379+
![Vertical motiongram](images/examples/motiongram_v.png)
380+
381+
*`mv.motiongrams()[1]` — motion collapsed over columns, time on the y-axis.*
382+
383+
**Motion history**
384+
385+
![Motion history](images/examples/motionhistory.png)
386+
387+
*`mv.motionhistory()` — accumulated motion trails in a single image.*
388+
389+
**Motion heatmap**
390+
391+
![Motion heatmap](images/examples/heatmap.png)
392+
393+
*Spatial distribution of where motion occurs across the clip.*
394+
395+
**Average image**
396+
397+
![Average image](images/examples/average.png)
398+
399+
*`mv.average()` — pixel-wise average of all frames.*
400+
401+
**History video**
402+
403+
![History video](images/examples/history.gif)
404+
405+
*`mv.history()` — recent frames overlaid on the current frame.*
406+
407+
**Motion descriptors**
408+
409+
![Motion descriptors](images/examples/motiondescriptors.png)
410+
411+
*`mv.motiondescriptors()` — quantity-of-motion time series and its power spectrum.*
412+
413+
**Motion tempo**
414+
415+
![Motion tempo](images/examples/motiontempo.png)
416+
417+
*Periodicity of the movement signal over time.*
418+
419+
### Space-time
420+
421+
**Stroboscope**
422+
423+
![Stroboscope](images/examples/stroboscope.png)
424+
425+
*`mv.stroboscope()` — successive poses superimposed in one frame.*
426+
427+
**Silhouette waterfall**
428+
429+
![Silhouette waterfall](images/examples/silhouette_waterfall.png)
430+
431+
*Stacked silhouettes revealing the body's path through time.*
432+
433+
### Motion vectors & optical flow
434+
435+
**Motion vectors**
436+
437+
![Motion vectors](images/examples/motionvectors.gif)
438+
439+
*`mv.motionvectors()` — block motion vectors overlaid on the video.*
440+
441+
**Dense optical flow**
442+
443+
![Dense optical flow](images/examples/flow_dense.gif)
444+
445+
*`mv.flow.dense()` — colour encodes direction, brightness encodes speed.*
446+
447+
**Sparse optical flow**
448+
449+
![Sparse optical flow](images/examples/flow_sparse.gif)
450+
451+
*`mv.flow.sparse()` — tracked feature points and their trajectories.*
452+
453+
### Eulerian
454+
455+
**Eulerian magnification**
456+
457+
![Eulerian magnification](images/examples/eulerian.gif)
458+
459+
*`mv.motion()`-style amplification of subtle, otherwise invisible motion.*
460+
461+
### Videograms & SSM
462+
463+
**Horizontal videogram**
464+
465+
![Horizontal videogram](images/examples/videogram_h.png)
466+
467+
*`mv.videograms()[0]` — raw frames collapsed over rows.*
468+
469+
**Vertical videogram**
470+
471+
![Vertical videogram](images/examples/videogram_v.png)
472+
473+
*`mv.videograms()[1]` — raw frames collapsed over columns.*
474+
475+
**Self-similarity matrix**
476+
477+
![Self-similarity matrix](images/examples/ssm.png)
478+
479+
*`mv.ssm()` — recurrence structure of the visual content over time.*
480+
481+
### Directograms & impacts
482+
483+
**Directogram**
484+
485+
![Directogram](images/examples/directograms.png)
486+
487+
*Distribution of motion directions over time.*
488+
489+
**Impacts**
490+
491+
![Impacts](images/examples/impacts.png)
492+
493+
*Detected movement impacts / accents.*
494+
495+
### Pose
496+
497+
**Pose average**
498+
499+
![Pose average](images/examples/pose_average.png)
500+
501+
*Average pose across the clip.*
502+
503+
**Pose trajectories**
504+
505+
![Pose trajectories](images/examples/pose_trajectories.png)
506+
507+
*Paths traced by each tracked landmark.*
508+
509+
**Pose waterfall (trajectories)**
510+
511+
![Pose waterfall trajectories](images/examples/pose_waterfall_trajectories.png)
512+
513+
*`mv.pose_waterfall(style='trajectories')` — 3D spatio-temporal landmark paths.*
514+
515+
**Pose waterfall (skeleton)**
516+
517+
![Pose waterfall skeleton](images/examples/pose_waterfall_skeleton.png)
518+
519+
*`mv.pose_waterfall(style='skeleton')` — stacked skeletons through time.*
520+
521+
**Pose segments**
522+
523+
![Pose segments](images/examples/pose_segments.png)
524+
525+
*`mv.pose_segments()` — circular statistics per body segment.*
526+
527+
**Pose center**
528+
529+
![Pose center](images/examples/pose_center.png)
530+
531+
*Trajectory of the body's centre of mass.*
532+
533+
**Pose distance**
534+
535+
![Pose distance](images/examples/pose_distance.png)
536+
537+
*Inter-landmark distances over time.*
538+
539+
### Audio
540+
541+
**Waveform**
542+
543+
![Waveform](images/examples/waveform.png)
544+
545+
*`mv.audio.waveform()` — amplitude envelope.*
546+
547+
**Spectrogram**
548+
549+
![Spectrogram](images/examples/spectrogram.png)
550+
551+
*`mv.audio.spectrogram()` — time-frequency energy.*
552+
553+
**MFCC**
554+
555+
![MFCC](images/examples/mfcc.png)
556+
557+
*Mel-frequency cepstral coefficients.*
558+
559+
**Chromagram**
560+
561+
![Chromagram](images/examples/chromagram.png)
562+
563+
*Pitch-class energy over time.*
564+
565+
**Tempogram**
566+
567+
![Tempogram](images/examples/tempogram.png)
568+
569+
*`mv.audio.tempogram()` — local tempo estimates.*
570+
571+
**Tempo**
572+
573+
![Tempo](images/examples/tempo.png)
574+
575+
*Estimated beat / tempo curve.*
576+
577+
**Audio descriptors**
578+
579+
![Audio descriptors](images/examples/descriptors.png)
580+
581+
*`mv.audio.descriptors()` — combined audio feature summary.*
582+
583+
### Audio–movement
584+
585+
**Tempo similarity**
586+
587+
![Tempo similarity](images/examples/tempo_similarity.png)
588+
589+
*`mv.tempo_similarity()` — audio tempo vs. movement tempo.*
590+
591+
**Phase synchrony**
592+
593+
![Phase synchrony](images/examples/phase_synchrony.png)
594+
595+
*`mv.phase_synchrony()` — phase-locking between audio and movement rhythm.*
596+
597+
**Structure comparison**
598+
599+
![Structure comparison](images/examples/structure_comparison.png)
600+
601+
*`mv.structure_comparison()` — audio SSM vs. movement SSM.*
602+
603+
**Body–audio coupling**
604+
605+
![Body–audio coupling](images/examples/body_audio_coupling.png)
606+
607+
*`mv.body_audio_coupling()` — which body parts track the music.*
608+
609+
**Dynamics coupling**
610+
611+
![Dynamics coupling](images/examples/dynamics_coupling.png)
612+
613+
*`mv.dynamics_coupling()` — loudness vs. quantity of motion.*
614+
346615
## Next Steps
347616

348617
- **[User Guide](user-guide/loading.md)** — Loading, preprocessing, and all analysis methods

0 commit comments

Comments
 (0)