At WWDC 2026, Apple introduced the Music Understanding framework, an on-device audio analysis tool for Apple platforms.

It allows apps to process audio and extract detailed, time-stamped information across six core musical dimensions: rhythm, key, loudness, pace, structure, and instrument activity:

• AnalyzedRhythm: Detects beat positions, bar boundaries, and tempo in beats per minute (BPM).
• Key: Identifies the musical key (tonic and mode) over specific time ranges.
• Loudness: Measures perceptual loudness (integrated, short-term, momentary, and peak) matching industry standards.
• Pace: Measures the perceived energy and momentum as events-per-minute, independent of the fixed tempo.
• Structure: Breaks down songs into structural sections, segments, and phrases.
• Instrument Activity: Identifies which instruments are playing and exactly when they are active.