🎵 Voice Note to MIDI

---

name: voice-note-to-midi

description: Convert voice notes, humming, and melodic audio recordings to quantized MIDI files using ML-based pitch detection and intelligent post-processing

author: Clawd

tags: [audio, midi, music, transcription, machine-learning]

---

# 🎵 Voice Note to MIDI

Transform your voice memos, humming, and melodic recordings into clean, quantized MIDI files ready for your DAW.

What It Does

This skill provides a complete audio-to-MIDI conversion pipeline that:

1. **Stem Separation** - Uses HPSS (Harmonic-Percussive Source Separation) to isolate melodic content from drums, noise, and background sounds

2. **ML-Powered Pitch Detection** - Leverages Spotify's Basic Pitch model for accurate fundamental frequency extraction

3. **Key Detection** - Automatically detects the musical key of your recording using Krumhansl-Kessler key profiles

4. **Intelligent Quantization** - Snaps notes to a configurable timing grid with optional key-aware pitch correction

5. **Post-Processing** - Applies octave pruning, overlap-based harmonic removal, and legato note merging for clean output

Pipeline Architecture

Audio Input (WAV/M4A/MP3)
    ↓
┌─────────────────────────────────────┐
│ Step 1: Stem Separation (HPSS)     │
│ - Isolate harmonic content          │
│ - Remove drums/percussion           │
│ - Noise gating                      │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│ Step 2: Pitch Detection             │
│ - Basic Pitch ML model (Spotify)    │
│ - Polyphonic note detection         │
│ - Onset/offset estimation           │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│ Step 3: Analysis                    │
│ - Pitch class distribution          │
│ - Key detection                     │
│ - Dominant note identification      │
└─────────────────────────────────────┘
    ↓
┌─────────────────────────────────────┐
│ Step 4: Quantization & Cleanup      │
│ - Timing grid snap                  │
│ - Key-aware pitch correction        │
│ - Octave pruning (harmonic removal) │
│ - Overlap-based pruning             │
│ - Note merging (legato)             │
│ - Velocity normalization            │
└─────────────────────────────────────┘
    ↓
MIDI Output (Standard MIDI File)

Setup

Prerequisites

Installation

**Quick Install (Recommended):**

cd /path/to/voice-note-to-midi
./setup.sh

This automated script will:

**Manual Install:**

If you prefer manual setup:

mkdir -p ~/melody-pipeline
cd ~/melody-pipeline
python3 -m venv venv-bp
source venv-bp/bin/activate
pip install basic-pitch librosa soundfile mido music21
chmod +x ~/melody-pipeline/hum2midi

5. **Add to your PATH (optional):**

echo 'export PATH="$HOME/melody-pipeline:$PATH"' >> ~/.bashrc
source ~/.bashrc

Verify Installation

cd ~/melody-pipeline
./hum2midi --help

Usage

Basic Usage

Convert a voice memo to MIDI:

./hum2midi my_humming.wav

This creates `my_humming.mid` with 16th-note quantization.

Specify Output File

./hum2midi input.wav output.mid

Command-Line Options

| Option | Description | Default |

|--------|-------------|---------|

| `--grid <value>` | Quantization grid: `1/4`, `1/8`, `1/16`, `1/32` | `1/16` |

| `--min-note <ms>` | Minimum note duration in milliseconds | `50` |

| `--no-quantize` | Skip quantization (output raw Basic Pitch MIDI) | disabled |

| `--key-aware` | Enable key-aware pitch correction | disabled |

| `--no-analysis` | Skip pitch analysis and key detection | disabled |

Usage Examples

#### Quantize to eighth notes

./hum2midi melody.wav --grid 1/8

#### Key-aware quantization (recommended for tonal music)

./hum2midi song.wav --key-aware

#### Require longer minimum notes

./hum2midi humming.wav --min-note 100

#### Skip analysis for faster processing

./hum2midi quick.wav --no-analysis

#### Combine options

./hum2midi recording.wav output.mid --grid 1/8 --key-aware --min-note 80

Processing MIDI Input

You can also process existing MIDI files through the quantization pipeline:

./hum2midi input.mid output.mid --grid 1/16 --key-aware

This skips the audio processing steps and goes directly to analysis and quantization.

Sample Output

═══════════════════════════════════════════════════════════════
  hum2midi - Melody-to-MIDI Pipeline (Basic Pitch Edition)
  [Key-Aware Mode Enabled]
═══════════════════════════════════════════════════════════════

Input:  my_humming.wav
Output: my_humming.mid

→ Step 1: Stem Separation (HPSS)
  Isolating melodic content...
  Loaded: 5.23s @ 44100Hz
  ✓ Melody stem extracted → 5.23s

→ Step 2: Audio-to-MIDI Conversion (Basic Pitch)
  Running Spotify's Basic Pitch ML model on melody stem...
  ✓ Raw MIDI generated (Basic Pitch)

→ Step 3: Pitch Analysis & Key Detection
  Notes detected: 42 total, 7 unique
  Note range: C3 - G4
  Pitch classes: C3, E3, G3, A3, C4, D4, G4
  Dominant note: G3 (23.8% of notes)
  Detected key: G major

→ Step 4: Quantization & Cleanup
  Octave pruning: removed 3 harmonic notes above 67 (median+12)
  Overlap pruning: removed 2 harmonic notes at overlapping positions
  Note merging: merged 5 staccato chunks into legato notes (gap<=60 ticks)
  Grid:   240 ticks (1/16)
  Notes:  38 notes
  Key:    G major
  Key-aware: 2 notes corrected to scale
  Tempo:  120 BPM
  ✓ Quantized MIDI saved

═══════════════════════════════════════════════════════════════
  ✓ Done! Output: my_humming.mid
═══════════════════════════════════════════════════════════════

📊 ANALYSIS SUMMARY
─────────────────────────────────────────────────────────────
  Detected Notes: C3, E3, G3, A3, C4, D4, G4
  Detected Key:   G major
  Quantization:   Key-aware mode (notes snapped to scale)

MIDI Info: 38 notes, 7 unique pitches, 120 BPM
Pitches: C3, E3, G3, A3, C4, D4, G4

Notes & Limitations

Audio Quality Matters

Musical Considerations

Technical Limitations

Post-Processing Recommendations

After generating MIDI, you may want to:

1. **Import into your DAW** and adjust tempo to match your original recording

2. **Quantize further** if stricter timing is needed

3. **Adjust note velocities** for dynamics

4. **Apply swing/groove** templates if the rigid grid sounds too mechanical

5. **Edit individual notes** that were misdetected (common with fast runs)

Supported Audio Formats

Input formats supported via FFmpeg:

Troubleshooting

No notes detected

Too many notes / messy output

Wrong key detected

Notes in wrong octave

References

License

This skill integrates Basic Pitch by Spotify, which is licensed under Apache 2.0. The pipeline script and documentation are provided under MIT license.