# Python for Scientific Audio

The aim of this repository is to create a comprehensive, curated list of python software/tools related and used for scientific research in audio/music applications.

## Contents

- Audio Related Packages
  - Read/Write
  - Transformations - General DSP
  - Feature extraction
  - Data augmentation
  - Speech Processing
  - Environmental Sounds
  - Perceptial Models - Auditory Models
  - Source Separation
  - Music Information Retrieval
  - Deep Learning
  - Symbolic Music - MIDI - Musicology
  - Realtime applications
  - Web - Audio
  - Audio related APIs and Datasets
  - Wrappers for Audio Plugins
- Tutorials
- Books
- Scientific Paper
- Other Resources
- Related lists
- Contributing
- License

## Audio Related Packages

- Total number of packages: 66

#### Read-Write

- audiolazy :octocat: :package: - Expressive Digital Signal Processing (DSP) package for Python.
- audioread :octocat: :package: - Cross-library (GStreamer + Core Audio + MAD + FFmpeg) audio decoding.
- mutagen :octocat: :package: - Reads and writes all kind of audio metadata for various formats.
- pyAV :octocat: - PyAV is a Pythonic binding for FFmpeg or Libav.
- (Py)Soundfile :octocat: :package: - Library based on libsndfile, CFFI, and NumPy.
- pySox :octocat: :package: - Wrapper for sox.
- stempeg :octocat: :package: - read/write of STEMS multistream audio.
- tinytag :octocat: :package: - reading music meta data of MP3, OGG, FLAC and Wave files.

#### Transformations - General DSP

- acoustics :octocat: :package: - useful tools for acousticians.
- AudioTK :octocat: - DSP filter toolbox (lots of filters).
- AudioTSM :octocat: :package: - real-time audio time-scale modification procedures.
- Gammatone :octocat: - Gammatone filterbank implementation.
- pyFFTW :octocat: :package: - Wrapper for FFTW(3).
- NSGT :octocat: :package: - Non-stationary gabor transform, constant-q.
- matchering :octocat: :package: - Automated reference audio mastering.
- MDCT :octocat: :package: - MDCT transform.
- pydub :octocat: :package: - Manipulate audio with a simple and easy high level interface.
- pytftb :octocat: - Implementation of the MATLAB Time-Frequency Toolbox.
- pyroomacoustics :octocat: :package: - Room Acoustics Simulation (RIR generator)
- PyRubberband :octocat: :package: - Wrapper for rubberband to do pitch-shifting and time-stretching.
- PyWavelets :octocat: :package: - Discrete Wavelet Transform in Python.
- Resampy :octocat: :package: - Sample rate conversion.
- SFS-Python :octocat: :package: - Sound Field Synthesis Toolbox.
- sound_field_analysis :octocat: :package: - Analyze, visualize and process sound field data recorded by spherical microphone arrays.
- STFT :octocat: :package: - Standalone package for Short-Time Fourier Transform.

#### Feature extraction

- aubio :octocat: :package: - Feature extractor, written in C, Python interface.
- audioFlux :octocat: :package: - A library for audio and music analysis, feature extraction.
- audiolazy :octocat: :package: - Realtime Audio Processing lib, general purpose.
- essentia :octocat: - Music related low level and high level feature extractor, C++ based, includes Python bindings.
- python_speech_features :octocat: :package: - Common speech features for ASR.
- pyYAAFE :octocat: - Python bindings for YAAFE feature extractor.
- speechpy :octocat: :package: - Library for Speech Processing and Recognition, mostly feature extraction for now.
- spafe :octocat: :package: - Python library for features extraction from audio files.

#### Data augmentation

- audiomentations :octocat: :package: - Audio Data Augmentation.
- muda :octocat: :package: - Musical Data Augmentation.
- pydiogment :octocat: :package: - Audio Data Augmentation.

#### Speech Processing

- aeneas :octocat: :package: - Forced aligner, based on MFCC+DTW, 35+ languages.
- deepspeech :octocat: :package: - Pretrained automatic speech recognition.
- gentle :octocat: - Forced-aligner built on Kaldi.
- Parselmouth :octocat: :package: - Python interface to the Praat phonetics and speech analysis, synthesis, and manipulation software.
- persephone :octocat: :package: - Automatic phoneme transcription tool.
- pyannote.audio :octocat: :package: - Neural building blocks for speaker diarization.
- pyAudioAnalysis² :octocat: :package: - Feature Extraction, Classification, Diarization.
- py-webrtcvad :octocat: :package: - Interface to the WebRTC Voice Activity Detector.
- pypesq :octocat: - Wrapper for the PESQ score calculation.
- pystoi :octocat: :package: - Short Term Objective Intelligibility measure (STOI).
- PyWorldVocoder :octocat: - Wrapper for Morise’s World Vocoder.
- Montreal Forced Aligner :octocat: - Forced aligner, based on Kaldi (HMM), English (others can be trained).
- SIDEKIT :package: - Speaker and Language recognition.
- SpeechRecognition :octocat: :package: - Wrapper for several ASR engines and APIs, online and offline.

#### Environmental Sounds

- sed_eval :octocat: :package: - Evaluation toolbox for Sound Event Detection

#### Perceptial Models - Auditory Models

- cochlea :octocat: :package: - Inner ear models.
- Brian2 :octocat: :package: - Spiking neural networks simulator, includes cochlea model.
- Loudness :octocat: - Perceived loudness, includes Zwicker, Moore/Glasberg model.
- pyloudnorm :octocat: - Audio loudness meter and normalization, implements ITU-R BS.1770-4.
- Sound Field Synthesis Toolbox :octocat: :package: - Sound Field Synthesis Toolbox.

#### Source Separation

- commonfate :octocat: :package: - Common Fate Model and Transform.
- NTFLib :octocat: - Sparse Beta-Divergence Tensor Factorization.
- NUSSL :octocat: :package: - Holistic source separation framework including DSP methods and deep learning methods.
- NIMFA :octocat: :package: - Several flavors of non-negative-matrix factorization.

#### Music Information Retrieval

- Catchy :octocat: - Corpus Analysis Tools for Computational Hook Discovery.
- chord-detection :octocat: - Algorithms for chord detection and key estimation.
- Madmom :octocat: :package: - MIR packages with strong focus on beat detection, onset detection and chord recognition.
- mir_eval :octocat: :package: - Common scores for various MIR tasks. Also includes bss_eval implementation.
- msaf :octocat: :package: - Music Structure Analysis Framework.
- librosa :octocat: :package: - General audio and music analysis.

#### Deep Learning

- Kapre :octocat: :package: - Keras Audio Preprocessors
- TorchAudio :octocat: - PyTorch Audio Loaders
- nnAudio :octocat: :package: - Accelerated audio processing using 1D convolution networks in PyTorch.

#### Symbolic Music - MIDI - Musicology

- Music21 :octocat: :package: - Toolkit for Computer-Aided Musicology.
- Mido :octocat: :package: - Realtime MIDI wrapper.
- mingus :octocat: :package: - Advanced music theory and notation package with MIDI file and playback support.
- Pretty-MIDI :octocat: :package: - Utility functions for handling MIDI data in a nice/intuitive way.

#### Realtime applications

- Jupylet :octocat: - Subtractive, additive, FM, and sample-based sound synthesis.
- PYO :octocat: - Realtime audio dsp engine.
- python-sounddevice :octocat: :package: - PortAudio wrapper providing realtime audio I/O with NumPy.
- ReTiSAR :octocat: - Binarual rendering of streamed or IR-based high-order spherical microphone array signals.

#### Web Audio

- TimeSide (Beta) :octocat: - high level audio analysis, imaging, transcoding, streaming and labelling.

#### Audio Dataset and Dataloaders

- beets :octocat: :package: - Music library manager and MusicBrainz tagger.

- musdb :octocat: :package: - Parse and process the MUSDB18 dataset.

- medleydb :octocat: - Parse medleydb audio + annotations.

- Soundcloud API :octocat: :package: - Wrapper for Soundcloud API.

- Youtube-Downloader :octocat: :package: - Download youtube videos (and the audio).

- audiomate :octocat: :package: - Loading different types of audio datasets.

- mirdata :octocat: :package: - Common loaders for Music Information Retrieval (MIR) datasets. #### Wrappers for Audio Plugins

- VamPy Host :package: - Interface compiled vamp plugins.

## Tutorials

- Whirlwind Tour Of Python :octocat: - fast-paced introduction to Python essentials, aimed at researchers and developers.
- Introduction to Numpy and Scipy :octocat: - Highly recommended tutorial, covers large parts of the scientific Python ecosystem.
- Numpy for MATLAB® Users - Short overview of equivalent python functions for switchers.
- MIR Notebooks :octocat: - collection of instructional iPython Notebooks for music information retrieval (MIR).
- Selected Topics in Audio Signal Processing - Exercises as iPython notebooks.
- Live-coding a music synthesizer Live-coding video showing how to use the SoundDevice library to reproduce realistic sounds. Code.

## Books

- Python Data Science Handbook - Jake Vanderplas, Excellent Book and accompanying tutorial notebooks.
- Fundamentals of Music Processing - Meinard Müller, comes with Python exercises.

## Scientific Papers

- Python for audio signal processing - John C. Glover, Victor Lazzarini and Joseph Timoney, Linux Audio Conference 2011.
- librosa: Audio and Music Signal Analysis in Python, Video - Brian McFee, Colin Raffel, Dawen Liang, Daniel P.W. Ellis, Matt McVicar, Eric Battenberg, Oriol Nieto, Scipy 2015.
- pyannote.audio: neural building blocks for speaker diarization, Video - Hervé Bredin, Ruiqing Yin, Juan Manuel Coria, Gregory Gelly, Pavel Korshunov, Marvin Lavechin, Diego Fustes, Hadrien Titeux, Wassim Bouaziz, Marie-Philippe Gill, ICASSP 2020.

## Other Resources

- Coursera Course - Audio Signal Processing, Python based course from UPF of Barcelona and Stanford University.
- Digital Signal Processing Course - Masters Course Material (University of Rostock) with many Python examples.
- Slack Channel - Music Information Retrieval Community.

## Related lists

There is already PythonInMusic but it is not up to date and includes too many packages of special interest that are mostly not relevant for scientific applications. Awesome-Python is large curated list of python packages. However, the audio section is very small.

## Contributing

Your contributions are always welcome! Please take a look at the contribution guidelines first.

I will keep some pull requests open if I’m not sure whether those libraries are awesome, you could vote for them by adding 👍 to them.

## License