ESP-beatsy Audio Library

Launch the Audio System Design Tool (GUI) · Source · GUI notes and tools

ESP-beatsy is an ESP32-oriented port and adaptation of the Teensy Audio Library. The main goal is to retain the familiar Teensy programming model and public APIs wherever practical, while adapting audio scheduling, I/O, memory handling and codec support to ESP32 hardware.

The project is work in progress. It is not a claim that every class has full Teensy feature parity or has been tested on every ESP32 target.

Teensy Audio Library compatibility

The library deliberately keeps the central Teensy concepts:

• Audio processing nodes derive from AudioStream and are connected with AudioConnection patch cords.
• Sketches use AudioMemory(n) to provide the shared pool of audio blocks.
• The familiar 16-bit PCM audio-block model and the usual block size of 128 samples are retained.
• Many established Teensy class names and control methods are preserved, making existing patch designs and sketches a useful starting point.

Compatibility is primarily source/API-level compatibility, not binary compatibility or a promise that every Teensy sketch will compile unchanged. Hardware initialization, board pins, drivers, memory requirements and some object-specific APIs differ. Features that are not listed below should not be assumed to be implemented.

Important implementation differences

• Scheduling: Unlike the Teensy implementation, ESP-beatsy uses FreeRTOS tasks and ESP32 timer/I2S callback mechanisms. When I2S output is active, the TX DMA callback can drive audio updates; otherwise a software timer can provide the update clock. Timing and processor-load numbers therefore cannot be compared directly with Teensy measurements.
• I2S hardware: The ESP32 implementation uses the ESP-IDF I2S driver and explicit pin configuration. AudioStream samples remain signed 16-bit PCM; conversion to the configured I2S slot representation is done in the output path. Mono output can be selected with setChannelCount(1).
• Memory: The shared AudioMemory block pool remains, but some ESP-specific objects allocate their own buffers. Delay and reverb implementations can prefer PSRAM and fall back to internal RAM, so memory consumption is not always equivalent to the Teensy implementation.
• Mixer extension: In addition to the familiar four-input AudioMixer4, AudioMixerN supports 4–32 inputs. It uses floating-point gains and accumulation, with saturation to 16-bit PCM only at the final output.
• Filter implementation: AudioFilterLadder replaces the Teensy/CMSIS-DSP FIR interpolation path with portable polyphase FIR routines and uses 4× internal oversampling.
• ESP-specific playback: MP3/AAC decoding, generic AudioSource objects and network-stream handling extend the original hardware-focused audio graph with ESP32 software-decoder functionality.
• WAV and memory-sample playback: AudioPlaySdWav::setLoop() repeats a WAV data chunk; loadToMemory() can decode a supported PCM WAV into a mono AudioSampleBuffer allocated in PSRAM after checking free and largest-block capacity while preserving a configurable reserve. AudioPlayMemory::play(buffer, true) plays that buffer without copying it and can loop it with variable playback rate. The buffer must outlive playback.
• Wavetable instruments: AudioSynthWavetable renders sample-based instruments from static sample/instrument data, commonly generated from an SF2 SoundFont by an external converter. It does not parse or load .sf2 files at runtime; use selected converted instruments rather than bundling a large complete SoundFont in a sketch.
• Hardware codecs: AudioControlES8311 and AudioControlES8388 are included for boards using those codecs.
• USB Audio: Experimental stereo-only 16-bit USB audio in both directions. Use AudioInputUSB usbInput; for PC playback into the graph and AudioOutputUSB usbOutput; for graph audio sent to the PC. The default constructors let the library configure the shared USBAudioCard, CDC interface and stream-enable callbacks during AudioMemory(); the USB environment enables AUDIO_USB_REDIRECT_SERIAL, so ordinary Serial.print() targets managed USB CDC while Serial0 remains the physical UART. Runtime ESP_LOGE/ESP_LOGW/ESP_LOGI messages are also sent to CDC while the host terminal is connected. Set AUDIO_SAMPLE_RATE_EXACT at compile time. Neither class resamples or requires polling in loop(). Advanced applications can still pass an explicitly configured USBAudioCard and manage startup themselves. The tested USB host playback example targets the ESP32-S3 PlatformIO environment freenove_esp32_s3_display_USBAUDIO. For Windows CDC + UAC1 on the tested Arduino-ESP32 3.3.12 setup, the version-specific Audio-IAD workaround is in scripts/patch_uac1_audio_iad.py.

Examples currently present

These examples are present in the repository. Their board-specific pin and codec settings are defined by the project configuration; presence in the tree alone does not imply that every example builds on every target.

Example	Description
AAC Player	Plays an AAC file from SD_MMC through I2S; reports playback, audio-block and processor statistics.
MP3 Player	Plays an MP3 file from SD_MMC through I2S and reports playback/decoder statistics.
Internet Radio	Reads an HTTP/HTTPS radio stream, selects the MP3 or AAC player from the response content type, adjusts the output sample rate when requested, and provides volume-dependent loudness compensation. Serial controls: l toggles loudness, +/- adjust volume. Requires local Wi-Fi credentials in secrets.h. Note: the sample uses TLS with certificate verification disabled; production applications should validate the server certificate.
P4-WIFI6 Wi-Fi station test	Checks ESP-Hosted Wi-Fi via the onboard ESP32-C6, then reports the DHCP IP address and RSSI. Use the esp32-p4-wifi6 PlatformIO environment and local src/secrets.h; see Wi-Fi setup.
P4-WIFI6 C6 Hosted OTA updater	Transfers a locally built, matching ESP-Hosted v2.13.0 firmware image to the onboard ESP32-C6 via a temporary local HTTP server; see the build and update steps in the Wi-Fi setup guide.
Stereo Pass-through	Routes the left/right I2S input to the matching output channel and reports received, erroneous and dropped input blocks.
USB Host Playback	Receives 16-bit stereo audio from the PC over USB and routes it to the board's I2S codec output. Use the ESP32-S3 freenove_esp32_s3_display_USBAUDIO PlatformIO environment.
Play Queue Demo	Generates a sine wave in application code and demonstrates AudioPlayQueue buffer/sample submission, including non-stalling operation.
Record to LittleFS	Generates a short 880 Hz sine ping with an envelope and records mono PCM WAV to LittleFS using AudioRecordFile. Requires PSRAM and a mounted/formatted LittleFS partition.
Record to SD	Generates the same short sine ping and records mono PCM WAV to an SD card using AudioRecordFile. SD_MMC uses the board-specific 4-bit pin mapping from the project configuration.
Delay effect	Generates a sine chirp, shapes it with an envelope and mixes several delayed taps to the opposite output channel.
FIR and Biquad filters	Sends white noise through FIR and low-pass biquad paths to the left and right outputs for comparison.
Guitar synthesis	Uses six Karplus–Strong string generators with per-string damping() settings, mixers and chord data to synthesize strummed chords. Exercises the 1024-sample buffer with the low E string at 44.1/48 kHz.
Polyphonic synth music	Plays a score stored in memory using 16 waveform voices, envelopes and left/right AudioMixerN instances.
Simple Drum	Triggers four simple-drum synthesizers and mixes them to stereo output.
Elisa's Hand Pan	A 30-second excerpt from Beethoven's public-domain Für Elise. The sketch configures just three body modes per pad and uses the synth's built-in soft finger-contact transient, with no sample data in application code. See the handpan synthesis notes.
Wavetable Drum Computer	Plays a four-bar, alternative-rock-style intro before looping an original groove. Uses one-shot wavetable playback with CC0 kick, snare, closed/open hi-hat and two tom samples prepared from Aspirin DX Soundbank — Fairytale Edition. Records about 60 seconds as mono PCM WAV to SD (/DrumComputer.wav); skips recording if /DrumComputer.wav or /DrumComputer.tmp already exists. Requires SD wiring, a writable card and PSRAM.
Doppler Helicopter	Loads a short mono rotor-loop WAV from SD_MMC in 4-bit mode into a PSRAM AudioSampleBuffer, then plays it from AudioPlayMemory with looping and Doppler playback-rate control, stereo panning and distance-dependent gain. Requires /Helicopter.wav to fit in available PSRAM.
Generative Melody	Creates C64-inspired, ever-changing melodies with a repeating motif, chord-aware notes, a bass line, a sustained chord pad and four tempo-synchronised melody echoes.
Deep House Groove	Generates a 125 BPM four-on-the-floor beat with kick, noise clap, closed/open hi-hats with occasional ghost notes, syncopated filtered bass, offbeat seventh-chord stabs and stereo Freeverb ambience.
Neon Noir	Original 76 BPM cinematic synth piece with slow detuned pads, pulsing sub-bass, a more audible low mechanical drone synthesized from layered oscillators and pink noise, a louder distant slowly sweeping siren heard through the reverb, a sampled saw-table lead with a slow 2.05–2.75 kHz filter sweep, an airy chorale wavetable derived from the CC0-licensed New Vintage Dreams SF2, tempo-synchronised echoes, stereo Freeverb and per-node CPU diagnostics. This checked-in sketch targets the ESP32 ES8311/ES8388 setup. Only compact 256-point tables are embedded; the SF2 file itself is not bundled.

Implemented filters

Class	Functionality and notes
AudioFilterBiquad	Multi-stage biquad filter. Supports low-pass, high-pass, band-pass, notch, all-pass, peaking EQ, low-shelf and high-shelf coefficient setup. setAllpass(stage, frequency, q) preserves magnitude while changing phase; setPeaking(stage, frequency, gain, q) controls a bell-shaped frequency band. Up to four stages; coefficient handling uses Q30 internally.
AudioFilterDCBlock	First-order high-pass/DC blocker for removing DC offset. Default cutoff is 10 Hz; frequency(cutoffHz) sets another positive cutoff. Its Q31 feedback coefficient is recalculated per audio block, including after sample-rate changes.
AudioFilterFIR	FIR filter with caller-provided coefficients; supports up to 200 coefficients/taps. Coefficients must be designed for the current audio sample rate and recalculated when it changes.
AudioFilterLadder	Nonlinear ladder low-pass filter based on the Huovilainen/New Moog model. Offers linear or polyphase-FIR interpolation and uses 4× internal oversampling.
AudioFilterStateVariable	State-variable filter providing low-pass, band-pass and high-pass outputs; also supports a control input for variable cutoff.

Implemented effects

Class	Functionality and notes
AudioEffectBitcrusher	Reduces bit depth and/or effective sample rate.
AudioEffectChorus	Multi-voice chorus. The application supplies and owns the delay-line memory.
AudioEffectDigitalCombine	Combines two inputs using OR, XOR, AND, modulo, addition or subtraction. ADD and SUBTRACT saturate to the signed 16-bit PCM range. Supports block-wise bypass(bool), forwarding input 0 unchanged.
AudioEffectDelay	Delay effect with up to eight delay taps. Its sample ring buffer is allocated lazily, prefers PSRAM and falls back to internal RAM. The default maximum delay is 4000 ms.
AudioEffectEnvelope	Delay/attack/hold/decay/sustain/release envelope with note-on/note-off control.
AudioEffectFade	Fade-in and fade-out over a specified duration.
AudioEffectFlange	Flanging effect; the application supplies the delay-line memory.
AudioEffectFreeverb	Mono Freeverb reverb.
AudioEffectFreeverbStereo	Stereo Freeverb variant sharing the ESP32-adapted implementation.
AudioEffectGranular	Granular freeze and pitch-shift modes using an application-owned sample buffer.
AudioEffectMidSide	Encodes stereo left/right to mid/side and decodes mid/side back to left/right. Supports block-wise bypass(bool), forwarding both inputs unchanged to their matching outputs.
AudioEffectMultiply	Multiplies two input signals sample by sample. Supports block-wise bypass(bool), forwarding input 0 unchanged.
AudioEffectRectifier	Full-wave rectification. Supports block-wise bypass(bool).
AudioEffectReverb	Reverb with configurable approximate decay time; delay-buffer allocation can use PSRAM.
AudioEffectWaveFolder	Wave-folding effect with two input connections. Supports block-wise bypass(bool), forwarding input 0 unchanged.
AudioEffectWaveshaper	Applies a caller-defined transfer curve. The curve is copied into effect-owned storage. Supports block-wise bypass(bool).

Mixers and gain

Class	Functionality and notes
AudioMixer4	Familiar four-input mixer with per-input gain control.
AudioMixerN	Extended mixer supporting 4–32 inputs. Accumulates in float and saturates once when converting the final output to 16-bit PCM.
AudioAmplifier	Single-input gain stage.

Audio inputs and outputs

Class	Functionality and notes
AudioInputI2S	Stereo I2S input using the ESP-IDF I2S driver. Pin assignments are supplied through the Pins configuration. Exposes RX block, error and dropped-block counters.
AudioInputI2S2	Convenience wrapper selecting the second I2S controller. Board-specific pins must be supplied.
AudioInputI2S3	Convenience wrapper selecting the third I2S controller, where supported by the SoC.
AudioOutputI2S	I2S master output using the ESP-IDF driver. Supports explicit BCLK/WS/DOUT/MCLK pins and stereo-to-mono output selection.
AudioOutputI2S2	Convenience wrapper selecting the second I2S controller. Board-specific pins must be supplied.
AudioOutputI2S3	Convenience wrapper selecting the third I2S controller, where supported by the SoC.
AudioOutputPDM	Hardware PCM-to-PDM DAC output on I2S0; supports mono or stereo data lines on compatible targets.
AudioInputUSB	Experimental stereo-only 16-bit PCM source receiving host playback through the USB speaker endpoint. The default AudioInputUSB usbInput; constructor lets the library configure USB audio and CDC. Connect outputs 0/1 to the graph's left/right inputs. A small two-block queue decouples the framework receive task from AudioTask. No runtime sample-rate setter or resampling is provided.
AudioOutputUSB	Experimental stereo-only 16-bit PCM sink sending graph audio to the host's UAC microphone input. The default AudioOutputUSB usbOutput; constructor lets the library configure USB audio and CDC. Connect left/right graph outputs to inputs 0/1; the sink interleaves them as L/R PCM. Set AUDIO_SAMPLE_RATE_EXACT at compile time. USB FIFO feeding is automatic from AudioTask; there is no runtime sample-rate setter or resampling.
AudioOutputSPDIF	Software IEC 60958 consumer S/PDIF output using a fixed BMC lookup table and I2S0. Requires an optical transmitter or a suitable coaxial line driver; do not connect GPIO directly to coax.
AudioOutputSPDIF2	Same software S/PDIF encoder routed through I2S controller 1; availability and pins depend on the SoC and board.
AudioOutputSPDIF3	Same software S/PDIF encoder routed through I2S controller 2; available only on SoCs that provide a third I2S controller.
AudioRecordFile	Stand-alone mono/stereo 16-bit PCM WAV recorder. Uses a PSRAM ring buffer (512 KiB by default) and a separate writer task, optionally pinned to a core. WAV sizes are finalized and the temporary file renamed on successful stop.

The I2S2/I2S3 wrappers do not add extra hardware: they select a controller index. Whether those controllers exist, can be used concurrently and expose suitable pins depends on the particular ESP32 variant and board. AudioControlES8311 and AudioControlES8388 are available as codec-control classes, not as audio-stream inputs or outputs themselves. PDM direct-DAC output requires a compatible SoC/ESP-IDF combination; see I2S.md for pin mapping and clock configuration.

USB audio capture from the ESP32 to the host is experimental through AudioOutputUSB, and host playback into the ESP32 is provided by AudioInputUSB. The default constructors automatically configure the required shared endpoints and route stream-enable events; AudioMemory() starts the managed USB device. The USB environment enables AUDIO_USB_REDIRECT_SERIAL, so ordinary Serial output uses managed CDC; Serial0 remains UART0. Runtime ESP_LOGE/ESP_LOGW/ESP_LOGI messages are redirected to CDC while the host terminal is connected. The explicit-USBAudioCard constructors remain available for manual/advanced setup. Keep the compile-time audio sample rate aligned with the USB rate. S/PDIF output variants AudioOutputSPDIF, AudioOutputSPDIF2 and AudioOutputSPDIF3 are exposed through Audio.h. Each output requires exclusive use of its selected I2S controller; physical controller availability varies by SoC. TDM is planned; PDM output is implemented on supported targets.

Players and audio sources

Class	Functionality and notes
AudioPlayMp3	MP3 decoder/player. Can play from an Arduino filesystem or a generic AudioSource; used by the radio example for MP3 streams.
AudioPlayAac	AAC decoder/player. Can play from an Arduino filesystem or a generic AudioSource; used by the radio example for AAC streams.
AudioPlaySdWav	WAV/PCM file player for filesystems and seekable AudioSource objects. Supports loop and playback-rate control, and can preload a supported PCM WAV into a mono PSRAM AudioSampleBuffer for AudioPlayMemory.
AudioPlayMemory	Plays legacy uncompressed PCM arrays from wav2sketch or mono PCM from AudioSampleBuffer without copying; μ-law-encoded legacy arrays are not decoded. Supports looping and variable playback rate. The buffer must outlive playback.
AudioPlayQueue	Supplies application-generated samples or blocks into the audio graph; includes ORIGINAL and NON_STALLING queue behaviours.
AudioPlayer	Common player interface used by the decoder/file-player architecture, with codec metadata, MIME matching, playback state, position and duration methods. It is an abstract interface, not a concrete decoder.

MP3 and AAC use the ESP-beatsy software-decoder/AudioDecoderStream path. The radio example currently selects between those two formats based on the HTTP content type. Although the AudioCodec enumeration also contains WAV and Opus values, there is no AudioPlayOpus implementation in the current inventory; an enum value does not mean the corresponding player is implemented.

Synthesizers

Class	Functionality
AudioSynthWaveformDc	Constant/DC signal source.
AudioSynthWaveformSine	Sine oscillator.
AudioSynthWaveformSineHires	High-resolution sine oscillator variant.
AudioSynthWaveformSineModulated	Linear frequency-modulated sine oscillator. Zero modulation gives the carrier frequency; negative full-scale input stops phase progression (0 Hz/DC), while positive full-scale input approaches twice the carrier frequency. The carrier is limited to one quarter of the active sample rate.
AudioSynthWaveform	General waveform oscillator.
AudioSynthWaveformModulated	Waveform oscillator with modulation input.
AudioSynthWaveformPWM	PWM-style waveform generator inside the audio graph. This is a PCM audio source, not a hardware GPIO PWM output class.
AudioSynthToneSweep	Swept-frequency tone generator.
AudioSynthNoiseWhite	White-noise generator.
AudioSynthNoisePink	Pink-noise generator.
AudioSynthWavetable	Wavetable-based synthesis.
AudioSynthSimpleDrum	Simple percussive/drum synthesis.
AudioSynthKarplusStrong	Plucked-string synthesis using the Karplus–Strong algorithm, with optional adjustable low-pass damping via damping(0.0..1.0). Its 1024-sample delay buffer uses 2048 bytes per instance and supports the low E string of a standard guitar at 44.1/48 kHz. Default damping 0.0 retains the original sound.
AudioSynthHandpan	Hybrid handpan model with a built-in procedurally generated PCM16 finger-contact transient (custom samples may override it), six damped resonator modes per pad, a 5 ms smooth onset ramp, variable velocity, up to 16 pads, 32 short-lived strike voices, and eight overlapping attack voices. See synthesis documentation.

The shared AudioSineTable helper uses a 513-entry interpolated Q15 table. AudioSynthWaveformSineModulated deliberately retains the Teensy-compatible 257-entry AudioWaveformSine table and fixed-point interpolation for its linear FM path.

AudioSynthWaveform and AudioSynthWaveformModulated also expose restart(), which requests a phase-accumulator reset on the next audio update. To synchronize multiple oscillators, call their restart() methods between AudioNoInterrupts() and AudioInterrupts(). The Audio System Design Tool documents the method in both oscillator nodes.

Audio analysis

• AudioAnalyzeFFT256 — 256-point FFT analysis.
• AudioAnalyzeFFT1024 — 1024-point FFT analysis.
• AudioAnalyzePeak — peak-level measurement.
• AudioAnalyzeRMS — RMS-level measurement.

Build and board configuration

Use the PlatformIO environments in platformio.ini as the starting point for the intended board and framework configuration. Before running an example, check its pin definitions and codec selection (ES8311 or ES8388 where applicable). Network examples also need local credentials/configuration.

All SD-card examples use SD_MMC in 4-bit mode. The ESP32-A1S Audio Kit and Freenove ESP32-S3 Display pin mappings (CLK, CMD, D0–D3) are defined in platformio.ini; the old SPI SD pin definitions have been removed. On the ESP32-A1S Audio Kit, set the SD-card switches CMD and DATA3 to ON.

For board-specific requirements, implementation details and buffer/task notes, see the library technical overview. The current implementation is evolving, so treat the source in the selected branch as authoritative when an API or hardware detail matters.