Riffusion

Riffusion is Developed by Seth Forsgren and Hayk Martiros, it was launched in December 2022 and employs the Stable Diffusion model, historically used for generating images from text prompts.

How Riffusion Works

1. Text to Spectrogram: Users input text prompts that describe the desired audio. Riffusion uses Stable Diffusion to generate spectrogram images based on these text prompts.
2. Spectrogram to Audio: The generated spectrograms, visual representations of audio frequencies, are converted back into audio files using an inverse Fourier transform.
3. Music Interpolation: Riffusion can interpolate between different spectrogram outputs, allowing it to create seamless transitions and extend music tracks by incorporating different styles and sounds within its latent space.

Features and Capabilities

• Text-to-Music: Allows users to generate songs by simply adding lyrics and describing the sound.
• Image-to-Music: Can analyze visual content to generate music that matches the images.
• AI Covers: Users can upload a track and create AI covers, arranging music around the sample.
• Dynamic Music Creation: Provides three main modes for music creation: prompt, compose, and radio mode, offering flexibility in generating music.

Platform and Accessibility

Riffusion offers a free web platform, making AI music generation accessible to anyone interested in exploring music creation without needing advanced skills or expensive software. The latest version of Riffusion, as of 2025, remains in the public beta phase, allowing users to experience its functionalities without cost.

Competitive Landscape and Ethical Concerns

Riffusion exists in a competitive field alongside platforms like Suno and Udio, offering core features such as AI cover generation but facing challenges related to audio quality and ethical concerns about training data licensing. The platform plays a pivotal role in democratizing music creation, sparking debates about AI's impact on creativity and the music industry.