The pipeline from prompt to audio has four stages, and each one introduces characteristic artifacts.
Raw audio is enormous — 44,100 numbers per second per channel. No model generates that directly at scale.
Instead, a neural audio codec compresses audio into a much smaller sequence of discrete tokens. Think of it as a learned, extremely efficient audio format: a few hundred tokens per second instead of 44,100 samples.
This compression is the first place quality is lost. The codec keeps what matters perceptually and discards the rest — which is why generated audio often sounds slightly glassy in cymbals and sibilance, the content hardest to represent compactly.
The model trains on large quantities of music, learning statistical relationships: which tokens follow which, how harmony typically moves, what a chorus sounds like relative to a verse.
It does not store songs. It stores patterns — the way a language model stores how sentences work rather than the sentences themselves. That distinction matters legally, though it is not a complete defence: models can and do reproduce distinctive training patterns closely.
Your text is encoded into a numerical representation by a text model, and that representation steers generation.
This is why prompting is imprecise. "Melancholy lo-fi with warm Rhodes" becomes a point in a mathematical space that pulls the output in a direction. There is no explicit instruction being followed — which is why the same prompt produces different results, and why asking for a small change often changes everything.
The generated tokens are converted back into a waveform by the codec's decoder. The decoder reconstructs plausible audio from a compressed representation, which is where the final texture quality is decided.
| What you hear | Where it comes from |
|---|
| Metallic cymbals, harsh sibilance | Codec compression |
| Structure drifting after 90 seconds | Model context limits |
| Same prompt, different result | Sampling randomness |
| Small prompt change, totally different song | Prompt conditioning is holistic |
| Slightly blurry stereo image | Codec + decoder reconstruction |
Every stage above produces rendered audio. There is no project, no tracks, no MIDI — the musical decisions are baked in at the moment of generation.
This is the structural reason "just re-prompt it" is the only edit a generator offers, and why re-prompting changes everything rather than the one thing you wanted changed.
The alternatives are tools that generate editable output — MIDI you can change note by note — or tools that take a generated track, separate it into real stems, and let you work on the actual audio. Veena is built around that second approach: bring the render, get real stems, edit from there.
Related reading: diffusion vs transformer music models, why AI music generators can't edit, and how to edit AI-generated music.
Start making music in Veena
Free, browser-based, no downloads required.
Try Veena Free