How to Generate Music with Vocals from Text and References on Your GPU

Cloud-based music generators like Suno or Udio produce excellent results, but they require a constant subscription, limit your credits, and tie you to external servers. If you want to run a similar tool locally, for a long time there were practically no adequate alternatives. Most open-source neural networks could only generate short background samples without vocals or required complex console manipulations.

Recently, developer BazedFrog released the SongGeneration Studio project. This is a convenient web interface for the LeVo model, created by researchers at Tencent AI Lab. The application is packaged for one-click launch via the Pinokio installer, so you can deploy it without deep knowledge of Python and PyTorch.

What the application can do

The project is built on a generative model capable of synthesizing a complete audio track from a text description and lyrics. The SongGeneration Studio web interface turns this engine into a full-featured mini studio.

Inside, there are several main features:

• Composition structure builder. The song lyrics can be divided into logical blocks: intro, verses, chorus, bridging section, and final outro. This helps the neural network build the correct musical form.
• Detailed style customization. You select the genre (from pop and hip-hop to metal and jazz), emotional tone, tempo in BPM, as well as the vocal type and set of lead instruments.
• Style cloning from an audio file. If you upload a reference track, the model will try to copy its character, rhythm, and overall sound when creating a new song.
• Multitrack separation. The output gives you not only a mixed track but also separate stems: clean vocals and instrumental arrangement.
• Project manager and export. All generated tracks are saved to the built-in library. The finished result can be exported in lossless FLAC format or as an MP4 video file with cover art.

Separating isolated vocals and instrumentals makes life easier for those accustomed to refining material in DAWs like Ableton, FL Studio, or Logic. Samples can be easily processed with effects, trimmed, or layered over your own beat.

Hardware requirements and installation

The main limitation of the project is related to resources. The LeVo model is demanding on video memory.

You'll need an NVIDIA GPU with at least 10 GB of VRAM. However, for stable generation of long tracks in high quality, the developers recommend having 24 GB of video memory. You'll need about 50 GB of free space on your hard drive, as the model weights themselves take up about 15 GB.

Deploying the system is extremely simple thanks to the Pinokio platform:

1. Launch Pinokio.
2. Search for SongGeneration Studio.
3. Click the install button.

The installer will automatically install the required Python version, download dependencies, and load the model weights. Once complete, just click Start, and the interface will open in your browser.

How the generation process works

After launching the web interface, creating a song consists of four steps:

1. Text input and markup. You paste lyrics and distribute them across song blocks.
2. Stylistic parameter selection. Genre, mood, vocalist voice, and instruments are set.
3. Adding a reference if needed. Uploading a short music excerpt helps lock in the desired sound design.
4. Starting generation. Calculating one track usually takes 3 to 6 minutes.

From personal observations: vocal quality directly depends on the rhythmic structure of the text. If you dump continuous text without rhyme and clear meter, the neural network starts to stumble, stretch vowels, or "swallow" endings. If you break the text into even four-line stanzas, the result turns out significantly cleaner.

The task queue function also works interestingly. You can prepare several text variations at once and send them for background generation, then choose the best result from the library.

Limitations and overall impression

The tool cannot yet fully replace studio recording or a professional arranger. Sometimes the vocals can sound slightly synthetic, and generation on cards with 10 GB of VRAM may hit the memory limit with overly long texts. Additionally, the project is still young — the repository has around 550 stars and about twenty open issues.

Nevertheless, SongGeneration Studio is becoming an excellent tool for quick experiments. It will be useful for indie game developers creating soundtracks, content creators getting background music without copyright issues, and musicians generating quick demo ideas.