Building an AI Music Composer

Whats poppa lockin?

I built a terminal-based AI that composes music autonomously. Let me tell you about it.

A few weeks ago a few coworkers proposed an agent build-off. I've been looking for an excuse to play with openai's agent sdk for more than just work. It took me a while to actually have the idea- But I thought - if AI can execute code - and you can write midi music with python - then an AI agent that can execute code - CAN PLAY MUSIC!

So what does the thing do?

Basically ------- you tell it what style of music you want (like "synthwave" or "jazz"), and it:

• Writes the MIDI Code
• Another Agent checks the code quality for musical coherence
• Once code passes, the code executes and a midi file is created -the python converts midi to wav and the code plays the file
• After it plays - another agent builds on the composition - adding more instruments - then plays again
• The arrangements build in complexity and duration as the loop continues

The Tech Behind It

I built this using some really practical tools:

• OpenAI's agent library for the brains
• pretty_midi for handling the actual music creation (I know its an old library and I should update)
• FluidSynth for playing the sounds
• Python becuase Python

What I Learned Building This

Some interesting things came up while working on this:

• AI agents can be SURPRISINGLY creative when you give them the right tools
• Lots of iteration on the prompts to even begin to get some MIDI code that sounded like music.
• It still has a hard time staying on beat with simple music
• The 40-year-old MIDI standard works perfectly with modern AI--- who knew?

What Still Needs Work

Being completely honest, there are some things I'm still figuring out:

• Making the musical improvements more focused
• Adding musical knowledge without overcomplicating things
• Finding better ways to evaluate if the music is actually "good"

Want to Try It Yourself?

The setup is pretty straightforward:

• Python 3.8+
• An OpenAI API key
• FluidSynth installed on your system
• A SoundFont file
• A few Python packages

I've included complete setup instructions in the repo----[https://github.com/williavs/agentic-composure]

Why This Matters

Here's what makes this project ACTUALLY interesting--- it shows how AI can be more than just a tool. It can be a creative partner that thinks out loud, explains its choices, and improves its work over time.

Whether you're into AI, music, creative coding, or just like building cool stuff, there's something here to explore. And the best part? It's all built with tools you can use right now.

The code's available if you want to check it out or build something similar. Let me know if you make anything cool with it!

#AIdev #MusicTech #CreativeCoding #Python #OpenAI