AI models producing music they can't hear
We made the FL Studio MCP and Producer Arena because we wanted to understand how frontier AI models understand and reason about music. They have no native way to parse and understand it. Instead they have to use smaller models and renders of waveforms (like spectrograms) to understand what an audio sample sounds like.
So we gave the models production software and asked them to produce music and be an assistant to a producer. That led us to build an MCP and to let the models use computer use on the FL Studio app, which let them use vision and other ways of seeing the music to understand and create it. With just this, they can create decent music. Super interesting.
We think one of the next big applications of AI is in audio (even in game dev and apps, finding the right audio and SFX matters), and we want to see how the models improve over time at understanding it. Check it out today: musicarena.io
“what if beethoven had access to modern production software and instruments? cook up a concept.”
The rankings are on the leaderboard