Artificial Intelligence for Musicians

Music theory began with a walk. Pythagoras walked past a blacksmith's shop, heard the hammers ringing off the anvil, and noticed the ones that sounded good together were in simple whole-number ratios, like a perfect fourth (4:3) or fifth (3:2). Johannes Kepler spent years calculating planetary orbits by hand, convinced the geometry of the solar system hid musical harmonies. Four centuries later, John Coltrane drew diagrams linking scales, symmetry, and cosmic patterns. For thousands of years, people have sensed that math and music are two ways of looking at the same thing. It took hard work like painstaking calculations, overcoming many failed experiments, and a lifetime of practice. The reward for all that work are the modern chords and scales that musicians use today.

Today, music is facing an existential crisis. AI can be a powerful partner, or it can straight-up be Dormammu from Doctor Strange, the force that shows up to consume everything and leave nothing of you behind. It's a tricky time to be a musician. You can now "make" a complete song in under a minute, prompted with a single sentence. But it comes with a catch. Hand everything to Generative AI; your learning, your playing, your creative work, and it will replace you before you know it. The smarter, and, frankly, much more fun move, is the opposite, use it to create music that wasn't possible before and push your own creativity and musicianship further. Understanding how AI actually works is what will decide that outcome.

So let me start with the idea my book Artificial Intelligence for Musicians is built on: you can outsource tasks, even parts of your thinking, but you can't outsource your understanding, your musicianship, or your creativity.

I first saw that idea from kache (@yacineMTB) on X, and it's stuck with me ever since. It's as true for engineering as it is for music.

Understanding is what enables you to identify what's inside an AI output, steer the tool, and keep the model from becoming the musician. Without it, AI can basically walk off with your musical ideas, your creative patterns, and secretly steal your rights.

The Musician and AI/Machine Learning Connection

I build AI systems and teach AI/ML engineering for a living, and I'm also a jazz musician, improviser, composer, and educator. That combination is rarer than it should be, and it's the whole point of this book: if done right, understanding both AI and music can be a superpower.

Many musicians already have the music part down but not the ML or AI foundation. At a recent workshop, I asked a room full of professionals what happens when an AI model recognizes a sound. Most said something like "it detects a pattern" or "it listens and makes predictions." Not wrong, but not the whole story. AI tools are built to abstract their complexity for convenience and to talk to you like a human behind a slick chatbot user interface.

So, when you use AI without understanding how it works, the speed of the output can feel exciting and creative. It can feel addictive. But if you can't explain how the model reads your audio or your prompt, what data shaped it, or how the AI listens, then you're not doing much at all. Instead, you're handing over creative decision-making to a black box you don't control.

AI doesn't actually hear the way you do. In a lot of AI audio systems, the sound gets turned into a picture first, usually a spectrogram, which is an image of how frequencies move across time. A kick drum, for example, makes a different image than a snare. A spoken word makes a different shape than a guitar chord. The model doesn't listen like a human, it learns patterns from those images and sees only the shape of the sound.

In Artificial Intelligence for Musicians, we explore how AI works before you build anything. Specifically, I cover how sound becomes data, how that data then becomes a training dataset with labels, how labels shape a model's output, and then, how these models can be incredibly useful but dangerous at the same time.

We also go hands-on and build a tiny model together with a no-code tool where your sound becomes an image, and the model learns the image's pattern. That's the technical reality behind the scenes, it's all machine learning.

Thinking About AI as Software

Here are three simple ways to think about AI, borrowed from AI researcher Andrej Karpathy.

Software 1.0 can be identified as traditional programming. You write explicit rules. Say, the chord has C, E, and G in it, we call it C major. You know the rule, so you know why you got the output. It feels a lot like music theory. It's deterministic.

Software 2.0 is machine learning. Instead of writing every rule, you give the model examples. Here are kick drums. Here are snares. Here are my favorite beats. The model then picks up the patterns on its own.

Deep learning is loosely modeled on the way the human brain works. It mimics our biological neurons with math and layers of artificial neurons, which is why we call it "deep." It is the most capable AI we have right now, and it's what is used to build most of the generative AI tools on the market.

Software 3.0 is language models. Tools like ChatGPT, Claude, and Grok let you request output from a large language AI model with plain language or prompts. This is where prompt engineering begins. You can think of prompt engineering as a kind of programming language, and Karpathy's tweet summarizes it best: "The hottest new programming language is English." It's non-deterministic.

So, that means generative AI is Software 3.0 and it's an end-to-end system, meaning you ask for an image and it'll create the image, you ask for music and it'll give you music. That's why prompting is like programming in plain language.

Why Understanding AI is So Important for Musicians

I see AI music diverging into two camps. One camp wants music to become generative. A single prompt as input and a finished song as the output, with years of practice and self-expression treated like a problem to be solved. The other wants AI to augment musicianship, so you understand more, build more, and own more. That's the camp my company, Barrios AI, is in.

Here's the problem with the first camp. Music has been a part of every community since the beginning of civilization. It has had an integral role in every gathering, celebration, funeral, and protest. When you take the human expression out of the process and there's nothing behind it, music becomes, well, pointless.

And this isn't just sentiment or opinion. Active music-making physically reshapes the brain in ways that passive listening and a prompt cannot. Gaser and Schlaug's 2003 study found professional musicians had measurably more gray matter in motor, auditory, and spatial regions than non-musicians, and the difference tracked with how much they had practiced. Years of playing builds that, not just years of listening. Live performance goes even further. Trost and colleagues found in 2024 that live music drove stronger, more consistent activity in the brain's emotion centers than recorded versions of the same pieces, partly because a performer reads the room and adjusts in real time. This is what Erykah Badu is protecting when she calls herself a performance artist, not a recording artist. The moment she becomes "one living breathing organism with people" in front of a live audience benefits herself and others in a way that generative output never will.

Music has always had a relationship with technology. Instruments, guitar pedals, loopers, studios, synths, samplers, DAWs, plugins. Every one of them changed the way musicians work, and every one still required you to understand what you were doing. AI is the first tool that can imitate creativity without being creative.

The future I want isn't AI supplanting musicians. It's musicians who understand AI well enough to build their own systems, protect their own work, and realize potential that neither human nor machine can reach alone. That's what Artificial Intelligence for Musicians is about, using AI to protect and deepen human creativity, not be replaced by it.