Brown University STEM JAZZ Lecture Notes Part 1

The following PART ONE writeup of my notes I prepared for a talk given to a multidisciplinary group of students and professors in hard sciences at Brown University. The original ask was to give a masterclass on some topic in music and math. However, there is an A.I. malaise hanging over both fields, I figured it would be good to address the topic. It was a long talk and so I’m splitting these lecture notes in to 2 parts. Part 1 gives a heuristic look at the the music industry in the 1900s and 2000s and how technology and law have shaped some of the underlying incentives. Part 2 takes the lessons of the influence of tech in Music and suggests how to understand our current mathematical moment. 

Thank you to Dr. Stephon Alexander Of The Physics Department for inviting me to speak.

So I was asked to give a talk about the relationship between math and music and usually I have a whole bag of tricks around this like analyzing a Coltrane diagram or symmetries in the Octatonic scale, or diagramming a Bach fugue or prattling on about frequencies, maybe with a Chladni plate so you could see some cool shapes. Maybe for this talk I’d even go into some Kirchoff-Love Theory. But actually, right now there is a bigger problem at hand which is that AI is coming for both of the things I like and threatening to upend the systems that uphold them as professions. As a mathematician, it doesn’t affect me much in a practical sense because at this point in my life I am hobbyist. I read Differential Geometry, Algebra, and Category theory books for fun and I like to keep up in new results in theoretical physics when I’m able. It is a language in which I enjoy reading, speaking, and thinking. 

But as a musician, AI may or may not be an actual problem for my own career.

Still in both situations AI’s use in these fields upset my own personal values as well as my sense of the world order. Those values include beauty, exploration, personal relationship to knowledge, and building something long lasting. I think its important to stick to values, however my sense of the world order can change with research, perspective, and time. Maybe AI will be a good thing for all of this. Im not closed to the possibility, but I’d make a bet that the costs will outweigh the benefits, with respect to my own values. Preparing this talk changed my perspective on how things work and that shift clarifies the character of the moment. When you look at the history of music in the 20th century, not from the perspective of albums, musicians, compositions, styles, or cultural movements, but from the perspective of the underlying business incentives established by Law and Tech the typical musician complaints about pay, record labels, and why music is “garbage” now begin to make more sense. This framing also gives some new insight to the AI question as someone in the field, and maybe some indication of where this is all going. 

These lessons port over to the mathematical community I believe, which has been decked in the face by AI. For those not keeping score at home, Open AI has been publishing solutions to research problems that many distinguished mathematicians have been working on for years. This has meaningfully changed the trajectory of careers, discouraged students and senior professors from continuing in the field, and has called into question the purpose of our system of human mathematics. I keep up with Terry Tao’s blog, which has been flush with guest posts about what should we do. I also talked to some of my friends who are pros in the field and I believe this question has some material value to them. 

A quick note about AI. I have compelled by the argument in Yudkowsky and Soares “If Anyone Builds It, Everyone Dies”. I think that collectively, we are monumentally stupid for pursuing ASI. Ultimately, it takes us out, in my opinion. My P(doom) is high. I think all of our great mythologies from antiquity have warned us of such a thing and that all of our great science fiction writers of the past 200 years have warned us again and yet, doofy, misanthropic, death cult tech bros and their cartoon supervillain-like bosses and investors are driving us to the cliff because they got zero-interest-rate fake money from quantitative easing and fake valuations based on fake consensus math and here we are. All of this could have probably been avoided if we made all Stanford dropouts read The Brothers Karamazov, but I digress. I bring this up because I don’t want to have a discussion about where things are going actually. The answer is high extinction risk, high doom probability, high disruption to society. But this talk is built on the assumption things continue on reasonably smoothly.

I’ll proceed as follows. I will outline the salients in my pet theory around the history of music in the 20th century, and map the key lessons homomorphically into the world of math which is undergoing similar existential and philosophical transformation. Then I’ll take some questions and we’ll all hang out for the rest of the night. I might play some saxophone as well, because I’m good at it and you will enjoy it. 

Music

The big thesis is that the age of the upper middle class professional musician in America was a historical accident, as was the preponderance of musicians who became very rich and influential. Music, in the grand scheme of things was never a stable career path, certainly not before the 20th century. However musicians in the 20th century were successful enough to at least convince their juniors that a career in music could work. This success was an emergent phenomenon of a strange confluence of law, technology, and American dominance in the post war era.

As a caveat, perhaps this is all reducible to:

“post WWII america was the biggest economic expansion in known world history, it peaked in the 80s or 90s and musicians, like everyone else are in the downturn” 

I don’t have evidence against this trivializing argument, but its too glib and too general, to be orienting as a musician trying to make sense of things, so I won’t explore this line of reasoning either. Anyway my story is more fun to think about than the low-hanging-fruit macroeconomics position.  

Let me outline some of the Key moments in the story that made this accident possible. Note that very little of my analysis will be about music itself. I will not say “Stravinsky brought in the age of such and such, which caused Dizzy Gillespie and Charlie Parker to investigate this and that; and Son House influenced Robert Johnson and Big Mama Thornton whom the Beatles and bands of the british invasion studied voraciously which put rock in the cultural center of thus and so” 

Nope. This is law, tech, and incentives. And the story begins with the copyright act of 1909.

The copyright act of 1909 arose from a dispute over piano rolls. Heretofore a composition was published in sheet music, but the new technology of player pianos allowed for a mechanical representation of the composition to be played automatically in pianos that could read them. Who should be compensated from the sale of piano rolls? The copyright act of 1909 established that the owner of a composition (maybe the composer, maybe not) should be paid a royalty of 2c for each mechanical reproduction of their composition. It also established something called a compulsory license, which granted permission for anyone to reproduce any composition, without having to obtain permission from the composer (or rights holder), in exchange for a blanket fee to a company that collects and administers these royalties and pays out rights holders. 

Technology forced a new law into existence with far reaching consequences. The copyright act makes songs into an asset class, a financial instrument that represents an income stream not only through publication of sheet music but through any mechanical reproduction. With the phonograph (Edison 1877) having been invented and gaining some traction there would be a myriad of ways to make money off of “owning” songs. 

Now owning a pile of songs is not enough because if no one buys their mechanical reproduction, you have no cash flow. So you might have the bright idea to find ways to make your songs very popular. And with the advent of the radio (Marconi 1894), which allows for more reach than any broadcast technology before it, you might see a way to get very rich indeed. 

You might even organize a company that collects songs and promotes them. Maybe you pay songwriters a flat fee or offer to promote their songs for a percentage. You might convince popular singers or bands to sing your songs (or sell you their originals) in exchange for your effort in recording, pressing, distributing records, leveraging their popularity to increase your cashflow and offering them an opportunity for brand expansion. By integrating the recording and distribution apparatus to your royalty collecting outfit you have the makings of a record label, a business entity that had never existed before and whose profits depend on the popularity of their IP. 

If you can keep costs of recording, distribution, promotion and artist compensation down, you can manufacture assets that have no natural limit to their potential cashflow. The key thing here is that the songs, not the acts, are your source of revenue.

Ok, Now that you have some songs, how can you make a song super popular so that it makes you super money? Find and cultivate performers. Again that’s a win win. An unproven performer can ride off the popularity of an established song and you get another bite at the royalty apple. Another thought occurs to you. What if you spend money on promoting your performers?

People, especially this new postwar demographic called “teenagers” really connect with stars. That guy Elvis Presley, for instance, seems like he has a hold on young girls. If he sings songs in your catalogue, that repertoire will print money for as long as people know him. And it doesn’t even matter if he wrote it, appropriated it, or even has the best version. His FAME, more than his creativity or skill, drives the value of the income stream. Then you think, If you control who gets to be a star, and then control the star with a deal that works in your favor, and make that star sing your songs (or just find a way to own the songs that they sing) then you’ve increased the value of your song catalogue and kept everything in house. 

Now pressing each record is a gamble because its very hard to tell what the public likes and recording records and making stars costs a lot up front. That means you have to be responsive to the public demand. You would love to just force the public to like what will give you the most return but you don’t have that much control until the Telecommunications act of 1996 and the rise of social media. You have to spend a lot of money to record and distribute a lot of music in order to find that big hit and its tough to keep the lights on. 

So you hire people who seem to have an instinct for what will be big. You segment your catalogue into genre to organize promotion a little more easily. (Over time people will self organize around these categories because they believe it’s heritage and culture) You also create (sometimes illicit) relationships with radio DJs. You really like thoughtful DJs like Todd Storz who was one of the first to investigate what songs were played most in regional jukeboxes and designed his radio program around that, creating the first hit radio format in 1951 and giving you valuable feedback. 

Later in the mid 70s, Lee Abrams would create a similarly structured format called AOR based on customer data that lets you know what the hits are and when. And when social media comes out, you’ll have immediate feedback. To them taste is intuition and magic, to you its probability. The expected value of an ensemble of bets is the same or better than the EV of an individual. Make a lot of bets at the same time and make sure the ones that hit pay for the ones that don’t.

Now I just skipped a lot of time! From 1909-1973 we have the blues, big band swing, the birth of the crooner (with the introduction of the Shure 55s Microphone), bebop, avant garde classical music, Jazz, Country, Rock and Roll, Bluegrass, and classic theater music. But thats just the point! To you, the rights holder, none of the content really matters as long as you have songs that make money. You can hire writers to match the mood, own across genres, bring in artists and bands that have the zeitgeist, and have no personal or artistic stake in the music at all. You have a simple rule, established in 1909: If the songs get big you get richer; and if you can make an artist famous, the song will probably get big. 

No actual record label thought purely like this, or had scale to control the industry in such a fashion. But this this incentive structure was always there. It could be the reason for our “American Songbook.” Not our early 20th century love of theater and cinema, not the use of “functional harmony” or nice melodies, or universal lyrical themes, or the sauce added by Jazz musicians to “corny” vanilla show tunes. Those musical and cultural considerations are important but they are just the means by which the incentive expresses itself. 

Said another way, the above reasons come from the perspective of the consumer, which is downstream from yours, who took the risk to pay for the music to exist in the first place. Seen your perspective, it now makes sense that musicians were encouraged to record “Standards.” Because the rights holders of the compositions, like you, were likely financing the record dates, or at least influencing, at a high level, what repertoire a smaller label believed would sell.

All the cultural movement, change in taste, evolution in musical skill and technique; all are orthogonal to your baseline natural incentive even through the tumultuous 60s and 70s, when music was so closely tied to identity. Note that this is not usually how we talk about the cultural evolution of music in America. Most of the discussion among music fans and critics is tied up in style, genre, identity, influence, art, commerce, and culture. But you don’t care about that, you care about the value of the IP you own. 

Here is the part that affects musicians. During this time, you have to hire good musicians to make good music and because you have made the lifestyle associated with the musician appear so exciting, and their messages so impactful, kids are growing up thinking that being a musician is a reliable or aspirational career move. For those with talent, it might be, because talented musicians capture the public attention. Or if they have no charisma, they can add something special in a recording or arrangement, or just keep studio costs down by not wasting time. Some want to be stars. But some just want help make your assets, whoops, I mean your songs, for a living, whether they are writing their own and offering them to you for promotion or they are helping bring them to life in the studio. 

They tend to be temperamental and don’t really understand your role in making everything happen, except that you cut their checks and sometimes those checks aren’t big enough, or that sometimes you tell them what to do and don’t let them express themselves. But its better that way because telling them what the game is would disrupt your operation. Plus, you have jobs for them because business is booming! Even better, they all know your songs because they grew up on them! Those jazz musicians especially really seem to like your old acquisitions from the the 20s-the 50s. They train their skills on them. They’ll be making you money forever.

But something happens in the 70s that changes your incentive structure. The Copyright act of 1976, developed by the great Barbara Ringer. This new change outlined the fair use doctrine, which allows songs to be used without explicit license or royalty for education, criticism, or parody. That messes up your cash flow a little because there is now a way for people to use your stuff for free, but it still raises awareness around your music, which is cool. 

But more importantly, it establishes a 2nd copyright in the sound recording, differing from the underlying composition. This creates another potential source of cashflow in something you already own! But optimizing it is subtle because there is more potential money in each individual version of a song. While you could continue with the current playbook, you surmise that there is slightly more value in promoting the ARTIST attached to the sound recording, because it is cheaper to license and administer a finished master recording than it is to record and promote several bands cover the same song. It’s also easier to control one artist than it is several, although you adopt some reputational risk, especially once we get to the cancel culture era down the road. But for now scandal can work in your favor, especially in the times of Sex, Drugs, and Rock and Roll.

Anyway, This means that you’ll want to change the creative priorities now that the relationship between the fame of the artist and success of the sound recording is tighter. You understand the influence celebrity has over young people in the current media environment and especially as hip hop develops (and pop begins to follow its format). You’ll want songwriting to become more styled to an artists brand and image, like a commercial for the artists brand, rather than a composition meant to be explored by others artists.

Accordingly we can trim the fat. We don’t need as much in terms of melody or harmony. Loops and simple melodies will suffice as the artist persona and sound design begin doing the heavy lifting. And new technology, like the synthesizer (Moog 1964), drum machine (Linn 1982), Samplers (Also Linn 1988) and Digital Audio Workstations (Digidesign ProTools 1989) reduce the cost of studio equipment and preclude the need to hire musicians and arrangers. People still like the IDEA of bands, especially in live performance, but so much of the sounds are crafted artificially that actual instruments can made strictly ornamental for some acts, especially once we hit the 2000s/2010s.

Of course some great songs, in the classic sense, still get written. You’ll gamble on those with the right act, (artists with really special voices) and for the right demographic. But for the most part after the turn of the century you can acquire “beats” from “producers" for cheap and make your artists (chosen for charisma, sex appeal or shock value as much as musical talent) create an unlimited supply of sound recordings; Similar enough to promote together but different enough to create a sense of variety for fans. 

If people still want “good music” there was more than enough recorded by great musicians in the 60s and 70s. Your “producers” can just sample that and (maybe) pay those rights holders, which might just be you anyway. Or they can spend their money on the old stuff you own. You don’t care.

The culture people will say we lost something and criticize your direction. They will complain that melody went away. They will lament the loss of harmony and structure. They will say musicianship is no longer important to culture, which you now have enormous control over due to the aforementioned telecom act of 1996, which eliminated limits on how many radio stations could be owned by one company and consolidated central control over what was being played everywhere. They will say everything feels disposable, even more so in the streaming era. That music feels cheaper and that there is less skill involved. That it lost its soul. 

But you never cared about all that, you want to increase the value of your assets. It is in fact cheaper, the process is far more efficient, because modern tech allows it to be so. Isn’t that what technology is supposed to do? Reduce costs, automate processes, create an abundance of end product? And frankly, you reduced costs because as physical sales, a cash cow, gave way to digital media and digital piracy, your margins started to contract. And the global economy turned sideways in the 80s, early 2000s, late 2000s and 2020s. You had to cut costs as much as possible to stay afloat.

Plus the culture people will complain and then just go right on consuming what you give them whether new or old. Either way, they increase the value of your IP. Fair use and licensing deals ended up working in your favor by the time you got to the 2020s because kids now post videos of themselves scored by songs that haven’t made you money in decades! The people who are really mad, not that you need to be concerned, are the musicians. They’ve applied brilliance and discipline to their craft. They’ve studied the best of their cultural heritage (recorded, promoted, and owned in part by you). They feel their hard work of learning an instrument and a bunch of your old catalogue is advancing culture in some vague way they can’t really put their finger on. 

Benighted fools. Poor unfortunate souls. Your business, recording, administering and promoting songs and people to goose an asset class you own, resulted in core memories and a sense of identity for these brilliant hardworking people and now they are irrelevant. You almost feel bad for them. They aren’t needed in this incentive structure because they are so expensive and sophisticated, when your business works on cost cutting and simplicity. Back when you needed their input things were sophisticated, but today’s audience? Nah… Your new artists can do glorified karaoke on stage to a packed stadium, they don’t need a band! Instrumental covers of a lot of new music wouldn’t even make sense because songs are wrapped up in the vibes and persona of the original artist. Some people still like it and so you apportion some small amount of your budget to please that market while focusing on the real moneymakers. 

And musicians are really mad about this AI thing because it threatens to make them fully redundant, especially if you can align public taste with AI. I mean, you can make an original recording in 2 seconds and own the IP. It’s all margin! 

Making AI songs stick is a little tricky though. With an oversaturated market, the cost of promoting and distinguishing new IP might get a little high. And the payout from each song might not last as long. You need songs to be big in order to make money and thats just hard in a crowded market.

Plus the culture people talk A LOT on the internet and they really don’t seem to like AI on principle, even if they like AI music when its good enough to trick them. You’ve changed peoples minds before but this one might be hard. Maybe some artists whom you’ve made rich will run interference for public opinion.

SUMMARY/TLDR

Anyway here’s the point. Musicians thought their skills mattered and they had permanent merit based careers that grew by network, and proximity to opportunity. They mistook their personal, artistic, and aesthetic beliefs for the value their counterparty, the rights holders, wanted from them. This misunderstanding has induced much weeping and gnashing of teeth when musicians value shrank due to changing incentives on the other side of the table. 

The real leverage was fame all along because popularity controlled the value of the underlying asset. Heritage, creativity, cleverness, and skill were always meant to be in support of fame given the natural incentives of the label. When music was more human and musical, then they were strong enough to build a life on for the talented. These incentives flowed directly from the copyright acts of 1909 and 1976 which made compositions and sound recordings into asset classes that could create cash flows over a long period of time. 

Consider that most of the American Songbook, as well as the many of biggest rock songs are all written before the 80s. After this time, songs become associated with an artist, rather than a shared public property. Think about Pop, Hip Hop, Alternative Rock. Most of this music is uncoverable for an artist looking to establish a brand. Sometimes it works like when Luke Combs covered Tracy Chapman but the exception proves the rule. Labels still make money off of covers anyway, but a master from a massive artist will be in commercials, movies, stadiums, with no extra charge or risk to the rights holder. 

Note that through the 70s we still need great musicians to make music and create iconic live performances. The late 40s through the early 2000s are in fact the golden era for solid instrumentalists to find their way into the middle class. Visual media needed performers, touring grew as an industry, studio work was frequent and protected by the union. All because you needed great musicians to make a great music product. 

This starts to shift around 1980 with the arrival of the drum machine, which begins to take away studio work from drummers as new popular genres are built around its sound. As the technology advances we start to see multitrack recording get to a point where preparation becomes more and more optional as musicians can do as many takes as they like. Soon as we get to the late 1990s and early 2000s, we will gain the ability to heavily process vocals well and make music completely on a computer with samplers and DAWs, lowering the bar for performance talent. Hip hop and modern pop, the dominant genres in the late 90s through the 2010s eschew the need for live instrumentation altogether. In 2009, the beat maker lex luger will proclaim that many of his hit songs with Rapper Waka Flocka Flame were produced a hit in 10 minutes or less of free software with stock sounds. Bands stop dominating pop. Rock dies. EDM takes over.

You as a rights holder LOVE this. Music is cheaper to make which allows you to keep costs low and focus on making artists as famous as possible, earning royalties from the sound recording, compositions, and now, due to the 360 deal, all of the artist’s income stream that has to do with their music. 

But to musicians, trained in a craft, honed in a heritage, and arriving into the world looking to innovate their fundamental values to an adoring public, in exchange for a slightly upper middle class lifestyle, they’re twisting in the wind because they didn’t realize the game that was being played. None of this mattered to the people with the biggest checks and bigger incentive to influence culture for their own reasons.

AI is natural next step in the technology motivated erasure of the craft and heritage of music performance, composition, theory and thought. However, it might backfire by over saturating the market with slop that people can’t be convinced to want. And we are still probably hard pressed to spend money on an AI concert. Musicians now have the tools to make themselves famous and influential to a certain point. The consensus is that major labels are still needed for stardom, but entrepreneurial musicians can create a respectable career by understanding and owning their income streams. And the technology allows people to create song assets and become famous on platforms outside of the music industry’s control. 

Next
Next

Brown University Lecture Notes Part 2