Avoiding Spectral Logjams

Avoiding Spectral Logjams

Having designed, edited, and mixed the sound for hundreds of action sequences, I can testify that the biggest problem to overcome is usually having too much sound.

The “too much sound” comes in several forms. The most basic one is simply having too many simultaneous, individual sounds competing for attention. We sound designers tend to be afraid of having produced too few sounds for a given sequence, and in trying to allay that fear, we produce too many.

The classic mistake is to throw several mediocre sounds at a given moment in the hope that when they play together some kind of magic will happen to turn the moment into something better than mediocre. It rarely does, and you wind up with a bunch of average sounds masking each other into noise. The solution is to have fewer sounds and better sounds. That’s where creativity is necessary, as opposed to throwing a ton of gunk at the canvas and praying the mixer will somehow make it beautiful. Yep. I’ve been guilty too.

"Too much sound" is something more specific than too many sounds...

But another form of “too much sound” is something more specific than too many sounds, it’s too much sound in a given spectrum. In my experience, the part of the audio spectrum between about 200Hz and 800Hz is usually not the problem. The two problem areas tend to be below 200Hz and above 1K, especially above 3K.

For some reason we often conclude that low frequencies and high frequencies are where most of the drama is in sound effects, that those parts of the spectrum are the ones that get people’s attention. So, we pile it on in those areas in action sequences: constant booming bass that feels like low-passed pink noise and constant highs that basically feel like unrelenting, uncorrelated white noise. It’s not dramatic, dynamic, or even coherent. It’s just damn loud.

My recommendation for how to avoid this common, nasty situation that doesn’t sound good and will usually get you into trouble with Directors (or whoever is creatively in charge) is to not only reduce the number of simultaneous sounds, and to come up with individual sounds that have more natural intrinsic drama, but also to listen carefully for unnecessary sonic competition in particular frequency ranges.

Certain sounds will have something stellar going on in the highest frequency bands. Showcase that sound by filtering or eq’ing down the high frequencies in any other simultaneous sound that could mask the star of the moment. Same for the low end. The subwoofer can shine, but not if you neutralize it by sending three, six, eight sounds to it simultaneously.

​Lots of this happens in the mix, of course, but my advice is to save the mixer some time by preparing sounds so that when all the faders are initially put up; the mixer isn’t confronted with a wall of blasting meaninglessness.

Good Luck!

Sound Design and AI

Sound Design and Artificial Intelligence

As I write this in late April of 2024, artificial intelligence has not had a significant impact on sound effects design for film, video, or games. But I specify the date because I expect the situation to change in the not-too-distant future. A.I. is already being used in various kinds of voice and music work, with varying degrees of success.

The most promising use of a.i. for sound effects that I know of has been done by a company called ElevenLabs. https://elevenlabs.io/ai-sound-effects-for-sora But their process is not available yet. They say they have about 11,000 people on their waiting list.

ElevenLabs have posted a demonstration in which they claim to have generated sound effects for a series of videos produced by Open A.I.’s “Sora” program, which creates moving images from a text input. The quality of the sound design in the ElevenLabs Sora demo is mediocre at best in my opinion, but the fact that they can do it at all is impressive.

Legal and Ethical questions...

As with all forms of A.I., there are lots of legal and ethical questions that need to be addressed. One of the foremost has to do with compensating the individual people and companies who produced the material used to train the a.i. Another major issue is the number of human jobs that will be eliminated if and when, most likely when, A.I. does begin to make a serious impact on sound design work.

To be honest, if I could make A.I. go away, I would. But it won’t go away. It will inevitably get more and more sophisticated, eliminating more human jobs in the process. My advice to people who have a career or want a career in sound design is to learn everything you can about A.I. for sound. That will put you in the best position to use it, instead of being replaced by it.

The jobs on projects at the low end of the budget scale will be the first lost to A.I. That’s a shame, because it’s those projects where people new to the industry usually get their foot in the door. The high-end, high-profile projects will be the last to adopt A.I. sound design because they can afford the added value they get from human creative involvement.

The creative level of the sound work in the existing Sora videos is what I would call “see a dog, hear a dog.” Obvious and not very interesting interpretations. You can outdo the current and near-term future level of A.I. sound design by going beyond the obvious.

If you are seeing a truck traveling down a road, don’t settle for finding or making an audio recording of that kind of truck. Make the truck a character by using non-obvious elements in addition to the obvious ones, and by finding ways to add believable dynamics to the sound that will contribute to the story by tweaking the imagination of the audience.

That level of sound design creativity won’t be in A.I.’s repertoire anytime soon.
Sound Ideas and A.I

Making It Real

Making It Real and Making It Feel

When I was working on the Bob Zemeckis film Cast Away it was very difficult coming up with sounds for the waves on the beach when the Tom Hanks character is alone on the island. The sound recorded while they were shooting those beach scenes in Fiji was almost all useless. It was just the boring “noise” of constant wave breaking in all those shots. It sounded like a giant pink noise generator.

We had to invent a sound for every single wave...

So, we had to invent the sound for every single wave we see in the movie, as well as the hundreds that we don’t see. The waves needed to be a character in the movie, an interesting and totally believable character. It turned out that both the interesting aspect and the believable aspect were harder to achieve than I thought they would be. The sound of the waves needed to mirror the Hanks character’s emotions. When he was calm, they needed to be calm, and when they were intense, he was intense. Each wave also needed to present its own little story with a beginning, middle, and end.

I started the wave work by gathering all kinds of water sounds, not just waves. That’s because I knew from the beginning that I would need to build each wave with elements, not just go out and record real ocean waves. Though I did that too. At least a month was spent going up and down the West Coast of the USA collecting wave sounds, from huge ones to tiny ones. The holy grail was always waves that were hitting the beach at ninety degrees, pretty close to shore. In that situation you get a nice pause between each wave break during which you can hear the fizz of all the bubbles settling into the sand. I made lots of recordings where I focused mainly on getting that fizzing. Though I also faked some fizz elements with things like Alka Seltzer tablets in a studio.

I wanted to make each wave dynamic, changing over time in volume and timbre. Each needed to sound unique, but still in the same family as its neighbors. That’s the “feel” part of the character I was looking for.

But the sound for each wave we see in the movie, and we see lots of them, needed to feel real too. You’d think that waves would be easy to edit. Nope. That’s because each wave is different: different amounts of bubbles, different amounts of impact, different amounts of curl, etc. That’s why we had to design the sound for each wave individually in order to make it believable and real.

​An average wave we see in the movie was made of at least three or four elements. Everything from actual ocean beach waves to water being poured into a swimming pool (to get the watery “curl and flow” sound), to Alka Seltzer, to cannons firing and explosions for the big wave impacts, and beyond. Myself and the sound effects editors would pass wave sequences back and forth, and each person would contribute something new in each iteration that would help sell the sound. It was difficult, fun, and wound up getting us an Oscar nomination that year the following spring. It’s still hard for me to look at a real wave and avoid thinking about how I could improve its sound.

Storytelling Functions of Sound

When someone asks “What does sound do in a movie” most of us are at least a little confused about how to answer that question. Obviously both music and sound effects can make you feel a certain way. Sound effects can clearly add realism to a scene; make it more exciting, more mysterious, or funny. Same for music. But beyond those obvious jobs, it can be hard to specifically say what sound is doing.

Suggesting a Mood or Defining a Place

Here are some of sounds jobs. Very often a given sound will be doing two or more of these at the same time:
    - suggest a mood, evoke a feeling
    - set or modify a pace
    - indicate a geographical locale
    - indicate a historical period
    - indicate changes in time
    - connect otherwise unconnected ideas, characters, places, images, or moments
    - define a character
    - clarify the plot
    - heighten realism or diminish it
    - heighten ambiguity or diminish it
    - draw attention to a detail, or away from it
    - smooth otherwise abrupt changes between shots or scenes
    - emphasize a transition for dramatic effect
    - describe an acoustic space
    - startle or soothe
    - exaggerate action or mediate it
    - indicate the geography within a scene*

                          *(for example: most jungle shots look the same, but sound can tell us
                             who is where relative to the camera when we cut from one jungle
                             location to another two hundred feet away)

If two different sounds are competing while trying to accomplish any one of these jobs, that can be a red flag, because unless they are orchestrated well together, there’s a good chance that each is undercutting the impact of the other rather than each amplifying the other.

This often happens when music and sound effects are attempting to make essentially the same statement at the same time. One of the most difficult aspects of sound mixing is finding ways to feature one sound or set of sounds in each moment and shifting that focus so that the audience feels they are hearing everything they are supposed to be hearing.

​Simply playing every sound loud is usually a disaster unless the goal is to create a wall of noise, which sometimes it is.

Exploding Bridges and Bowling Pins

In Sam Wasson’s fantastic new book about Francis Coppola and American Zoetrope, he quotes Walter Murch ruminating on the sound of colliding bowling pins. I remember those pins well.
When the helicopters attack the village in “Apocalypse Now,” a wooden bridge gets blown up by American rockets. As one of Walter’s assistants, my main job for many months was finding sounds to use in the film. We didn’t have access to any of the big movie studio libraries, and there were pretty close to zero commercial sound libraries in those days, certainly none that had Vietnam War related sounds, so we had to go out and record a high percentage of the sound effects we needed for the film.

We could have found a large wooden structure to blow up

Walter showed me the sequence where the bridge is attacked, a section of it shattered. It looked like several tons of timbers and planks splintered and flying through the air, banging into each other. We had recorded lots of explosions for the movie… grenades, bombs, gasoline, etc., but nothing where big pieces of wood get exploded. We could have found a large wooden structure to blow up, but that would have been expensive, and there was no guarantee that we’d get the wood aspect that we wanted. Very little useful sound was recorded on set while they were shooting the movie. It was way too chaotic, so we had to invent nearly all of the sound in post production. To make the sound of that moment feel authentic, I needed to find woody elements to add to the basic explosion sounds we had already made in post.
I tried lots of things: trees being chopped down and falling, hitting wooden telephone poles with sledge hammers, throwing pieces of lumber around, etc. I think any of those sounds could have been useful layers, and a couple of them were helpful, but they didn’t quite sell the moment. I wracked my brain to come up with situations in which lots of pieces of wood collide with each other. The idea of bowling pins bubbled to the surface.
I suspected it could be a waste of time. Bowling pins are small compared to the size of the structural parts of the bridge in the movie. Bowling pins don’t typically shatter like that bridge wood did. And the sound of bowling pins is so identifiable and familiar… wouldn’t people in the audience recognize that sound immediately, and wonder why they were hearing a bowling alley in a battle scene?
What the heck, I spent about an hour recording in a San Francisco bowling alley. They let me put microphones in various places, including at the back of the alley, behind the pins. When I got the recordings to the studio, they definitely sounded woody, and it was cool that there were a bunch of hits in close time proximity, so I wouldn’t have to build a mass of them individual hit by individual hit. But to my ears they were so obviously bowling pins, and the sound of them felt too small to be elements of so large a bridge.

We didn’t have digital sound processing in those days, but we could make a recording on tape sound bigger by slowing down the tape playback speed, then transferring it at the slower speed. I played the bowling pin recordings at half speed and one quarter speed. Now the pins sounded like they were about ten feet tall. What wound up in the movie was a mix of regular speed, half, and quarter speed pins colliding. As to whether it’s too recognizable as bowling pins… the idea seems never to have occurred to anybody but me. I still hear that bowling alley when I watch the scene, as I did yesterday. To everybody else, it feels like we had microphones all over that bridge. Such is the power of sound design.