Directionality in How We Perceive Sound: Part 1
Human hearing is really terrible compared to the rest of the animal kingdom. With the exception of birds, we have some of the smallest ears in nature. I know that sounds like a really primitive way to measure hearing - by ear size - but try this: next time you're outside, cup your hands behind your ears, palms forward. You will suddenly hear all these strange mid and high frequencies, and catch sounds further away. It matters. Any animal you think of will most likely have larger, or at least prominent ears - they stick out like cups or cones. Ours do not, and that makes directionality complicated to create in games and film.
Regarding left and right, game engines are starting to really take the potential of our hearing to its fullest. Even just a few years ago, most engines and even popular games used sound in a very limited way: if a sound was to your left, the left ear/channel would be at 100%, the right at 0%. We don't hear that way, at all (more next blog post about how we hear left and right) and it's embarrassing how long it took games to catch up.
Luckily now, bigger companies are putting more time into sound in their custom engines, and even engines like Unity and Unreal have really great audio systems going on. I particularly love UE4's new experimental audio system. Really worth checking out. I hear great things about Steam's room-based audio but I haven't used it myself.
The purpose for this post is to examine why we can't hear up and down. We're terrible at it. I'm not a scientist, but my guess is that we rarely have encountered threats from above or below, in terms of survival, so we never really needed our ears to handle sound directionality in a vertical manner.
Up and down is hard. I'm posting about it because engines don't even try to do it, at all, but I wanted to give you a tip on how to get there.
AND HERE IT IS: use this lack of human awareness to your advantage. It confuses us, to try to place a sound above us. What this means is - you don't have to achieve vertical sound. You only have to confuse your gamer's ears. This will have to be done out of engine, but it's actually pretty simple. The first trick is to duplicate the track your sound is on, and reverse the phase of the second track. Leave the first as is. This is really confusing to the mind, so the mind will place it directly behind you, or above you. Whichever one of these you want can be aided with visuals. Another trick for sound placed behind the player is to create a very wide panned sound, using a stereo expander if you have a good one, and have that sound pan to center quickly. This creates the sensation that something has moved to directly behind us.
Modern audio technology is awesome and all, but having a toolbox full of psychoacoustic tricks can really make your game shine.