At the Coachella music festival two weeks ago, Youtube introduced live 360-degree video streaming, WITH 3-D sound. 3D movies, 3D printing, and 360 virtual reality videos have become common terms… but 3D sound? As Alison Vicrobeck reports, it’s the next step in reproducing a life-like virtual experience.
VICROBECK 1: 3D audio isn’t easy to describe – one one hand it’s like what we hear naturally with our ears. But when it’s being recreated…
…it’s a bit like trying to describe a 3D movie to someone who has never seen one…
The best thing to do is to experience it yourself.
If you can, you should put on some headphones right now…..so I can play you a 3D audio demonstration especially made for headphones…
Close your eyes and imagine you’re getting a haircut at Luigi and Manuel’s barber shop.
VICROBECK 2: If you didn’t have headphones on, you likely didn’t get the full effect. As Luigi’s was cutting your virtual hair, you’d should have be able to hear more than just a random cutting sound. You would’ve heard it on your left, on your right, and more impressively behind you!
Luigi’s barber shop is a demo of 3D sound created by Qsound Labs.
3D audio allows you to hear sound the way you would if you were actually in the room. It’s really popular among video gamers. Technically, this 3D sound is called binaural audio – bi for two and aural for ears – and it takes a sort of jerry-rigged recorder to capture it.
GENOVESE 1: The dummy head, it’s a plastic head with plastic ear molds, and it has with two tiny microphone capsules placed inside the plastic ears. (0:08)
VICROBECK 4: Andrea Genovese is getting his PhD in Music Technology at NYU. He specializes in 3D audio.
GENOVESE 2: So what happens is that the sound environment, the natural sound atmosphere, get modified by the head shape and the plastic ears in a similar manner that we perceive sound. (0:11)
VICROBECK 5: Using a dummy head helps recreate the way sound coming from the left hits our left ear first – and our right ear a fraction of a second later. Then the sound bounces differently off each part of the ear lobe. But the dummy head technique isn’t quite perfect.
GENOVESE 3: The dummy head it has a fixed head shape, head size, head width, and the ears are a standardized kind of shape (0:10)
VICROBECK 6: But we’re all different, and so the sound isn’t perfectly adapted to each person. Gregg Wilkes is the CEO of a company called VisiSonics.
WILKES 1: The trojan horse of our technology is we have the ability to capture your HRTF or hearing print in our lab (0:06)
VICROBECK 7: …The HRTF is Head Related Transfer Function. By that, he means he can measure how your head and ears affect the way you hear
WILKES 2: We believe we’ll be able to do this in your living room in 12-18 months, so we’ll be able to personalize every one of these audio experiences (0:07)
VICROBECK 8: VisiSonics is doing this with one goal
WILKES 3: ….To create or simulate sound in stereo headset so that your brain almost believes it’s not wearing headphones. (0:08)
VICROBECK 9: There’s a reason why sound engineering companies are working to develop 3D audio right now… it’s VR. Or Virtual Reality video. Wilkes estimates by the year 2020 virtual reality will be a 120 billion dollar industry and over two-thirds of the products will require 3D audio. He says it’s a critical piece for the future….because without it…
WILKES 6: then your brain fatigues, your brain get sick, falls out of it. The immersive experience just doesn’t happen. (0:06)
DURAISWAMI 1: so now the goal of VR, is essentially allow you to be present somewhere else, as if you were there, really. (0:15)
VICROBECK 13: Ramani Duraiswami is the co-founder of VisiSonics. He’s teaches in the computer science department at the University of Maryland and has been working on 3D audio for over a decade. He says that 3D audio also helps us to not get too distracted the virtual reality.
DURAISWAMI 2: Then the only way they will be able to direct your attention to something behind you is by having a sound happen there. (0:09)
VICROBECK 14: But 3D audio, can also be used without a visual component, like for storytelling and music.
(Pearl Jam plays)
Pearl Jam was one of the first bands to experiment with 3D sound – more than 15 years ago. They released an entire album of 3D songs. Now, entire concerts, like the one at Coachella a few weeks ago are also being recorded in 3D sound.
The possibilities for 3D audio are near endless, engineers are currently developing head tracking technology, so that when you move, the sound appears to be coming from the same place. They’re also working on making 3D sound accessible on speakers.
At this point, if you can imagine it, the audio technology is probably not far off. So I asked Andrea Genovese, the PhD student at NYU, what is the craziest thing we could imagine doing with this technology in the future..
GENOVESE: What if you could listen through the ears of a great composer? (Beethoven’s Moonlight Sonata plays) So if we can record in the way a great composer perceives music, which supposedly is more refined, and apply that signal to your ear, maybe you’re able to listen to music with the same sensitivity than a great musician does (0:20)
VICROBECK 15: … except maybe Beethoven. He was able to compose the Moonlight Sonata you’re listening to binaurally, without being able to hear. Because sound isn’t just processed by the ears. They’re a conduit to the biggest auditory organ: the brain.
Alison Vicrobeck, Columbia Radio News