Adding spatial audio synced  

Got some great idea or a feature request? Post it here and discuss it.
no avatar
Ronaldvdw
New member
 
Topic author
Posts: 4
Likes: 0 post
Liked in: 2 posts
Joined: Fri Jan 20, 2017 10:36 am
Info

Adding spatial audio synced

by Ronaldvdw » Fri Jan 20, 2017 10:47 am

When stitching the files, also add & sync the spatial audio right under!

Eric Fassbender likes this post.

User avatar
decurgia
Member
 
Posts: 17
Likes: 0 post
Liked in: 0 post
Joined: Sun Mar 13, 2016 10:25 pm
Location: Switzerland
Info

Re: Adding spatial audio synced

by decurgia » Fri Jan 20, 2017 4:23 pm

I would also like to be able to add spatial audio to the video. But I think there is no real standard yet. YouTube and Facebook do have got some resources about that topic and there are some tutorials using Reaper and Kolor eyes...

I do own a GoPro Omni and a Sennheiser VR Mic. The 4 mono tracks recorded by a Zoom F8 recorder have to be converted from A to B-format using Sennheisers A-B converter plugin. In order to upload it to YouTube the output has to follow the rules described in "YouTube Spatial Audio" below (First Order Ambisonic (FOA) audio channel layouts, ACN channel ordering, SN3D normalization, FOA components W, Y, Z, X should have channel map indexes 0, 1, 2, 3, respectively, 4.0 layout).

Currently I fail to validate if my audio output (generated with Logic Pro X and the Sennheiser A-B format converter) is compliant. I do also not know yet how to replace the audio track from Autopano Video Pro with the spatial audio track from Logic Pro X.

From my point of view it would be great if I could add the 4 mono audio files to Autopano Video Pro as input and then having AVP adding it to the video. Ideally I could also use the audio plugin to configure the spatial audio.

- Can you share a workflow which outputs video and spatial audio compatible to YouTube?
- Where would you like to use spatial audio?

Cheers
-Luca.

YouTube Spatial Audio: https://support.google.com/youtube/answer/6395969?hl=en
YouTube Spatial Media Metadata Injector: https://github.com/google/spatial-media ... atialmedia
YouTube Spatial Audio RFC: https://github.com/google/spatial-media ... dio-rfc.md

no avatar
Skaven252
Member
 
Posts: 53
Likes: 4 posts
Liked in: 4 posts
Joined: Thu Jan 26, 2017 9:03 am
Info

Re: Adding spatial audio synced

by Skaven252 » Thu Jan 26, 2017 9:10 am

I would appreciate this feature. I use a Zoom H2n recorder which saves directly to B-format (AmbiX order), with a mute vertical channel since it's not a true tetra.

Since AutoPano is primarily for stitching and not really for editing, it would probably be ideal to include the audio stream in a lossless format (PCM) to improve quality in the final edit.

The editing program I use (Vegas Pro) does not support directly rendering 4-channel audio into an MP4 video file, so render the audio out separately, then use FFMpeg (I bought MyFFMpeg to make it easier) to replace the audio track in the video file.

User avatar
decurgia
Member
 
Posts: 17
Likes: 0 post
Liked in: 0 post
Joined: Sun Mar 13, 2016 10:25 pm
Location: Switzerland
Info

Re: Adding spatial audio synced

by decurgia » Thu Jan 26, 2017 1:57 pm

Hi again

I was able to replace the audio track of one of the source video files using Logic Pro X (LPX). I used LPX in order to sync the audio recorded by the GoPro and the Zoom F8. If you open a video in LPX you can select "Export Audio to Movie" (1).
The output is the same video footage with replaced audio (4 channel, ambiX order).

In Autopano Video Pro I then selected the modified video as audio source. After rendering I got a nice, stitched video with a 4 channel audio track. Currently uploading the footage to YouTube in order to validate the spatial audio...

Cheers
-Luca.

(1) https://support.apple.com/kb/PH13342?locale=en_US

User avatar
decurgia
Member
 
Posts: 17
Likes: 0 post
Liked in: 0 post
Joined: Sun Mar 13, 2016 10:25 pm
Location: Switzerland
Info

Re: Adding spatial audio synced

by decurgia » Tue Jan 31, 2017 3:54 pm

I think my workflow didn't work out at all.. The uploaded video does not sound nice: https://youtu.be/rjbfsW7eqA0
The .mov file says it's 4 channel audio, and the YouTube "360 Video Metadata"-App did work with ambiX.. but the final result in YouTube is wrong.

Does anyone know how to fix audio?
Is there somewhere a documented workflow available which I could copy?

Cheers
-Luca.

no avatar
Skaven252
Member
 
Posts: 53
Likes: 4 posts
Liked in: 4 posts
Joined: Thu Jan 26, 2017 9:03 am
Info

Re: Adding spatial audio synced

by Skaven252 » Tue Jan 31, 2017 4:42 pm

Yeah... all i hear there is raspy clicking. It's possible that YouTube converts it wrong if you upload it as a mov? (what was the audio codec in that uploaded file?)

I uploaded this earlier 360 video of mine, a direct Zoom H2n recording, as an mp4 with the audio encoded from wav to AAC using FFMPeg, then the metadata injector tool used to make it 360 + spatial audio. It seemed to work OK.

https://youtu.be/ONwWgyOc7Ok

User avatar
decurgia
Member
 
Posts: 17
Likes: 0 post
Liked in: 0 post
Joined: Sun Mar 13, 2016 10:25 pm
Location: Switzerland
Info

Re: Adding spatial audio synced

by decurgia » Fri Feb 03, 2017 5:14 pm

Hi again

I think my workflow does not work. I've downloaded a tool (Atom inspector) to validate the .mov container.

Faulty workflow:
1. Replace stereo audio-track from camera1 with 4 channel ambiX audio-track --> Inspector reports 4 channels, 96kHz, 24bit
2. Render spherical video with AVP using camera1 as audio source --> Inspector reports 4 channels but weird values for sample rate
3. Inject metadata required for YouTube playback --> Inspector reports 4 channels with weird values for sample rate plus SN3D
4. Video has no/bad sound on YouTube

Does AVP do some sound-processing? Ideally it would just pass it through the output.. but it seems like it does modify the audio track.

New workflow:
1. Render spherical video with AVP using camera1 as audio source --> Inspector reports 2 channels
2. Replace stereo audio-track with 4 channel ambiX audio-track --> Inspector reports 4 channels, 96kHz, 24bit
3. Inject metadata required for YouTube playback --> Inspector reports 4 channels, 96kHz, 24bit plus SN3D
4. Video upload pending (120GB), have to validate tomorrow @ https://youtu.be/kDeuQOANZns

The bad thing about the new workflow is that I would have to cut the video using a video editing tool (Final Cut Pro X) after rendering, which would again add some processing time. I use the three claps technology to sync both audio sources (gopro camera 1 / Sennheiser VR Mic). But I do not want to see myself clapping on the video...

User avatar
Emeric
Kolor Team
 
Posts: 156
Likes: 3 posts
Liked in: 14 posts
Joined: Tue Apr 12, 2016 9:45 am
Info

Re: Adding spatial audio synced

by Emeric » Thu Feb 23, 2017 11:36 am

Yes you should definitely replace the audio by ambisonic audio AFTER the render.

no avatar
Skaven252
Member
 
Posts: 53
Likes: 4 posts
Liked in: 4 posts
Joined: Thu Jan 26, 2017 9:03 am
Info

Re: Adding spatial audio synced

by Skaven252 » Sun Feb 26, 2017 8:29 am

It would be useful if you could add a 4-channel audio track and sync it to the vido already in AVP. It would make it easier to make further editing afterwards, with intermediate footage that already contains the audio tracks synced. Presuming, of course, that the audio can be added to the intermediate files in a lossless format.


Who is online

Users browsing this forum: No registered users and 2 guests

cron