Page 1 of 1

Adding spatial audio synced

PostPosted: Fri Jan 20, 2017 10:47 am
by Ronaldvdw
When stitching the files, also add & sync the spatial audio right under!

Re: Adding spatial audio synced

PostPosted: Fri Jan 20, 2017 4:23 pm
by decurgia
I would also like to be able to add spatial audio to the video. But I think there is no real standard yet. YouTube and Facebook do have got some resources about that topic and there are some tutorials using Reaper and Kolor eyes...

I do own a GoPro Omni and a Sennheiser VR Mic. The 4 mono tracks recorded by a Zoom F8 recorder have to be converted from A to B-format using Sennheisers A-B converter plugin. In order to upload it to YouTube the output has to follow the rules described in "YouTube Spatial Audio" below (First Order Ambisonic (FOA) audio channel layouts, ACN channel ordering, SN3D normalization, FOA components W, Y, Z, X should have channel map indexes 0, 1, 2, 3, respectively, 4.0 layout).

Currently I fail to validate if my audio output (generated with Logic Pro X and the Sennheiser A-B format converter) is compliant. I do also not know yet how to replace the audio track from Autopano Video Pro with the spatial audio track from Logic Pro X.

From my point of view it would be great if I could add the 4 mono audio files to Autopano Video Pro as input and then having AVP adding it to the video. Ideally I could also use the audio plugin to configure the spatial audio.

- Can you share a workflow which outputs video and spatial audio compatible to YouTube?
- Where would you like to use spatial audio?

Cheers
-Luca.

YouTube Spatial Audio: https://support.google.com/youtube/answer/6395969?hl=en
YouTube Spatial Media Metadata Injector: https://github.com/google/spatial-media ... atialmedia
YouTube Spatial Audio RFC: https://github.com/google/spatial-media ... dio-rfc.md

Re: Adding spatial audio synced

PostPosted: Thu Jan 26, 2017 9:10 am
by Skaven252
I would appreciate this feature. I use a Zoom H2n recorder which saves directly to B-format (AmbiX order), with a mute vertical channel since it's not a true tetra.

Since AutoPano is primarily for stitching and not really for editing, it would probably be ideal to include the audio stream in a lossless format (PCM) to improve quality in the final edit.

The editing program I use (Vegas Pro) does not support directly rendering 4-channel audio into an MP4 video file, so render the audio out separately, then use FFMpeg (I bought MyFFMpeg to make it easier) to replace the audio track in the video file.

Re: Adding spatial audio synced

PostPosted: Thu Jan 26, 2017 1:57 pm
by decurgia
Hi again

I was able to replace the audio track of one of the source video files using Logic Pro X (LPX). I used LPX in order to sync the audio recorded by the GoPro and the Zoom F8. If you open a video in LPX you can select "Export Audio to Movie" (1).
The output is the same video footage with replaced audio (4 channel, ambiX order).

In Autopano Video Pro I then selected the modified video as audio source. After rendering I got a nice, stitched video with a 4 channel audio track. Currently uploading the footage to YouTube in order to validate the spatial audio...

Cheers
-Luca.

(1) https://support.apple.com/kb/PH13342?locale=en_US

Re: Adding spatial audio synced

PostPosted: Tue Jan 31, 2017 3:54 pm
by decurgia
I think my workflow didn't work out at all.. The uploaded video does not sound nice: https://youtu.be/rjbfsW7eqA0
The .mov file says it's 4 channel audio, and the YouTube "360 Video Metadata"-App did work with ambiX.. but the final result in YouTube is wrong.

Does anyone know how to fix audio?
Is there somewhere a documented workflow available which I could copy?

Cheers
-Luca.

Re: Adding spatial audio synced

PostPosted: Tue Jan 31, 2017 4:42 pm
by Skaven252
Yeah... all i hear there is raspy clicking. It's possible that YouTube converts it wrong if you upload it as a mov? (what was the audio codec in that uploaded file?)

I uploaded this earlier 360 video of mine, a direct Zoom H2n recording, as an mp4 with the audio encoded from wav to AAC using FFMPeg, then the metadata injector tool used to make it 360 + spatial audio. It seemed to work OK.

https://youtu.be/ONwWgyOc7Ok

Re: Adding spatial audio synced

PostPosted: Fri Feb 03, 2017 5:14 pm
by decurgia
Hi again

I think my workflow does not work. I've downloaded a tool (Atom inspector) to validate the .mov container.

Faulty workflow:
1. Replace stereo audio-track from camera1 with 4 channel ambiX audio-track --> Inspector reports 4 channels, 96kHz, 24bit
2. Render spherical video with AVP using camera1 as audio source --> Inspector reports 4 channels but weird values for sample rate
3. Inject metadata required for YouTube playback --> Inspector reports 4 channels with weird values for sample rate plus SN3D
4. Video has no/bad sound on YouTube

Does AVP do some sound-processing? Ideally it would just pass it through the output.. but it seems like it does modify the audio track.

New workflow:
1. Render spherical video with AVP using camera1 as audio source --> Inspector reports 2 channels
2. Replace stereo audio-track with 4 channel ambiX audio-track --> Inspector reports 4 channels, 96kHz, 24bit
3. Inject metadata required for YouTube playback --> Inspector reports 4 channels, 96kHz, 24bit plus SN3D
4. Video upload pending (120GB), have to validate tomorrow @ https://youtu.be/kDeuQOANZns

The bad thing about the new workflow is that I would have to cut the video using a video editing tool (Final Cut Pro X) after rendering, which would again add some processing time. I use the three claps technology to sync both audio sources (gopro camera 1 / Sennheiser VR Mic). But I do not want to see myself clapping on the video...

Re: Adding spatial audio synced

PostPosted: Thu Feb 23, 2017 11:36 am
by Emeric
Yes you should definitely replace the audio by ambisonic audio AFTER the render.

Re: Adding spatial audio synced

PostPosted: Sun Feb 26, 2017 8:29 am
by Skaven252
It would be useful if you could add a 4-channel audio track and sync it to the vido already in AVP. It would make it easier to make further editing afterwards, with intermediate footage that already contains the audio tracks synced. Presuming, of course, that the audio can be added to the intermediate files in a lossless format.