Live Pano chroma key request

[Xander Berkenhagen]

Live Pano chroma key request

by Xander Berkenhagen » Thu Nov 14, 2013 1:29 am

It would be a great feature to add the possibility to use chroma key recordings in Live Pano.
Then we could record a person or certain situation in front of a green screen and then easily add it in the panorama at a location.

Then it would be also possible to let that person move with the pano or even float on top and stay on a certain and don't move with the pano.

Think this would be a killer add-on and we could do some pro video inserts.

Regards,

Xander Berkenhagen

[jschrader]

Re: Live Pano chroma key request

by jschrader » Thu Nov 14, 2013 11:55 am

Should be no difference to current Krpano.
So I guess with the right video codec this already is possible.

[Xander Berkenhagen]

Re: Live Pano chroma key request

by Xander Berkenhagen » Fri Nov 15, 2013 9:30 pm

Nope not possible yet.

[Morgan]

Re: Live Pano chroma key request

by Morgan » Mon Nov 18, 2013 9:25 am

Would be great if we could get Kolor's thoughts on that really interesting question as I have the same for a while but never received reply yet...

I use Adobe After Effect and even it's not filmed with a green backgroung, with the rotorbrush we can create a nice Livepano with a chormakey ready background, but kid to use it with PPT 2.0...

Regards.

Morgan

[ygilquin]

Re: Live Pano chroma key request

by ygilquin » Mon Dec 09, 2013 2:16 pm

The essence of LivePano is to merge video and panorama taken from the same point of view. BUT, there is a simple trick that could allow you to reach your goal: merge your chroma key recording with a part of the panorama projected as a plan.

I assume you have two things:

• a chroma key video recording with a subject on transparent background (for my test I use some animated gif)
• an equirectangular panorama to be used as background

First make sure the panorama is centered around the point where you intend to merge your recording, else correct it with Autopano. Then open the panorama in Panotour and extract cube faces (you need some planar background).

Then only keep the frame to be used as background. Use your prefered editing software to merge your animated character in the background for each frame of your video, and export as a video. (On Linux I used ImageMagick and ffmpeg to do this).

Here you have your video merged with a portion of the planar panorama that can be used as a LivePano input. You will also need to recreate a panorama project to include video frame (grabbed from obtained video) in the panorama before proceeding with LivePano.

I hope this will help you.

[reinhardkiss]

Re: Live Pano chroma key request

by reinhardkiss » Tue Jan 28, 2014 1:55 pm

Hello GIL,

I really do have big problems with LIVEPANO. Maybe you, or someone else, can help me.

Your input is very good. I need to put a livepano in a equirectangular pano (4096x2048) . This panorama is made of a render .

Qubeface extracted , made ​​keying , mov file exported from AE, made ​​snapshot .

So far so good .

Now the problems begin in APG . I import the 2:1 panorama as JPEG and snapshot to APG . The result is , however, very strange. The aspect ratio is no longer true , the size has been changed and the snapshot does not fit . Now I save the APG Project. Back in PTP2 I import the original render as JPEG (not any APG output), import a Livepano, give the path to the APG project-file and to the mov-file and try to render the Livepano after adapting all settings . Result is only a black Blakely at the position where the video should be. I am grateful for any help , we would also take a paid Support Service by Kolor claim in order to solve this issue.

To all other forum members : please no quotes from China .

A tutorial would be great (present all Chroma-Key belongs and settings - photo and render)

I'll make a screencast of my steps and this - put on my blog - for better understanding, an will post URL.

https://vimeo.com/85243708

Thanks in advance .

Alex , Vienna - AUSTRIA

Last edited by reinhardkiss on Tue Jan 28, 2014 2:52 pm, edited 2 times in total.

[Destiny]

Re: Live Pano chroma key request

by Destiny » Tue Jan 28, 2014 2:18 pm

In that case, if you do not want forum member advice, and only request Support Services from Kolor, then why not send them a PM??..

http://www.destinyvirtualtours.com/PTPV ... oTour.html

BTW.. I think you win the record for the longest url.. I will pass on opening it since it looks a big dodgy..

Destiny..

[reinhardkiss]

Re: Live Pano chroma key request

by reinhardkiss » Tue Jan 28, 2014 2:48 pm

Hello Desteny,

of course I want forum member advice - what I do not want are those strange quotes from Morgan. Sorry for the long URL, will change it.

Any suggestion is welcome

Alex

[klausesser]

Re: Live Pano chroma key request

by klausesser » Tue Jan 28, 2014 4:41 pm

Hello Desteny,

of course I want forum member advice - what I do not want are those strange quotes from Morgan. Sorry for the long URL, will change it.

Any suggestion is welcome

Alex

Hey Alex!

Maybe it´s because you didn´t shoot the lady from the same point and using the same lens as you used for the pano.
As i see it´s an example from Primatte - so it´s not the correct lens and perspective.

You need to shoot the video from the point you shoot the first panorama-picture. Only then you get the same perspective and
the video gets un-distorted the same way as the pano. This is not the case when you use another clip!

I´m actually also experimenting with keying for LivePano . .

best, Klaus

[reinhardkiss]

Re: Live Pano chroma key request

by reinhardkiss » Tue Jan 28, 2014 4:52 pm

Hello Klaus,

thanks for your reply. I use to shoot the video from the same angle in the "photographic world", and it works almost fine. But this is a different approach. I try to achieve implementing a LivePano into a 3D Rendering (360° equirectangular).

Of course the Godiva Video is only a try and maybe not the best example.

The goal should be filming a Salesperson in front of a Greenscreen and implementing it into the 3D Pano. So no angle comes in play. But if I understand the post of Gil right, it can be done. I only don't know how.

Best Greetings from Vienna

Alex

[klausesser]

Re: Live Pano chroma key request

by klausesser » Tue Jan 28, 2014 5:17 pm

Hello Klaus,

thanks for your reply. I use to shoot the video from the same angle in the "photographic world", and it works almost fine. But this is a different approach. I try to achieve implementing a LivePano into a 3D Rendering (360° equirectangular).

Of course the Godiva Video is only a try and maybe not the best example.

The goal should be filming a Salesperson in front of a Greenscreen and implementing it into the 3D Pano. So no angle comes in play. But if I understand the post of Gil right, it can be done. I only don't know how.

Best Greetings from Vienna

Alex

Hi Alex!

The point is: the key-clip needs to be exactly the same (outline) size as the photographed images! That´s what i understand from the documentation - and that´s what makes the most sense
Regardless of what you later keep due to masking - the overall size counts!

Best to Vienna (wonderful, mysterious city, i loved to be there for some weeks on a video-assignment)

Klaus

[reinhardkiss]

Re: Live Pano chroma key request

by reinhardkiss » Tue Jan 28, 2014 5:50 pm

Hello,

the point is that I do not have any photographic images in this case. I have 3D Rendering (3ds Max Virtualization). The VRay Renderer is able to render in a photographic Quality, and it is able to produce a 360° spherical image as TIFF or JPEG or HDR or whatever, but it is not a stitched photograph. And therefore also no tripot-outline. This is my point of despair.

Greetings to Dusseldorf, love this city as well

Best Regrads

Alex

[klausesser]

Re: Live Pano chroma key request

by klausesser » Tue Jan 28, 2014 10:19 pm

Hello,

the point is that I do not have any photographic images in this case. I have 3D Rendering (3ds Max Virtualization). The VRay Renderer is able to render in a photographic Quality, and it is able to produce a 360° spherical image as TIFF or JPEG or HDR or whatever, but it is not a stitched photograph. And therefore also no tripot-outline. This is my point of despair.

Greetings to Dusseldorf, love this city as well

Best Regrads

Alex

Yep - understand. I rendered some sperical panos in Cinema4D using the QTVR-camera option some time ago.

The panorama-camera in Max, Cin or whatever uses a size of the "photographs" it uses for "shooting" the pano. You need to set the same size for your video - when i say "the same size" i mean the exact pixel-size.

I´m new to using Live-Pano - just evaluating it. I suggest to give it a try this way.

best, Klaus

[reinhardkiss]

Re: Live Pano chroma key request

by reinhardkiss » Wed Jan 29, 2014 6:41 am

Hi Klaus,

thanks for your answer, I will play around with it and post the results.

But I do have another problem since RC2, and - as I know - you are very experienced - so maybe you have a hint. I made a almost finished production in PTP2, autorotation is on. After rotating aprox. 300° PTP2 changes automatically to the next pano, an so on (like a kind of loop). There are 2 groups in the production. Any idea ?

If you like to see it, I could publish the production on a testserver or on my blog.

Thank you for your answer

Alex

[Destiny]

Re: Live Pano chroma key request

by Destiny » Wed Jan 29, 2014 7:53 am

I will let the "very experienced" Klaus answer your direct question aimed at him regarding the Auto Rotation..

Regarding your use of a 3D rendered panos to use with LivePano. Sorry to tell you but this is actually impossible.. For one, your single image needs to be registered into the pano using control points. That single image is set to Fisheye if you use a Fisheye or normal is you used a normal lens to capture your pano..

The next reason why its impossible.. During the processing of your LivePano in PTP V2, your pano images are spit up and your video is added as well as your APG files. There is nothing for LivePano to register your video....

Destiny....

[reinhardkiss]

Re: Live Pano chroma key request

by reinhardkiss » Wed Jan 29, 2014 8:09 am

you answer the second time very rude and sarcastic. I ask you polite to keep distance in the future

[ygilquin]

Re: Live Pano chroma key request

by ygilquin » Wed Jan 29, 2014 10:22 am

The fact is Destiny is perfectly right.

In a nutshell, Livepano has been designed to work with 2 MANDATORY inputs: a video and an Autopano project used for stitching the background. Background and video are supposed to be shot with the same lens in the exact same conditions.

In your case, you have neither the stitched panorama, nor the video shot in same conditions. So I really wonder why you want to use Livepano for this project. This is nonsense.

But I can try to suggest 2 options to help you:

• you really really want to use Livepano: try to generate two spherical images, with a 10° shift e.g. Feed Panotour with them, and extract cube faces for both. There you have 12 planar pictures of the same place, with some overlap. Use the 12 pictures as Autopano input to stitch the panorama. Now you should have a panorama stitched with Autopano (project file + rendered image) and a video that you can integrate following my previous explanations.
• ok Livepano is a very cool tool, but you don't really need it there: center your panorama where you want to merge your video. Extract cube faces, keep centered one and merge your video with this. Then in Panotour, create a video hotspot with the merged video that will be easily positionned.

[reinhardkiss]

Re: Live Pano chroma key request

by reinhardkiss » Wed Jan 29, 2014 10:50 am

Gil, thanks a lot. Will follow your suggestions and keep you informed.

Best regards and let me thank you indeed.

Alex

[ygilquin]

Re: Live Pano chroma key request

by ygilquin » Wed Jan 29, 2014 10:53 am

You're welcome, but who is Gil ??? In fact, my firstname is Yann.

[reinhardkiss]

Re: Live Pano chroma key request

by reinhardkiss » Wed Jan 29, 2014 10:57 am

oh, sorry Yann

[klausesser]

Re: Live Pano chroma key request

by klausesser » Wed Jan 29, 2014 11:48 am

In a nutshell, Livepano has been designed to work with 2 MANDATORY inputs: a video and an Autopano project used for stitching the background. Background and video are supposed to be shot with the same lens in the exact same conditions.

Hey Yann!

That´s what i said before.

In your case, you have neither the stitched panorama, nor the video shot in same conditions. So I really wonder why you want to use Livepano for this project. This is nonsense.

True - but it´s a question whether that is really impossible. Just give it a closer look:

You see: Alex produces a panorama using a 3D application. I don´t know whether you ever did such a thing?

Let´s see the basics: when you generate a virtual qtvr camera for shooting the virtual scene in Cinema4D or Max or whatever, this camera basically behaves exactly like a real camera on a NPP-head! The only diffgerence is: it doesn´t do single images but renders an equirectangular straight. But we can make cubefaces from it as usual.
Each time you use a camera in a 3D app you set a focal-lens and a film/sensor size. That´s what you also do using a real camera!
So my idea is: doing that you might use a real camera providing the very same sensor size and focal-length - resp. you set the VR-camera to the very same values the real camera has.

The point is: you need to shoot the video using the absolute identical settings as the virtual camera: lens (doable), size (doable), perspective (doable).
Whether the video is keyed or not i guess doesn´t matter as long as it provides the same outline-size!
My time is somewhat limited theses days - but i´ll give it a try again . .

What we need to pin down: is my thinking correct? Ist the basic thing really the format-SIZE, the LENS and the perspective which need to be identical? Nothing else?
Because: THAT is possible to achieve using a 3D application! And given this is correct it must be possible to do what Alex asked for - and what i experimented with some time ago.

But I can try to suggest 2 options to help you:

• you really really want to use Livepano: try to generate two spherical images, with a 10° shift e.g. Feed Panotour with them, and extract cube faces for both. There you have 12 planar pictures of the same place, with some overlap. Use the 12 pictures as Autopano input to stitch the panorama. Now you should have a panorama stitched with Autopano (project file + rendered image) and a video that you can integrate following my previous explanations.
• ok Livepano is a very cool tool, but you don't really need it there: center your panorama where you want to merge your video. Extract cube faces, keep centered one and merge your video with this. Then in Panotour, create a video hotspot with the merged video that will be easily positionned.

[/quote]

Yann - you made my day (and i guess you made also Alex´day ) for giving such a great input

Thank you for your really valuable words!

best, Klaus

[klausesser]

Re: Live Pano chroma key request

by klausesser » Wed Jan 29, 2014 12:05 pm

The fact is Destiny is perfectly right.

Regarding the usual and already well known items, yes. But what Alex tries to achieve is somewhat special - and i think it´s definitely worth a try!

Klaus