[TO PLAN] Force The Insertion Of Cps?  

Got some great idea or a feature request? Post it here and discuss it. The most requested concepts are usually implemented, as Autopano Pro / Giga is very community driven.
no avatar
hankkarl
Member
 
Topic author
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

[TO PLAN] Force The Insertion Of Cps?

by hankkarl » Mon Nov 06, 2006 7:35 pm

APP needs a way to force the insertion of a point into a pair of pictures.

When testing a set of snapshots from my cellphone camera, I've found a pair of pictures that have a slight overlap. I can identify this area by eye, but APP cannot find any points, even using the link editor. The pictures were taken at night and were a test of what the cellphone camera could do, so there's a lot of noise, and the photos are only 640x480 (I think thats what the Treo 650 has for its native resolution).

On the other hand, the pictures were taken handheld and from different points. Its amazing that APP can get any of the pictures into a pano.

no avatar
hankkarl
Member
 
Topic author
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Tue Nov 07, 2006 9:49 pm

'[bo wrote:']hankkarl, try selecting smaller area in the CP Editor. I've found out that using smaller selection gives better results when trying to detect points on "difficult" material.

Tried that. It didn't work. The issue may be that the pictures are only cellphone quality and APP can't find a good match, where I can find an ok match. Its not a big problem--I'll do serious work with a serious camera.

I don't always carry the 5D and a set of lenses around, but my cellphone is almost always with me. And sometimes I'll need to take a picture in parts because the cellphone cam isn't wide enough. So it would be nice if I could use APP with my cellphone, but it's not necessary. And so far, the problem is only a few images on one pano of 4 or 5 panos that I have tried.
Last edited by hankkarl on Wed Nov 08, 2006 12:08 am, edited 1 time in total.

no avatar
ssprengel
Member
 
Posts: 34
Likes: 0 post
Liked in: 0 post
Joined: Fri Sep 01, 2006 3:20 pm
Location: Nebraska, USA
Info

by ssprengel » Wed Nov 08, 2006 3:26 am

Another thing to try would be to add artificial details in a photo-editor, like a couple # or @ characters, for APP to match on and then heal/clone them away after the pano is built.

no avatar
hankkarl
Member
 
Topic author
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Wed Nov 08, 2006 5:50 pm

OK, here are a selection of the images I had trouble with. Eventually, I got APP to produce the pano shown in the first picture.

These shots were handheld and I was walking around, not even considering doing a pano, so the fact that APP was able to do anything with them is amazing! I was testing my Treo650 to see how good the camer was in low light.

APP can't find good control points between the last image (image 11) and the third one (image 7). the second image is image 6, and APP finds some not terribly bad control points here, which allowed me to create the pano shown in the first picture.








Last edited by hankkarl on Wed Nov 08, 2006 6:10 pm, edited 1 time in total.

no avatar
hankkarl
Member
 
Topic author
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Wed Nov 08, 2006 6:02 pm

At first, APP didn't find any CP between the "left half" and the "right half" Using the CP editor, APP fouind some points between image 7 and 11. Note that the points probably not correct because the picture is "bent"



no avatar
hankkarl
Member
 
Topic author
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Wed Nov 08, 2006 6:04 pm

Her are the CP APP found for the above image



no avatar
hankkarl
Member
 
Topic author
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Wed Nov 08, 2006 6:06 pm

This selection did not find any points in APP (note that I'm using a feature of the building, not a light).



no avatar
hankkarl
Member
 
Topic author
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Wed Nov 08, 2006 6:08 pm

This selection found some points, but they're not even close to being correct.





User avatar
AlexandreJ
Kolor Team
 
Posts: 5988
Likes: 7 posts
Liked in: 10 posts
Joined: Mon Nov 14, 2005 4:56 pm
Location: Francin, France
Info

by AlexandreJ » Wed Nov 08, 2006 6:13 pm

Yes, that's a case where the image are so blurry and noise full that we cannot really do it without the need of manual control point placement. Or using ssprengel tips of adding a letter to the picture so that autopano will find a match ...

no avatar
JohnM
Member
 
Posts: 188
Likes: 0 post
Liked in: 0 post
Joined: Tue Oct 31, 2006 9:47 pm
Info

by JohnM » Wed Nov 08, 2006 6:31 pm

I will second this request. I shot a 40 images full 360*180 yesterday, with a painted white cement roof, and even if I knew where some of the roof images should go APP had real problems finding CP`s. Not strange since the white painted cement roof has the same structure/texture in several of the images.

John M

no avatar
hankkarl
Member
 
Topic author
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Thu Nov 09, 2006 3:06 am

AlexandreJ wrote:Yes, that's a case where the image are so blurry and noise full that we cannot really do it without the need of manual control point placement. Or using ssprengel tips of adding a letter to the picture so that autopano will find a match ...

Thanks Alexandre,

I learn something new about APP every day.

The amazing thing is that control points are found for most of the pictures (there are 11)

BTW, I think the problem was the noise, not the blurryness--I ran NoiseNinja on the input photos, and APP behaves much better.
Last edited by hankkarl on Thu Nov 09, 2006 3:50 am, edited 1 time in total.

User avatar
AlexandreJ
Kolor Team
 
Posts: 5988
Likes: 7 posts
Liked in: 10 posts
Joined: Mon Nov 14, 2005 4:56 pm
Location: Francin, France
Info

by AlexandreJ » Thu Nov 09, 2006 7:34 am

Really ? That's a nice hint : running noiseninja or another noise remover tools before detection.

User avatar
[bo]
Moderator
 
Posts: 1226
Likes: 0 post
Liked in: 0 post
Joined: Fri May 05, 2006 8:16 am
Location: Bulgaria
Info

by [bo] » Thu Nov 09, 2006 8:40 am

What JohnM says is a somewhat common scenario. Having some images that should be place at specific locations in a pano, but not being able to place them there due to CP limitation.

For example, a top row of a pano, consisting of uniform clear blue skies - you can tell which photo goes where according to filename or time of shoot (sequence), but APP cannot place them properly. Or the example JohnM gives.

Maybe some kind of "hard link" could be helpful. Instructing APP "this image fits here, no matter what".
Some of my panoramas, posted in the Autopano Pro flickr group.

no avatar
hankkarl
Member
 
Topic author
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Thu Nov 09, 2006 4:59 pm

'[bo wrote:']For example, a top row of a pano, consisting of uniform clear blue skies - you can tell which photo goes where according to filename or time of shoot (sequence), but APP cannot place them properly. Or the example JohnM gives.

Maybe some kind of "hard link" could be helpful. Instructing APP "this image fits here, no matter what".

Hi Bo,

At least one of the other pano tools lets you grab and move an image within the pano (RealViz stitcher). Do you mean this kind of functionality (grab an image, move and rotate it, then let APP find CPs on the images it overlaps)? Or do you just mean the ability to force a couple of CPs by hand?

no avatar
hankkarl
Member
 
Topic author
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Thu Nov 09, 2006 5:09 pm

AlexandreJ wrote:Really ? That's a nice hint : running noiseninja or another noise remover tools before detection.

It shows APP doesn't use the same criteria for a match that I do :-) And its probably a good thing to include in the FAQ.

But would NoiseNinja hide details if you had a sharp photo so that you get a less-good pano than if you didn't use it?

You had a demo of APP where it found lots of good CPs on a wood panel--I never would have found them. In the above case, I used gross features (the pattern of the lighted windows) to find a match, where APP was concerned about the smaller features, which were blown away by the noise.

While I wouldn't be to concerned about my example above, there are a lot of cases where repeated patterns of small features like JohnM's cement wall, a textured (sandpainted) ceiling or wall that takes up the whole frame, etc may confuse APP in the same manner that noise does.
Last edited by hankkarl on Thu Nov 09, 2006 5:10 pm, edited 1 time in total.

User avatar
[bo]
Moderator
 
Posts: 1226
Likes: 0 post
Liked in: 0 post
Joined: Fri May 05, 2006 8:16 am
Location: Bulgaria
Info

by [bo] » Fri Nov 10, 2006 8:59 am

hankkarl, I don't have RealViz, so I cannot confirm, but I was thinking something like a hybrid solution. Dragging an image to position and then, using similar to CP Editor interface, set "a zone" or "a point" where you want the images to match. In your example (Chicago1Selection1), imagine using the same selection tool not to detect point, but rather to say to APP - this here single lighted window is this one in the other image and that's that.
Some of my panoramas, posted in the Autopano Pro flickr group.

User avatar
AlexandreJ
Kolor Team
 
Posts: 5988
Likes: 7 posts
Liked in: 10 posts
Joined: Mon Nov 14, 2005 4:56 pm
Location: Francin, France
Info

by AlexandreJ » Fri Nov 10, 2006 9:12 am

I think for future version, some tool can be added to handle such case :
- Manual location of a single picture : drag'n'drop of a single picture to an approximate location inside the panorama view => for sky hole. A full blue picture without anything in it. Just drag it approximately where a hole is located in sky. It don't have to be accurate.
- Having a manual placement of an image relative to another one like in realviz can also be a good idea. It's easy to setup : it's a drag and rotate tool. You validate when both pictures are good enough placed for you.
- Secondary control points algorithm : For some hard case, like this one, we could think of a secondary algorithm for control point detection. Actually APP is only relying on SIFT which have wonderful properties. But is some case, like this noisy picture, it fails. Having a second type of control point detector could improve the manual add of control point.

no avatar
hankkarl
Member
 
Topic author
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Fri Nov 10, 2006 5:13 pm

'[bo wrote:']hankkarl, I don't have RealViz, so I cannot confirm, but I was thinking something like a hybrid solution. Dragging an image to position and then, using similar to CP Editor interface, set "a zone" or "a point" where you want the images to match. In your example (Chicago1Selection1), imagine using the same selection tool not to detect point, but rather to say to APP - this here single lighted window is this one in the other image and that's that.

Yes, and I think you'd need at least two points to get the pictures to be rotated properly with each other. Three points may be better because you may get a clue as to image size differences, camera rotation, etc

RealViz has a free demo. Its a nice program, but even the $500 version doesn't seem to be as good as APP. The only exception is that RealViz lets you select an object to use in the final pano or to eliminate from the final pano, so you could say "use this view out of the window instead of any other photo for this region" or "don't use this part of the picture, I intentionally want a different exposure level here".
Last edited by hankkarl on Fri Nov 10, 2006 5:16 pm, edited 1 time in total.

no avatar
hankkarl
Member
 
Topic author
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Fri Nov 10, 2006 5:29 pm

AlexandreJ wrote:I think for future version, some tool can be added to handle such case :
- Manual location of a single picture : drag'n'drop of a single picture to an approximate location inside the panorama view => for sky hole. A full blue picture without anything in it. Just drag it approximately where a hole is located in sky. It don't have to be accurate.
- Having a manual placement of an image relative to another one like in realviz can also be a good idea. It's easy to setup : it's a drag and rotate tool. You validate when both pictures are good enough placed for you.

And a button to say "find CP with the photos that overlap" would be good. Often, APP cannot place an image, but I can go into the CP editor and find points.

Imagine a part of a spherical pano where one picture overlaps eight others in a "tic-tac-toe" grid (or 3x3 section of a chessboard). If the middle photo doesn't get placed by APP at first, the user could drag and rotate it to about the right position. One button would then find the overlaping areas of each photo and use overlap to run SIFT and find CPs. This should be a manual operation, because sometimes (e.g. blue sky) its not needed or will result in bad CPs.

AlexandreJ wrote:- Secondary control points algorithm : For some hard case, like this one, we could think of a secondary algorithm for control point detection. Actually APP is only relying on SIFT which have wonderful properties. But is some case, like this noisy picture, it fails. Having a second type of control point detector could improve the manual add of control point.

The manual editor would be good enough for me, I'm not too concerned about noisy pictures. The only reason to add it would be that it improves what SIFT does and helps SIFT give better initial results. On second thought, this may help out for panos that have a photo not linked to any other, or two or more groups of photos that have no common links.
Last edited by hankkarl on Fri Nov 10, 2006 5:31 pm, edited 1 time in total.

no avatar
hankkarl
Member
 
Topic author
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Sun Nov 26, 2006 10:58 pm

'[bo wrote:']hankkarl, I don't have RealViz, so I cannot confirm, but I was thinking something like a hybrid solution. Dragging an image to position and then, using similar to CP Editor interface, set "a zone" or "a point" where you want the images to match. In your example (Chicago1Selection1), imagine using the same selection tool not to detect point, but rather to say to APP - this here single lighted window is this one in the other image and that's that.

Kind of an obvious comment, but:

Another thing to consider here is that you probably wouldn't use the manual editor if the existing CP editor worked well.

And we know that what humans see and what SIFT sees are two different things--SIFT gets points humans never could and humans get points SIFT does not.

So if there's an automatic fine tune or multipoint selection sort of thing under manual, it should probably use something very different than SIFT. It may use contrast or some sort of line finding, etc.
Last edited by hankkarl on Sun Nov 26, 2006 11:00 pm, edited 1 time in total.


Who is online

Users browsing this forum: No registered users and 2 guests