Geometric Analysis distance limiter  

Got some great idea or a feature request? Post it here and discuss it. The most requested concepts are usually implemented, as Autopano Pro / Giga is very community driven.
no avatar
hankkarl
Member
 
Topic author
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

Geometric Analysis distance limiter

by hankkarl » Mon Nov 27, 2006 1:10 am

When using geometric analysis on a 360x180 pano, I got a lot of "long" links that may or may not be correct (areas of ceiling or carpet in particular).

I'd like to see some sort of (optional or user selectable) limit placed on geometric analysis. Assuming a full spherical pano taken on a pano head with equal spacing in all axisis will have a regular arrangement of the individual images in an almost rectangular matrix. I'd like an option to limit geometric analysis to a certain length of link, or a given percent coverage (this can be tricky with vertex and nadir shots)

Assuming the shots are numbered as follows:

Vertex

A1 A2 A3 A4 A5 A6 A7 A8

B1 B2 B3 B4 B5 B6 B7 B8

C1 C2 C3 C4 C5 C6 C7 C8


Nadir

Ideally, I'd like geometric analysis to look at a rectangle around the current image. Assuming we're looking at B4, then go right until you find a shot, and go 50% further (in case bracketed exposures are used) Go left, up and down the same way. This should eliminate all the very long links.

Alternatly, it may be better to take the closest "n" shots to the current shot (where the user enters "n") to cover the handheld case.

no avatar
GURL
Member
 
Posts: 2943
Likes: 0 post
Liked in: 0 post
Joined: Tue Dec 06, 2005 1:57 pm
Location: Grenoble
Info

by GURL » Mon Nov 27, 2006 11:24 am

Which mode of geometric analysis did you use ?
(1) Find control points where none exist: Looks for control points between overlapping images that have no image link created. Use this mode when there are some missing image links.
(2) Find control points everywhere and merge with old CP: Look for control points in every image, then add the found points to those, already present.
(3) Find control points everywhere and remove old CP: Look for control points in every image, then remove the current points and use only the new ones. Use this mode when the overall control points quality is weak.

The idea behind my question is that when huge errors exist in the placement of some images then using current image placement can't improve things. I have yet to gain experience in using this tool but see it as some sort of multipass optimization where the user is helping APP by removing wrong links established by APP and by setting good links where none exist yet.

Being able to forbid APP to place links between some images could help for that, but the way the user could state those "do not link" rules looks difficult to devise. Perhaps we could have "anti-CPs" which, rather than attract the corresponding images, would repel them and would tend to move them on two opposite sides of the sphere...

In case of bracketed exposures I tend to select first a set of non duplicated images and to stitch that set first. When the result is OK, it's easy to add the remaining images and to help their placement by forcing a one to one correspondence with the previous ones.

Here is an example where your sheme could be applied (this is a series of 16 fisheye defished images, 8 for the ground and 8 for the sky):


Last edited by GURL on Mon Nov 27, 2006 11:55 am, edited 1 time in total.
Georges

no avatar
hankkarl
Member
 
Topic author
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Mon Nov 27, 2006 5:31 pm

I used the default (find CPs where none exist). The pano in question was pretty good--overall CP quality was 3.x or 4.x, I forget which. The pano looked correct, but a lot of images didn't have links -- in your exampls, imangine the link 11-12 was missing, 4-12 and 5-11 were also missing. The pano could "bend" between 4 and 5 and still have a low overall CP quality if similar links were missing elsewhere.

Using geometry analysis often results in worse overall CP quality, probably because the found CPs need to be weeded out.

The issue I have is that my panohead has several "click stops" but they don't always match up well with the lens parameters, so I may have an option to overlap by 51% or 27% (I made these numbers up for the example). If I overlap by 51%, picture 2 (in your image) will overlap picture 4 by a slight amount. picture 9 and picture 4 will have a slight overlap in one corner. I don't want APP to use these areas in geometric analysis--the lens is just too distorted at the edges (Especially on a full frame sensor)

I'm thinking that the "do not link" rules may be easy for the user to understand, especially if a graphical tool is used. One way is to allow the user to select an image, then draw (or type) a radius. A circle would be shown on the pano, in your example let's say that you select image 11 and draw a radius to cover 10, 12, and 4. (This could be a square or rectangle selection also).

Alternately, the user could be given a selection "only perform geometric analysis when the pictures overlap by more than xx percent" and the user fills in xx.

I like your "anit-CP" idea, but I don't know that they should repel. I find that APP sometimes picks points on the pano head (nadir and the next level up). I'd rather that APP allows me to define the pano head in the nadir shot, then use that to eliminate all links that fall on the pano head--this includes links between the nadir and bottom row, and links between two pictures on the bottom row (or any row that contains a picture of the pano head)

I'll have to try your idea about bracketed exposures. But I'm lazy--I use autobracketing and would have to pick out all the images from one given exposure. I'd rather have a feature, perhaps another checkbox in the layer editor, that says "show only this layer" or "work on only this layer" If it is "show only this layer" it could work as a toggle, the user could display or hide each layer, like in photoshop. If it was "work on only this layer" there would have to be a button that says "find all the images behind the active layer and align them behind the current image"

This brings up another issue--its tough to see when there's a link missing, especially between bracketed images that are stacked on top of each other. Perhaps we need another option to the link editor that shows only links that are not present between two images. Of course, we don't want two pictures that don't overlap to have links shown. Instead of link "quality", APP could show the percent the images overlap for these missing links.

User avatar
[bo]
Moderator
 
Posts: 1226
Likes: 0 post
Liked in: 0 post
Joined: Fri May 05, 2006 8:16 am
Location: Bulgaria
Info

by [bo] » Tue Nov 28, 2006 9:14 am

I thought about that one too... or at least of something similar. That's why I posted this some time ago:
http://forum.autopano.net/p4970-2006-10-31-10-07-59#p4970

The "length" of links as I see it is an option to force APP to create links only in adjacent images, after the initial detection. So, in a grid like this:
Code: Select all
1    2    3
4    5    6
7    8    9

APP will create a missing 1-4 or 5-9 link, but not 1-9 or 2-8. At least in the material I get to work with, those "overlapping" links have a negative effect on the whole panorama.
Some of my panoramas, posted in the Autopano Pro flickr group.

no avatar
GURL
Member
 
Posts: 2943
Likes: 0 post
Liked in: 0 post
Joined: Tue Dec 06, 2005 1:57 pm
Location: Grenoble
Info

by GURL » Tue Nov 28, 2006 11:58 am

The "length" of links should be expressed as an 'angular distance'.

Here is an example of a 'grid' for a 360° x 180° pano:
Code: Select all
          7
1   2   3   4   5   6
          8

..where 1, 2, ... 6 is a series of horizontal shots, 7 is zenith shot and 8 is nadir shot.
We want links like:
1 - 2, 2 - 3, etc, where angular distance is 60° (360°/6)
7 - 1, 7 - 2, etc and 8 - 1, 8 - 2, etc, where angular distance is 90°
We don't want links like:
7 - 8 where angular distance is 180°
1 - 3, 2 - 4, etc where angular distance is 120°

Actually, using a grid looks like a good idea when using a pano-head which have clicks stops and either the pitch variation or the yaw variation is limited. When a large part of the sphere is photographed or when a pano-head is not used, I'm quite sure "expressing to the stitcher where source images should go" is much more difficult...

PTgui have a special tool where users can (but are not forced to) place the source images at their approximative location: when the number of shots is low this is practical, when the shot number is high this is not possible.

Perhaps, a Link Editor tool to remove links where angular distance from image center to image center is higher than the value the user specifies could help?
Last edited by GURL on Tue Nov 28, 2006 12:05 pm, edited 1 time in total.
Georges

User avatar
[bo]
Moderator
 
Posts: 1226
Likes: 0 post
Liked in: 0 post
Joined: Fri May 05, 2006 8:16 am
Location: Bulgaria
Info

by [bo] » Tue Nov 28, 2006 12:32 pm

Yes, that should be like a CP cleaner, but more of a Image Link cleaner. I really liked that angular distance explanation. Although I'm not sure how it is calculated and if it will work with free-form grids, like taken with handheld camera, etc.
Some of my panoramas, posted in the Autopano Pro flickr group.

no avatar
hankkarl
Member
 
Topic author
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Tue Nov 28, 2006 6:24 pm

OK, the link cleaner is a good idea, but IMO it should apply to geometric analysis also--APP knows beforehand the distance between two images, and shouldn't waste time calculating a link that will later be cleaned.

Angular distance is better, the linear distance is distorted on the spherical panos.

Angular distance should work with freeform grids, especially if the minimum (or maximum) angle is specified by the user. The idea is to not use two pictures that only overlap at the very edges. So if a freeform grid is taken with all images the same focal length, there shouldn't be a problem (but the user may have to tweak the angular distance.)

But will angular distance work in a mixed size pano? If somone takes a photo at 12mm and then uses a 300mm lens to fill in details this may not work. But a percent overlap would work, if the percent was used based on the smaller (ie longer focal length) picture.

User avatar
AlexandreJ
Kolor Team
 
Posts: 5988
Likes: 7 posts
Liked in: 10 posts
Joined: Mon Nov 14, 2005 4:56 pm
Location: Francin, France
Info

by AlexandreJ » Tue Nov 28, 2006 6:29 pm

In fact, GA is not working with either the angular nor the linear distance. It's a projection based approach. If some points of picture A can be projected on a picture B and some points of picture B can be projected on picture A, these pictures are linked. So a sift detection is performed on zones where points were projected.
There are no distortion is this kind of approach. It should be possible to prove it mathematically good.

no avatar
hankkarl
Member
 
Topic author
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Tue Nov 28, 2006 11:32 pm

Thanks Alexandre,

I think the biggest thing you point out is that a lot of us are guessing at how GA and SIFT work, and are constructing myths and superstitions to explain it. I hope you will have a quick explanation for the layman in the documentation. Please! :-)

In another thread in this forum, the number of CPs and weighting was discussed. It brought up a thought that because some lenses distort more toward the edges, a large number of CPs in two images that only overlap in their corners (say, 5 and 9 in Bo's example) may throw off optimization because of the lens distortion.

Perhaps a simple way to do the distance limitation is to only use points near the center of the lens -- that is, have the user specify the radius of a circle and only use the points inside the circle (specify the radius as a percentage of the maximum diagonal?)

Alternately, giving gradually more weight to the points near the center of the image may lead to a more accurate optimization but the extra processing probably wouldn't be worth it.
Last edited by hankkarl on Tue Nov 28, 2006 11:35 pm, edited 1 time in total.

User avatar
[bo]
Moderator
 
Posts: 1226
Likes: 0 post
Liked in: 0 post
Joined: Fri May 05, 2006 8:16 am
Location: Bulgaria
Info

by [bo] » Wed Nov 29, 2006 8:47 am

AlexandreJ wrote:In fact, GA is not working with either the angular nor the linear distance.

I'm not sure I understand, but does this mean that no such "Image Link Cleaning" or "restricting" is possible?

And Karl is right, we should put some more explanation on this in the Wiki - at least to say that GA can produce too many links, which, in some cases, degrade the overall RMS. I think the option that lately (hopefully) was rename to "Find control points where none exist" should become "Create every link possible" or something like this.
Some of my panoramas, posted in the Autopano Pro flickr group.

User avatar
AlexandreJ
Kolor Team
 
Posts: 5988
Likes: 7 posts
Liked in: 10 posts
Joined: Mon Nov 14, 2005 4:56 pm
Location: Francin, France
Info

by AlexandreJ » Wed Nov 29, 2006 8:48 am

I'm pretty sure, it's the inverse that should be implemented : more points on the edge and less in the center.
It comes from the fact that the lens has a radial distortion. Having CP near the center doesn't give good candidates for the lens distortion calculation. The farest from they are, the best they are for lens distortion.

---

You're right, perhaps a theorical documentation on SIFT and lens distortion is welcomed.

no avatar
GURL
Member
 
Posts: 2943
Likes: 0 post
Liked in: 0 post
Joined: Tue Dec 06, 2005 1:57 pm
Location: Grenoble
Info

by GURL » Wed Nov 29, 2006 2:02 pm

AlexandreJ wrote:I'm pretty sure, it's the inverse that should be implemented : more points on the edge and less in the center.

I'm quite sure that the best situation for any optimizer to find out distortion is when the side on one image is crossing the center of another image. I did that using PanoTools to establish a, b, c, d, e coefs for a wide-angle lens and it worked very well: I believe placing CPs along a straight line of the subject is not mandatory for that !
Georges

no avatar
hankkarl
Member
 
Topic author
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Wed Nov 29, 2006 7:22 pm

AlexandreJ wrote:I'm pretty sure, it's the inverse that should be implemented : more points on the edge and less in the center.
It comes from the fact that the lens has a radial distortion. Having CP near the center doesn't give good candidates for the lens distortion calculation. The farest from they are, the best they are for lens distortion.

So for lens distortion, radial distance is important, the further the better? Note that not all lenses distort in a linear manner--see http://www.16-9.net/lens_tests/14mm/14mm_test_3.html

But is this true for optimization? Seems you have a difficult balancing act-getting the lens distortion and picture placement correct at the same time!


---
AlexandreJ wrote:You're right, perhaps a theorical documentation on SIFT and lens distortion is welcomed.

Please, but not too technical :-)

User avatar
AlexandreJ
Kolor Team
 
Posts: 5988
Likes: 7 posts
Liked in: 10 posts
Joined: Mon Nov 14, 2005 4:56 pm
Location: Francin, France
Info

by AlexandreJ » Wed Nov 29, 2006 7:31 pm

The linearity has nothing to do with the fact that the distortion is still radial. Do the picture change when you switch from portrait to lansdscape orientation. If it changes, you have a non radial distortion.

no avatar
GURL
Member
 
Posts: 2943
Likes: 0 post
Liked in: 0 post
Joined: Tue Dec 06, 2005 1:57 pm
Location: Grenoble
Info

by GURL » Thu Nov 30, 2006 12:11 am

AlexandreJ wrote:Do the picture change when you switch from portrait to landscape orientation.

No lens element is perfectly centered nor perfectly aligned so that the answer is : alas, yes, it changes!
The place where the lens axis cross the sensor never is the center of said sensor (and this cause no harm in usual photography.) Besides that, a well known reflex camera manufacturer sells a very inexpensive "kit lens" where changes are visible in the viewfinder when you touch the lens barrel...
Georges


Who is online

Users browsing this forum: No registered users and 2 guests