Can global RMS be used to indicate quality?  

Share your tips and tricks here or get help with any Autopano Pro / Giga problem!
No bug reports (of any kind) in this forum!
no avatar
HansKeesom
Member
 
Topic author
Posts: 2210
Likes: 1 post
Liked in: 2 posts
Joined: Mon Jul 19, 2010 8:53 pm
Info

Can global RMS be used to indicate quality?

by HansKeesom » Fri Sep 02, 2011 12:07 am

I realise that outside this forum not many will know the meaning of RMS.
Still I would like to have a hard number to indicate quality. Is RMS the number to use.
I do the following :
- I process photos for (an)other photographer. If a group, even after a number of detections with different settings and some general improvements, still has a high RMS, does it mean that photographer delivered bad quality? What is bad then? A RMS of 6? 5? 4?
- I deliver to all kind of clients. Some are not so keen on quality and pay a lower price. Some are very keen on quality and pay a higher price. If I would hire someone to do the editing, can I tell them for the lower paying customer add control points until your reach a RMS of 3 and for the higher paying customer go on until a RMS of 1.0 or less?

regards,

Hans Keesom
Regards, Hans Keesom
I stitch and render for other photographers. Price: 25 euro or less, no cure no pay. If you want to concentrate on your business let me do the stitching for you. Free TB of Dropbox space when you have more then 250 euro business a year.

no avatar
digipano
Member
 
Posts: 582
Likes: 1 post
Liked in: 0 post
Joined: Sat Feb 16, 2008 9:07 am
Info

by digipano » Fri Sep 02, 2011 4:31 am

HansKeesom wrote:- I process photos for (an)other photographer. If a group, even after a number of detections with different settings and some general improvements, still has a high RMS, does it mean that photographer delivered bad quality? What is bad then? A RMS of 6? 5? 4?

In a earlier reply Alex mentioned that RMS is not a guarantee of the good stitch, we cant solely rely on those numbers but keeping them below 4 is going to get you good results, the least you can get is 1, going below 1 is practically difficult to impossible. So any RMS above 5-6 is showing bad calibration.

- I deliver to all kind of clients. Some are not so keen on quality and pay a lower price. Some are very keen on quality and pay a higher price. If I would hire someone to do the editing, can I tell them for the lower paying customer add control points until your reach a RMS of 3 and for the higher paying customer go on until a RMS of 1.0 or less?

This is not that simple, for example if I pay you higher price of $20 per stitch will you get the RMS below 2-3 when my pano head is not actually calibrated & the scene I have shot is of a bathroom with repeating pattern on to the wall ? you cant just do that if I am not calibrated correctly.

User avatar
AlexandreJ
Kolor Team
 
Posts: 5987
Likes: 7 posts
Liked in: 10 posts
Joined: Mon Nov 14, 2005 4:56 pm
Location: Francin, France
Info

by AlexandreJ » Fri Sep 02, 2011 8:25 am

RMS is an indication of quality of a stitch, but not an absolute valuation. For example, you could just have one CP per image pair and with that having RMS close to 0. Perfect in the sense of error reprojection ( RMS law ), but probably bad when you inspect visually the panorama.

I know we need to think of a new indicator that will correct the RMS which is too geometrical. It should be corrected by the average number of cp / image pair, and does reflect that CP are over the whole overlapping zone and not only a small part of it.

no avatar
HansKeesom
Member
 
Topic author
Posts: 2210
Likes: 1 post
Liked in: 2 posts
Joined: Mon Jul 19, 2010 8:53 pm
Info

by HansKeesom » Fri Sep 02, 2011 10:55 am

AlexandreJ wrote:RMS is an indication of quality of a stitch, but not an absolute valuation. For example, you could just have one CP per image pair and with that having RMS close to 0. Perfect in the sense of error reprojection ( RMS law ), but probably bad when you inspect visually the panorama.

I know we need to think of a new indicator that will correct the RMS which is too geometrical. It should be corrected by the average number of cp / image pair, and does reflect that CP are over the whole overlapping zone and not only a small part of it.

Hi Alexandre.

For me you don't have to give high priority to making RMS an better indicator for quality. But the general idea is that when adding control points I like to know whether things got better or not. But problably the only way to tell is by looking at the rendered picture.
Regards, Hans Keesom
I stitch and render for other photographers. Price: 25 euro or less, no cure no pay. If you want to concentrate on your business let me do the stitching for you. Free TB of Dropbox space when you have more then 250 euro business a year.

no avatar
hankkarl
Member
 
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Wed Sep 07, 2011 6:29 pm

AlexandreJ wrote:RMS is an indication of quality of a stitch, but not an absolute valuation. For example, you could just have one CP per image pair and with that having RMS close to 0. Perfect in the sense of error reprojection ( RMS law ), but probably bad when you inspect visually the panorama.

I know we need to think of a new indicator that will correct the RMS which is too geometrical. It should be corrected by the average number of cp / image pair, and does reflect that CP are over the whole overlapping zone and not only a small part of it.

RMS can be meaningless. Imagine a one row pano with all CPs on clouds so you get a really low RMS. But the clouds have moved in relation to the ground in each picture.

A better measure of quality may be to compare the overlaping parts of each two images. Perhaps by using edge detection, or by subtracting the pixels in one image from the other and looking for a uniform colored field.

Part of the issue is that AFIK APG uses RMS as part of its positioning algorithm. So you're using RMS to position, then using the same method (RMS) to say its good.

IMO, the quality check has to be by some method that replicates what people look for. I think people look for edges and discontinuities, and also color issues between images. But we also look for lines that are sharp in one place, and get blurrier in another (could be caused by blending after a misalignment.)
Last edited by hankkarl on Wed Sep 07, 2011 6:30 pm, edited 1 time in total.

no avatar
HansKeesom
Member
 
Topic author
Posts: 2210
Likes: 1 post
Liked in: 2 posts
Joined: Mon Jul 19, 2010 8:53 pm
Info

by HansKeesom » Wed Sep 07, 2011 6:38 pm

What people look for....

What disturbs me in a pano are broken lines.

This means that if two photos overlap, the overlapping area is the result of putting an area of one photo on top of an likewise area of another photo. If in both photos there are long unbroken lines, then there should be no broken line in the result. If there is, that would get a bad number from me.
Regards, Hans Keesom
I stitch and render for other photographers. Price: 25 euro or less, no cure no pay. If you want to concentrate on your business let me do the stitching for you. Free TB of Dropbox space when you have more then 250 euro business a year.

no avatar
HansKeesom
Member
 
Topic author
Posts: 2210
Likes: 1 post
Liked in: 2 posts
Joined: Mon Jul 19, 2010 8:53 pm
Info

by HansKeesom » Wed Sep 07, 2011 9:02 pm

Say you set your camera to store every photos as both RAW and jpeg.
Say detection on the jpeg's is 4.73 and on the RAW is 4.50. This is not surprising right? It would be if its the jpegs would score better, right?

Now imaginge you have an external programm converting the RAW's to tiff and you get on the tiff's a RMS of 5.29. Does this mean your RAW-conversion is bad? Should it be at least ast good as the 4.50?
Regards, Hans Keesom
I stitch and render for other photographers. Price: 25 euro or less, no cure no pay. If you want to concentrate on your business let me do the stitching for you. Free TB of Dropbox space when you have more then 250 euro business a year.

User avatar
AlexandreJ
Kolor Team
 
Posts: 5987
Likes: 7 posts
Liked in: 10 posts
Joined: Mon Nov 14, 2005 4:56 pm
Location: Francin, France
Info

by AlexandreJ » Thu Sep 08, 2011 8:10 am

I never found myself any explanation about image format and RMS relation. Because image format often affects what is invisible in the image ( sharpen, dust, crusty, single pixels, etc ). Or these little changes that are invisible to us, is often visible to our control point detector. So the RMS will be affected. In bad or good ? I don't know.

no avatar
HansKeesom
Member
 
Topic author
Posts: 2210
Likes: 1 post
Liked in: 2 posts
Joined: Mon Jul 19, 2010 8:53 pm
Info

by HansKeesom » Thu Sep 08, 2011 2:57 pm

hi Alexandre,

You never found and explanation for the relation. Does this mean you agree there seems to be a relation?

Now my second question is not about the relation between formats. It is more about whether a bad RMS on a processed set of photos compares to a good RMS on the original raw ones, would that mean we conclude the processing was bad?
Regards, Hans Keesom
I stitch and render for other photographers. Price: 25 euro or less, no cure no pay. If you want to concentrate on your business let me do the stitching for you. Free TB of Dropbox space when you have more then 250 euro business a year.

User avatar
AlexandreJ
Kolor Team
 
Posts: 5987
Likes: 7 posts
Liked in: 10 posts
Joined: Mon Nov 14, 2005 4:56 pm
Location: Francin, France
Info

by AlexandreJ » Thu Sep 08, 2011 3:39 pm

HansKeesom wrote:You never found and explanation for the relation. Does this mean you agree there seems to be a relation?

Honestly, this need a deeper analysis. As I said, location of control point can vary a little depending on image format ( when I mean varying, it's the decimal part inside a single pixels ).
So, there is a relation, of course. But it has no sense to say it's good or bad. It just gives another solution.

HansKeesom wrote:Now my second question is not about the relation between formats. It is more about whether a bad RMS on a processed set of photos compares to a good RMS on the original raw ones, would that mean we conclude the processing was bad?

Again, this is not really relevant. We cannot measure the quality of the processing just by RMS. If we imagine a white wall with some really little details, by analysing the raw but with noise reduction, you'll get a bad result whereas if you developed the raw nicely with crispy details, the results will be better because of image quality.


Who is online

Users browsing this forum: No registered users and 2 guests