Page 1 of 3

hdr workflow test

PostPosted: Wed Feb 20, 2008 2:07 pm
by DrSlony
I'm carrying out certain tests for a hdr workflow and I have some questions for the developers (Hello Alexandre! ;] ):
a- Does APP treat Radiance RGBE & OpenEXR identically? Which one would you recommend and why?
b- What about 32bit TIFF?

These are the workflows I will carry out and report results here:
1- render layers in APP, generate hdr and tonemap in Photomatix or Qtpfsgui
2- batch hdr in PM and save as openexr, stitch in APP, tonemap in PM or Q
3- batch hdr in PM and save as tiff 16bit, stitch in APP, tonemap in PM or Q
4- batch hdr and tonemap in PM and save as tiff 16bit, stitch in APP

The problem with PM is that max image size allowed for tonemapping is limited by physical RAM, no scratch disk usage is possible and they told it it wont be possible anytime soon, so with 4GB RAM I can tonemap images up to about 15 000 x 7500. For bigger images only workflow 4 is possible.

OpenEXR stores 48bits per pixel
Radiance RGBE stores 32bits per pixel
TIFF16 stores 16 bits per pixel

Theoretically the best workflows are 1 and 2 because tonemapping is supposed to take global lighting into account, which is what happens if I tonemap the whole pano and not just source images. Only regional lighting is used in step 4. Step 3 is worse than 1 and 2 because TIFF16 stores less color info than OpenEXR does.

Practically could be different, that is why I want to test this.

After all that, I will postprocess the pano by adjusting levels, cloning, etc. I don't think I need the 48 bits of info that OpenEXR provides to adjust levels - this will after all be a small adjustment. Also perhaps Photoshop and the future GIMP versions won't support editing and saving OpenEXR so Im doomed to TIFF16 (unless I choose to use another image editor, which I don't). Other factors are swap space, final hdr image size [MB] and stitching time - will there be a significant time difference between stitching a set of OpenEXR images and a set of TIFF16 images? If there won't then obviously I will go for OpenEXR. Will an OpenEXR pano be bigger than a TIFF16 pano? If not significantly (e.g. 10% bigger) then again I choose OpenEXR. Same with RAM requirements and the rest of those factors I mentioned.

And then there is the future version of APP to consider, the one that will allow for high quality smartblend tonemapping.

Anyone here who already tested this?

PostPosted: Wed Feb 20, 2008 2:20 pm
by AlexandreJ
DrSlony wrote:I'm carrying out certain tests for a hdr workflow and I have some questions for the developers (Hello Alexandre! ;] ):

I'm here ;)

DrSlony wrote:a- Does APP treat Radiance RGBE & OpenEXR identically? Which one would you recommend and why?
b- What about 32bit TIFF?

Only HDR for the moment. OpenEXR is planned the same time as TIFF float format.

DrSlony wrote:These are the workflows I will carry out and report results here:
1- render layers in APP, generate hdr and tonemap in Photomatix or Qtpfsgui
2- batch hdr in PM and save as openexr, stitch in APP, tonemap in PM or Q
3- batch hdr in PM and save as tiff 16bit, stitch in APP, tonemap in PM or Q
4- batch hdr and tonemap in PM and save as tiff 16bit, stitch in APP

hum, interesting. 2 is not possible because of openEXR not supported, but could be done in HDR.
I think you should study 1 and 4 first. It's the two limits : stitch first, hdr after or hdr and tonemap first, then stitch.

DrSlony wrote:The problem with PM is that max image size allowed for tonemapping is limited by physical RAM, no scratch disk usage is possible and they told it it wont be possible anytime soon, so with 4GB RAM I can tonemap images up to about 17 000 x 8500. For bigger images only workflow 4 is possible.
OpenEXR stores 48bits per pixel
Radiance RGBE stores 32bits per pixel
TIFF16 stores 16 bits per pixel

I have a prediction for this year. A new HDR star is born : enfuse / tufuse. This is a new way to do HDR : http://panospace.wordpress.com/

PostPosted: Wed Feb 20, 2008 3:55 pm
by DrSlony
AlexandreJ wrote:Only HDR for the moment

I assume that by this you mean Radiance RGBE, because Radiance RGBE gets saved with the extension .hdr.

I always used 1 because theoretically it's the best option (global brightness etc.) but practically it's not because I can only render ~ 15 000 x 7 500 px @ 4GB RAM. I am trying 4 now. Since global lighting wasnt taken into account, the tonemapped source images have color and brightness differences easily visible with the eye, I hope APP will do a good job.

Thank you for the reply and link, I will eagerly dive into tufuse when I get back home to Poland to my Linux in March! And hopefully taste sweet 64bit APP by then! ;] I know I know, we users are terrible buggers :D

ps. how can I edit the topic of this thread? 'test' isn't enough, I want to change it to 'hdr workflow test'.

PostPosted: Wed Feb 20, 2008 4:11 pm
by AlexandreJ
DrSlony wrote:ps. how can I edit the topic of this thread? 'test' isn't enough, I want to change it to 'hdr workflow test'.

Agreed. Done it for you.

PostPosted: Mon Feb 25, 2008 1:58 pm
by DrSlony
1- render layers in APP, generate hdr and tonemap in Photomatix or Qtpfsgui
2- batch hdr in PM and save as openexr, stitch in APP, tonemap in PM or Q
3- batch hdr in PM and save as tiff 16bit, stitch in APP, tonemap in PM or Q
4- batch hdr and tonemap in PM and save as tiff 16bit, stitch in APP

Results:
1- best for small panos which, for some reason, require you to tonemap in the orthodox way (ie. by using global light values of the whole pano, not just local light values as workflow 4 uses). Required RAM in PM when tonemapping: ram = megapixels x 32. (more than 32 if "360" option is checked, forgot how much exactly, possibly 64). Useful only for small panos that you want to tonemap or for huge panos that you dont need to tonemap, for example if you want to use the HDR in 3d rendering, but since 3d rendering doesnt require high resolution HDRs (low resolution will do) this workflow is pretty pointless.
2- APP doesn't support openexr, only radiance, so I tested with that. Slow as hell. Rendered hdr needs to be tonemapped so same limitations and uses apply as to workflow 1.
3- Cannot batch save HDRs as tiff16 (or 32) in PM so abandoned this test. Even if you could, it would be better to use openexr than tiff (workflow 2) since tiff16 files are bigger than openexr and tiff16 files have less bits per pixel (16) than openexr (48).
4- good for all panos big and small. Working with 16bit files is slow but not as slow as with radiance. I recommend this workflow for now. Once APP will offer better tonemapping quality and once I test TuFuse I will update my tests and report.



Other findings:

[uli]HDR files created from 8bit input files (-2, 0, +2) are slightly smaller than those created from 16bit input files. This is pretty obvious as 16bit input files offer more info (bits per pixel). The difference in the HDR's size created from 8bit input images and the one created from 16bit input images is very small, about 1-5%, and there is no visual difference after tonemapping them using the same settings, so if your camera shoots acceptible quality jpegs then using them is sufficient. Obviously shooting in RAW still has its advantages, such as being able to use better demosaicing algorithms than those used by your camera, so your processed-from-raw images can be sharper than camera-produced jpegs.[/uli]
[uli]The smallest HDR file format is OpenEXR using piz compression (so APP should support this). Coincidentally, OpenEXR offer the most bits per pixel - 48.[/uli]
[uli]The biggest HDR file format is floating point TIFFs, so avoid them.[/uli]
[uli]Tested on Windows XP using AMD Athlon 64 X2 Dual 4200+ (2.21GHz), 2.75GB RAM, 1 SATA HD.[/uli]
[uli]APP-140_2007-12-18 settings: cache size 500MB, memory used during rendering high, 2CPU, detection quality high, 87 keypoints per image pair, strong algorithm with adjust lens distortion.[/uli]
[uli]Images from Pentax k10d 3872x2592px. 3 bracketed shots.[/uli]
[uli]For workflows 2 and 4 I used 24 hdr/stitched images.[/uli]
[uli]For workflow 1 I used 72 8bit TIFFs (the same images that were HDR'd or tonemapped for workflows 2 and 4).[/uli]
[uli]Workflow 2 (.hdr) detection time 13 minutes, workflow 4 detection time 7 minutes.[/uli]

Example file sizes - HDR saved from PM as:
[uli]Floating Point TIFF [48bits per pixel] 117MB[/uli]
[uli]Radiance RGBE [32bpp] 30MB[/uli]
[uli]OpenEXR ZIP [48bpp] 24MB[/uli]
[uli]OpenEXR PIZ [48bpp] 21MB[/uli]

Why do I save tonemapped images as 16bit tiffs instead of 8bit? Because using local light values means that the tonemapped images will have very visible seams when stitched without color correction. Thanks to APP's magic, the color correction makes transitions invisible, and to preserve image quality and to prevent posterization I use 16bit since strong color or luminosity correction to an 8bit image is easily noticable. Also when postprocessing - correcting levels, cloning and such - I lose far less chroma/luminance data and I'm sure I won't get any posterizing when working on 16bit images. Far more room to manoeuvre in.

As I wrote ealier, I recommend using workflow 4 for now. The orthodox approach tells us that we should tonemap the whole pano because global light values are used when tonemapping and if we dont do that, if we tonemap individual image sets before stitching, then we will get "strange" results. In practice, however, this strange approach can be very pleasing, because it will have more local contrast than the orthodox version. Compare these two images below. (Now).
The first was done by batch creating .hdr files from each image pair, stitching them, rendering as .hdr and tonemapping in PM. The orthodox way. This is very slow and requires lots of ram so only good for small panos. The left window (with the red car outside) has an even sky color but the pano in general looks a bit washed out.
The second one was done using workflow 4. Requires little ram and was much faster than the previous one. Although this isn't orthodox, it looks visually more pleasing. Note that the left window here doesn't look as good as on the orthodox one because the edges are lighter than the center, but the whole pano in general is nicer, not washed out, more local contrast. Also I'm sure I could avoid those lighter edges around the window frame if I would change the settings a bit when tonemapping.

PostPosted: Thu Feb 28, 2008 2:47 pm
by DrSlony
One word: TuFuse!

TuFuse is a freeware command-line program currently only for M$ Windows (but runs well in WINE) that blends several exposure bracketed shots. It allows the photographer to portray a much higher dynamic range in the image than his camera can capture, same as Photomatix does, but without the hassle of going into HDR and then tonemapping. Output quality is impressive and seems very natural, no halos!

As using command-line is slow, I recommend this GUI:
Enfuse GUI http://software.bergmark.com/enfuseGUI/

So far this program accepts only TIFF files as input. I listed supported input TIFF compression types in the tabel below.

Photos below are:
1- -2EV
2- 0EV
3- Enfused result
4- +2EV

PostPosted: Wed Mar 05, 2008 5:19 pm
by DrSlony
I wrote to Max Lyons, author of TuFuse. Below are my questions and his answers:


> > 1- I noticed that when I use 16bit input TIFF files then I get a 16bit
> > output. However there is no option to use 8bit input files and save
> > the result as 16bit tiff. I understand that this is because TuFuse
> > does not create a higher dynamic range image, just blends pixels. If
> > TuFuse could save 16bit output files from 3 or more 8bit input files,
> > would the 16bits of potential give any advantage?

Yes, I think it might be useful. I plan to add this ability (16 bit output
from 8 bit source files) in a future version (soon).

> > 2- Secondly, do you have support for JPEG input files planned? I
> > suppose to have quick and easy support for different input files you
> > could use any of the several available libraries to internally convert
> > whatever input file into tiff and then process the result.

I plan to add this as well.

> > 3- Do you have linux support planned?

Not in the immediate future...

PostPosted: Wed Mar 05, 2008 6:07 pm
by GURL
DrSlony wrote:One word: TuFuse!

I'm using Enfuse and EnfuseGUI with good results. Some reasons to prefer TuFuse?

PostPosted: Wed Mar 05, 2008 7:29 pm
by digipano
Does Tufuse have a GUI, or can I run EnfuseGUI & use tufuse instead?

PostPosted: Wed Mar 05, 2008 7:38 pm
by DrSlony
GURL I asked Max Lyons to answer that question as he would seem the best person to ask :]
digipano: TuFuse does not have a gui, but you can use EnfuseGUI and just set it up to use tufuse.exe in the options!

PostPosted: Wed Mar 05, 2008 8:00 pm
by digipano
DrSlony thanks,
I am downloading it now.

PostPosted: Wed Mar 05, 2008 8:28 pm
by digipano
after installing enfuseGUI It ask for enfuse.exe path, so how do I get Tufuse to work with it.
Do I rename the tufuse> as enfuse.exe?

PostPosted: Wed Mar 05, 2008 10:07 pm
by GURL
Well, as usual when Max Lyons is involved in software, the most obvious difference is the documentation ;)
Examples: http://www.tawbaware.com/ptasmblr_help_stack_example.htm
Parameters, etc: http://www.tawbaware.com/tufuse.htm

PostPosted: Thu Mar 06, 2008 2:08 pm
by maxlyons
Hello all...I'm the author of TuFuse.

I've updated TuFuse so that it can now be configured to create 8 or 16 bit images, and can also read JPEG and RAW files as input.

Some reasons to prefer TuFuse?

There are a number of differences. Here are some of the reasons I prefer TuFuse:

1. Two stage image fusion allows for both focus blending and exposure blending in one iteration. TuFuse figures out if the images you feed it ought to be focus blended or exposure blended or both. It picks the right settings for each/both operation and performs fusion as necessary. Enfuse must be configured to perform one or the other.

2. "Auto-bracketing". Feed Tufuse a single image and it can create a "pseudo HDR" file by creating lighter and/or darker versions of that file, and then fusing those lighter and/or darker versions with the original.

3. Support for RAW files. The latest version (version 1.30) can read RAW files if DCRaw is installed. This works nicely in combination with the "auto-bracketing" feature mentioned to create a converted file with as much dynamic range information as possible extracted from the RAW file.

4. More configurable. For example, TuFuse offers adjustment over the "sigma" parameter which configures how much/little weights is given to light/dark regions compared to mid-tones. In Enfuse, this is hardcoded.

5. I listed a few other differences here.


Does Tufuse have a GUI

Yes...it is called PTAssembler. PTAssembler can also align (see the "auto-align" feature) images before invoking TuFuse.

Max

PostPosted: Thu Mar 06, 2008 2:27 pm
by digipano
Ah well PTassembler is quite well known but I knew it as a panorama stitching tool & last I tried few years ago it did not have an panorama editor/preview window which I felt was needed for beginners so I can get some sense of which way I am going wrong & correct the parameters.

Can you or do you plan to make a seperate GUI so Tufuse can be used separately just for blending?

PostPosted: Thu Mar 06, 2008 2:57 pm
by maxlyons
PTassembler ... did not have an panorama editor/preview window

It does now!

Can you or do you plan to make a seperate GUI so Tufuse can be used separately just for blending?

I can, but I'm not sure if I will. If I was to make a seperate GUI, it would not include the image registration/alignment portion. That feature is non-trivial to build and is a core part of functionality in PTAssembler. It wouldn't make sense to me to reinvent that wheel by writing another program, when I've already written one explicity for that purpose!

Max

PostPosted: Thu Mar 06, 2008 3:57 pm
by [bo]
Whoooa, all this focus blending the exposure blending examples are amazing! I've always found PhotoMatix to produce overdone and unrealistic results and CombineZ - very complicated... If the workflow with TuFuse is so simple as described on the page linked above - that's great!

I wonder how APP fits in all this? The way I see it, it does not fit at all.

PostPosted: Thu Mar 06, 2008 4:05 pm
by DrSlony
Thank you for the answers!

I have a problem though. I tried hard but I couldnt get EnfuseGUI to run in wine (a linux program for running windows programs). I cannot really use tufuse as a photographer if I have to use command line, that just takes too much time on a 150 photo (5 brackets per fov) panorama, I need a gui.

I know running tufuse using the command line from wine works.
Do you know of any way that linux users can use tufuse from a gui?
Do you know if PTAssembler can run from wine?
Is compiling tufuse to run on linux planned at all, or just on a 'some day' basis? Because if you will do that soon, Im seriously considering writing a gui in python myself. I will have to learn it first of course, because so far the only pythoning I did was a simple batch script thing for gimp, but this would be a good investment, I think. I would like to make it opensource and all that, but then I'd have to spend more time learning about svn or git.

PostPosted: Thu Mar 06, 2008 5:38 pm
by mediavets
'[bo wrote:']I wonder how APP fits in all this? The way I see it, it does not fit at all.

I think anything to do with HDR/blending/bracketing is relevant to (some) APP users.

And I guess if Hugin incorporates Enfuse soon - which I understand it will - then something similar could be considered for APP? There's seems to be general agreement that APP's current in-built HDR-related features are inadequate. But then again there's the risk of feature bloat/overload - maybe the HDR stuff should remain a separate application?

Some APP users use Photomatix on bracketed image sets before stitching/rendering with APP - so any alternatives to this workflow must also be relevant.

Tufuse looks very competent but command line operation would not suit me.

This is the Open Talk topic - so surely everything need not immediately relate to APP usage?

PostPosted: Thu Mar 06, 2008 7:31 pm
by digipano
As of now using enfuseGUI & then using APP works perfectly fine & gives far better results than APP hdr method. May be we can have enfuse as a plugin within APP that would be best of the both worlds.

PostPosted: Thu Mar 06, 2008 8:31 pm
by [bo]
mediavets wrote:
'[bo wrote:']I wonder how APP fits in all this? The way I see it, it does not fit at all.

I think anything to do with HDR/blending/bracketing is relevant to (some) APP users.

A slight misunderstanding here. I did not imply this whole discussion is off-topic. Rather I pondered if you needed Autopano at all, using the process described on Max Lyons' site.

The way I see it, PTA+TuFuse produces much better looking HDR stuff than APP+PM, it does not require two separate purchases and it happens in a single interface. AND it allows amazing quality focus blending!

Alex is talking about some HDR 2.0 thing in a next-gen APP and I really hope it's better and simpler than what I see from Max Lyons. Right now HDR in APP is pretty much guesswork with those cryptic sliders and unexpected results. That's why I (and I suppose plenty of other users, judging by the comments on this forum) prefer to work Shoot->PhotoMatix->APP.

But that still leaves the focus blending out.

Maybe there is a way to use TuFuse as a plugin of some sort, I don't know. A few years back I was the main proponent of SmartBlend integration in APP and in the end - I got it. We got it :)! So nagging about TuFuse in APP could potentially help :D :rolleyes: !

PostPosted: Thu Mar 06, 2008 8:54 pm
by mediavets
'[bo wrote:'A slight misunderstanding here. I did not imply this whole discussion is off-topic. Rather I pondered if you needed Autopano at all, using the process described on Max Lyons' site.

The way I see it, PTA+TuFuse produces much better looking HDR stuff than APP+PM, it does not require two separate purchases and it happens in a single interface. AND it allows amazing quality focus blending!
.........
Maybe there is a way to use TuFuse as a plugin of some sort, I don't know. A few years back I was the main proponent of SmartBlend integration in APP and in the end - I got it. We got it :)! So nagging about TuFuse in APP could potentially help :D :rolleyes: !

Would I switch to PTA+TuFuse? No way! For me APP beats all the others in so many (other) ways. And I guess the majority of APP users don't and won't ever be doing AEB and blending/HDR. But...wouldn't it be great if it was possible to have something like TuFuse's apparent capabilities integrated into APP.

I am grateful you managed to get Smartblend integrated I'd probably not have chosen APP without it.

PostPosted: Fri Mar 07, 2008 8:54 am
by AlexandreJ
Yes, I agree with [bo], tufuse ( or enfuse based product ) is easier to use than any full HDR workflow ( ldr to hdr then tonemapping, photomatix, fdrtool, picturenaut, etc ).
I always found that HDR has 2 big issues :
- not easy at all,
- ghost.
First problem is now solved with enfuse system. The big remaining issue is still ghost removal. I fear that this one will stay a long time in the area.

PostPosted: Fri Mar 07, 2008 9:07 am
by AlexandreJ
About HDR2.0 ( I like the name ). It's been 6 months now we are working on this new system and we already have good result. Comparing to enfuse : it's also really easy to use but you have far more control on the result. There's still the problem with ghost that we are studying. Standard HDR ghost removal approach are not good at all ( example in photomatix, it's doesn't work as well as a smartblend technology ). So we are investigating to solve this.

PostPosted: Fri Mar 07, 2008 9:18 am
by digipano
Why not avoid HDR completely in favour of Tufuse/enfuse which does not require HDR processing, thus saving time & the processing is quite faster, so an integrated LDR enfusing method for panoramic creation will avoid the HDR/smartblend slow processing.

2nd problem with enfuse is also solved I don't see any ghosting at all with enfused files.