hdr workflow test  

All things panoramic.
An open forum for every topic on photography, panoramas, IT or related.
User avatar
DrSlony
Moderator
 
Topic author
Posts: 1893
Likes: 0 post
Liked in: 0 post
Joined: Sat Nov 03, 2007 6:30 pm
Location: Sweden
Info

by DrSlony » Tue Mar 18, 2008 8:35 am

THIS SCRIPT IS OUTDATED. See this link for a current version:
http://www.autopano.net/wiki/action/view/Bashfuse



This script below is kept here for archival and educational purposes, the script linked to above is recommended.

note: we are trying to get the script to be able to process photos in any dir, this script is temporary only, check back soon for an update

HOWTO tufuse.exe in Linux
v0.1

This will show you how to get tufuse.exe running in wine. You will be able to select a number of input images and have tufuse.exe process them, saving a 16bit tiff file (irrelevant of input bit depth) using the "blend across 180° border" setting. You will not be able to specify any parameters without editing the script. Functionality for this can be added, but its not worth the hassle, I hope soon to have a REAL gui. The photos you want to process must be in the same dir as tufuse.exe!

What you need:
tufuse.exe - tested on version 1.34
wine - for runing tufuse.exe. Tested on version 0.9.57
gqview - an image viewer. Tested on version 2.1.5
bash - for running the script. Tested on version 3.2_p17-r1

1- Download tufuse, extract the file tufuse.exe to /home/user/

2- In a text editor, create a file called tufuse in /home/user/ and paste the following into it:
Code: Select all
#!/bin/bash
set -x
wine tufuse.exe -o "${1%.*}_tufused.tif" -a 16 --compression DEFLATE -w "${@##*/}"

3- Make the file executable.
Code: Select all
chmod a+rwxrwxrwx /home/user/tufuse

4- Open gqview. Go to Edit > Preferences > Editors, in a blank cell in the Menu Name column write the name of this action, eg. tufuse. In the corresponding blank cell in the Command Line column, paste this:
Code: Select all
%v/home/user/tufuse %f

You're ready. To tufuse some files, select them in gqview, right click on them, "Edit" > "in tufuse...". A window will pop up.
If you do not want it to pop up, remove the "%v" part in step 4, but we want it to pop up to give use info in case something goes wrong. The first line in the window shows the exact command that was run. "set -x" in our script (step 2) shows it.
After a short while, tufuse.exe will load, give us many errors, but hopefully none critical. I get plenty of this, "fixme:msvcrt:MSVCRT__sopen : pmode 0x01b6 ignored". After a successful run, the tufused output file will be created in the same directory as the input files. The name of the tufused file will be the name of the first of the input files with a "_tufused" suffix, so if your input files are 0001.tif 0002.tif and 0003.tif, the resulting file will be called 0001_tufused.tif.

PROBLEMS:
Behaviour is unpredictable. Here are some of the tests I did, in the same order:
Tried to tufuse _IGP1370.tif _IGP1371.tif _IGP1372.tif in /home/drslony/temp/test/, failed.
Renamed them to 0001.tif 0002.tif 0003.tif, failed.
Copied 0001.tif 0002.tif 0003.tif to /home/drslony/, success.
Tried the 000x ones in /home/drslony/temp/test again, success.
Tried the _IGP137x ones in /home/drslony/temp/test again, success.
I'm pretty sure I didn't change anything between these tests, I even tried pasting the command from the popup window into a shell and running it manually, it also failed. Why did it suddenly start working for files not in the same dir as tufuse and tufuse.exe? I don't know. Maybe I messed up. Note that it is 08h23 now (morning) and I've been up all night, that could have been part of the problem :] Absentmindedness.

If you find any repeatable problems, report.

LIMITATIONS:
You cannot tufuse two sets of files simultaneously with this script using two processors as you can with EnfuseGUI in Windows - there will be a conflict of temporary files and both runs will fail. You cannot select a whole dir for processing either.
The photos you want to process must be in the same dir as tufuse.exe!

ACKNOWLEDGEMENTS:
fma38 and prince_jammys (irc.freenode.net #bash) for the script development and idea.
Last edited by DrSlony on Sat Mar 29, 2008 4:04 pm, edited 1 time in total.

User avatar
fma38
Moderator
 
Posts: 5850
Likes: 2 posts
Liked in: 2 posts
Joined: Wed Dec 07, 2005 6:21 pm
Location: Grenoble, France
Info

by fma38 » Tue Mar 18, 2008 9:06 am

tufuse expects input files to be in the current dir; in my script, I first do a cd in the dir where images belong, and it works fine. Here is my version:

Code: Select all
#!/bin/sh

dir=`dirname $1`
base=`basename $1`
cd $dir

files=""
for file in $@; do
    files=$files" "`basename $file`
done

base_output=`echo $base | cut -d '.' -f 1`
num=1
output=$base_tufuse.tif
if [ -e $output ]; then
    output=$base_tufuse$num.tif
    while [ -e $output ]; do
        num=`echo $num+1 | bc`
        output=$base_tufuse$num.tif
    done
fi

wine /opt/tufuse.exe -o $output $files

The ouput file has the name of the first image (in fact, the first image selected in gqview).

BTW, do you know what '${1%.*}' and '${@##*/}' exactly do?
Frédéric

no avatar
GURL
Member
 
Posts: 2943
Likes: 0 post
Liked in: 0 post
Joined: Tue Dec 06, 2005 1:57 pm
Location: Grenoble
Info

by GURL » Tue Mar 18, 2008 3:56 pm

AlexandreJ wrote:To GURL. I would be interested in any results you made with images from the HDRI handbook.
The book results will serve as a background and we will be able to compare enfuse / tufuse to that.

Not from HDRI Handbook, but I sent you some clouds ( Nuages_APP_Enfuse_TuFuse.zip )
It's about ghosts: Smartblend is great for blending clouds but the *fuse brothers had a problem because clouds moved between one layer and the next one:


Georges

User avatar
DrSlony
Moderator
 
Topic author
Posts: 1893
Likes: 0 post
Liked in: 0 post
Joined: Sat Nov 03, 2007 6:30 pm
Location: Sweden
Info

by DrSlony » Tue Mar 18, 2008 4:56 pm

fma38 I started off using your first script, but I got errors when I tried tufusing files in "/home/drslony/pictures/20080316 flats/_IGPxxxx.tif" (tufuse.exe and tufuse were in /home/drslony), also got errors when I tried tufusing "/home/drslony/pictures/panoramas/20080316 flats 78 127.12x64.12 C 000x.tif", so I went to #bash for help, we started off by just making all output verbose and the script I pasted is what we arrived at. They arrived at, to be precise :]

I see that you now have modified the script, from what I understand it splits up the pathname into one variable and filename into the other, then uses the pathname to save the output. Does it handle deep dir structures and spaces in the names? From what I learned yesterday, shouldn't some of your variables be in " "?

I do now know what '${1%.*}' and '${@##*/}' does exactly, looks uber hacked to me :] But it worked for deep structures, I will test again soon after a nice long sleep and see.

Is there any reason code-wise why your version is better? I'd be glad to use it if it is :)

User avatar
fma38
Moderator
 
Posts: 5850
Likes: 2 posts
Liked in: 2 posts
Joined: Wed Dec 07, 2005 6:21 pm
Location: Grenoble, France
Info

by fma38 » Tue Mar 18, 2008 5:13 pm

Oh, I didn't say that my version is better! In fact, it does not handle deep dirs, neither spaces in file names (which is always a very bad idea... Most of unix tools does not like much spaces neither...).

I'll try to understand how the other version exactly works...
Frédéric

User avatar
DrSlony
Moderator
 
Topic author
Posts: 1893
Likes: 0 post
Liked in: 0 post
Joined: Sat Nov 03, 2007 6:30 pm
Location: Sweden
Info

by DrSlony » Fri Mar 21, 2008 7:46 pm

fma38 what is that extra stuff in your script?
You know what I see when i look at it? I see potential of making it read the whole dir and parse images into groups, send one group to tufuse.exe, wait for it to return no error, and then send the next! :D

User avatar
fma38
Moderator
 
Posts: 5850
Likes: 2 posts
Liked in: 2 posts
Joined: Wed Dec 07, 2005 6:21 pm
Location: Grenoble, France
Info

by fma38 » Fri Mar 21, 2008 7:58 pm

I think it should not be too hard, but I need to know how to group images... What are the criterions?
Frédéric

User avatar
DrSlony
Moderator
 
Topic author
Posts: 1893
Likes: 0 post
Liked in: 0 post
Joined: Sat Nov 03, 2007 6:30 pm
Location: Sweden
Info

by DrSlony » Fri Mar 21, 2008 8:37 pm

well, grouping them by number will do. I can run a batch rename before sending them to the scrip, so they will be from eg. 0000.whatever to 0120.whatever, and the script will create groups of 3 or 5, and send one group to tufuse.exe with whatever settings the users wants in the script, for example I use 16bit output. In DOS a program used to send an error code 0 if it completed its job successfully, so the script could wait for something like that and then send the next group. If detecting such an exit code is impossible, an ugly workaround would be to just wait xx seconds and then send the next one. For example lets say it takes 30 seconds to tufuse 3 10MP images, so it could wait 35 or 40 seconds to be sure.

So basically it must divide images in the folder into groups of 3 or 5 based on the incremental filenames, and use whatever method to feed tufuse.exe groups one by one (two at the same time cause tufuse.exe to fail because temp files use the same name. I suppose enfusegui somehow tells tufuse to make a separate temp directory for each group - enfusegui processes two sets simultaneously on my core 2 duo pc).

User avatar
fma38
Moderator
 
Posts: 5850
Likes: 2 posts
Liked in: 2 posts
Joined: Wed Dec 07, 2005 6:21 pm
Location: Grenoble, France
Info

by fma38 » Fri Mar 21, 2008 8:47 pm

I still don't understand what are the groups for... What is the common point of images (3 to 5) in the same group? Are they different expositions of the same shot (bracketing)? Are these images directlty from the camera? So, they are not perfeclty aligned, are they? How can tufuse blend them correctly?

If you could provide me an example (some pictures, defining groups, with the name of images in the same group)...
Frédéric

User avatar
DrSlony
Moderator
 
Topic author
Posts: 1893
Likes: 0 post
Liked in: 0 post
Joined: Sat Nov 03, 2007 6:30 pm
Location: Sweden
Info

by DrSlony » Fri Mar 21, 2008 9:42 pm

Yes they are bracketed and they are perfectly aligned. I use an infrared remote to fire the camera, and if the setup is long/high and therefore shakey, i also use a delay between the flipping of the mirror and the capturing of sensor data. Tufuse blends them perfectly, just blending eg. 150 shots manually is overkill.

The names of the images can be anything, the camera calls them _PFGxxxx or something like that, I can rename batch them to whatever.
The names of the groups dont matter, the groups dont have to be permanent, just temporary groups that the script will divide the photos into.

So: I have a dir with lets say 150 shots. They are bracketed. I can bracket -2 0 +2 or -4 -2 0 +2 +4, so each group should have either 3 or 5 images in it. How to check? Well thats not important, the user can set this manually. You can use xdialog to make a selection box, or the user can change that parameter in the script, or he can save two versions of the same script - one for brackets of 3 and one for brackets of 5.

The script sends a group of images to tufuse.exe
wine tufuse.exe -o 0001_tufused.tif -a 16 --compression DEFLATE -w 0001.tif 0002.tif 0003.tif

and then the next group would be
wine tufuse.exe -o 0002_tufused.tif -a 16 --compression DEFLATE -w 0004.tif 0005.tif 0006.tif

User avatar
fma38
Moderator
 
Posts: 5850
Likes: 2 posts
Liked in: 2 posts
Joined: Wed Dec 07, 2005 6:21 pm
Location: Grenoble, France
Info

by fma38 » Sat Mar 22, 2008 9:18 am

Ok, I understood, know! It should not be difficult... In fact, if your images are in the good order, then there is no need to rename them. You just have to select them in the good order, ie first selecting the first image in gqview, then selecting the last image, so they will be passed to the script in that order. Then, the script just has to split them by group (using the user param 1, 3, 5 or any other number), and launch tufuse on each group.

I'll try to make that this week-end...

About selection, it may be possible to sort the images...
Frédéric

User avatar
DrSlony
Moderator
 
Topic author
Posts: 1893
Likes: 0 post
Liked in: 0 post
Joined: Sat Nov 03, 2007 6:30 pm
Location: Sweden
Info

by DrSlony » Sat Mar 29, 2008 4:05 pm

You can now batch tufuse photos in linux!

See this wiki article:
http://www.autopano.net/wiki/action/view/Bashfuse

User avatar
fma38
Moderator
 
Posts: 5850
Likes: 2 posts
Liked in: 2 posts
Joined: Wed Dec 07, 2005 6:21 pm
Location: Grenoble, France
Info

by fma38 » Sat Mar 29, 2008 9:51 pm

:)
Frédéric

User avatar
DrSlony
Moderator
 
Topic author
Posts: 1893
Likes: 0 post
Liked in: 0 post
Joined: Sat Nov 03, 2007 6:30 pm
Location: Sweden
Info

by DrSlony » Tue Apr 01, 2008 10:03 pm

I wanted to test the following workflow:
1- feed APP bracketed photos (-2, 0, +2, 99 10MP RAW files), run customized fine tuner and optimize, render to layers, tufuse.exe
2- feed APP only the 0 photos (33 10MP RAW photos), customized fine tuner, optimize, render @16bpp, tufuse.exe
3- feed tufuse.exe 99 10MP RAW photos, save as TIFF16, feed them to APP, customized fine tuner, optimize, render


Results
Detection settings: high quality, 120CP/image pair, strong algorithm, adjust lens distortion
1- Detection and primary projection took 53 minutes!!! Since this obviously is way too long, I decided to see if it was worth it. I set the RH4 tonemapper, pushed brightness to the very left, then adjusted levels manually. Then I wanted to see the difference between this tonemapped version and the prior one. I opened undo, clicked two steps back, and APP crashed.
2- in progress
3- in progress

I might try 1 again, doubt it, but I might.

User avatar
DrSlony
Moderator
 
Topic author
Posts: 1893
Likes: 0 post
Liked in: 0 post
Joined: Sat Nov 03, 2007 6:30 pm
Location: Sweden
Info

by DrSlony » Thu Nov 27, 2008 7:36 pm

Update

Time has passed and things have changed, here are some things that are new and/or have changed.

Enfuse 3.2 is now complete and it supports both exposure and focus blending. I prefer it now over Tufuse because it works much faster (perhaps due to me being able to compile it on my 64bit system) and it allows me to process bigger images. For example I process three 15 000 x 7 500 px panoramas @ 16 bits per channel using Enfuse quickly and without any problems. I have 8GB RAM and I set the "m" parameter (controls how much RAM Enfuse can use) to 7000 (MB). What's more, Enfuse is open source, which not only in itself pleases me but it also gets more attention than the alternative program. I haven't tested Enfuse (nor EnfuseGUI) in Windows though, so someone else who has should advise other Windows users here, or if you are a Windows user you should try both EnfuseGUI and TufusePro.
To read more about using Enfuse in Linux on a batch of files, read Batch Enfuse for Linux (APP forum thread).

DrSlony wrote:4- batch hdr and tonemap in PM and save as tiff 16bit, stitch in APP
4- good for all panos big and small. Working with 16bit files is slow but not as slow as with radiance. I recommend this workflow for now. Once APP will offer better tonemapping quality and once I test TuFuse I will update my tests and report.

NOT good for all panos. Local fusion (and local HDR tonemapping) does not work well on photos which consist of a large plain area and a bright spot somewhere in it, such as a photo of the sky with the sun in it. Fusing such images will produce a washed out photo with bad exposure and with bad colors (in relation to others shots in the pano series).
See this thread: Why its better to tonemap (*fuse) AFTER stitching

Enfuse/Tufuse still do not have a tool for handling ghosts. Enfuse has the advantage of being open source and has more people working on it (in theory, I hope in practice too :] ) so we might see ghost handling in some future version. Or perhaps Hugin handles that already, not sure.

Of course all the benefits of local fusion still apply - no memory/pano size limit and no APP misalignments if your source images are well aligned, but I now use global fusion whenever I can because it produces more realistic results and handles the sun perfectly.

I am still looking for a way to produce HDR (not tonemapped) files from a batch of input files using open source software, and then for a good way (open source) to tonemap the resulting HDR panorama.

As of today, APP-1.4.2 still sometimes places smartblend blending borders in different areas on each of the bracketed layers, as described here, bracketed shots alignment trouble. It should of course plot the blending border in the exact same area on all layers. Hopefully APP-2 will do that with its notion of stacks.

Previous

Who is online

Users browsing this forum: No registered users and 2 guests