Exposure fusion - problems with the preview  

Share your tips and tricks here or get help with any Autopano Pro / Giga problem!
No bug reports (of any kind) in this forum!
User avatar
DrSlony
Moderator
 
Topic author
Posts: 1892
Likes: 0 post
Liked in: 0 post
Joined: Sat Nov 03, 2007 6:30 pm
Location: Sweden
Info

Exposure fusion - problems with the preview

by DrSlony » Fri Jun 17, 2011 12:57 am

Hey!

I have an indoor panorama shot near a large window with strong sunlight pouring in through it - a high dynamic range scene.

Bracketed -2, 0, +2 EV.
16 bit TIFF source files, EXIF intact.
Three layers grouped by bracket, all bracket levels are identical (manual mode).
Auto color correction, optimize by layers.
Gamma 1.0.

I want to get a nice looking LDR output from these bracketed LDR inputs. That means that workflow B, example 1 applies to me, as described here: http://www.autopano.net/wiki-en/action/view/LDR_/_HDR_-_Workflow_case_B

Issues:
1- "Example 1" doesn't specify whether I should combine these bracketed shots into sets or not. I would assume I should, but they are not combined into sets in that screenshot. Does it matter? I combined them into sets in these screenshots.

2- The main issue: the preview I get when using Exposure Fusion is ugly and nothing like the final rendered image, see the screenshots below. Without a good preview, the rendered result is guesswork. What's wrong? Is this APG's fault, or am I the only one with this problem?

Using a stable Gentoo 64 bit system, 8GB RAM, GeForce GTX 285M using Nvidia drivers 270.41.06.

Code: Select all
OGL Version: 3.3.0 NVIDIA 270.41.06
Render: GeForce GTX 285M/PCI/SSE2
Vendor: NVIDIA Corporation
Shader Version: 3.30 NVIDIA via Cg compiler
MaxTextureSize: 8192
MaxTextureUnits: 4
MaxColorAttachement: 8
MaxRenderBufferSize: 8192
Extentions:
    GL_ARB_ES2_compatibility
    GL_ARB_blend_func_extended
    GL_ARB_color_buffer_float
    GL_ARB_compatibility
    GL_ARB_copy_buffer
    GL_ARB_depth_buffer_float
    GL_ARB_depth_clamp
    GL_ARB_depth_texture
    GL_ARB_draw_buffers
    GL_ARB_draw_elements_base_vertex
    GL_ARB_draw_instanced
    GL_ARB_explicit_attrib_location
    GL_ARB_fragment_coord_conventions
    GL_ARB_fragment_program
    GL_ARB_fragment_program_shadow
    GL_ARB_fragment_shader
    GL_ARB_framebuffer_object
    GL_ARB_framebuffer_sRGB
    GL_ARB_geometry_shader4
    GL_ARB_get_program_binary
    GL_ARB_half_float_pixel
    GL_ARB_half_float_vertex
    GL_ARB_imaging
    GL_ARB_instanced_arrays
    GL_ARB_map_buffer_range
    GL_ARB_multisample
    GL_ARB_multitexture
    GL_ARB_occlusion_query
    GL_ARB_occlusion_query2
    GL_ARB_pixel_buffer_object
    GL_ARB_point_parameters
    GL_ARB_point_sprite
    GL_ARB_provoking_vertex
    GL_ARB_robustness
    GL_ARB_sampler_objects
    GL_ARB_seamless_cube_map
    GL_ARB_separate_shader_objects
    GL_ARB_shader_bit_encoding
    GL_ARB_shader_objects
    GL_ARB_shading_language_100
    GL_ARB_shading_language_include
    GL_ARB_shadow
    GL_ARB_sync
    GL_ARB_texture_border_clamp
    GL_ARB_texture_buffer_object
    GL_ARB_texture_compression
    GL_ARB_texture_compression_rgtc
    GL_ARB_texture_cube_map
    GL_ARB_texture_env_add
    GL_ARB_texture_env_combine
    GL_ARB_texture_env_crossbar
    GL_ARB_texture_env_dot3
    GL_ARB_texture_float
    GL_ARB_texture_mirrored_repeat
    GL_ARB_texture_multisample
    GL_ARB_texture_non_power_of_two
    GL_ARB_texture_rectangle
    GL_ARB_texture_rg
    GL_ARB_texture_rgb10_a2ui
    GL_ARB_texture_swizzle
    GL_ARB_timer_query
    GL_ARB_transpose_matrix
    GL_ARB_uniform_buffer_object
    GL_ARB_vertex_array_bgra
    GL_ARB_vertex_array_object
    GL_ARB_vertex_buffer_object
    GL_ARB_vertex_program
    GL_ARB_vertex_shader
    GL_ARB_vertex_type_2_10_10_10_rev
    GL_ARB_viewport_array
    GL_ARB_window_pos
    GL_ATI_draw_buffers
    GL_ATI_texture_float
    GL_ATI_texture_mirror_once
    GL_EXTX_framebuffer_mixed_formats
    GL_EXT_Cg_shader
    GL_EXT_abgr
    GL_EXT_bgra
    GL_EXT_bindable_uniform
    GL_EXT_blend_color
    GL_EXT_blend_equation_separate
    GL_EXT_blend_func_separate
    GL_EXT_blend_minmax
    GL_EXT_blend_subtract
    GL_EXT_compiled_vertex_array
    GL_EXT_depth_bounds_test
    GL_EXT_direct_state_access
    GL_EXT_draw_buffers2
    GL_EXT_draw_instanced
    GL_EXT_draw_range_elements
    GL_EXT_fog_coord
    GL_EXT_framebuffer_blit
    GL_EXT_framebuffer_multisample
    GL_EXT_framebuffer_object
    GL_EXT_framebuffer_sRGB
    GL_EXT_geometry_shader4
    GL_EXT_gpu_program_parameters
    GL_EXT_gpu_shader4
    GL_EXT_multi_draw_arrays
    GL_EXT_packed_depth_stencil
    GL_EXT_packed_float
    GL_EXT_packed_pixels
    GL_EXT_pixel_buffer_object
    GL_EXT_point_parameters
    GL_EXT_provoking_vertex
    GL_EXT_rescale_normal
    GL_EXT_secondary_color
    GL_EXT_separate_shader_objects
    GL_EXT_separate_specular_color
    GL_EXT_shadow_funcs
    GL_EXT_stencil_two_side
    GL_EXT_stencil_wrap
    GL_EXT_texture3D
    GL_EXT_texture_array
    GL_EXT_texture_buffer_object
    GL_EXT_texture_compression_dxt1
    GL_EXT_texture_compression_latc
    GL_EXT_texture_compression_rgtc
    GL_EXT_texture_compression_s3tc
    GL_EXT_texture_cube_map
    GL_EXT_texture_edge_clamp
    GL_EXT_texture_env_add
    GL_EXT_texture_env_combine
    GL_EXT_texture_env_dot3
    GL_EXT_texture_filter_anisotropic
    GL_EXT_texture_format_BGRA8888
    GL_EXT_texture_integer
    GL_EXT_texture_lod
    GL_EXT_texture_lod_bias
    GL_EXT_texture_mirror_clamp
    GL_EXT_texture_object
    GL_EXT_texture_sRGB
    GL_EXT_texture_shared_exponent
    GL_EXT_texture_swizzle
    GL_EXT_texture_type_2_10_10_10_REV
    GL_EXT_timer_query
    GL_EXT_vertex_array
    GL_EXT_vertex_array_bgra
    GL_IBM_rasterpos_clip
    GL_IBM_texture_mirrored_repeat
    GL_KTX_buffer_region
    GL_NVX_conditional_render
    GL_NVX_gpu_memory_info
    GL_NV_alpha_test
    GL_NV_blend_minmax
    GL_NV_blend_square
    GL_NV_complex_primitives
    GL_NV_conditional_render
    GL_NV_copy_depth_to_color
    GL_NV_copy_image
    GL_NV_depth_buffer_float
    GL_NV_depth_clamp
    GL_NV_explicit_multisample
    GL_NV_fbo_color_attachments
    GL_NV_fence
    GL_NV_float_buffer
    GL_NV_fog_distance
    GL_NV_fragdepth
    GL_NV_fragment_program
    GL_NV_fragment_program2
    GL_NV_fragment_program_option
    GL_NV_framebuffer_multisample_coverage
    GL_NV_geometry_shader4
    GL_NV_gpu_program4
    GL_NV_half_float
    GL_NV_light_max_exponent
    GL_NV_multisample_coverage
    GL_NV_multisample_filter_hint
    GL_NV_occlusion_query
    GL_NV_packed_depth_stencil
    GL_NV_parameter_buffer_object
    GL_NV_parameter_buffer_object2
    GL_NV_pixel_data_range
    GL_NV_point_sprite
    GL_NV_primitive_restart
    GL_NV_register_combiners
    GL_NV_register_combiners2
    GL_NV_shader_buffer_load
    GL_NV_texgen_reflection
    GL_NV_texture_barrier
    GL_NV_texture_compression_vtc
    GL_NV_texture_env_combine4
    GL_NV_texture_expand_normal
    GL_NV_texture_lod_clamp
    GL_NV_texture_multisample
    GL_NV_texture_rectangle
    GL_NV_texture_shader
    GL_NV_texture_shader2
    GL_NV_texture_shader3
    GL_NV_transform_feedback
    GL_NV_vdpau_interop
    GL_NV_vertex_array_range
    GL_NV_vertex_array_range2
    GL_NV_vertex_buffer_unified_memory
    GL_NV_vertex_program
    GL_NV_vertex_program1_1
    GL_NV_vertex_program2
    GL_NV_vertex_program2_option
    GL_NV_vertex_program3
    GL_OES_depth24
    GL_OES_depth32
    GL_OES_depth_texture
    GL_OES_element_index_uint
    GL_OES_fbo_render_mipmap
    GL_OES_get_program_binary
    GL_OES_mapbuffer
    GL_OES_packed_depth_stencil
    GL_OES_rgb8_rgba8
    GL_OES_standard_derivatives
    GL_OES_texture_3D
    GL_OES_texture_float
    GL_OES_texture_float_linear
    GL_OES_texture_half_float
    GL_OES_texture_half_float_linear
    GL_OES_texture_npot
    GL_OES_vertex_array_object
    GL_OES_vertex_half_float
    GL_S3_s3tc
    GL_SGIS_generate_mipmap
    GL_SGIS_texture_lod
    GL_SGIX_depth_texture
    GL_SGIX_shadow
    GL_SUN_slice_accum













no avatar
GURL
Member
 
Posts: 2943
Likes: 0 post
Liked in: 0 post
Joined: Tue Dec 06, 2005 1:57 pm
Location: Grenoble
Info

by GURL » Fri Jun 17, 2011 10:05 am

Hello DrSlony
DrSlony wrote:Bracketed -2, 0, +2 EV.
16 bit TIFF source files, EXIF intact.
Three layers grouped by bracket, all bracket levels are identical (manual mode).
Auto color correction, optimize by layers.
Gamma 1.0.

As you used manual mode you should use no color correction (NONE) rather than LDR correction (AUTO) but as you selected the Optimize by layers option this should not cause problems provided yellow anchors are well located and selected anchor type is the right one (whether it should be Exposure, Gamma or Gamma and Exposure is not clear for me...)

DrSlony wrote:Issue 1- "Example 1" doesn't specify whether I should combine these bracketed shots into sets or not. I would assume I should, but they are not combined into sets in that screenshot. Does it matter? I combined them into sets in these screenshots.

I will suppose "set" means "stack".

Stacks are needed for detection and optimization.
Layers are needed for fusion.
Using stacks causes Autopano to automatically create layers and this is helpful for fusion but (though this is not recommended) for fusion one could create empty layers and manually place source images using drag and drop.

DrSlony wrote:Issue 2- The main issue: the preview I get when using Exposure Fusion is ugly and nothing like the final rendered image, see the screenshots below. Without a good preview, the rendered result is guesswork. What's wrong? Is this APG's fault, or am I the only one with this problem?

There are many conditions for a correct preview:
- correct settings in Anchors tool
- selecting Enable and Preview for Exposure Fusion Settings tool
- selecting Preview in Levels tool

;) don't forget to select Exposure Fusion in the Render window (... I just made a fusion test and forgot this one.)
Last edited by GURL on Fri Jun 17, 2011 10:32 am, edited 1 time in total.
Georges

User avatar
klausesser
Member
 
Posts: 8830
Likes: 5 posts
Liked in: 63 posts
Joined: Mon May 22, 2006 12:18 am
Location: Duesseldorf, Germany
Info

by klausesser » Fri Jun 17, 2011 1:50 pm

Honestly: i also didn´t dig it still. Tried and tried but never had a result matching my expectations.

best, Klaus
Simplicity is the keynote of all true elegance. Coco Chanel

no avatar
hermer-blr
Member
 
Posts: 237
Likes: 0 post
Liked in: 0 post
Joined: Thu May 06, 2010 1:01 pm
Location: Near Paris - France
Info

by hermer-blr » Fri Jun 17, 2011 7:20 pm

I have already complained several times that the documentation is not complete. The documentation by workflow (case B in this case) was a very good idea from Kolor. Unfortunately, they have not done an accurate and complete work to describe the workflows....

And the result is.... lot of time wasted and frustration for the users !!!

Can I say that I don't care about version 2.6 or 3, if there is no complete and clear documentation to support the users. The developers should understand that there are times where they should stop developing and improving their software and better write comprehensive documentation.

Will Kolor understand that one day ??????????
Nikon D5100 (formerly Nikon D60) - Sigma 10-20 - 24 shots Panoramas in 3 raws
Windows 32 bits - APG
website htt://jmh.trp.free.fr

no avatar
hankkarl
Member
 
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Fri Jun 17, 2011 8:21 pm

I've always thought that we have to handle this case (bright daylight through the window) as two separate HDRed images: When I've tried it with PM, PM wants to keep the windows brighter than the interior (ie the area around the window.)

1. create an HDR for the interior, let the window get blown out. Color correct for Tungsten or whatever light is prevalent.
2. Create an HDR for the outside, let the interior go dark. Color correct for sunlight (or clouds, ...)

Merge the two by cut and paste, but perhaps exposure fusion would be better.

I supose you could go one better and have another HDR for fluorescent lights in addition to the tungsten ones. But that starts to get very difficult.

no avatar
GURL
Member
 
Posts: 2943
Likes: 0 post
Liked in: 0 post
Joined: Tue Dec 06, 2005 1:57 pm
Location: Grenoble
Info

by GURL » Fri Jun 17, 2011 9:43 pm

klausesser wrote:Honestly: i also didn´t dig it still. Tried and tried but never had a result matching my expectations.

I can get result which match my own expectations but ...
- I know this is possible so that when I'm not pleased I try again (I'm retired and see this as a pleasant hobby, :P)
- I believe the present interface as a too complex one. For example having to adjust anchors (exposue? - gamma? - both?), levels (black point matters, white point matters, gamma matters) and the Exposure Fusion Settings tool 3 sliders to get what you want needs some practice and is not really that easy to explain.

To test and learn fusion my recommendation would be to avoid stitching a real pano (!)
- use a single series of source image (shot using the very same camera orientation, not a pano, no stacks needed, no anchors needed )
- create empty layers and use drag&drop to place a single source image in each layer
- use color correction: NONE
- go to Exposure Fusion Settings
- first leave the Exposure target slider at its default position but place Dark enhancement and Bright enhancement to the far right.
- don't forget to select the Enable option
- move the three above sliders by small steps
- when what you see is improved adjust the Levels tool 3 sliders
- return to Exposure Fusion Settings, etc
- when what you see is next to what you want don't forget to select Exposure fusion in the Render settings (50% rendering is often enough to judge the result)

I insist:
- using LDR color option and associated anchors is making the whole process more tricky
- using the Levels tool to set the black point and the white point and avoid an improper global contrast is a necessary step in the process

In my opinion the learning curve is steep but the feature works. Adding an Auto button in the Exposure Fusion Settings tool to preset its 3 sliders plus black points and white points could help...
Last edited by GURL on Sat Jun 18, 2011 7:06 am, edited 1 time in total.
Georges

User avatar
DrSlony
Moderator
 
Topic author
Posts: 1892
Likes: 0 post
Liked in: 0 post
Joined: Sat Nov 03, 2007 6:30 pm
Location: Sweden
Info

by DrSlony » Fri Jun 17, 2011 10:46 pm

Thank you for the response everyone.

As you know I've been doing panos for a few years now, using Autopano as well as getting down and dirty with some raw tools like command-line enfuse. I've tested this Exposure Fusion feature in 2.5 quite a bit before posting this question. I'm very impressed with how APG's Exposure Fusion performs. Along with the appropriate multiband value setting (-5 to 0) you can get some very pleasing and natural looking results.

GURL yes, by set (bracketed set) I meant stack.
I used LDR color correction, one yellow anchor per layer. The chosen yellow anchor stack made no difference as far as this buggy preview issue is concerned.
All non-yellow anchors were gamma. I tried others as well, made no difference as far as this buggy preview issue is concerned.
The levels were unchanged (0,00 - 1.00), simply because the histogram was filled out nicely.
Gamma changes made no difference as far as this buggy preview issue is concerned.

The documentation has improved a lot, and it needs only minor additions to be complete, at least that's how I see it with my prior knowledge of Autopano gathered during the last four years. The situation might be less colorful for new users.

Hankkarl I'm not clear what you meant by "HDR" in "as two separate HDRed images". If you really mean HDR, I have a problem with that, as I can't find a decent tonemapping program that works in Linux. The last time I used Photomatix in Windows (and I used it quite a bit!) I wasn't very impressed. It somehow felt a little like stitching bracketed panoramas in Autopano-1.4, if you know what I mean, hehe :] APG-2.5 is phenomenally better, I had no trouble at all stitching this ~90 image bracketed pano; very good stitching quality, low system spec requirements - my only problem is the Exposure Fusion preview. It would be nice if there was a HDR/Tonemapping program that was cross-platform and as easy to use, stable and low memory consuming as Autopano 2.5 is. Even with 4GB RAM, I still ran into memory limitations with Photomatix very often, despite using that strip-by-strip mode, or whatever it was called. Speaking of HDR, I'd like to add that I use it when I want to add some magic into a photo/pano. Enfuse does an amazing job (fast, low mem usage, etc) at creating natural looking photos, but they don't have the magical quality that moderate tonemapping can render.

User avatar
AlexandreJ
Kolor Team
 
Posts: 5988
Likes: 7 posts
Liked in: 10 posts
Joined: Mon Nov 14, 2005 4:56 pm
Location: Francin, France
Info

by AlexandreJ » Mon Jun 20, 2011 12:17 pm

Just some notes
-----

Preview rendering does not use all algorithms of final rendering.

Some differences come from the antighost which is not applied in preview.
The main issue comes from the multiband blending which allows to reduce irregularity of exposure weighting (neighbors weights can be very different).
In GPU mode the multiband blending does not exist. So the preview can be noisy and different from final rendering.
In CPU mode the multiband is applied but this algorithm is dependent to pictures size. So the preview can be a little bit different.

So in GPU mode you can have a preview of the global illumination but with noisy blending.
In CPU mode the preview will be closer to final result but computing time is longer.

We hope that in the future, we will be able to give you a better estimation of final rendering, but know that this task is really hard.

no avatar
hankkarl
Member
 
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Mon Jun 20, 2011 9:56 pm

DrSlony wrote:Hankkarl I'm not clear what you meant by "HDR" in "as two separate HDRed images". If you really mean HDR, I have a problem with that, as I can't find a decent tonemapping program that works in Linux.

I meant take the bracketed shots, and make one image that shows the interior exposed correctly. This would be done by creating an HDR image, then tonemapping it to a TIFF Or you could use exposure fusion if you can get it to ignore the outside parts. Then take the same images, and make an image that shows the outside exposed correctly (you may have one image that shows the outside exposed correctly, and can skip the HDR/tonemapping step).

Imagine two histograms:

Interior: ........IIIIIIIIIIIIIIIII.......................................................................
Exterior .......................................................................EEEEEEE........

When you are looking out a window, there can be shadows directly under the window, so the darkest and lightest part of the pano can be quite close. You don't want to compress it all and have this:

Compressed: ..IIIII...........EEE...

Everytime I try HDR or Tonemapping, I think I get something like:

.IIIIIIIIEEEEEEE Where the dark parts of the exterior seem brigher than the bright parts of the interior.

By treating them as two separate images, then combining them, I think I get:

..IIIIIIIIIIIIIIIII..
..EEEEEEE..

And I can shift them a bit so that I get:

...IIIIIIIIIIIIIII........
......EEEEEEE....

Or I can stretch or compress either interior or exterior, correct the color temperature, etc. for each of the two parts.

no avatar
Nanard
Member
 
Posts: 145
Likes: 0 post
Liked in: 3 posts
Joined: Sun Jul 29, 2007 11:00 pm
Location: France
Info

by Nanard » Wed Jun 22, 2011 5:34 pm

AlexandreJ wrote:Just some notes
-----

Preview rendering does not use all algorithms of final rendering.

Some differences come from the antighost which is not applied in preview.
The main issue comes from the multiband blending which allows to reduce irregularity of exposure weighting (neighbors weights can be very different).
In GPU mode the multiband blending does not exist. So the preview can be noisy and different from final rendering.
In CPU mode the multiband is applied but this algorithm is dependent to pictures size. So the preview can be a little bit different.

So in GPU mode you can have a preview of the global illumination but with noisy blending.
In CPU mode the preview will be closer to final result but computing time is longer.

We hope that in the future, we will be able to give you a better estimation of final rendering, but know that this task is really hard.

Alex,
What you say here is quite important, and I hope it will be mentioned in the documentation.
Why not letting the user make a choice between 'quick preview' (as now) , and 'complete but slow preview' (with anti-ghost applied,....) ?

Bernard

User avatar
AlexandreJ
Kolor Team
 
Posts: 5988
Likes: 7 posts
Liked in: 10 posts
Joined: Mon Nov 14, 2005 4:56 pm
Location: Francin, France
Info

by AlexandreJ » Thu Jun 23, 2011 1:06 pm

Nanard wrote:Alex,
What you say here is quite important, and I hope it will be mentioned in the documentation.
Why not letting the user make a choice between 'quick preview' (as now) , and 'complete but slow preview' (with anti-ghost applied,....) ?
Bernard

Simple, because it's not done yet. We will work on this feature as well as anti-ghost preview in editor windows for v3.

no avatar
hankkarl
Member
 
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Fri Jun 24, 2011 4:28 pm

Nanard wrote:What you say here is quite important, and I hope it will be mentioned in the documentation.
Why not letting the user make a choice between 'quick preview' (as now) , and 'complete but slow preview' (with anti-ghost applied,....) ?

Bernard

I'd like this to be easily changed from the pano editor--I can see using a quick preview for stitching, and the complete preview for color correction.

no avatar
KreAture
Member
 
Posts: 188
Likes: 0 post
Liked in: 0 post
Joined: Sun Feb 04, 2007 4:22 pm
Location: Norway
Info

by KreAture » Sat Jul 16, 2011 9:28 pm

v3 ???
So you are saying this usability fix won't come untill another licence upgrade?


Who is online

Users browsing this forum: No registered users and 5 guests