CS4 RAM Management with Neutralhazer

[ame.mjs]

CS4 RAM Management with Neutralhazer

by ame.mjs » Mon Apr 30, 2012 9:49 am

Hi to all,

I tried Neutralhazer since a couple of day as CS4 plugin on Win 7 Ultimate 32 bits 4GB RAM running on a quad core kernel.
No problems with small pics (< 20 Mpix) but CS4/Neutralhazer crashes on bigger ones (I suspect RAM crash).
I tried rebooting, resetting CS4 Prefs, freeing max of RAM, changing CS4 memory allowance, changing virtual memory, even disabling Open GL (NVIDIA GeForce 8800 GTX)

==> with no better results, remains systematic crashes.

Does somebody know how Neutralhazer handling its RAM ?

It seems to me :

* When starting up the Neutralhazer takes +/- 3x as RAM as the size of the loaded picture to treat.
* During adjustments the plugin freed some (sometimes a lot) of memory.
* Immediately after launching the final treatment (by clicking “ok”), CS4/Neutralhazer rapidly crashes even when not exceeds the 2 GB of RAM standard 32 bit usage limit.

It seem also Neutralhazer does not use my virtual memory.

Thanks if you have some ideas or experiences.

Alain

Last edited by ame.mjs on Mon Apr 30, 2012 12:14 pm, edited 1 time in total.

[klausesser]

by klausesser » Mon Apr 30, 2012 12:11 pm

No problems with small pics (< 20 Mpix) but CS4/Neutralhazer crashes on bigger ones (I suspect RAM crash).

Hi!

What does it mean: "on bigger pics"? HOW big? I guess NH should be used *before* stitiching. On the
camera-size images - not on the rendered panorama.

best, Klaus

[AlexandreJ]

by AlexandreJ » Mon Apr 30, 2012 12:22 pm

We manage RAM inside neutralhazer in 2 different ways : on through Photoshop, and the rest through general memory manager.
Now, as it is really a huge mathematical algorithm, if you want to apply the effect on big images or panorama, you definitively need a 64bits OS and Photoshop.

It does work well even on gigapixels if you have the right computer configuration. It won't work on a modest machine with big images.

[ame.mjs]

by ame.mjs » Mon Apr 30, 2012 12:39 pm

Hi Klaus,

5616 x 3744 pics (21 Mpix) taken with my EOS 5D Mk II seems to be the treatable limit for NH on my system.
Sometimes NH works it out perfectly to the end but sometimes crashes.

Does anybody experienced NH with a lot of other plugins and filters installed (I remember some previous well known difficulties on Photoshop to handle with many plugins).

Regards,

Alain

[ame.mjs]

by ame.mjs » Mon Apr 30, 2012 12:49 pm

Hi Alexander,

Do you think a 64bit OS is really mandatory to run NH even for 20 Mpix images or some other parameters could be adjusted ?

Thanks,

Alain

We manage RAM inside neutralhazer in 2 different ways : on through Photoshop, and the rest through general memory manager.
Now, as it is really a huge mathematical algorithm, if you want to apply the effect on big images or panorama, you definitively need a 64bits OS and Photoshop.

It does work well even on gigapixels if you have the right computer configuration. It won't work on a modest machine with big images.

[AlexandreJ]

by AlexandreJ » Mon Apr 30, 2012 2:47 pm

For 20 MPix images, it should be mandatory. But you need to allocate carefully ressources on this setup.
Just for your information : a 20 MPix is 240 MB in RAM ( for quality we do every calculation in 32bits space, each pixel needs 12 bytes ). We need a 3x free space around that, meaning, you should have 720 MB free to treat this image.
So depending on photoshop undo level, cache size, other software launched, it could run or not depending on free RAM available.
One note : there is a real limit : you won't be able correct on a 32bits OS images bigger than 1 GB / 3 / 12 = 28 MPix. For that, you'll need 64bits OSes.

[ame.mjs]

by ame.mjs » Mon Apr 30, 2012 5:58 pm

Hi all,

Thanks AlexandreJ for this answer. it confirms my latest tests.
NH works sometimes on 21 Mpix images with carefully memory allocation.

This size will probably be the practical limit for NH on a 32 bit system knowing Photoshop and some other software also need a little bit of RAM.
My problem is solved, I know what to do, thanks to all for our input.

Alain.

For 20 MPix images, it should be mandatory. But you need to allocate carefully ressources on this setup.
Just for your information : a 20 MPix is 240 MB in RAM ( for quality we do every calculation in 32bits space, each pixel needs 12 bytes ). We need a 3x free space around that, meaning, you should have 720 MB free to treat this image.
So depending on photoshop undo level, cache size, other software launched, it could run or not depending on free RAM available.
One note : there is a real limit : you won't be able correct on a 32bits OS images bigger than 1 GB / 3 / 12 = 28 MPix. For that, you'll need 64bits OSes.

Last edited by ame.mjs on Mon Apr 30, 2012 5:59 pm, edited 1 time in total.

[PeterE.]

by PeterE. » Sun Jun 24, 2012 2:02 pm

We manage RAM inside neutralhazer in 2 different ways : on through Photoshop, and the rest through general memory manager.
Now, as it is really a huge mathematical algorithm, if you want to apply the effect on big images or panorama, you definitively need a 64bits OS and Photoshop.

It does work well even on gigapixels if you have the right computer configuration. It won't work on a modest machine with big images.

Sorry, it does not work with CS5 on 64bit Win7 on a 7i with 8 GB Ram for a 7200 x 28400 px scanned picture. Is "the right confguration" a mainframe??? I really like the kolor programmes, but NH is a bit of a dissappointment with this memory problems. Make it slow with big files, that's o.k., but please make it work! I can render really big files with autopano on my machine, even big files made of big files, please make NH work!!!
Kind regards, Peter E.

Last edited by PeterE. on Sun Jun 24, 2012 2:04 pm, edited 1 time in total.

[tived]

by tived » Mon Jun 25, 2012 3:06 am

Hi Peter,
when you say you can work with really big files on your machine, can you specify how big the images are in pixels.
btw, I thought they had stopped making mainframes, or is it just me that are not keeping up.
Also 8GB of ram is not sufficient today - its very 2005 - sorry, sell a photo and buy some more ram, and a couple of SSD disks and make your computer a bit more snappy ;-)

Henrik

We manage RAM inside neutralhazer in 2 different ways : on through Photoshop, and the rest through general memory manager.
Now, as it is really a huge mathematical algorithm, if you want to apply the effect on big images or panorama, you definitively need a 64bits OS and Photoshop.
It does work well even on gigapixels if you have the right computer configuration. It won't work on a modest machine with big images.

Sorry, it does not work with CS5 on 64bit Win7 on a 7i with 8 GB Ram for a 7200 x 28400 px scanned picture. Is "the right confguration" a mainframe??? I really like the kolor programmes, but NH is a bit of a dissappointment with this memory problems. Make it slow with big files, that's o.k., but please make it work! I can render really big files with autopano on my machine, even big files made of big files, please make NH work!!!
Kind regards, Peter E.

[PeterE.]

by PeterE. » Mon Jun 25, 2012 7:26 am

Hi Henrik,
a single image out of camera for me is usually 7200 x 28400 px 16 bit (as written above).
Take some of them and render a bigger picture...
Sorry, you are right, "mainframes" come out of use, today they are called "supercomputer"; in colloquial German, both are called "Grossrechner" (big calculator), and I use the word "mainframe" for all big calculators, because a "super-computer" would be something more than a machine - which hopefully will never come to existence and "maiframe" for me descibes very good what even "supercomputers" do...
But all this doesn't help for the problem mentioned above: NH doesn't work even with a single shot out of my camera.
The algorithm should be able to use virtual instead of real memory, that's the only way to work on big files for "standard users". The ability to calculate the picture is more important than the speed of the calculation.
Kind regards, Peter E.

[gavinfarrell]

by gavinfarrell » Fri Sep 07, 2012 6:40 pm

AlexandreJ, I am shocked these limitations are not advertised as many of your customers are like me, gigapixel photographers, who use AutoPano Giga to render very large images. I have a small Gigapixel image that is only 1.6 Gig at 29593 x 19508 I tried allocating more RAM to CS6 giving it a total of 15468MB and reducing history states to 1 (200TB of scratch disk) it did not work. I then reduced the image to only 700MB and the filter still did not work. I don't agree with your memory allocation description. The ONLY thing I have discovered that works (as posted elsewhere in this forum) is if you reduce the image to only 12427 pixels high. Then suddenly it works. This is a VERY REAL pixel height file restriction. I think you need to have your programmers take a closer look at this. Until you get this problem fixed please change the description on your product page and be honest/open with your customers about the product's very real limitation/drawbacks.

[gavinfarrell]

by gavinfarrell » Mon Sep 10, 2012 6:19 am

No problems with small pics (< 20 Mpix) but CS4/Neutralhazer crashes on bigger ones (I suspect RAM crash).

Hi!

What does it mean: "on bigger pics"? HOW big? I guess NH should be used *before* stitiching. On the
camera-size images - not on the rendered panorama.

best, Klaus

Hey Klaus,

NH analyzes the overall / final image to create a artificial depth map to make it's calculations so applying the filter to single images and then stitching the panorama later would not be using the filter correctly.

[klausesser]

by klausesser » Mon Sep 10, 2012 2:09 pm

No problems with small pics (< 20 Mpix) but CS4/Neutralhazer crashes on bigger ones (I suspect RAM crash).

Hi!

What does it mean: "on bigger pics"? HOW big? I guess NH should be used *before* stitiching. On the
camera-size images - not on the rendered panorama.

best, Klaus

Hey Klaus,

NH analyzes the overall / final image to create a artificial depth map to make it's calculations so applying the filter to single images and then stitching the panorama later would not be using the filter correctly.

Hi Gavin!

Indeed i experienced it as more effective using the filter on the input images inside APG. That´s my personal experience and so it is my personal opinion.

Another point is the size of the image to get filtered when it´s a very big one - as you experienced yourself recently . . - Photoshop needs VERY much RAM anyway for working fluently.
I sugst at least 16 GB RAM for handling gigapixels without having cake-and-tea-time after each command and also when saving a, let´s say, 9GB file . .

The worst case i had was selecting a sky using a soft lasso of 200px and running a Gauss blur . . took about 25 min. to show only the preview . . . and another 20 min to fine-render it.

In CS6 it´s a bit easier: you can keep on working while CS6 saves in the background - big step ahead in terms of workflow.

(i use a MacPro, 2x3GHz Xeons, 8 cores, 16GB RAM and 8TB of fast HDs inside).

So when it comes to gigapixel panos: yes, i definitely prefer to de-haze the images before stitching.

best, Klaus

[gavinfarrell]

by gavinfarrell » Mon Sep 10, 2012 5:30 pm

I tried the de-haz inside Auto-Pano GIGA is looked terrible! It applied the filter to each image individually in the image set on the left column of the interface. When it stitched panorama together it looked like checkerboard quilt - Uck - terrible! Does anyone know if there a way to apply the filter inside AutoPano-Giga on the right column after the image control point have been calculated and the preview image is built? Klaus with my system I can move around a 30Gig .PSB file in Photoshop CS 6 no problem. Saving takes about 20 minutes that the only slow part. I have CS6 running on a SSD and I dedicate 15GB of RAM to it, I also have a RAID which I can go up to 600TB of scratch disk if I needed. My machine is a little older now it's the 2009 2 x 2.26 GHz Quad-Core Intel Xeon. Everything is running at 64 bit.

[klausesser]

by klausesser » Mon Sep 10, 2012 6:07 pm

I tried the de-haz inside Auto-Pano GIGA is looked terrible! It applied the filter to each image individually in the image set on the left column of the interface. When it stitched panorama together it looked like checkerboard quilt - Uck - terrible! Does anyone know if there a way to apply the filter inside AutoPano-Giga on the right column after the image control point have been calculated and the preview image is built? Klaus with my system I can move around a 30Gig .PSB file in Photoshop CS 6 no problem. Saving takes about 20 minutes that the only slow part. I have CS6 running on a SSD and I dedicate 15GB of RAM to it, I also have a RAID which I can go up to 600TB of scratch disk if I needed. My machine is a little older now it's the 2009 2 x 2.26 GHz Quad-Core Intel Xeon. Everything is running at 64 bit.

Hi Gavin!

When i used the haze-fiter in APG i tried one image, find the settings and then selected all images to apply the settings.
Stitching them resulted in an evenly de-hazed stitch.

I´m trying to find it - didn´t use it often because i always take care not to do too hazy shots . .

best, Klaus

PS: here an example - squeezed to 50MPx from 700MPx:

http://360impressions.de/Dra_DeHazed.jpg

shot somewhat earlier:

http://gigapan.com/gigapans/97639

Both panos have around 120pics.

De-Hazing took place in APG before stitching.

Last edited by klausesser on Mon Sep 10, 2012 7:28 pm, edited 1 time in total.

[gavinfarrell]

by gavinfarrell » Tue Oct 02, 2012 6:12 pm

So because I refuse to take "NO" for an answer,,,,
Just added my Dogmountain Gigapixel Panorama to my site, top image here: http://gavinfarrell.com/#item=columbia-river-gorge
What's you might find interesting is that this image was basically unusable because it was such a hazy day, however I ran it through KOLOR's neutralhazer and Viola! MAGIC! One thing to note is that Neutralhazer can't handle .PSB images over 1.6 Gb unless you have 64GB of RAM and even then it doesn't give you much more wiggle room. I solved for this by cutting the entire 30GB file into overlapping 1.6GB pieces and processing them separately and then recombining the image. Here's the fullscreen link (works on iPad & iPhone) http://www.gigapan.com/mobile/iOS/1.0/?id=115624

[tived]

by tived » Wed Oct 03, 2012 12:58 am

That just goes to show there is always a way :-)
Thanks for sharing
Henrik