should a GPU , RAM and solid stateHDto speed up giga panorama process?

[tived]

by tived » Tue Jan 10, 2012 8:05 am

I can tell I need to upgrade my SSD's and HDD's :-) ... waiting for the prices to drop after the flooding in Thailand

Leif is that just 4 SSD's and wich ones are they?

Henrik

PS: Looking at 8 drives to fill the controller... but also condemplating a 16 or 24 channel controller - currently have two 8 channel controller installed

[HansKeesom]

by HansKeesom » Tue Jan 10, 2012 10:32 am

Well, bad news, the SSD now shows up in BIOS but not in windows anymore. As it is the OCZ120 vertex II which is rather critised online, I am sceptic about whether it will get to work again. O well, should have read the forums a bit better and turn to a better brand.
In general, I would not store the only copy of photos on a SSD and use SSD only for fast processing.

[leifs]

by leifs » Tue Jan 10, 2012 4:08 pm

I can tell I need to upgrade my SSD's and HDD's :-) ... waiting for the prices to drop after the flooding in Thailand
Leif is that just 4 SSD's and wich ones are they?

Henrik

This is a "small" scratch/temp-disk made out of 8 old SSDs 32 GB each, RAID 0 on a LSI raid-controller.

leifs

Last edited by leifs on Tue Jan 10, 2012 4:10 pm, edited 1 time in total.

[tived]

by tived » Wed Jan 11, 2012 7:56 am

Thanks / Tak Leif,
It's not always the size but the number of disks and quality of controller.
The two I have r IBM 1015 aka LSI 9240/9211 just very basic controllers

[tived]

by tived » Wed Jan 11, 2012 8:01 am

Leif,
With the new gen 3 SSD @ 550 mb/s one should be able to reach
Close to 3+ GB/s with 8 disks

Now having said that small disks are ok, is not entirely true if one makes very large pano, at least
this is my experience. I am aiming for 8x 120gb which should give the best speed vs cost
Henrik

[leifs]

by leifs » Wed Jan 11, 2012 11:34 am

I had a 3ware controller until recently, but it didn't cope with SSD's.
So I swapped in a LSI 9280. It can take 6GB/s disks and it is made to handle SSD's
The speed from my 8 old SSD's more than doubled with the LSI controller :-)

leifs

[HansKeesom]

by HansKeesom » Wed Jan 11, 2012 8:54 pm

So guys, should I disconnect the (two) ssd I have from the mainboard and connect them to a nice 6 gb/s controller. Will that bring much increase in speed to the standard sata controllers on the board. Is it enough to have a two port controller as my i7 920 was already having trouble keeping up with the ocz vertex 2 120& GB

[leifs]

by leifs » Wed Jan 11, 2012 9:10 pm

Maybe you don't need a controller at all. The next level of speed will come from devices like the Revodrive, where the drive goes right into the bus without the SATA bottlenecks. Have a look at http://www.tomshardware.com/news/OCZ-RevoDrive3-max-iops-PCIe-SSD,13919.html
You can't boot from it, but it would be a nice scratch-disk

leifs

[HansKeesom]

by HansKeesom » Wed Jan 11, 2012 9:19 pm

240 gb version seems like great price compared with capacity. For scratch large enough, copy of all sourcefiles would even fit in my case

[tived]

by tived » Thu Jan 12, 2012 5:55 am

check for compatability with the revo drives as they are a bit flaky with some boards

Also be aware that some onboard Sata III controllers are not all what they are cracked up to be!!

Leif, that is a very nice RAID controller ;-) very nice:-)

Henrik

[dhoomdon]

by dhoomdon » Sat Jan 14, 2012 12:35 pm

the GPU will only speed up the previews of the panos, not the rendering. Having the temp drive on a SSD is great if you are shooting raw images and for blending images larger than your RAM. The RAM upgrade will help the most.
[url=motorola xpr 455]motorola xpr 4550[/url]

[HansKeesom]

by HansKeesom » Sat Jan 14, 2012 4:09 pm

Yes during the whole discussion here the consensus is to buy RAM first. that will give autopano lots of breathing room.
If you installed max rampossible and you notice the pafefile/swapfile is still very active, buy a ssd for it.
If you then find reading sourcefiles is slowing things down, you might want to copy them first to another ssd.

Of course ,wherever iwrite ssd you can read alternatively a raid-0 config with harddisk or ssd(preferable)

[HansKeesom]

by HansKeesom » Mon Feb 27, 2012 3:47 pm

Got my bigger ssd running again. Great to have a system that is no longer making much noise anymore. But it is sad to notice autopano running just 13% of cpu problably because the programm was written this way ( during colorequialisation)
Perfmon is showing diskactivity in kB/sec.

Groups are waiting to be detected, only 5 out of 16 gb ram is used.....kind of sad to see this.

I think I have done what I can to upgrade my system, time for Kolor to upgrade their programm?

Last edited by HansKeesom on Mon Feb 27, 2012 5:53 pm, edited 1 time in total.

[tived]

by tived » Mon Feb 27, 2012 11:22 pm

Hans,

you are just not pushing your system enough :-) i can max out my 48gb of ram which i will push to 96gb next month. then in the coming months replace my current 2nd gen SSD raid with 3rd gen but add more disks 6-8 in each array for 2+ gb/sec throughput (currently its 3-500mb/sec)
What am i expecting to happen? i will be able to do more editing without waiting on the computer, less chance of running out of fast temp disk space 200gb vs 1Tb, plus it will be 4x faster to off load temp data from disk.

it also means that i will be able to process and render panos, while being able to also juggle large pano's in photoshop.

but it does come at a cost

Henrik

ps: i m happy for you that its working so well

[HansKeesom]

by HansKeesom » Tue Feb 28, 2012 12:15 am

Well you surely win when it cames to the specs. I will have to do with 16 gb for the moment which is supposed to be the max of my motherboard at thenmoment. Wonder if I can slip in some more GB's in the four slots I have. But it is not really that I do large pano's that need so much. 45 images is normally the maximum.
To test the new setup I am running a job I got from someone else. But the first group of 900 photos already seems to lock the system or at least slow progress by using only 12% of cpu capacity and 40 % of memory. no hefty harddisk i/o so no bottleneck there. i will let it run through the night and see what it comes up with in the morning!

So no need for

[tived]

by tived » Tue Feb 28, 2012 1:44 am

Hi Hans

When you say "lock the system", are you not able to do much else on it while this is taking place? are you looking at the Resource monitor in Task Manager (Performance tab)
How big in total is the 900 images? and are you only having SSD's in your system.
Could you write up your specs, please.

You basically need to look at how the data is travelling through the system, as in from device to device...eg. from HDD to Memory to CPU to memory, temp storage (disk), memory to CPU to memory (this can loop through till the data set has been completed) then it goes off to your rendering (CPU) and saving (back to disk).

As you know the CPU is the fastest device in your machine, pumping at the highest frequency, followed by your memory and then its the disks be it HDD's or SDDs are the slowest components. The frequency is translated into bandwidth which is limited by the PCIe 2.0 x16 to 8Gb/s if memory serves me right, now your onboard devices are most likely capped at max PCIe x8 so 4GB/s which I think you need something like 12-16 of the fastest SSD's to saturate, maybe even 24 SSD's so you are a fair way away from this point. (It takes about 8x SSD 2nd Gen to reach close to 2Gb/s).
I have no idea of how many HDD's it will take, plus latancy delays, to reach these numbers. (Currently 4x WD Black Caviar Sata-III will give me 3-500 mb/s in RAID-0 on a LSI 9240 controller PCIe x8).

I honestly think we are a fair way off to reach the softwares limitation, but I will try :-) to find it.

thanks

Henrik

[HansKeesom]

by HansKeesom » Tue Feb 28, 2012 5:03 pm

The term "lock the system"was used wrongly by me. It is autopano giga seems to lock. It runs at 12/13% and show no progress. Only after many hours it does. There is no data travelling through the system much, as is show by perfmon.exe. So my conclusion, as done before, is that some parts of autopano just run on one core only. It basicly is a waiste of 87% of my cpu power without a clear bottleneck to point at...

I can use the system for other tasks without problem, but I rather would have it run 7-8 as fast and be unusable for other tasks.

[lumelix]

by lumelix » Tue Feb 28, 2012 11:50 pm

Hello Hans
Have You check your system with sysinternals suite (processexplorer, processmonitor, filemonitor) when it's in such a low load while apg should work ?
With this tools you can look into every process and task and what it is doing (or not).

[Ronald]

by Ronald » Wed Feb 29, 2012 12:29 am

Hans,

I did notice the performance situation a while ago http://www.kolor.com/forum/t13878-app2.6.0-xw7-64-local-optimization-cpu-performance and that's why I decide to put in hold the HDD upgrades at least for some time, because in my opinion for the most intense part of APP/APG, the HDD performance is not relevant, so I can live waiting couple minutes extra during rendering with my current raid-0 sata configuration...

Ron

Last edited by Ronald on Wed Feb 29, 2012 12:33 am, edited 1 time in total.

[pavka]

by pavka » Sat Mar 17, 2012 7:16 pm

;-))

Last edited by pavka on Sat Mar 17, 2012 7:17 pm, edited 1 time in total.

[HansKeesom]

by HansKeesom » Sat Mar 17, 2012 8:13 pm

Looking nice Pavka, is autopano using all cores here?

[HansKeesom]

by HansKeesom » Sat Mar 17, 2012 8:46 pm

Hello Hans
Have You check your system with sysinternals suite (processexplorer, processmonitor, filemonitor) when it's in such a low load while apg should work ?
With this tools you can look into every process and task and what it is doing (or not).

Got the second ssd working and balanced the temp directories over them. Things are running good now, cpu above 80 most of the time. Have not tested with projects that caused problems.

[pavka]

by pavka » Sat Mar 17, 2012 9:36 pm

Yes, screenshot show batch rendering autopano only.

When prestiching (preview) on HIGH settings there is 100% cpu almost all the time.. all cores.

Anyway... still not fast enough :-)

Looking nice Pavka, is autopano using all cores here?

[HansKeesom]

by HansKeesom » Sat Mar 17, 2012 9:53 pm

Can you give us an indication of the cost of your system?

[pavka]

by pavka » Sat Mar 17, 2012 10:43 pm

Hans: 2x xeon cpu expensive (x5660 or higher) , MB is evga SR-2 , ram is standard, gpu standard /evga can supply easy 4 GPU with dual PSU/