Please suggest settings for my hardware  

In the panorama field, hardware is also part of the success. You can discuss here about it: camera, computer, pano head, anything
no avatar
orland
Member
 
Posts: 12
Joined: Sun May 25, 2008 1:56 pm

Please suggest settings for my hardware

by orland » Sun May 25, 2008 2:17 pm

I have been using APP with good results on my vista64 machine which is a q6600 cpu with 8GB ram. Most of my panoramas are in the 100-200 images range so I thought I would try running APP on my server presuming it should run much faster. My server is a supermicro winxp64 with a dual xeon q5355 with 32GB ECC DDR2 667 ram, three raid arrays (OS is on a two-disk raptor raid 0, swap drive is on a two-disk raptor raid 0, storage drive is a four-disc raid 10 using four seagate 1TB ES2 drives) I'm using the latest 64 bit version of APP on both machines.

I ran my first render on the server and was very disappointed in the time. I set APP to use 8 cores at max memory which was 11GB and at hightest priority. I have the temp directory set up to use my swap array. (BTW, the raid arrays are all controlled by 3ware hardware controllers) I concluded that I must not be setting the program for best use. I've played with the settings all weekend with no real improvement. I then set up the exact same render on my vista64 single cpu machine and ran it at default settings against my server which is also now set at default settings. As of this writing my desktop machine looks to be running about 30% faster. Desktop machine shows all four cores in use at about 30% with a page file size of 2.4GB. Server shows all eight cores running 18% with a page file size of aroudn 700MB.

I must be doing something wrong. Does anyone have any suggestions on how to best configure APP to run on my server?

Thank you in advance for the help.

no avatar
Ath
Member
 
Posts: 36
Joined: Mon May 05, 2008 10:58 am
Location: Brunswick Heads, Australia

by Ath » Sun May 25, 2008 2:51 pm

Not sure... out of interest, how long does it take APP to detect and stitch the 100-200 images on each machine? I'm interested in building a workstation and want to know what kind of performance id get out of APP with it.

In Windows you can optimise the machine for applications or background services, you want to switch it to Applications I would think.. (processor scheduling), and also set memory usage to programs rather than system cache...

edit: if all else fails try grabbing a 64-bit Linux Live CD and put APP Linux 64-bit and images on a usb thumb drive or something, and configure its scratch/temp space for the ram drive the live CD creates
Last edited by Ath on Sun May 25, 2008 2:52 pm, edited 1 time in total.

no avatar
orland
Member
 
Posts: 12
Joined: Sun May 25, 2008 1:56 pm

by orland » Sun May 25, 2008 7:28 pm

I'm running a render test on both machine right now. It's 96 high res jpegs from my canon 1Ds Mark II, ~17 megapixel each. I'll post the complete specs of the render once it's done.

I would run a linux cd but I don't think I could get it to recognize my raid arrays. I might be mistaken as I'm not nearly as versed in Linux as I should be.

There is definitely something off in my configuration. My desktop machine is currently at least twice as far along as my server is. This makes no sense to me.

no avatar
orland
Member
 
Posts: 12
Joined: Sun May 25, 2008 1:56 pm

by orland » Sun May 25, 2008 9:07 pm

My desktop render is complete. 96 images took 6h40m and rendered to a final size of 37523x12390 pixels. I chose a psb file, 8bit, and it is 1.73GB in size. I used bicubic w/ smartblend. Memory was set to normal and 256MB, 4cpu, one instance, temp directory was on my local drive, quality of detection standard, 50 key points, optimization strong, adjust lens distortion, auto color and auto level.

In contrast my server looks to only be about 25% complete.

Thoughts anyone?

no avatar
Ath
Member
 
Posts: 36
Joined: Mon May 05, 2008 10:58 am
Location: Brunswick Heads, Australia

by Ath » Mon May 26, 2008 3:11 am

Dude I think my Pentium D is matching your desktop, I use Spline 64, with Smartblend, advanced distortion, adjust lens distortion, strong algorithm, for both detection and render, my time is only like an hour and a half for 68 photos with 1/3 overlap from my 30D.

And my mum's 2GHz Core 2 Duo laptop was even better at it...

Pentium D, 1GB RAM, XP SP2, 32-bits OS + App.

Although whenever I render out to an 8-bit file, I'll render it out to a JPEG on the maximum setting of 12, unless I want layers.

Linux should see your RAID if its a hardware RAID, besides, a Live Linux CD will turn your 32GB ram into a ram disk on your home folder, render straight to the ram disk as the temp folder :)

This way you can judge two things, if it's basically the same, APP isn't or cannot take full advantage of the hardware, if it isn't then it's your OS to blame, running it off a ram disk in a live linux cd will bypass hard drive performance too.


Also access to 32GB of ECC ram, is obviously going to be a bit slower than than it is in the desktop, but I do not think APP is being bottlenecked by the RAM, as it's certainly not pushing that much data through anywhere near it's maximum transfer rate (well it certainly doesn't sound like it to me).

no avatar
Ath
Member
 
Posts: 36
Joined: Mon May 05, 2008 10:58 am
Location: Brunswick Heads, Australia

by Ath » Mon May 26, 2008 3:13 am

orland wrote:My desktop render is complete. 96 images took 6h40m and rendered to a final size of 37523x12390 pixels. I chose a psb file, 8bit, and it is 1.73GB in size. I used bicubic w/ smartblend. Memory was set to normal and 256MB, 4cpu, one instance, temp directory was on my local drive, quality of detection standard, 50 key points, optimization strong, adjust lens distortion, auto color and auto level.

In contrast my server looks to only be about 25% complete.

Thoughts anyone?

Also try switching it on the batch render to 2nd highest priority, see if that helps.

no avatar
Ath
Member
 
Posts: 36
Joined: Mon May 05, 2008 10:58 am
Location: Brunswick Heads, Australia

by Ath » Mon May 26, 2008 1:14 pm

Dude, I just remembered, Vista and Server 2k8, are both superior (more efficient) at making usage of multiple processors/cores, and i noticed you have XP on your server, this -might- be an issue, in which case I still recommend trying a linux live cd/dvd.

no avatar
orland
Member
 
Posts: 12
Joined: Sun May 25, 2008 1:56 pm

by orland » Mon May 26, 2008 1:52 pm

Ath wrote:Dude I think my Pentium D is matching your desktop, I use Spline 64, with Smartblend, advanced distortion, adjust lens distortion, strong algorithm, for both detection and render, my time is only like an hour and a half for 68 photos with 1/3 overlap from my 30D.

That's exactly what's so discouraging. My old pentium 4 machine was nearly as fast as what I'm seeing now. I'm downloading the linux live cd and will post my results as soon as I can. Thanks for the help.

I really wonder if it's actually a bug in the 64bit version of APP. Every other software I have is blazing fast on my server.

no avatar
Ath
Member
 
Posts: 36
Joined: Mon May 05, 2008 10:58 am
Location: Brunswick Heads, Australia

by Ath » Mon May 26, 2008 5:30 pm

Im concerned about it, i want to build a similar machine, just for doing photographic work, which APP will be part of.

Let us know the results!

no avatar
orland
Member
 
Posts: 12
Joined: Sun May 25, 2008 1:56 pm

by orland » Mon May 26, 2008 6:36 pm

Tell me about it. I just purchased a US$10k server to run APP and it runs slower than on my desktop machine.

no avatar
Ath
Member
 
Posts: 36
Joined: Mon May 05, 2008 10:58 am
Location: Brunswick Heads, Australia

by Ath » Tue May 27, 2008 12:15 am

Let us know how you get on with the test

User avatar
[bo]
Member
 
Posts: 1226
Joined: Fri May 05, 2006 8:16 am
Location: Bulgaria

by [bo] » Tue May 27, 2008 10:02 am

Erm... something is wrong here...

My last desktop is a Q6600@3Ghz, 4GB ram and a single Raptor (not RAID). XP64, not Vista. Last week I did a 30-to-40 image pano (Canon 5D JPEGs), render to PSB, multiband, bicubic and it took very short time... less than one hour I think.

"Memory used during rendering" was set to "Normal".
Some of my panoramas, posted in the Autopano Pro flickr group.

no avatar
orland
Member
 
Posts: 12
Joined: Sun May 25, 2008 1:56 pm

by orland » Tue May 27, 2008 2:46 pm

'[bo wrote:']Erm... something is wrong here...

My last desktop is a Q6600@3Ghz, 4GB ram and a single Raptor (not RAID). XP64, not Vista. Last week I did a 30-to-40 image pano (Canon 5D JPEGs), render to PSB, multiband, bicubic and it took very short time... less than one hour I think.

"Memory used during rendering" was set to "Normal".

I just did another test on my desktop machine, single q6600 w/ 8GB ram running vista64. I took 12 of the 96 images I've been working with and APP made the pano in about four minutes. All 96 images took over 6.5 hours. Settings were four cores, normal priority, spline64 w/ smartblend, file output to 8bit psb file. This makes no sense to me. To make matters worse, I tested my server running xp64, vista64, and windows server 2003 enterprise. In all cases APP wasn't even at the halfway point when my desktop machine was finished rendering. All other software I have runs much faster on my server than on my desktop and my many benchmarks support this.

I just invested over $10k on my new server so I could speed up APP with my large panos and its beginning to look like a complete waste of money.

Has anyone been successful in getting any meaningful support from Kolor? I have yet to get a response to my inquiries.

no avatar
hankkarl
Member
 
Posts: 1284
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA

by hankkarl » Tue May 27, 2008 2:55 pm

Watch your AntiVirus settings.

I turn the run time stuff off for my APP temporary directory, and for any drives that live on the server (the server has its own AV).

So on my desktop, if I host the APP source and output files on the server, the desktop doen't do AV on the files. The server does, and there is some latency, but most of the work (AFIK) is done in the temporary directory.

Also, both desktop and server may be doing other things, even if nothing is connected to either. I couldn't guess which (server or desktop) has more cycle-stealers.

no avatar
orland
Member
 
Posts: 12
Joined: Sun May 25, 2008 1:56 pm

by orland » Tue May 27, 2008 8:38 pm

I might not have been clear when I say "server". I have a server that I'm using as a workstation in the hopes of having maximum performance with APP. Essentially it's a dual-xeon machine with 32GB ram and two raid 0 arrays made of raptors and running xp64. I've also tried vista64 and windows server 2003 enterprise. The issue I'm having is that APP is running much slower on this machine than on my normal desktop which is only a single q6600 and 8GB ram.

The reason I'm thinking its an APP problem is that my server benchmarks are what you would expect and all of my other software run much faster than on my standard desktop.

no avatar
Ath
Member
 
Posts: 36
Joined: Mon May 05, 2008 10:58 am
Location: Brunswick Heads, Australia

by Ath » Wed May 28, 2008 12:57 pm

how did the linux disc go?

no avatar
orland
Member
 
Posts: 12
Joined: Sun May 25, 2008 1:56 pm

by orland » Wed May 28, 2008 1:33 pm

I tried linux and couldn't get app to load the images. It kept telling me that cmyk photos were not supported. The photos are not in cmyk. I even converted the jpegs a second time to be absolutely certain and APP still thought they were four color for some reason.

no avatar
Ath
Member
 
Posts: 36
Joined: Mon May 05, 2008 10:58 am
Location: Brunswick Heads, Australia

by Ath » Wed May 28, 2008 1:55 pm

sounds like a bug, APP supports Canon raw files, if you shot in RAW, try dropping the RAW straight onto it.

User avatar
AlexandreJ
Kolor Team
 
Posts: 5918
Joined: Mon Nov 14, 2005 4:56 pm
Location: Francin, France

by AlexandreJ » Thu May 29, 2008 9:31 am

There's definitively something wrong in the setup (or in the software), not in the hardware which seems really up to date.
---
Apparently, you are comparing the rendering time more than the detection, right ?
* during detection, the robust algorithm can be a time eater on big panorama.
* during rendering, it's disk IO that is the most important factor.
So what I would suggest you is first to compare the rendering alon. I'm pretty sure you did that but just want to be sure : open the .pano and render ( it's no use to time the detection process yet ).
About IO :
- make sure you have the write cache enabled on the temporary folder drive.
- with the amount of memory you have, you don't need to put input images on a fast drive, they will be read once and put in memory by our cache system.

Make this test :
- render in multiband without layer outputing any format : this is a case where the drive are not used at all.
- render in smartblend without layer outputing : this is a case where the drive is used a lot.
You should expect a 2x up to 4x slow down. Not more. If it's more, it's a disk problem.

Don't hesitate to get back to me directly if you need more advice.

no avatar
orland
Member
 
Posts: 12
Joined: Sun May 25, 2008 1:56 pm

by orland » Thu May 29, 2008 11:41 pm

I very much appreciate the advice. The render time is my concern. Detection time seems fine. I'll perform the render test tonight and post my results. Thanks again.

no avatar
Ath
Member
 
Posts: 36
Joined: Mon May 05, 2008 10:58 am
Location: Brunswick Heads, Australia

by Ath » Tue Jun 03, 2008 1:06 pm

orland: I just got a Nodal Ninja 5L, and even without the Nodal Point found correctly on it with my 30D and Sigma 12-24mm, even with max settings in AutoPano Pro, I found a huge speed increase in APP.

And I haven't even started using the detent click stops yet.

no avatar
orland
Member
 
Posts: 12
Joined: Sun May 25, 2008 1:56 pm

by orland » Fri Jun 06, 2008 2:55 am

Sadly I already have a Really Right Stuff pano rig already and have taken all my recent photos with meticulous precision. I have run many tests putting my server/workstation against my desktop machine and I just can't get APP to speed up with the dual quad-core xeon rig.

There is definitely a bottle neck somewhere.

Unless one already exists, I propose that one of us post a set of perhaps 50 photos so that we can all download and render the exact same jpegs to compare times.

no avatar
Ath
Member
 
Posts: 36
Joined: Mon May 05, 2008 10:58 am
Location: Brunswick Heads, Australia

by Ath » Fri Jun 06, 2008 11:08 am

I've got a set from my 30D/Sigma 12-24mm and NN5L i recently did I'll see if i can zip up the jpegs converted from RAW files, they're all bracketed.

The way I use "HDR" in AutoPano Pro is I set HDR colour correction, with multiband blending.

Settings:
Quality of Detection: Normal.
Optimization: Strong Algorithm (yes), Adjust lens distortion (yes - since these were shot with the sig 12 @ 12mm, this can be set to no, as it basically has no distortion, but i have no probs leaving it on), advanced distortion (no).

Auto Level and Auto Colour Correction on (initially).

I'll convert the RAW since I do mine off the RAW as I shoot small jpegs, not full ones.

no avatar
orland
Member
 
Posts: 12
Joined: Sun May 25, 2008 1:56 pm

by orland » Fri Jun 06, 2008 10:57 pm

Wonderful. Let me know when you are set and I will happily render these on my server/workstation. It will be very interesting to see what happens.


Return to Hardware

Who is online

Users browsing this forum: No registered users and 2 guests