Specific question on building a gigapixel rendering machine  

In the panorama field, hardware is also part of the success. You can discuss here about it: camera, computer, pano head, anything
no avatar
Zurich.Gigapixel
Member
 
Posts: 36
Joined: Sun Jul 01, 2012 6:49 pm
Location: Zurich, Switzerland

Specific question on building a gigapixel rendering machine

by Zurich.Gigapixel » Sat Aug 25, 2012 3:30 pm

Hi

I'm building/upgrading a computer just to render nearly 10'000 Pictures, 36.2 MP each.

Some of you already know about our project other don't, but that's NOT the topic here.
Also I do NOT want to go into discussions if it will be possible to render it or not.

All I want to know and have some opinions on is this:

We're able to connect four harddrives to the system.
Some of them will be SSD others HDD.

Which system should be better for this project:
A) 3 SSD (128GB each) and a 3TB HDD
B) 2 SSD (128GB each) and a 3TB + 2TB HDD
?

btw: Consider the temporary space used should be nearly 1.5TB I think.

Second question:
Should the operating system and autopano be installed on a SSD or HDD? Does it make a difference? Or would it just take unneccessary space away from the valuable SSD space?

Thanks for your help!
Kind Regards from Switzerland

no avatar
HansKeesom
Member
 
Posts: 2169
Joined: Mon Jul 19, 2010 8:53 pm

by HansKeesom » Sat Aug 25, 2012 3:49 pm

first question : System A is better
Second question : OS on SSD is nice on startup but rather unimportant when running. Therefore OS on HDD is fine.
Regards, Hans Keesom
I stitch and render for other photographers. Price: 25 euro or less, no cure no pay. If you want to concentrate on your business let me do the stitching for you. Free TB of Dropbox space when you have more then 250 euro business a year.

no avatar
Zurich.Gigapixel
Member
 
Posts: 36
Joined: Sun Jul 01, 2012 6:49 pm
Location: Zurich, Switzerland

by Zurich.Gigapixel » Sat Aug 25, 2012 4:00 pm

HansKeesom wrote:first question : System A is better
Second question : OS on SSD is nice on startup but rather unimportant when running. Therefore OS on HDD is fine.

Thanks for the fast response!

I forgot one questions (I think I already know the answer but I want to be sure)
Does it make a difference where (SSD vs HDD) the .jpgs are, which are loaded into autopano?

no avatar
HansKeesom
Member
 
Posts: 2169
Joined: Mon Jul 19, 2010 8:53 pm

by HansKeesom » Sat Aug 25, 2012 4:16 pm

Zurich.Gigapixel wrote:
HansKeesom wrote:first question : System A is better
Second question : OS on SSD is nice on startup but rather unimportant when running. Therefore OS on HDD is fine.

Thanks for the fast response!

I forgot one questions (I think I already know the answer but I want to be sure)
Does it make a difference where (SSD vs HDD) the .jpgs are, which are loaded into autopano?

It does not matter other then speed but to save space on SSD I would place them on HDD.
Last edited by HansKeesom on Sat Aug 25, 2012 4:17 pm, edited 1 time in total.
Regards, Hans Keesom
I stitch and render for other photographers. Price: 25 euro or less, no cure no pay. If you want to concentrate on your business let me do the stitching for you. Free TB of Dropbox space when you have more then 250 euro business a year.

no avatar
Zurich.Gigapixel
Member
 
Posts: 36
Joined: Sun Jul 01, 2012 6:49 pm
Location: Zurich, Switzerland

by Zurich.Gigapixel » Sat Aug 25, 2012 4:25 pm

HansKeesom wrote:... to save space on SSD I would place them on HDD.

I thought so, because the .jpgs which are loaded into autopano are converted into .tiffs and put in the first temporary space anyway (SSD in this case) and are used from there. That's how I atleast imagine it to work.

Regards and thanks again!

no avatar
klausesser
Member
 
Posts: 7884
Joined: Mon May 22, 2006 12:18 am
Location: Duesseldorf, Germany

by klausesser » Sat Aug 25, 2012 4:43 pm

Zurich.Gigapixel wrote:. . because the .jpgs which are loaded into autopano are converted into .tiffs . .

I´m afraid i don´t understand . . !?

best, Klaus
Simplicity is the keynote of all true elegance. Coco Chanel

no avatar
Zurich.Gigapixel
Member
 
Posts: 36
Joined: Sun Jul 01, 2012 6:49 pm
Location: Zurich, Switzerland

by Zurich.Gigapixel » Sat Aug 25, 2012 4:47 pm

klausesser wrote:I´m afraid i don´t understand . . !?

I think I've read this here on the forum, I could be wrong, please correct me in this case:

When you load your .jpgs into Autopano to render the panorama, autopano will convert all your pictures into .tiffs (as soon as you start rendering it), save it on the temporary space and work with those instead of the original .jpgs.
Last edited by Zurich.Gigapixel on Sat Aug 25, 2012 4:48 pm, edited 1 time in total.

User avatar
gkaefer
Member
 
Posts: 3387
Joined: Tue Jun 09, 2009 1:01 pm
Location: Salzburg

by gkaefer » Sat Aug 25, 2012 7:19 pm

Zurich.Gigapixel wrote:
klausesser wrote:I´m afraid i don´t understand . . !?

I think I've read this here on the forum, I could be wrong, please correct me in this case:

When you load your .jpgs into Autopano to render the panorama, autopano will convert all your pictures into .tiffs (as soon as you start rendering it), save it on the temporary space and work with those instead of the original .jpgs.

a link to that thread would be interesting for me... to read that piece of information from one kolor staff...
if this is true it would implify that working with jpgs as source images would take longer than instead using tiffs as source images (and that no more needed to be converted inside autopano to "autopano-tiffs").
raw images are getting converted to tiffs by autopano using the dcraw algorithm and thats the reasons why a) it takes longer to process them internally (compared with using tiffs from beginning) and b) its wise to use a external raw converter because of better converting results ...

I use 2 x 120 ozctech SSDs for temp and as source and destination paths. thats ok for smaller projects.
and I never want to go back to HDs because of read/write times. so if this project does have the budget, I would take 2 LSI RAID controller where 8 SSds each can be connected to a RAID0 and would use 160 GB SSDs so 2.8TB in sum are available.
... or one LSI equipped with three/four OCZ OCZSSD2-1CLS500G in one RAID0 as temp, and two further 500GB SSDs for source images and pano destination.

Georg
Last edited by gkaefer on Sat Aug 25, 2012 7:20 pm, edited 1 time in total.

no avatar
HansKeesom
Member
 
Posts: 2169
Joined: Mon Jul 19, 2010 8:53 pm

by HansKeesom » Sat Aug 25, 2012 7:25 pm

You can copy a temp-file and rename it to be a .tiff. It will then load nicely in a viewer.

The temp-files, whether tiff or not is what APG seems to work on so the originals can be elsewhere.
Still I do load my originals (RAW) onto SSD, but that has a lot to do with the way I work :-)
Regards, Hans Keesom
I stitch and render for other photographers. Price: 25 euro or less, no cure no pay. If you want to concentrate on your business let me do the stitching for you. Free TB of Dropbox space when you have more then 250 euro business a year.

no avatar
Zurich.Gigapixel
Member
 
Posts: 36
Joined: Sun Jul 01, 2012 6:49 pm
Location: Zurich, Switzerland

by Zurich.Gigapixel » Sat Aug 25, 2012 7:53 pm

gkaefer wrote:a link to that thread would be interesting for me...

http://www.kolor.com/forum/p83295-2011-05-18-06-12-47#p83295

gkaefer wrote:so if this project does have the budget, I would take 2 LSI RAID controller where 8 SSds each can be connected to a RAID0 and would use 160 GB SSDs so 2.8TB in sum are available.
... or one LSI equipped with three/four OCZ OCZSSD2-1CLS500G in one RAID0 as temp, and two further 500GB SSDs for source images and pano destination.

This project doesn't have this kind of budget :)

no avatar
klausesser
Member
 
Posts: 7884
Joined: Mon May 22, 2006 12:18 am
Location: Duesseldorf, Germany

by klausesser » Sat Aug 25, 2012 8:14 pm

Zurich.Gigapixel wrote:
gkaefer wrote:a link to that thread would be interesting for me...

http://www.kolor.com/forum/p83295-2011-05-18-06-12-47#p83295

Read again . . . "the RAW files are converted to TIFF". ;):cool:

best, Klaus
Simplicity is the keynote of all true elegance. Coco Chanel

no avatar
Zurich.Gigapixel
Member
 
Posts: 36
Joined: Sun Jul 01, 2012 6:49 pm
Location: Zurich, Switzerland

by Zurich.Gigapixel » Sat Aug 25, 2012 8:39 pm

klausesser wrote:Read again . . . "the RAW files are converted to TIFF".

good point. I should read slower ;)

But that makes me think what autopano does when I'm using .jpgs....
Does it use the .jpgs directly? Without converting them (to .tiffs or whatever) into the temp space?
Should in this case the folder with the .jpgs be on the faster disk (SSD)? (So it renders faster...)

no avatar
HansKeesom
Member
 
Posts: 2169
Joined: Mon Jul 19, 2010 8:53 pm

by HansKeesom » Sat Aug 25, 2012 8:47 pm

if you input jpgs it problably still converts to tiff's.

But give it a try, converts your RAW files to jpg, delete all temp-files, start APG and load your jpg's
Regards, Hans Keesom
I stitch and render for other photographers. Price: 25 euro or less, no cure no pay. If you want to concentrate on your business let me do the stitching for you. Free TB of Dropbox space when you have more then 250 euro business a year.

User avatar
MikeMcFarlane
Member
 
Posts: 54
Joined: Sat Feb 26, 2011 4:16 pm
Location: Moscow, Russia

by MikeMcFarlane » Wed Aug 29, 2012 7:56 am

Does the amount of RAM make a difference for Autopano? Going from 8GB to 16GB on my MBP made a huge difference, reduced stitching time by about 40%. Having more in a desktop may help, but it depends on the OS and Autopano if it can actually use it.
Bio: Generation fluxer - farmer, mechatronics engineer, project manager, climbing shop manager, landscape photographer, programmer, virtual panographer, cyclist, life lover.
Twitter: MikeJMcFarlane
email: moo@moosooboo.com
www: www.moosooboo.com

no avatar
HansKeesom
Member
 
Posts: 2169
Joined: Mon Jul 19, 2010 8:53 pm

by HansKeesom » Sun Sep 16, 2012 4:28 pm

Mike,

Depends on the resolution of the input images and their number.
See here for calculating the RAM you need

http://www.kolor.com/forum/p87033-2011-08-11-11-53-58#p87033
Last edited by HansKeesom on Sun Sep 16, 2012 4:30 pm, edited 1 time in total.
Regards, Hans Keesom
I stitch and render for other photographers. Price: 25 euro or less, no cure no pay. If you want to concentrate on your business let me do the stitching for you. Free TB of Dropbox space when you have more then 250 euro business a year.

no avatar
tived
Member
 
Posts: 796
Joined: Fri Jul 11, 2008 3:49 pm
Location: Dane in Western Australia

by tived » Mon Sep 17, 2012 1:29 pm

Zurich.Gigapixel wrote:
klausesser wrote:Read again . . . "the RAW files are converted to TIFF".

good point. I should read slower ;)

But that makes me think what autopano does when I'm using .jpgs....
Does it use the .jpgs directly? Without converting them (to .tiffs or whatever) into the temp space?
Should in this case the folder with the .jpgs be on the faster disk (SSD)? (So it renders faster...)

this is probably a little late to reply, but the advantage of TIFF is that its lossless and you can use 16-bit, for better color quality.

So how is your project going? I have just added a 24 channel RAID controller to my system, which i slowly will add to.
Next is to source 16GB RAM sticks and hope its compatible ;-)

Henrik

no avatar
leifs
Member
 
Posts: 612
Joined: Sun Sep 06, 2009 12:08 pm
Location: Ørsta Norway

by leifs » Mon Sep 17, 2012 11:08 pm

I'm planning to "reorganize" my disks and need a confirmation. Is this what I need for the ultimate disk setup for a APG-machine:
1) OS - hdd (once started not very much disk-access)
2) OS pagefile - SSD1
3) OS temp -SSD2
4) APG sourcefiles - SSD3
5) APG destinationfile - SSD4
6) APG temp/scratch - SSD RAID as fast I/O as my $$$ can buy

Is there anything I have forgot ? Is there anything not needed ?
I know that enough RAM makes some of the above obsolete:
image MB x4 x number of pics for processing everything in RAM (16MB x4 x2000images - > 128GB RAM)
maybe, some day, I will build this computer
http://www.asus.com/Server_Workstation/Server_Motherboards/Z9PED16/
and give it 512GB RAM :-)

leifs
Last edited by leifs on Mon Sep 17, 2012 11:09 pm, edited 1 time in total.
Olympus OM-D E-M1, Panasonic 8mm f3.5 fisheye, Leica 25mm f1.4, Olympus 75mm f1.8, Canon 100-400mm f/4.5-5.6 L
Seitz VRdrive2
Intel i7 980X, 48GB RAM, Win7 64bit, SSD RAIDs

no avatar
tived
Member
 
Posts: 796
Joined: Fri Jul 11, 2008 3:49 pm
Location: Dane in Western Australia

by tived » Mon Sep 17, 2012 11:54 pm

Hi Leif,

I haven't tried with just single drives/SSD - how much ram do you have at the moment? ok just saw that 48gb....;-)

i don' think you will need more 3 sets OS/APPS - WIN/AGP/PTGUI/TEMP/PS/SCRATCH/PAGE - STORAGE

splitting it up further may make a small difference, but i haven't tried.

just remember it will run as fast as the slowest part.

hej fra Australien

Henrik


Return to Hardware

Who is online

Users browsing this forum: No registered users and 1 guest