Distributed Processing  

Got some great idea or a feature request? Post it here and discuss it. The most requested concepts are usually implemented, as Autopano Pro / Giga is very community driven.
no avatar
mctrivia
New member
 
Topic author
Posts: 9
Likes: 0 post
Liked in: 0 post
Joined: Sun Aug 06, 2006 11:09 pm
Info

Distributed Processing

by mctrivia » Mon Aug 07, 2006 1:11 am

I have 4 computers on a gigabit network. Would be nice if I could use the idel time of the other computers to help speed up the processing time. Would also be nice if when doing a single job it could use both processors on my computer.

User avatar
AlexandreJ
Kolor Team
 
Posts: 5987
Likes: 7 posts
Liked in: 10 posts
Joined: Mon Nov 14, 2005 4:56 pm
Location: Francin, France
Info

by AlexandreJ » Mon Aug 07, 2006 7:46 am

Distributed rendering is definitively not easy to do. That's could fit in v2.0 feature list.
The 1.3 branch uses any processor you have on your computer : the rendering is fully multithreaded. You can have a look at the current beta here to see what's soon coming : http://www.autopano.net/beta

no avatar
hankkarl
Member
 
Posts: 1284
Likes: 0 post
Liked in: 0 post
Joined: Tue Feb 21, 2006 5:32 pm
Location: Connecticut, USA
Info

by hankkarl » Wed Sep 13, 2006 2:38 am

How about just sending the entire rendering off to some other computer on the network, so you can go back to editing another pano on your machine (or, heaven forbid, do some usefull work to earn a living :-)

Are the layers independent? If so, could each layer in a multilayer project be sent to a different machine for rendering?

User avatar
AlexandreJ
Kolor Team
 
Posts: 5987
Likes: 7 posts
Liked in: 10 posts
Joined: Mon Nov 14, 2005 4:56 pm
Location: Francin, France
Info

by AlexandreJ » Wed Sep 13, 2006 7:31 am

There are many strategies for network rendering / distributed processing :
- Cell based network rendering. A cell per cpu per machine. Our rendering engine is based on cell, so each cell is quite isolated from anything else and could be rendered anywhere else. When I did the multithreading for a single rendering, I prepared the structure to allow such a rendering.
- Layer based. For a multi-layer panorama, we could imaging making a layer per computer.
- Panorama based. The batch stitcher have the logic to distribute panorama from the current list to different computers : a panorama, a computer.

In fact, the hard work is not really on the way we will do the job, any listed solution here are pretty easy to add. All the hard work is on the network detection / setup / transport. I studied some solutions for distributed computing, but they don't fit our project. Here's a list of what is needed for a network rendering system ( don't forget it needs to be crossplatform ) :
* Automatic detection of any available computer on the current LAN,
* Login / Passworded launch of APP daemon on client computer ( to prevent the need to setup something there one by one ),
* Strong tolerance to network crash. If a computer stop responding, if some packet are lost, etc.
To create this network rendering, that's the hard part.

no avatar
l_archi
Member
 
Posts: 162
Likes: 0 post
Liked in: 0 post
Joined: Fri Dec 02, 2005 9:05 pm
Info

by l_archi » Wed Sep 13, 2006 11:53 am

Very complex solution is : Kernel space (http://www.kerrighed.org/)

Distributed threads across network
Autopano configured with 2 cpu, create 2 wine process under linux.

Features of kerrighed :

* GFS (images files, cells)
* High availability

You can try it with live cd :

http://www.kerrighed.org/download.php

install wine and autopano, set 8 cpus :)
Last edited by l_archi on Wed Sep 13, 2006 11:54 am, edited 1 time in total.


Who is online

Users browsing this forum: No registered users and 1 guest

cron