Low GPU-load on new high end GPU

[HansKeesom]

Low GPU-load on new high end GPU

by HansKeesom » Wed Feb 05, 2014 7:14 am

I installed a new faster GPU card, a EVGA Titan superclocked, in the hope to see a massive faster preview.
My workstation is a x79 motherboad with a i7-3930 and 64 GB and a PCIE-SSD

Although I do see that the APG fills up the GPU-memory much faster(384 instead of 128 bit?) and CPU-memory much slower (so more is done on GPU?), I notice only a very low GPU-load while previewing. Most of the time it stays at 0% and averages at 4%, peaking at 22%

In practical terms this means that most previewing is not faster, only with big panoramas I can see sometimes half of the time is needed.

What can be done to make better use of the GPU. Should I reinstall APG?

Or should I move the Titan GPU to my slower i7-920 + 16 GB RAM machine where it will be of better use?

I am rather disappointed, 969 euro disappointed....

Last edited by HansKeesom on Thu Feb 06, 2014 1:41 am, edited 1 time in total.

[AlexandreJ]

Re: Low GPU-load on new high end GPU

by AlexandreJ » Wed Feb 05, 2014 9:12 am

There are several aspect in how GPU can be used efficiently or not.
We had lately a study on how we use it during previewing and what came out is that it is largely related to the way we use Qt that is not behaving well with opengl. This is gonna to change radically with qt 5.2 which is out recently and has perfect integration of Opengl into the UI. So large gain should be expected from that change first.

Then, about AVP and Titan. This works really nice together We use that power in that context.

[HansKeesom]

Re: Low GPU-load on new high end GPU

by HansKeesom » Wed Feb 05, 2014 10:46 am

There are several aspect in how GPU can be used efficiently or not.
We had lately a study on how we use it during previewing and what came out is that it is largely related to the way we use Qt that is not behaving well with opengl. This is gonna to change radically with qt 5.2 which is out recently and has perfect integration of Opengl into the UI. So large gain should be expected from that change first.

Then, about AVP and Titan. This works really nice together We use that power in that context.

QT? Quicktime?
Can I install something to make it work NOW and with APG. I don't use AVP

[mediavets]

Re: Low GPU-load on new high end GPU

by mediavets » Wed Feb 05, 2014 10:49 am

There are several aspect in how GPU can be used efficiently or not.
We had lately a study on how we use it during previewing and what came out is that it is largely related to the way we use Qt that is not behaving well with opengl. This is gonna to change radically with qt 5.2 which is out recently and has perfect integration of Opengl into the UI. So large gain should be expected from that change first.

Then, about AVP and Titan. This works really nice together We use that power in that context.

QT? Quicktime?
Can I install something to make it work NOW and with APG. I don't use AVP

Qt:
http://qt-project.org/

[HansKeesom]

Re: Low GPU-load on new high end GPU

by HansKeesom » Wed Feb 05, 2014 12:06 pm

There are several aspect in how GPU can be used efficiently or not.
We had lately a study on how we use it during previewing and what came out is that it is largely related to the way we use Qt that is not behaving well with opengl. This is gonna to change radically with qt 5.2 which is out recently and has perfect integration of Opengl into the UI. So large gain should be expected from that change first.

Then, about AVP and Titan. This works really nice together We use that power in that context.

QT? Quicktime?
Can I install something to make it work NOW and with APG. I don't use AVP

Qt:
http://qt-project.org/

Lookes like something for developers. Is there something I can do as end user?

[HansKeesom]

Re: Low GPU-load on new high end GPU

by HansKeesom » Wed Feb 05, 2014 2:07 pm

There are several aspect in how GPU can be used efficiently or not.
We had lately a study on how we use it during previewing and what came out is that it is largely related to the way we use Qt that is not behaving well with opengl. This is gonna to change radically with qt 5.2 which is out recently and has perfect integration of Opengl into the UI. So large gain should be expected from that change first.

Then, about AVP and Titan. This works really nice together We use that power in that context.

To be practical, can a version of APG 3.5.1 RC1 be generated with QT5.2 ?
I am more then willing to test its performace with my new Titan card. I have a number of projects coming up which where the reason to invest bigtime in this card.

Alexander, to be honest, as a customer I am not interested in why a problem exist, I am interested in a solution.

[HansKeesom]

Re: Low GPU-load on new high end GPU

by HansKeesom » Wed Feb 05, 2014 10:49 pm

To be sure my I tested my GPU with furmark. Great to see it can do 99% and the graphics it shows at HD-quality. So the card seems to be fine.

[HansKeesom]

Re: Low GPU-load on new high end GPU

by HansKeesom » Thu Feb 06, 2014 12:35 am

While doing some work on a panorama I was positively surprised by what GPU-z was showing, a GPU-load bar with a good amount of red, moving between 8 and 27% and not on 0 anymore while running preview.
I did use 3.0.8 as 3.5. RC1 is crashing during preview...

[HansKeesom]

Re: Low GPU-load on new high end GPU

by HansKeesom » Thu Feb 06, 2014 9:35 am

just noticed the card is running pci-express x16 v1.1 while capable to do v2.0.
Can anyone comment on this.

Correction : when running higher then low speed it becames v2.0. I think this is sort of powersaving mode of the motherboard

[HansKeesom]

Re: Low GPU-load on new high end GPU

by HansKeesom » Thu Feb 06, 2014 10:06 pm

Thing are getting better though the preview is still not much faster.

Attachments

[HansKeesom]

Re: Low GPU-load on new high end GPU

by HansKeesom » Sat Feb 08, 2014 12:37 pm

A few days of changing settings and I am doing already better.

Used a 971 photo panorama to test and improve my setup. I choose this set because it is the kind of panorama that took a lot of time (30 minutes per preview, 12 hours of rendering ) before having this GPU, the type I hope to do a lot this year and is therefore the reason why I bought the GPU.
I don't mind much about how long rendering take, I need to sleep 8 hours each day anyway...... Editing time, including preview is much more important for me. It needs to be reduced as much as possible.

I started with a panorama of 971 photos on my c-drive which is a normal HD. I do have a PCIE-SSD but until now I used it for pagefile and temp-directories, which despite my 64 GB RAM both were needed a lot.

Installing the GPU was not without worries. The card is massive and feel like a brick. Fitting it into my computer required me to move a harddisk to another location because the titan card is long, very long. But to my surprise after half an hour of carefull fitting and thinking it clicked into it's place and the computer started and recognised the card.

Not unimportant is to mention that I have my computer connected to a little device that tells me how many watts the system is using. In the past I had only a 450 Watt powersupply and that forced me to run the motherboard in first gear only. Now I have a 850 watt powersupply on board but knowing the Titan can do 350 Watt I was carefull. Do set the card to less then 100 % power if the total usage is getting close to what the powersupply in your computer can deal with. The card will slow itself down when using more watts then you want.

After installing the needed drivers it was time to get the best out of this card.
First thing I looked at was the fan of the titan-card. As the titan-card was installed at the bottom of my computer I wanted to prevent heat from going up and making life difficult for CPU and other parts of the computer. The titan-card blows out it's hot air at the back of the computer through it's own exit.

Using it's own software I gave the fan of the titan-card a curve to follow where at it's minimum temperature of 37 degrees the fan-speed would be 0%, but as soon the temperature would go up the % would quickly go up such that at 40 degrees it would be at 30% and at 50 degrees at 100%.
Using http://www.overclock.net/t/1368182/over ... recision-x I also set the card to 100% power, a max temp of 80 degrees, 140 mhz GPU clock off sett and 200 MHZ memory offset.
The above delivered me a stable card, where GPU-Z showed that when needed there would be a perfcap to keep things stable and within acceptable limits.

Now let's get a few things clear here, the goal of Autopano (pro or giga) is :
-first of all to deliver high quality panoramas
-secondly to do so in a way that makes it simple to use for the photographer
-thirdly to be able to run on all kind of machine
-fourtly to do so in reasonable time and maybe little time as possible.

The goal is not to make my GPU run at 100%. Still it is a bit hard to wait for the programm while the GPU is only used at 25% and most of time only 0-1%. The question whether it was wise to spend 969 euro on this card comes to mind.

When upgrading a system by replacing a part, one is likely to replace one bottleneck with another. It is like driving a car that can only go 150 KM/H. Replacing the engine with a version with more HP might allow you to go over 200 KM/H if it wasn't for the tires that are only certified for speeds until 170 KM/H. So then you replace the tires and find out that at speeds over 200 KM/H your windscreen wipers will not be able to stay .......But at least we are driving 50 KM/H faster then before.

The first botleneck I hit was easy to solve. With the new GPU installed it seems like APG is not using normal RAM and GPU-RAM so much. At least, not for the first half an hour it seems. Once you start zooming in RAM and GPU-RAM fill up. During that first period sourcefiles are read a lot which makes it more important to be able to load them fast. So moving the 45 GB of sourcefiles from a normal harddisk to a PCIe-SSD caused the reading of them to go from max. 80 MB/sec to 275 MB/sec. Bottleneck solved...or at least reduced, because a bottleneck is only solved when there is another component that is the new bottleneck.

The new bottleneck is now the CPU. It is not running on 100% but it does hit 80 degrees at which the motherboard is instructed to reduce the multiplier from a max of 46 back to as low as 33 to have things cool down a bit. This is all happening while the GPU is not much most of the time, same to disk I/O. So now the CPU and it's cooling is now the bottleneck....time to find some fan to improve the airflow....
The computer start with a multiplier of 38. For the preview option, the rotating circle in the upperleft corner of the editor, that is more then enough because after a few seconds of starting the preview temperature hits 80 and the multiplier is decreased. However, while working in the editor, placing markers, it can be usefull to have a higher multiplier. I found that 46 is still stable for my system. Other systems and motherboards might have different values.

Conclusions :
1. APG is not using this GPU to it's full potential and is still using a lot of CPU-power for calculating a preview.*
2. A much cheaper card would have done just as well in this configuration.
3. The 969 euro would problably have been better spend if I had used it to buy a XEON processor instead of the the i7 3960 I have now. But given the fact I decided for the 3960 last year, this card is at least helping this CPU**

*I still have to look into the bios upgrade that will allow for PCIE 3.0 which should double the speed of datatransfer for both the GPU as the PCIE-SSD.
But I believe not the speed of PCIE 2.0 is being used so hesitate to do any upgrade the bios.

**This card might be better at it's place in my other machine which is a i7 920 with 16 GB of memory. If I find the time I test the titan in that machine.



to be continued

[gkaefer]

Re: Low GPU-load on new high end GPU

by gkaefer » Sat Feb 08, 2014 10:45 pm

Hans, a sidenote & remark to your "space & heat/CPU Problem" with your Titan...
350$ investment worth each Cent:
http://www.corsair.com/us/pc-cases/obsi ... -case.html
dont underestimate the size you see on the Images.
to Transport the package from postal Office I needed each single cm (all directions) space in my VW Golf ;-))
I took also the 1200w power supply from corsair.
it is stated to not suport my Motherboard, but my E-ATX Xeon board finally fitted very well to the cases screwhowls/Pins (not all Pins did fit, but bit more than 50%, still everything stable mounted)
I took finally the Supermicro - X9DRi-F - Motherboard because of the 16 Memory Slots... so I can use the cheaper 16GB RAM modules and still achieve 256GB ram....
the Titan still has more than 5cm place before the HD/SD Cages begin...
(my System is not ready to run, only 2 ram modules, no Xeons until now... will take until summer....)

Liebe Grüße,
Georg

[HansKeesom]

Re: Low GPU-load on new high end GPU

by HansKeesom » Sun Feb 09, 2014 10:51 am

Hi Georg,
The thing with botlenecks is, you solve one and immediately you hit the next one....The trick is to find the right order in which you solve them, spending little money and making big progress, so called cherry-picking.

The space problem was not really a big issue. Of course it will be more of an issue when I want to add an extra harddisk or SSD.

The heat problem is partly the result of APG using the GPU not as much as I had hoped. Actually I think they did something wrong there or at least they could have done a better job,

As I write this suddenly the GPU does kick in, showing higscore of 91 % but also lots of 0 % causing a striped pattern in GPU-Z. Wow, I even get a long lasting perfcap because of VRel. which means that the GPU is slowed down to keep things reliable......

I opened the case to reverse the only casefan I am using at the moment. Like all other fans (WC of CPU, cooling of GPU and Powersupply) it was blowing outwards of the case. Now at least one fan is blowing inwards, together with all the cracks and holes. Let's see what that will bring.


half an hour later....that brought me 18 degrees lower temperature and running 7 point higher multiplier.....that took only 5 minutes work and was a lot cheaper then buying a titan ;-)
So another bottleneck "solved", but still a titan that is not being run on it maximum speed

[HansKeesom]

Re: Low GPU-load on new high end GPU

by HansKeesom » Sun Feb 09, 2014 9:59 pm

What I do notice is that the last of the twelve cores of my CPu is often running close to 100%. The first 11 cores are running only about half of that or less causing an average cpu of 10%. GPu is alternating between 0% and something higher.
Seems like only one process is working to feed the GPU. Not sure is that is the bottleneck because at the same time the GPu is also showing a perfcap for VREL

[HansKeesom]

Re: Low GPU-load on new high end GPU

by HansKeesom » Mon Feb 10, 2014 2:52 pm

Hans, a sidenote & remark to your "space & heat/CPU Problem" with your Titan...
350$ investment worth each Cent:
http://www.corsair.com/us/pc-cases/obsi ... -case.html
dont underestimate the size you see on the Images.
to Transport the package from postal Office I needed each single cm (all directions) space in my VW Golf ;-))
I took also the 1200w power supply from corsair.
it is stated to not suport my Motherboard, but my E-ATX Xeon board finally fitted very well to the cases screwhowls/Pins (not all Pins did fit, but bit more than 50%, still everything stable mounted)
I took finally the Supermicro - X9DRi-F - Motherboard because of the 16 Memory Slots... so I can use the cheaper 16GB RAM modules and still achieve 256GB ram....
the Titan still has more than 5cm place before the HD/SD Cages begin...
(my System is not ready to run, only 2 ram modules, no Xeons until now... will take until summer....)

Liebe Grüße,
Georg

Maybe I settle for this and place it in the middle of the living room as a kind of hightech fireplace

http://www.coolermaster.nl/product.php?product_id=6851

Or nice furniture like
http://azerty.nl/producten/product_deta ... #kenmerken

All yoking aside, good cooling due to a good big case is gonna help more then placing an expensive GPU in a small case that is badly ventilated

[klausesser]

Re: Low GPU-load on new high end GPU

by klausesser » Tue Feb 11, 2014 1:23 am

All yoking aside, good cooling due to a good big case is gonna help more then placing an expensive GPU in a small case that is badly ventilated

A good adress for informations about render-hardware:
http://www.3dpowerstore.de/html.php/mod ... es/pid/436

best, klaus

[HansKeesom]

Re: Low GPU-load on new high end GPU

by HansKeesom » Tue Feb 11, 2014 9:15 pm

Great stuff, but for the price of the single unit I build one with SSD and 64 GB RAM

The price of the other units are unclear but likely much higher.

But again, great stuff

[tived]

Re: Low GPU-load on new high end GPU

by tived » Tue Mar 04, 2014 2:01 am

Hi Klaus,

That is a nice toy store you have there :-)

Henrik

All yoking aside, good cooling due to a good big case is gonna help more then placing an expensive GPU in a small case that is badly ventilated

A good adress for informations about render-hardware:
http://www.3dpowerstore.de/html.php/mod ... es/pid/436

best, klaus

[tived]

Re: Low GPU-load on new high end GPU

by tived » Tue Mar 04, 2014 2:05 am

Hans,

What you use a test set, always use the same set, to test with so you can see if you have moved forward or backwards - which is why the PTGui Speedtest is very good as its a fixed set of images, you are not attached to the images. You can compare yourself to others as well as yourself of course. Also others can see what you have done and what improvement if any it has done. At least until someone writes another cross platform benchmark test for both AGP and PTGui.

All the best

Henrik

[HansKeesom]

Re: Low GPU-load on new high end GPU

by HansKeesom » Wed Mar 05, 2014 10:15 pm

Henrik, my point is not about rendering speed, it is about preview speed and using the GPU that is there.

Hans,

What you use a test set, always use the same set, to test with so you can see if you have moved forward or backwards - which is why the PTGui Speedtest is very good as its a fixed set of images, you are not attached to the images. You can compare yourself to others as well as yourself of course. Also others can see what you have done and what improvement if any it has done. At least until someone writes another cross platform benchmark test for both AGP and PTGui.

All the best

Henrik

[tived]

Re: Low GPU-load on new high end GPU

by tived » Wed Mar 05, 2014 11:39 pm

hans,
What are you specs again for your system.

With a Xeon, you can use more ram >64GB
Your PCIe being version 3 over version 2 - is your PCIe-SSD PCIe v3?

I look at my own system and even with my Areca 1882ix and 12x Intel SSD 530, I am still only just seeing over 1.2GB/s sustained transfer and peak at 4GB/s

regarding tests - you may be looking at preview render - but if you are not using the same test, you can not get consistency over the results and they do no compare.

because I usually work on large multi gigabyte images, I find it hard to find reference if what I am experiencing is normal or not - as I really have nothing to compare with.

I am wondering if the Titan will work for me (my board is only PCIe v2) will it give the boost or am I better investing in other area's

hope you will find a solution

Henrik

[HansKeesom]

Re: Low GPU-load on new high end GPU

by HansKeesom » Thu Mar 06, 2014 6:25 pm

PCIE 3.0 is possible on the board with updated bios. Should bring some for the GPU, not sure about the SSD-PCIE.

Xeon is in back of my mind, painfull as a motherboard with two-xeons and >64 GB.....a dream.....but only after this year achievable.

Last edited by HansKeesom on Fri Mar 07, 2014 7:35 pm, edited 1 time in total.

[tived]

Re: Low GPU-load on new high end GPU

by tived » Fri Mar 07, 2014 5:23 am

what board do you have?

everything with dual cpus is more expensive its a huge jump in price over a single cpu setup
but it does offer more flexibility IMHO

Henrik

[HansKeesom]

Re: Low GPU-load on new high end GPU

by HansKeesom » Fri Mar 07, 2014 7:34 pm

An Asus Saubertooth x79 http://www.asus.com/Motherboards/SABERTOOTH_X79/

Yes dual CPU's makes you think you could just as well afford a mac ;-)

[tived]

Re: Low GPU-load on new high end GPU

by tived » Sat Mar 08, 2014 1:19 am

I really don't understand this obsession with Mac? Like I have said many times before they are nice computers, they are the best looking boxes - and yes that bin does look pretty :-) and it is probably a good improvement over the past Mac's but they are still behind.

I think a fully kitted out Mac is about $10k

back on topic

Could you give me a description as to what you have in your box and where they are connected - this might be able help you identify your bottlenecks.

All the best

Henrik