I do worry. My other card is a ATI HD6850 and it gave me a Windows rating at 7.2. It had only 1 GB memory though (I have 3 monitors).
Now I have 4GB and a windows rating of 6.5 and the wheel is spinning and spinning.
To me it seems that there is no gain at all in APG adding memory to the GPU.
If you use the windows rating to evaluate your GPU-card, I think you are being a bit unfair to the cards. There are better test for GPU-cards.
As discussed with AlexandreJ in other threads, memory is important. It allows APG to store all informations about a pano in the GPU memory so it can work. I had a 512 MB card before and often saw the card could only hold 1/4 of what APG would like it to hold. Now I have a 85 euro card with 4 GB on it an the editor really flies and can hold much larger panoramas.
Yes APG might have some possibilities for improvement on using the APG, we are all looking forward to it.