Neutralhazer is very greedy with memory and with very big images as you observed and as we stated in product documentation. Crashs are definitely not acceptable, and we are working to find a solution to handle these tough cases.
Waiting for a solution, please be careful only running Photoshop on your machine when applying NH (check in task manager if another process is not eating much RAM).
You can also try to run Neutralhazer in XnView image processing software which has a lower memory footprint. You can get it there for free: http://www.xnview.com/en/download.html