Out-of-Memory Errors Running Photoshop CS3

[rogerb]

I run CS3 on Windows XP with 3GB ram. I find that after working on 3 or 4 photos using multiple layers, I start having "out-of-memory" errors and I have to kill CS3 and start it again.
My dng files are typically around 30MB and once loaded in photoshop, they are 110 - 120MB. It's not unusual to see the "info" window showing 300 - 500MB for the composite of the layers. In the photoshop preferences, I have played with the ram allocation setting all the way to 1.5GB but nothing helps. The memory utilization as displayed in the Windows Task Manager shows it approaching the limit then bang! ....that's as far as I can go. Purging history or closing other image files does nothing to reduce it.

One final thing: I have plenty of hard disk space and I let Windows handle the virtual memory requirements.

Is anyone else experiencing this?

[Colin Southern]

Is anyone else experiencing this?

Hi Roger,

Yes - I get it too sometimes. I'm not sure if it's a bug or a configuration issue.

Your RAW file size sounds normal, as does your opened file size. Adjustment layers don't normally increase the file size much, but if you duplicate the image layers that will bump it up; also, increasing the number of history states will use more - AND - there's also a setting (forget the name) that determines how many copies of the image Photoshop maintains to speed up the displaying of the image at various zoom levels that can multiply the overall amount of memory used by the number of levels set (so that's one you might like to check) (I think it's called "Levels" of the like - it's in prefs).

If your on a 32 Bit PC you need to set the /3GB switch in the boot.ini file otherwise Photoshop (or in fact any 32 bit program) will be limited to a max of 2GB RAM) (set automatically on 64 bit OS versions).

Hope this helps!

[rogerb]

Yes Colin, thanks for your response.
I will try the "Levels" setting to see if that helps. I have not done the /3GB switch in the ini file. My concern is that there appears to be a memory leak in that photoshop keeps taking more and more ram and never seems to give it up. I would have thought that the OS would just assign more & more virtual memory and the system would just slow down but that does not seem to be the case. If I'm close to my memory limit and I try to run the Neat Image plug-in, it fails, sometimes not very gently.

I've been reluctant to try the /3GB switch but may have to do it. Right now I can't handle more than about three or four images when doing HDR or Panos!

[Colin Southern]

Yes Colin, thanks for your response.
I will try the "Levels" setting to see if that helps. I have not done the /3GB switch in the ini file. My concern is that there appears to be a memory leak in that photoshop keeps taking more and more ram and never seems to give it up. I would have thought that the OS would just assign more & more virtual memory and the system would just slow down but that does not seem to be the case. If I'm close to my memory limit and I try to run the Neat Image plug-in, it fails, sometimes not very gently.

I've been reluctant to try the /3GB switch but may have to do it. Right now I can't handle more than about three or four images when doing HDR or Panos!

Let me know how you get on. I suspect the levels thing might make a difference. Personally, I just save the image and then restart Photoshop when it happens (which isn't very often).

You might get a more specific answer in the Adobe forums?

[rogerb]

I've posted the question on the Adobe forum. Thanks for that suggestion.

So far the responses I've seen indicate that it's just the nature of the beast. My images are too big and I'll need to go to 64-bit CS4 on Vista with 8GB ram!

Somehow I have a hard time believing this. Surely Adobe wouldn't write software that continually consumes ram until its exhausted?

I'll let you know if I someone posts a more constructive response on the Adobe forum.

[Colin Southern]

I've posted the question on the Adobe forum. Thanks for that suggestion.

So far the responses I've seen indicate that it's just the nature of the beast. My images are too big and I'll need to go to 64-bit CS4 on Vista with 8GB ram!

Somehow I have a hard time believing this. Surely Adobe wouldn't write software that continually consumes ram until its exhausted?

I'll let you know if I someone posts a more constructive response on the Adobe forum.

Well the good news - you don't HAVE to use Vista ... you can run CS4 64 bit edition on XP64 as far as I know

Did you try changing the levels setting?

[rogerb]

I did try Cache Levels = 2 and it just prolonged the agony.

I am trying something though that could make a difference: It seems like Photoshop might need to have contiguous space on the scratch disk. Although my scratch disk has 75GB of free space on it and there are some wide open spaces, if I interpret it right, the PS scratch disk area may be "boxed in" on the disk by some other files.

I have run CheckDisk to make sure I don't have a problem with that drive and now I am defragging which is taking its sweet time to complete. I'll let you know if that's made any difference. Actually what I might do is create a new partition (maybe 20GB) and then allocate this to PS to use as scratch disk with nobody else having access to it. That should keep PS happy.

I'll keep you advised.

[rogerb]

WOW!!!
My confidence in Adobe has been restored.
After running Check Disk I did a defrag on the PS scratch disk. Low and behold the problem appears to have gone away.

I loaded and a 120MB file then made 7 copies of this background layer then added masks for each, added several adjustment layers, and even overlaid two more images using the place function. The info window showed "Doc: 120.3M/1.34G"

I then tried various plugins: Neat Image, ColorFex, and Topaz labs and all of them worked. The only plugin that didn't work was Focus Magic, complaining about no memory. Since this is generally the very last thing I do, I don't have a problem restarting PS to do that. Actually, Neat Image has pretty well taken over my sharpening needs anyway.

Before the defrag, I could see PS memory utilization in Windows Task Manager climb quite a bit faster than what was shown in the PS Info window. Now it's the other way around.

I think the problem had started a while back but wasn't really an issue until I went from images from a 30D to 120MB images from the 5Dmk2. After a few months of that the fragged scratch disk became progressively worse.

I know I should know better than to get excited at this early stage. We'll have to see how it goes going forward but right now, I would strongly recommend defragging your scratch disk if you haven't done it in a while.

Thanks for your help Colin, and I hope you and others find this useful.

[Colin Southern]

Thanks for your help Colin, and I hope you and others find this useful.

Your welcome Roger.

I probably should defrag my drives ... it's been a while!

[Steaphany]

Roger,

Thanks for this tip !

The other day while trying to align several images in Photoshop, I ran into these same errors. At the time I was fortunate to get enough done to run a focal blend in TuFuse Pro. See Water Lily for the results that I achieved.

I have a Western Digital 1TB drive for my photos and set as the Photoshop scratch disk, at less than 10% utilized, it should be more then enough room. When I analyzed the space utilization, I found a remnant Photoshop temp file of 1.3GB, plus enough fragmentation to cause problems. I have the defrag running as I post this.

A Suggestion to the Moderators/Admins: Create a tutorial for Photoshop addressing performance and include this jewel.

[rogerb]

Well, I'm afraid I should have listened to my rational self instead of my emotional self before I made my last post.

Somehow when I thought I was really stress testing this thing, it turns out that, while it passed that test with flying colors, it is still failing the real test: processing photos!

There must be something in my workflow that messes with PS's memory management. Last night I loaded one image, cropped, copied the layer, adjusted levels, and can't exactly remember what else. The Info box showed the composite at less than 300MB. I then ran the Neat Image plug-in and it said there was not enough memory to display the entire image so it gave me a reduced image to work with. I completed that task and looked at memory utilization in Windows Task Manager and PS was at 1.2GB (I had set PS to use up to 1.4GB). Looking at history I only had about a dozen entries. Just for laughs I tried to create a Black & White adjustment layer and got a pop-up saying something about " can't complete this task because of a program error". This has been a recurring problem.

When I went to save my file, all the normal little icons in the save-as box for each file were gone but I could still see the filenames and it looked like it still saved it ok.

I think defragging the disk is still a very good thing to do. However it has not solved my problem like I had originally thought. I'm leaving on a trip for a week in the morning but will get back at it when I return. I'll try to monitor memory utilization after every step to see if there is one step that is causing the problem.

Somehow the solution just can't be "Get a 64-bit OS"!

[Steaphany]

It has been my observation that Photoshop and plugins employ separate memory management schemes. The only real interface between Photoshop and any installed plugin looks like nothing more than Photoshop invoking the execution of the plugin and a data pipe to exchange the image data. Apart from that, each installed plugin is an autonomous and self contained program.

Photoshop's own ability to utilize scratch disk space appears to be private. Unless a particularly memory intensive plugin can provide a similar scratch disk memory management, you'll run the risk of all going fine until the plugin begins to process the handed off image data.

Another area that may contribute to memory problems is Photoshop has no idea of what the memory requirements are for any particular plugin. Photoshop does not appear to release it's own memory allocations in anticipation to invoking a plugin.

[Colin Southern]

Somehow the solution just can't be "Get a 64-bit OS"!

"Interesting" developments (I'm sure you have a few other words for them!).

As a bit of a side note, the 64 bit world has a few subtle differences ...

In 32 bit architectures the processor can only utilize a maximum of 4GB of address space - however - that (unfortunately) doesn't correspond to the ability to address 4GB of RAM, as parts of that range are also needed for other things. So in reality, a 32 bit OS will see a maximum of around 3.3GB of RAM - but wait, there's less ... that's for the whole system. Although you can still set the /3GB switch, the reality is that - with other processes and services that are running - you'll get a LOT less available to any application. And of course - depending on how you've got Photoshop setup - it could be a lot less again. Also, if the memory arcitecture that Steaphany outlined is correct (about plugins running in their own address space) then that would be (potentially) quite a big ask on the systems resources.

In a 64 bit architecture however, the game changes - EACH PROCESS can enjoy up to 3GB of RAM - in fact, you can set Photoshop to use 100% of available RAM because of the way memory management works (The programs will think that they each have the whole machine to themselves) - so if your going to go that route, then don't muck around - whack in 8GB (or up to 12MB with the right motherboard). Also, in this configuration, you don't need to use the 64 bit edition of CS4 (in fact I'm talking about the 32 bit edition).

Thinking more about the problem, I don't think that I have experienced what your seeing - yes, I've had out of memory issues (where it slows to a crawl as it swaps the living daylights out of the drives and I have to re-open the image - but I haven't had the issue with plugins (by and large I don't use them).

Hope this helps!

[rogerb]

As far as PS plug-ins, I'm not sure if they really get spawned or if they run in the same memory space. In Linux, a spawned (detached) program would show up as a separate process. Here I don't see them as a separate process in the Windows Task Manager.

And Colin, yes I would expect the machine to behave exactly as you've experienced with your own machine: everything slows down as the disk is busy swapping out memory. This is quite different. Maybe I have a hardware or a driver problem with this machine that doesn't show up with any other application or even when running diagnostics for that matter. I've done a little more testing (before leaving on my trip) and I can definitely see PS just eating up memory. I'm afraid I've become way too reliant on plug-ins but I will try to go through a complete edit without accessing any 3rd party plugins to see what happens.

Colin: Do I understand you correctly that if I were to run a 64-bit OS (Vista?) that I could still use my 32-bit CS3/4 and still get the benefit of much expanded memory addressing space? Would Vista be running in XP emulation mode?

[rogerb]

Or how about Windows 7?

[Steaphany]

Windows Task Manager has it's limitations.

Here is something to try: Microsoft's Windows XP Process Monitor v2.5

Unlike Task Manager, Process Monitor is an advanced monitoring tool for Windows that shows real-time file system, Registry and process/thread activity, including monitoring of process and thread startup and exit, including exit status codes

[Colin Southern]

Colin: Do I understand you correctly that if I were to run a 64-bit OS (Vista?) that I could still use my 32-bit CS3/4 and still get the benefit of much expanded memory addressing space? Would Vista be running in XP emulation mode?

Yes ... No ... and Maybe

Yes you can use a 32 bit app under a 64 bit OS (I do it with CS3), but in terms of memory management - because of the way the memory is virtualised - 32 bit programs can't access more than 4GB of space (an in practice 2GB is usually given to the app and another 2gb reserved for communication between the app and OS - but rising to a 3GB / 1GB partition on 32 bit OS systems with the /3GB switch set, or in 64 bit systems where /3GB is set by default.

So to answer the question, if you have a 64 bit OS - and heaps of RAM (say 6 to 8 GB) then you can tell the 32 bit edition os Photoshop to use 100% of memory - and it'll see the full 3GB (it won't see or try to use the 6 or 8GB).

[Colin Southern]

Windows Task Manager has it's limitations.

Here is something to try: Microsoft's Windows XP Process Monitor v2.5

Unlike Task Manager, Process Monitor is an advanced monitoring tool for Windows that shows real-time file system, Registry and process/thread activity, including monitoring of process and thread startup and exit, including exit status codes

Another in the same suite is Process Explorer - (also great for advanced malware removal)

[rogerb]

Colin & Steaphany:
Thank you for your inputs.

I'll definitely try that Process Monitor. Sounds like it'll provide a lot more information than I've been getting from Task Manager.

Going from about 1.5GB to 3GB would obviously be a big help. According to the responses on the Adobe Forum, it's all about memory. However I'm still suspicious that my current system does not handle memory management correctly for whatever reason. If "side-grading" to a 64-bit OS solves that problem then I guess that's my only way out. However I will spend some time with the Process Monitor before doing so. Maybe I'll discover the one thing I'm doing that sends the system into the weeds.
Talk to you in a week.
Roger

[Colin Southern]

Colin & Steaphany:
Thank you for your inputs.

No worries Roger.

If your considering running a 64 bit OS on your existing hardware, then why not take the existing RAM up to - say - 4GB and see how PS goes? - nothing to lose, and it may well cure the issue?

The other "standard operating procedure" to try is trashing the prefs file (hold down ctrl + alt + shift while PS is starting), although be aware that it WILL delete all of your customisations / actions etc.