00473: Memory problems with 2.0 (Search, AllRecentChanges)

Summary: Memory problems with 2.0 (Search, AllRecentChanges)
Created: 2005-07-18 13:23
Status: Suspended
Category: Bug
From: Henning
Assigned:
Priority: 2
Version: 2.0b55, 2.0b48, 2.0b43
OS: Linux/Apache 2

Description:

Suspending as this has been reported a long time ago. If there are problems with the current version please create a new entry. --Petko

Since migrating to 2.0 from 1.0, I'm experiencing memory problems in the form Fatal error: Allowed memory size of x bytes exhausted (tried to allocate y bytes).

They occurred first during WikiSearches (where it's still most frequent), later in AllRecentChanges, too. Switching to 2.0b48, the sidebar and all other pages were affected as well.

Generally, the messages appear irregularly, but in 2.0b48 very frequently (making usage of the Wiki almost impossible).

I switched back to 2.0b43 now which mellowed the symptoms, but the problem is still there.

Insufficiency of the old server hardware and the rapid growth of the site might be valid explanations of the problem.

However, could it be that PmWiki is a bit more performance-hungry in the newer versions, or that there's some bug affecting the memory allocation?


Can you add $EnableDiag=1 to your config.php, then run the wiki with ?action=phpinfo and tell me the value of the "memory_limit" parameter? (Also it would be good to know what version of PHP you're running.)

Or, perhaps you can tell me what the value of 'x' is in the error message you received above...?

PmWiki can indeed use a fair amount of memory when doing its processing, I've tried to keep an eye on it from time to time but have never managed to hit the limits myself (except when doing searches on sites with tens of thousands of pages).

--Pm


Here are the details:

  • memory_limit 8M
  • PHP Version 4.2.2
  • From AllRecentUploads: Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to allocate 1296648 bytes) in /srv/www/htdocs/pmwiki/pmwiki.php on line 988

I've got 1235 pages and maybe 2000 - 2500 uploaded files in the wiki.

--Henning July 22, 2005, at 09:53 AM


Heavy use of the wiki (by the administrator = me) seems to induce a state of continued memory shortage. Changing about 100 pages manually in rapid succession, I couldn't even save them any more without Fatal Errors.

After a few minutes without any activity, the situation returned to normal though. As the problem seems to be transient and no user ever has complained to me about it, I guess it's not serious after all.

The server admin increased the PHP memory limit to 16M on my request, and I expect that will reduce or even eliminate the problem :-) --Henning August 31, 2005, at 05:27 AM


Actually, the symptoms you're describing indicate to me a problem with overall server memory availability, not the amount available to PmWiki. The PHP memory limit is supposed to be on a per-request basis, not aggregated over multiple simultaneous queries. If you're getting fatal errors for saving pages that would indicate that you're reaching resource limits on the server and not just within PHP.

--Pm


We've hit this twice now with gNewSense. Digging into it, looks like the code is trying to load lots of stuff into memory (such as the name of every wiki page), rather than streaming it. We have ~16k pages, and about ~120MB of data. Here's my first patch against 2.1.27, which unbreaks our forums:

===================================================================
RCS file: pmwiki.php,v
retrieving revision 1.2
diff -u -r1.2 pmwiki.php
--- pmwiki.php  2008/03/30 11:16:39     1.2
+++ pmwiki.php  2008/06/28 09:31:04
@@ -792,11 +792,9 @@
         if ($pagefile{0} == '.') continue;
         if (is_dir("$dir/$pagefile"))
           { array_push($dirlist,"$dir/$pagefile"); continue; }
-        $o[] = $pagefile;
+        $out = array_merge($out, MatchPageNames(array($pagefile), $pats));
       }
       closedir($dfp);
-      StopWatch("PageStore::ls merge {$this->dir}");
-      $out = array_merge($out, MatchPageNames($o, $pats));
     }
     StopWatch("PageStore::ls end {$this->dir}");
     return $out;