[H-GEN] Squid cache dirty, eats all my memory.

[Ian Pulsford]

[ Humbug *General* list - semi-serious discussions about Humbug and  ]
[ Unix-related topics.  Please observe the list's charter.           ]
[ Worthwhile understanding: http://www.humbug.org.au/netiquette.html ]

Hi,

My PC was working rather slowly this morning and was swapping a lot so I
had a look at top and discovered squid was eating about 300M of RAM and
200M of swap.  Here is a log entry:

2001/06/01 13:44:28| Starting Squid Cache version 2.3.STABLE4 for
i386--freebsd4.2...
2001/06/01 13:44:28| Process ID 1328
2001/06/01 13:44:28| With 4136 file descriptors available
2001/06/01 13:44:28| Performing DNS Tests...
2001/06/01 13:44:28| Successful DNS name lookup tests...
2001/06/01 13:44:28| DNS Socket created on FD 5
2001/06/01 13:44:28| Adding nameserver 127.0.0.1 from /etc/resolv.conf
2001/06/01 13:44:28| helperOpenServers: Starting 3 'wrapzap' processes
2001/06/01 13:44:28| Unlinkd pipe opened on FD 13
2001/06/01 13:44:28| Swap maxSize 524288 KB, estimated 40329 objects
2001/06/01 13:44:28| Target number of buckets: 806
2001/06/01 13:44:28| Using 8192 Store buckets
2001/06/01 13:44:28| Max Mem  size: 8192 KB
2001/06/01 13:44:28| Max Swap size: 524288 KB
2001/06/01 13:44:28| Store logging disabled
2001/06/01 13:44:28| Rebuilding storage in /export0/squid/cache (DIRTY)
2001/06/01 13:44:28| Rebuilding storage in /export1/squid/cache (DIRTY)
2001/06/01 13:44:28| Set Current Directory to /export0/squid/cache
2001/06/01 13:44:28| Loaded Icons.
2001/06/01 13:44:28| Accepting HTTP connections at 0.0.0.0, port 3128,
FD 16.
2001/06/01 13:44:28| WCCP Disabled.
2001/06/01 13:44:28| Ready to serve requests.
2001/06/01 13:44:28| Done reading /export0/squid/cache swaplog (0
entries)
2001/06/01 13:44:29| Done reading /export1/squid/cache swaplog (0
entries)
2001/06/01 13:44:29| Finished rebuilding storage from disk.
2001/06/01 13:44:29|         0 Entries scanned
2001/06/01 13:44:29|         0 Invalid entries.
2001/06/01 13:44:29|         0 With invalid flags.
2001/06/01 13:44:29|         0 Objects loaded.
2001/06/01 13:44:29|         0 Objects expired.
2001/06/01 13:44:29|         0 Objects cancelled.
2001/06/01 13:44:29|         0 Duplicate URLs purged.
2001/06/01 13:44:29|         0 Swapfile clashes avoided.
2001/06/01 13:44:29|   Took 1.1 seconds (   0.0 objects/sec).
2001/06/01 13:44:29| Beginning Validation Procedure
2001/06/01 13:44:30|   Completed Validation Procedure
2001/06/01 13:44:30|   Validated 0 Entries
2001/06/01 13:44:30|   store_swap_size = 21k
2001/06/01 13:44:31| storeLateRelease: released 0 objects

In the cache directories there were files swap.state.new as well as the
swap.state files.  I am guessing that squid was rebuilding the dirty
cache piece by piece.  Sick of waiting I just killed squid and built
fresh cache directories.

So I guess what I want to know is how to limit system resources
available to squid so if it happens again it won't hog the system.

I have this in /etc/login.conf:

nobody:\
        :memoryuse=128m:\
        :openfiles=1024:\
        :maxproc=512:\

but it didn't prevent squid eating 500M virtual memory.
This line from the log suggests squid will use up to 512M swap (or is it
just referring to the squid cache size?), 
2001/06/01 13:44:28| Max Swap size: 524288 KB
If it's referring to memory swap size any way to reduce it?  I couldn't
find anything in the config file.

Any thoughts appreciated.


Ian Pulsford

--
* This is list (humbug) general handled by majordomo at lists.humbug.org.au .
* Postings to this list are only accepted from subscribed addresses of
* lists 'general' or 'general-post'.