[Varnish] #273: running out of virtual memory on 32 bit server

[Varnish]

#273: running out of virtual memory on 32 bit server
------------------------+---------------------------------------------------
 Reporter:  chrisrixon  |        Owner:  des        
     Type:  defect      |       Status:  reopened   
 Priority:  normal      |    Milestone:  Varnish 1.2
Component:  build       |      Version:  1.1.2      
 Severity:  major       |   Resolution:             
 Keywords:              |  
------------------------+---------------------------------------------------
Changes (by chrisrixon):

  * status:  closed => reopened
  * resolution:  worksforme =>

Comment:

 Replying to [comment:2 des]:
 > What do you mean by "running out of virtual memory"?  We need to know
 the symptoms, not your interpretation.

 Well I monitor it in top and the VIRT column for varnishd reaches just
 over 3000m at which point varnish stops responding to requests. Here's
 what top looks like with -s set to 128M (malloc or file seems to make
 little difference). Note how VIRT is 1964m and rising.

 {{{
   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 29526 admin     16   0 1964m  23m 8984 S  3.3  1.2   0:05.66 varnishd
 }}}

 > How big is your cache file?
 128M

 We have now commissioned a 64 bit server and it works much better although
 VIRT (on top) can reach 20G !

 Here's a stat for the 64 bit:

 {{{
 0+00:19:26
 Hitrate ratio:       10      100      141
 Hitrate avg:     0.9573   0.9603   0.9604

       179615       271.73       154.04 Client connections accepted
       341046       547.45       292.49 Client requests received
       321617       517.48       275.83 Cache hits
         1763         0.00         1.51 Cache hits for pass
        15671        25.97        13.44 Cache misses
        19427        29.97        16.66 Backend connections success
            0         0.00         0.00 Backend connections failures
            0         0.00         0.00 Backend connections reuses
            0         0.00         0.00 Backend connections recycles
            8        -3.00         0.01 Backend connections unused
         6788          .            .   N struct srcaddr
         2510          .            .   N active struct srcaddr
         3306          .            .   N struct sess_mem
         3226          .            .   N struct sess
        10069          .            .   N struct object
        10059          .            .   N struct objecthead
         9222          .            .   N struct smf
         1040          .            .   N small free smf
            1          .            .   N large free smf
           77          .            .   N struct vbe_conn
         1892          .            .   N worker threads
         1892         0.00         1.62 N worker threads created
            0         0.00         0.00 N worker threads not created
            0         0.00         0.00 N worker threads limited
            0         0.00         0.00 N queued work requests
         1892         0.00         1.62 N overflowed work requests
            0         0.00         0.00 N dropped work requests
         7440          .            .   N expired objects
          392          .            .   N objects on deathrow
            0         0.00         0.00 HTTP header overflows
            0         0.00         0.00 Objects sent with sendfile
       291708       467.53       250.18 Objects sent with write
       178540       260.74       153.12 Total Sessions
       339195       539.46       290.90 Total Requests
          591         0.00         0.51 Total pipe
            0         0.00         0.00 Total pass
        18692        28.97        16.03 Total fetch
    111559849    180511.33     95677.40 Total header bytes
   1514209533   2136196.92   1298635.96 Total body bytes
         5947        12.99         5.10 Session Closed
 }}}

 We are puzzled as to why N worker threads slowly climbs to 2000 odd with
 500 requests per second.

 Thanks.

 PS sorry for late reply, I dont seem to be getting notification emails.'

-- 
Ticket URL: <http://varnish.projects.linpro.no/ticket/273#comment:4>
Varnish <http://varnish.projects.linpro.no/>
The Varnish HTTP Accelerator