r4873 - in trunk/varnish-cache: bin/varnishd doc/sphinx/reference
phk at varnish-cache.org
phk at varnish-cache.org
Tue Jun 1 11:12:00 CEST 2010
Author: phk
Date: 2010-06-01 11:11:59 +0200 (Tue, 01 Jun 2010)
New Revision: 4873
Added:
trunk/varnish-cache/doc/sphinx/reference/shmem.rst
Modified:
trunk/varnish-cache/bin/varnishd/mgt_shmem.c
trunk/varnish-cache/doc/sphinx/reference/index.rst
Log:
Second stab at making dynamic shared memory understandable.
Modified: trunk/varnish-cache/bin/varnishd/mgt_shmem.c
===================================================================
--- trunk/varnish-cache/bin/varnishd/mgt_shmem.c 2010-06-01 07:59:13 UTC (rev 4872)
+++ trunk/varnish-cache/bin/varnishd/mgt_shmem.c 2010-06-01 09:11:59 UTC (rev 4873)
@@ -91,25 +91,25 @@
seq = loghead->alloc_seq;
loghead->alloc_seq = 0;
- MEMORY_BARRIER();
+ VWMB();
memset(sha2, 0, sizeof *sha2);
sha2->magic = SHMALLOC_MAGIC;
sha2->len = sha->len - size;
bprintf(sha2->class, "%s", "Free");
- MEMORY_BARRIER();
sha->len = size;
bprintf(sha->class, "%s", class);
bprintf(sha->type, "%s", type);
bprintf(sha->ident, "%s", ident);
- MEMORY_BARRIER();
- loghead->alloc_seq = seq++;
- MEMORY_BARRIER();
+ VWMB();
+ if (seq != 0)
+ do
+ loghead->alloc_seq = seq++;
+ while (loghead->alloc_seq == 0);
return (SHA_PTR(sha));
-
}
return (NULL);
}
@@ -304,6 +304,7 @@
loghead->head.len =
(uint8_t*)(loghead) + size - (uint8_t*)&loghead->head;
bprintf(loghead->head.class, "%s", "Free");
+ VWMB();
VSL_stats = mgt_SHM_Alloc(sizeof *VSL_stats,
VSL_CLASS_STAT, VSL_TYPE_STAT, "");
@@ -319,10 +320,16 @@
vsl_log_end = vsl_log_start + s1;
vsl_log_nxt = vsl_log_start + 1;
*vsl_log_nxt = SLT_ENDMARKER;
- *vsl_log_start = random();
+ VWMB();
- VWMB();
- loghead->alloc_seq = random();
+ do
+ *vsl_log_start = random();
+ while (*vsl_log_start == 0);
+
+ do
+ loghead->alloc_seq = random();
+ while (loghead->alloc_seq == 0);
+
}
void
Modified: trunk/varnish-cache/doc/sphinx/reference/index.rst
===================================================================
--- trunk/varnish-cache/doc/sphinx/reference/index.rst 2010-06-01 07:59:13 UTC (rev 4872)
+++ trunk/varnish-cache/doc/sphinx/reference/index.rst 2010-06-01 09:11:59 UTC (rev 4873)
@@ -1,7 +1,14 @@
-%%%%%%%%%%%%%%%%%%%%%
-The Varnish Reference
-%%%%%%%%%%%%%%%%%%%%%
+.. _reference-index:
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+The Varnish Reference Manual
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+.. toctree::
+
+ varnishadm.rst
+ shmem.rst
+
.. todo::
The programs:
. varnishd manual page
@@ -32,4 +39,6 @@
. - rollback
Varnishtest
. syntax etc.
+ Shared Memory
+ . internals
Added: trunk/varnish-cache/doc/sphinx/reference/shmem.rst
===================================================================
--- trunk/varnish-cache/doc/sphinx/reference/shmem.rst (rev 0)
+++ trunk/varnish-cache/doc/sphinx/reference/shmem.rst 2010-06-01 09:11:59 UTC (rev 4873)
@@ -0,0 +1,59 @@
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+Shared Memory Logging and Statistics
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+Varnish uses shared memory for logging and statistics, because it
+is faster and much more efficient. But it is also tricky in ways
+a regular logfile is not.
+
+Collision Detection
+-------------------
+
+When you open a file in "append" mode, the operating system guarantees
+that whatever you write will not overwrite existing data in the file.
+The neat result of this is that multiple procesess or threads writing
+to the same file does not even need to know about each other it all
+works just as you would expect.
+
+With shared memory you get no such seatbelts.
+
+When Varnishd starts, it could find an existing shared memory file,
+being used by another varnishd, either because somebody gave the wrong
+(or no) -n argument, or because the old varnishd was not dead when
+some kind of process-nanny restarted varnishd anew.
+
+If the shared memory file has a different version or layout it will
+be deleted and a new created.
+
+If the process listed in the "master_pid" field is running,
+varnishd will abort startup, assuming you got a wrong -n argument.
+
+If the process listed in the "child_pid" field is (still?) running,
+or if the file as a size different from that specified in the -l
+argument, it will be deleted and a new file created.
+
+The upshot of this, is that programs subscribing to the shared memory
+file should periodically do a stat(2) on the name, and if the
+st_dev or st_inode fields changed, switch to the new shared memory file.
+
+Also, the master_pid field should be monitored, if it changes, the
+shared memory file should be "reopened" with respect to the linked
+list of allocations.
+
+Allocations
+-----------
+
+Sections inside the shared memory file are allocated dynamically,
+for instance when a new backend is added.
+
+While changes happen to the linked list of allocations, the "alloc_seq"
+header field is zero, and after the change, it gets a value different
+from what it had before.
+
+Deallocations
+-------------
+
+When a section is freed, its class will change to "Cool" for at
+least 10 seconds, giving programs using it time to detect the
+change in alloc_seq header field and/or the change of class.
+
More information about the varnish-commit
mailing list