<div dir="ltr">Looks pretty decent. Definitely makes sense to have some kind of JSON support in Varnish, especially on the parsing and reading end. I too had to implement my own JSON lib just because the options out there just dont hit all requirements, libraries generally optimize for 1 thing and somehow do everything else poorly, like navigation vs performance or strictness vs performance. What I got is ~1600 LOC, but it has very strict JSON grammer parsing. Very fast on both ends too, parsing is done in 1 pass (streamed) and the result is a nice search index which acts very close to a hashtable when searching. Infact, the search index get stored into cache (see below).<div><br></div><div>I could definitely see open sourcing this in the form of a VMOD. Yes, I said VMOD. Check out the following VCL. Sorry for the hijack, and yes, im digging up object support in VCL again :)</div><div><br></div><div>---</div><div><div><font face="monospace, monospace" color="#0b5394">import request;</font></div><div><font face="monospace, monospace" color="#0b5394">import objcore;</font></div><div><font face="monospace, monospace" color="#0b5394">import json;</font></div><div><font face="monospace, monospace" color="#0b5394">import types;</font></div><div><font face="monospace, monospace" color="#0b5394"><br></font></div><div><font face="monospace, monospace" color="#0b5394">sub vcl_recv {</font></div><div><font face="monospace, monospace" color="#0b5394"> new json_req = request.new();</font></div><div><font face="monospace, monospace" color="#0b5394"><br></font></div><div><font face="monospace, monospace" color="#0b5394"> if (req.url == "/") {</font></div><div><font face="monospace, monospace" color="#0b5394"> json_req.copy_headers();</font></div><div><font face="monospace, monospace" color="#0b5394"> json_req.set_host(local.ip);</font></div><div><font face="monospace, monospace" color="#0b5394"> json_req.set_port(std.port(local.ip));</font></div><div><font face="monospace, monospace" color="#0b5394"> json_req.set_url("/some/data.json");</font></div><div><font face="monospace, monospace" color="#0b5394"><br></font></div><div><font face="monospace, monospace" color="#0b5394"> </font><span style="color:rgb(11,83,148);font-family:monospace,monospace"># Tell the VMOD if this request goes back into Varnish</span></div><div><span style="color:rgb(11,83,148);font-family:monospace,monospace"> # safely grab the objcore, if possible</span></div><div><span style="color:rgb(11,83,148);font-family:monospace,monospace"> # (there is a more elegant way to do this</span></div><div><span style="color:rgb(11,83,148);font-family:monospace,monospace"> # but I rather get the point across)</span></div><div><font face="monospace, monospace" color="#0b5394"> json_req.want_objcore();</font></div><div><font face="monospace, monospace" color="#0b5394"><br></font></div><div><font face="monospace, monospace" color="#0b5394"> json_req.send();</font></div><div><font face="monospace, monospace" color="#0b5394"> }</font></div><div><font face="monospace, monospace" color="#0b5394">}</font></div><div><font face="monospace, monospace" color="#0b5394"><br></font></div><div><font face="monospace, monospace" color="#0b5394">sub vcl_deliver {</font></div><div><font face="monospace, monospace" color="#0b5394"> new oc = objcore.new();</font></div><div><font face="monospace, monospace" color="#0b5394"> new json_doc = json.new();</font></div><div><font face="monospace, monospace" color="#0b5394"> new json_field = types.string();</font></div><div><font face="monospace, monospace" color="#0b5394"><br></font></div><div><font face="monospace, monospace" color="#0b5394"> if (json_req.sent()) {</font></div><div><font face="monospace, monospace" color="#0b5394"> json_req.wait();</font></div><div><font face="monospace, monospace" color="#0b5394"><br></font></div><div><font face="monospace, monospace" color="#0b5394"> # Load the objcore from the json request</font></div><div><font face="monospace, monospace" color="#0b5394"> oc.get_reference(json_req.get_objcore());</font></div><div><br></div><div><font face="monospace, monospace" color="#0b5394"> if (!oc.is_valid()) {</font></div><div><font face="monospace, monospace" color="#0b5394"> return(synth(401));</font></div><div><font face="monospace, monospace" color="#0b5394"> }</font></div><div><font face="monospace, monospace" color="#0b5394"><br></font></div><div><font face="monospace, monospace" color="#0b5394"> </font><span style="color:rgb(11,83,148);font-family:monospace,monospace"># Similar to ESI data</span></div><div><font face="monospace, monospace" color="#0b5394"> json_doc.load_index(oc.get_attribute("json"));</font></div><div><font face="monospace, monospace" color="#0b5394"><br></font></div><div><font face="monospace, monospace" color="#0b5394"> if (!json_doc.is_valid()) {</font></div><div><font face="monospace, monospace" color="#0b5394"> </font><span style="color:rgb(11,83,148);font-family:monospace,monospace"># Parse the response text into JSON</span></div><div><font face="monospace, monospace" color="#0b5394"> json_doc.load_text(json_req.get_body());</font></div><div><font face="monospace, monospace" color="#0b5394"> </font><span style="color:rgb(11,83,148);font-family:monospace,monospace"># Write back the index into cache</span></div><div><font face="monospace, monospace" color="#0b5394"> oc.set_attribute("json", json_doc.get_index());</font></div><div><font face="monospace, monospace" color="#0b5394"> }</font></div><div><font face="monospace, monospace" color="#0b5394"><br></font></div><div><font face="monospace, monospace" color="#0b5394"> json_field.set(json_doc.get("some_json_field"));</font></div><div><font face="monospace, monospace" color="#0b5394"><br></font></div><div><font face="monospace, monospace" color="#0b5394"> # We now read a JSON field from the body of an object in cache</font></div><div><font face="monospace, monospace" color="#0b5394"> # Future lookups will hit the JSON search index directly</font></div><div><font face="monospace, monospace" color="#0b5394"> }</font></div><div><font face="monospace, monospace" color="#0b5394">}</font></div></div><div>---<br></div><div><br></div><div>Basically, make a request back to Varnish, grab the objcore, and then load/store/cache JSON into it. That JSON could have been a cache miss, it gets cached, it could have been a 404, this all plays really nice with how Varnish works. A client can even request that JSON back out (minus the search index, obviously). Dont get all up in arms, this all works, this is pretty much how Edgestash works over at Varnish Software :) Also, remember, the point of VCL is to allow people to write what they want, so if someone wants to write the above VMODs/VCL, good for them.</div><div><br></div><div>So ya, if I can write a JSON vmod object, then that would be a good case to release JSON code. The flipside is that with objects, JSON parsing is just the tip of the iceburg in terms of providing a nice standard library with all the things dev expect, plus interop with Varnish internals, like objcore above.</div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr">--<br>Reza Naghibi<br>Varnish Software</div></div></div>
<br><div class="gmail_quote">On Fri, May 19, 2017 at 10:48 AM, Poul-Henning Kamp <span dir="ltr"><<a href="mailto:phk@phk.freebsd.dk" target="_blank">phk@phk.freebsd.dk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">--------<br>
In message <CADe=<a href="mailto:ujYbWs09K_UgsB%2BA6%2Bu97OnTf3xR9HYk1pwKoNNr99EBjQ@mail.gmail.com">ujYbWs09K_UgsB+A6+<wbr>u97OnTf3xR9HYk1pwKoNNr99EBjQ@<wbr>mail.gmail.com</a>><br>
<span class="">, "Devon H. O'Dell" writes:<br>
<br>
</span><span class="">>> [2] I my check for control-chars in strings I forgot that char is signed.<br>
><br>
>Except when it isn't! Whether "plain" char is signed or unsigned is<br>
>implementation-defined<br>
<br>
</span>Yeah, compliments to the ISO-C people for that bit of insanity...<br>
<span class="im HOEnZb"><br>
--<br>
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20<br>
phk@FreeBSD.ORG | TCP/IP since RFC 956<br>
FreeBSD committer | BSD since 4.3-tahoe<br>
Never attribute to malice what can adequately be explained by incompetence.<br>
<br>
</span><div class="HOEnZb"><div class="h5">______________________________<wbr>_________________<br>
varnish-dev mailing list<br>
<a href="mailto:varnish-dev@varnish-cache.org">varnish-dev@varnish-cache.org</a><br>
<a href="https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev" rel="noreferrer" target="_blank">https://www.varnish-cache.org/<wbr>lists/mailman/listinfo/<wbr>varnish-dev</a><br>
</div></div></blockquote></div><br></div>