<div dir="auto"><div>Hi Jakob,<div dir="auto"><br></div><div dir="auto">(Sorry i didn't see that email sooner, it was in my spam folder)</div><div dir="auto"><br></div>Looking at the log, I'm not sure what varnish should be loud about :-)</div><div dir="auto">204 is a success code, and more importantly it's generated by the backend, so varnish is happily passing it along.</div><div dir="auto"><br></div><div dir="auto">At the http level, everything looks about right, but I can guess from your apparent irritation that something wrong one level up, let's try to debug that.</div><div dir="auto"><br></div><div dir="auto">What kind of response are you expecting, if not a 204? And maybe, what is that endpoint supposed to do? Given that the method was GET, and that there's no body, my only guess is that there's something happening with the TeamCity-AgentSessionId header, maybe?</div><div dir="auto">Is the 27 seconds processing time expected?</div><div dir="auto"><br></div><div dir="auto">Cheers,<br><br><div class="gmail_quote" dir="auto"><div dir="ltr" class="gmail_attr">On Tue, May 9, 2023, 15:12 Jakob Bohm <<a href="mailto:jb-varnish@wisemo.com">jb-varnish@wisemo.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Dear Varnish mailing list,<br>
<br>
When testing varnish as a reverse proxy for multiple services<br>
including a local JetBrains TeamCity instance, requests to that<br>
teamcity server get corrupted into "204 No Content" replies.<br>
<br>
Once again, Varnish fails to say why it is refusing to do its job.<br>
Any sane program should explicitly and loudly report any fatal error<br>
that stops it working. Loudly means the sysadmin or other user<br>
invoking the program receives the exaxt error message by default<br>
instead of something highly indirect, hidden behind a debug option<br>
or otherwisse highly non-obvious.<br>
<br>
Here's a relevant clip from the VCL:<br>
<br>
# Various top comments<br>
vcl 4.1;<br>
<br>
import std;<br>
import proxy;<br>
<br>
# Backend sending requests to the teamcity main server<br>
backend teamcity {<br>
.host = "2a01:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx";<br>
.port = "8111";<br>
}<br>
<br>
# IP ranges allowed to access the build server and staging server<br>
acl buildtrust {<br>
"127.0.0.0"/8;<br>
"::"/128;<br>
"various others"/??;<br>
}<br>
<br>
# IP ranges allowed to attempt login to things that use our common login<br>
# database<br>
acl logintrust {<br>
"various others"/??;<br>
}<br>
<br>
sub vcl_recv {<br>
# Happens before we check if we have this in cache already.<br>
#<br>
# Typically you clean up the request here, removing cookies you <br>
don't need,<br>
# rewriting the request, etc.<br>
if (proxy.is_ssl()) {<br>
set req.http.Scheme = "https";<br>
set req.http.ssl-version = proxy.ssl_version();<br>
set req.http.X-Forwarded-Proto = "https";<br>
set req.http.X-SSL-cipher = proxy.ssl_cipher();<br>
std.log("TLS-SSL-VERSION: " + proxy.ssl_version());<br>
} else {<br>
set req.http.X-Forwarded-Proto = req.http.Scheme;<br>
unset req.http.ssl-version;<br>
unset req.http.X-SSL-cipher;<br>
std.log("TLS-SSL-VERSION: none");<br>
}<br>
unset req.http.X-SSL-Subject;<br>
unset req.http.X-SSL-Issuer;<br>
unset req.http.X-SSL-notBefore;<br>
unset req.http.X-SSL-notAfter;<br>
unset req.http.X-SSL-serial;<br>
unset req.http.X-SSL-certificate;<br>
<br>
set req.http.X-Forwarded-For = client.ip;<br>
<br>
call vcl_req_host;<br>
<br>
if (req.url ~ "^/something") {<br>
set req.backend_hint = be1;<br>
} else if (req.url !~ "^/somethingelse" &&<br>
!(client.ip ~ logintrust) &&<br>
!(client.ip ~ buildtrust)) {<br>
# Treat as unknown by redirecting to public website<br>
if ((req.url ~ "^/yeatanother") ||<br>
(req.url ~ "^/yetsomeother")) {<br>
return (synth(752));<br>
} else if (req.url ~ "^/yetsomethird") {<br>
return (synth(753));<br>
}<br>
return (synth(751));<br>
} else if (req.http.Scheme && req.http.Scheme != "https") {<br>
# See example at <br>
<a href="https://www.varnish-software.com/developers/tutorials/redirect/" rel="noreferrer noreferrer" target="_blank">https://www.varnish-software.com/developers/tutorials/redirect/</a><br>
return (synth(750));<br>
} else if (req.url ~ "^/somethingelse") {<br>
set req.backend_hint = be1;<br>
} else if (req.url ~ "^/somethingfourth") {<br>
set req.backend_hint = be2;<br>
} else if (req.url ~ "^/somethingfifth") {<br>
set req.backend_hint = be2;<br>
} else if (!(client.ip ~ buildtrust)) {<br>
# Treat as unknown by redirecting to public website<br>
if ((req.url ~ "^/yeatanother") ||<br>
(req.url ~ "^/yetsomeother")) {<br>
return (synth(752));<br>
} else if (req.url ~ "^/yetsomethird") {<br>
return (synth(753));<br>
}<br>
return (synth(751));<br>
} else if (req.url ~ "^/teamcity") {<br>
set req.backend_hint= teamcity;<br>
return (pass);<br>
# } else if (req.http.host ~ "^somethingsixths") {<br>
# set req.backend_hint= be4;<br>
} else {<br>
set req.backend_hint = be5;<br>
}<br>
call vcl_req_method;<br>
call vcl_req_authorization;<br>
call vcl_req_cookie;<br>
return (hash);<br>
}<br>
<br>
sub vcl_backend_response {<br>
# Happens after we have read the response headers from the backend.<br>
#<br>
# Here you clean the response headers, removing silly Set-Cookie <br>
headers<br>
# and other mistakes your backend does.<br>
<br>
# The Java webserver in teamcity is incompatible with varnish <br>
connection<br>
# pooling<br>
if (beresp.backend == teamcity) {<br>
if (beresp.http.Connection &&<br>
beresp.http.Connection !~ "keep-alive") {<br>
set beresp.http.Connection += ", close";<br>
} else {<br>
set beresp.http.Connection = "close";<br>
}<br>
}<br>
}<br>
<br>
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++<br>
First 43 lines of varnishlog -v 2>&1<br>
<br>
* << BeReq >> 9<br>
- 9 Begin b bereq 8 pass<br>
- 9 VCL_use b boot<br>
- 9 Timestamp b Start: 1681484803.177212 0.000000 0.000000<br>
- 9 BereqMethod b GET<br>
- 9 BereqURL b /teamcity/app/agents/v1/commands/next<br>
- 9 BereqProtocol b HTTP/1.1<br>
- 9 BereqHeader b TeamCity-AgentSessionId: <br>
L6juFAAt1awJDt6UKToPIxQq7wpBF89C<br>
- 9 BereqHeader b User-Agent: TeamCity Agent 2021.2.3<br>
- 9 BereqHeader b Host: <a href="http://vmachine.example.com" rel="noreferrer noreferrer" target="_blank">vmachine.example.com</a><br>
- 9 BereqHeader b Via: 1.1 vmachine (Varnish/7.2)<br>
- 9 BereqHeader b Scheme: https<br>
- 9 BereqHeader b ssl-version: TLSv1.3<br>
- 9 BereqHeader b X-Forwarded-Proto: https<br>
- 9 BereqHeader b X-SSL-cipher: TLS_AES_256_GCM_SHA384<br>
- 9 BereqHeader b X-Forwarded-For: 192.168.2.112<br>
- 9 BereqHeader b X-Varnish: 9<br>
- 9 VCL_call b BACKEND_FETCH<br>
- 9 VCL_return b fetch<br>
- 9 Timestamp b Fetch: 1681484803.177227 0.000014 0.000014<br>
- 9 Timestamp b Connected: 1681484803.177603 0.000390 <br>
0.000375<br>
- 9 BackendOpen b 24 teamcity <br>
2a01:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx 8111 2a01:yyyy:yyyy:yyyy::yyyy <br>
59548 connect<br>
- 9 Timestamp b Bereq: 1681484803.177645 0.000432 0.000042<br>
- 9 BerespReason b No Content<br>
- 9 Timestamp b Beresp: 1681484830.672487 27.495274 <br>
27.494842<br>
- 9 BerespProtocol b HTTP/1.1<br>
- 9 BerespStatus b 204<br>
- 9 BerespReason b No Content<br>
- 9 BerespHeader b TeamCity-Node-Id: MAIN_SERVER<br>
- 9 BerespHeader b Date: Fri, 14 Apr 2023 15:07:10 GMT<br>
- 9 VCL_call b BACKEND_RESPONSE<br>
- 9 BerespHeader b Connection: close<br>
- 9 VCL_return b deliver<br>
- 9 Timestamp b Process: 1681484830.672563 27.495350 <br>
0.000075<br>
- 9 Filters b<br>
- 9 Storage b malloc Transient<br>
- 9 Fetch_Body b 0 none -<br>
- 9 BackendClose b 24 teamcity close Backend/VCL requested <br>
close<br>
- 9 Timestamp b BerespBody: 1681484830.672926 27.495713 <br>
0.000362<br>
- 9 Length b 0<br>
- 9 BereqAcct b 345 0 345 85 0 85<br>
- 9 End b<br>
<br>
<br>
<br>
<br>
Enjoy<br>
<br>
Jakob<br>
-- <br>
Jakob Bohm, CIO, Partner, WiseMo A/S. <a href="https://www.wisemo.com" rel="noreferrer noreferrer" target="_blank">https://www.wisemo.com</a><br>
Transformervej 29, 2860 Søborg, Denmark. Direct +45 31 13 16 10<br>
This public discussion message is non-binding and may contain errors.<br>
WiseMo - Remote Service Management for PCs, Phones and Embedded <br>
<br>
_______________________________________________<br>
varnish-misc mailing list<br>
<a href="mailto:varnish-misc@varnish-cache.org" target="_blank" rel="noreferrer">varnish-misc@varnish-cache.org</a><br>
<a href="https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc" rel="noreferrer noreferrer" target="_blank">https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc</a><br>
</blockquote></div></div></div>