Enhanced PHP File Manager

⬆️ Go Up (..)

📊 Server Information

Server:

nginx/1.28.0

System:

Linux 6.8.0-90-generic

PHP Version:

8.4.15

User:

alqudsvolunteers

Directory:

/home/alqudsvolunteers/rootlink/proc/thread-self/root/usr/share/doc/varnish/html/users-guide

Free Space:

80.18 GB

🔧 Quick Actions

✏️ Editing: troubleshooting.html

<!DOCTYPE html>

<title>Troubleshooting Varnish &#8212; Varnish version 7.5.0 documentation</title>
    <link rel="stylesheet" type="text/css" href="../_static/pygments.css?v=fa44fd50" />
    <link rel="stylesheet" type="text/css" href="../_static/classic.css?v=e2714048" />
    
    <script src="../_static/documentation_options.js?v=8214db13"></script>
    <script src="../_static/doctools.js?v=888ff710"></script>
    <script src="../_static/sphinx_highlight.js?v=dc90522c"></script>
    
    <link rel="index" title="Index" href="../genindex.html" />
    <link rel="search" title="Search" href="../search.html" />
    <link rel="next" title="The Varnish Reference Manual" href="../reference/index.html" />
    <link rel="prev" title="Content composition with Edge Side Includes" href="esi.html" /> 
  </head><body>
    <div class="related" role="navigation" aria-label="related navigation">
      <h3>Navigation</h3>
      <ul>
        <li class="right" style="margin-right: 10px">
          <a href="../genindex.html" title="General Index"
             accesskey="I">index</a></li>
        <li class="right" >
          <a href="../reference/index.html" title="The Varnish Reference Manual"
             accesskey="N">next</a> |</li>
        <li class="right" >
          <a href="esi.html" title="Content composition with Edge Side Includes"
             accesskey="P">previous</a> |</li>
        <li class="nav-item nav-item-0"><a href="../index.html">Varnish version 7.5.0 documentation</a> &#187;</li>
          <li class="nav-item nav-item-1"><a href="index.html" accesskey="U">The Varnish Users Guide</a> &#187;</li>
        <li class="nav-item nav-item-this"><a href="">Troubleshooting Varnish</a></li> 
      </ul>
    </div>

<div class="document">
      <div class="documentwrapper">
        <div class="bodywrapper">
          <div class="body" role="main">
            
  <section id="troubleshooting-varnish">
<span id="users-trouble"></span><h1>Troubleshooting Varnish<a class="headerlink" href="#troubleshooting-varnish" title="Link to this heading">¶</a></h1>
<p>Sometimes Varnish misbehaves or rather behaves the way you told it to
behave but not necessarily the way you want it to behave. In order for
you to understand whats going on there are a couple of places you can
check. <a class="reference internal" href="../reference/varnishlog.html#varnishlog-1"><span class="std std-ref">varnishlog</span></a>, <code class="docutils literal notranslate"><span class="pre">/var/log/syslog</span></code>,
<code class="docutils literal notranslate"><span class="pre">/var/log/messages</span></code> are all good places where Varnish might leave
clues of whats going on. This section will guide you through basic
troubleshooting in Varnish.</p>
<section id="when-varnish-won-t-start">
<h2>When Varnish won’t start<a class="headerlink" href="#when-varnish-won-t-start" title="Link to this heading">¶</a></h2>
<p>Sometimes Varnish wont start. There is a plethora of possible reasons why
Varnish wont start on your machine. We’ve seen everything from wrong
permissions on <code class="docutils literal notranslate"><span class="pre">/dev/null</span></code> to other processes blocking the ports.</p>
<p>Starting Varnish in debug mode to see what is going on.</p>
<p>Try to start Varnish with the same arguments as otherwise, but <code class="docutils literal notranslate"><span class="pre">-d</span></code>
added. This will give you some more information on what is going
on. Let us see how Varnish will react when something else is listening
on its port.:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># varnishd -n foo -f /usr/local/etc/varnish/default.vcl -s malloc,1G -T 127.0.0.1:2000  -a 0.0.0.0:8080 -d</span>
<span class="n">storage_malloc</span><span class="p">:</span> <span class="nb">max</span> <span class="n">size</span> <span class="mi">1024</span> <span class="n">MB</span><span class="o">.</span>
<span class="n">Using</span> <span class="n">old</span> <span class="n">SHMFILE</span>
<span class="n">Platform</span><span class="p">:</span> <span class="n">Linux</span><span class="p">,</span><span class="mf">2.6.32</span><span class="o">-</span><span class="mi">21</span><span class="o">-</span><span class="n">generic</span><span class="p">,</span><span class="n">i686</span><span class="p">,</span><span class="o">-</span><span class="n">smalloc</span><span class="p">,</span><span class="o">-</span><span class="n">hcritbit</span>
<span class="mi">200</span> <span class="mi">193</span>
<span class="o">-----------------------------</span>
<span class="n">Varnish</span> <span class="n">Cache</span> <span class="n">CLI</span><span class="o">.</span>
<span class="o">-----------------------------</span>
<span class="n">Type</span> <span class="s1">&#39;help&#39;</span> <span class="k">for</span> <span class="n">command</span> <span class="nb">list</span><span class="o">.</span>
<span class="n">Type</span> <span class="s1">&#39;quit&#39;</span> <span class="n">to</span> <span class="n">close</span> <span class="n">CLI</span> <span class="n">session</span><span class="o">.</span>
<span class="n">Type</span> <span class="s1">&#39;start&#39;</span> <span class="n">to</span> <span class="n">launch</span> <span class="n">worker</span> <span class="n">process</span><span class="o">.</span>
</pre></div>
</div>
<p>Now Varnish is running but only the master process is running, in debug
mode the cache does not start. Now you’re on the console. You can
instruct the master process to start the cache by issuing “start”.:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">start</span>
<span class="n">bind</span><span class="p">():</span> <span class="n">Address</span> <span class="n">already</span> <span class="ow">in</span> <span class="n">use</span>
<span class="mi">300</span> <span class="mi">22</span>
<span class="n">Could</span> <span class="ow">not</span> <span class="nb">open</span> <span class="n">sockets</span>
</pre></div>
</div>
<p>And here we have our problem. Something else is bound to the HTTP port
of Varnish. If this doesn’t help try <code class="docutils literal notranslate"><span class="pre">strace</span></code> or <code class="docutils literal notranslate"><span class="pre">truss</span></code> or come find us
on IRC.</p>
</section>
<section id="varnish-is-crashing-panics">
<h2>Varnish is crashing - panics<a class="headerlink" href="#varnish-is-crashing-panics" title="Link to this heading">¶</a></h2>
<p>When Varnish goes bust the child processes crashes. Most of the
crashes are caught by one of the many consistency checks we have
included in the Varnish source code. When Varnish hits one of these
the caching process will crash itself in a controlled manner, leaving
a nice stack trace with the mother process.</p>
<p>You can inspect any panic messages by typing <code class="docutils literal notranslate"><span class="pre">panic.show</span></code> in the CLI.:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">panic</span><span class="o">.</span><span class="n">show</span>
<span class="n">Last</span> <span class="n">panic</span> <span class="n">at</span><span class="p">:</span> <span class="n">Tue</span><span class="p">,</span> <span class="mi">15</span> <span class="n">Mar</span> <span class="mi">2011</span> <span class="mi">13</span><span class="p">:</span><span class="mi">09</span><span class="p">:</span><span class="mi">05</span> <span class="n">GMT</span>
<span class="n">Assert</span> <span class="n">error</span> <span class="ow">in</span> <span class="n">ESI_Deliver</span><span class="p">(),</span> <span class="n">cache_esi_deliver</span><span class="o">.</span><span class="n">c</span> <span class="n">line</span> <span class="mi">354</span><span class="p">:</span>
  <span class="n">Condition</span><span class="p">(</span><span class="n">i</span> <span class="o">==</span> <span class="n">Z_OK</span> <span class="o">||</span> <span class="n">i</span> <span class="o">==</span> <span class="n">Z_STREAM_END</span><span class="p">)</span> <span class="ow">not</span> <span class="n">true</span><span class="o">.</span>
<span class="n">thread</span> <span class="o">=</span> <span class="p">(</span><span class="n">cache</span><span class="o">-</span><span class="n">worker</span><span class="p">)</span>
<span class="n">ident</span> <span class="o">=</span> <span class="n">Linux</span><span class="p">,</span><span class="mf">2.6.32</span><span class="o">-</span><span class="mi">28</span><span class="o">-</span><span class="n">generic</span><span class="p">,</span><span class="n">x86_64</span><span class="p">,</span><span class="o">-</span><span class="n">sfile</span><span class="p">,</span><span class="o">-</span><span class="n">smalloc</span><span class="p">,</span><span class="o">-</span><span class="n">hcritbit</span><span class="p">,</span><span class="n">epoll</span>
<span class="n">Backtrace</span><span class="p">:</span>
  <span class="mh">0x42cbe8</span><span class="p">:</span> <span class="n">pan_ic</span><span class="o">+</span><span class="n">b8</span>
  <span class="mh">0x41f778</span><span class="p">:</span> <span class="n">ESI_Deliver</span><span class="o">+</span><span class="mi">438</span>
  <span class="mh">0x42f838</span><span class="p">:</span> <span class="n">RES_WriteObj</span><span class="o">+</span><span class="mi">248</span>
  <span class="mh">0x416a70</span><span class="p">:</span> <span class="n">cnt_deliver</span><span class="o">+</span><span class="mi">230</span>
  <span class="mh">0x4178fd</span><span class="p">:</span> <span class="n">CNT_Session</span><span class="o">+</span><span class="mi">31</span><span class="n">d</span>
  <span class="p">(</span><span class="o">..</span><span class="p">)</span>
</pre></div>
</div>
<p>The crash might be due to misconfiguration or a bug. If you suspect it
is a bug you can use the output in a bug report, see the “Trouble
Tickets” section in the Introduction chapter above.</p>
</section>
<section id="varnish-is-crashing-stack-overflows">
<h2>Varnish is crashing - stack overflows<a class="headerlink" href="#varnish-is-crashing-stack-overflows" title="Link to this heading">¶</a></h2>
<p>Bugs put aside, the most likely cause of crashes are stack overflows,
which is why we have added a heuristic to add a note when a crash
looks like it was caused by one. In this case, the panic message
contains something like this:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Signal</span> <span class="mi">11</span> <span class="p">(</span><span class="n">Segmentation</span> <span class="n">fault</span><span class="p">)</span> <span class="n">received</span> <span class="n">at</span> <span class="mh">0x7f631f1b2f98</span> <span class="n">si_code</span> <span class="mi">1</span>
<span class="n">THIS</span> <span class="n">PROBABLY</span> <span class="n">IS</span> <span class="n">A</span> <span class="n">STACK</span> <span class="n">OVERFLOW</span> <span class="o">-</span> <span class="n">check</span> <span class="n">thread_pool_stack</span> <span class="n">parameter</span>
</pre></div>
</div>
<p>as a first measure, please follow this advise and check if crashes
still occur when you add 128k to whatever the value of the
<code class="docutils literal notranslate"><span class="pre">thread_pool_stack</span></code> parameter and restart varnish.</p>
<p>If varnish stops crashing with a larger <code class="docutils literal notranslate"><span class="pre">thread_pool_stack</span></code>
parameter, it’s not a bug (at least most likely).</p>
</section>
<section id="varnish-is-crashing-segfaults">
<h2>Varnish is crashing - segfaults<a class="headerlink" href="#varnish-is-crashing-segfaults" title="Link to this heading">¶</a></h2>
<p>Sometimes a bug escapes the consistency checks and Varnish gets hit
with a segmentation error. When this happens with the child process it
is logged, the core is dumped and the child process starts up again.</p>
<p>A core dumped is usually due to a bug in Varnish. However, in order to
debug a segfault the developers need you to provide a fair bit of
data.</p>
<blockquote>
<div><ul>
<li><p>Make sure you have Varnish installed with debugging symbols.</p></li>
<li><p>Check where your operating system writes core files and ensure that
you actually get them. For example on linux, learn about
<code class="docutils literal notranslate"><span class="pre">/proc/sys/kernel/core_pattern</span></code> from the <cite>core(5)</cite> manpage.</p></li>
<li><p>Make sure core dumps are allowed in the parent shell from which
varnishd is being started. In shell, this would be:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">ulimit</span> <span class="o">-</span><span class="n">c</span> <span class="n">unlimited</span>
</pre></div>
</div>
<p>but if varnish is started from an init-script, that would need to
be adjusted or in the case of systemd, <code class="docutils literal notranslate"><span class="pre">LimitCORE=infinity</span></code> set
in the service’s <code class="docutils literal notranslate"><span class="pre">[Service]]</span></code> section of the unit file.</p>
</li>
</ul>
</div></blockquote>
<p>Once you have the core, <code class="docutils literal notranslate"><span class="pre">cd</span></code> into varnish’s working directory (as
given by the <code class="docutils literal notranslate"><span class="pre">-n</span></code> parameter, whose default is
<code class="docutils literal notranslate"><span class="pre">$PREFIX/var/varnish/$HOSTNAME</span></code> with <code class="docutils literal notranslate"><span class="pre">$PREFIX</span></code> being the
installation prefix, usually <code class="docutils literal notranslate"><span class="pre">/usr/local</span></code>, open the core with
<code class="docutils literal notranslate"><span class="pre">gdb</span></code> and issue the command <code class="docutils literal notranslate"><span class="pre">bt</span></code> to get a stack trace of the
thread that caused the segfault.</p>
<p>A basic debug session for varnish installed under <code class="docutils literal notranslate"><span class="pre">/usr/local</span></code> could look
like this:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ cd /usr/local/var/varnish/`uname -n`/
$ gdb /usr/local/sbin/varnishd core
GNU gdb (Debian 7.12-6) 7.12.0.20161007-git
Copyright (C) 2016 Free Software Foundation, Inc.
[...]
Core was generated by `/usr/local/sbin/varnishd -a 127.0.0.1:8080 -b 127.0.0.1:8080&#39;.
Program terminated with signal SIGABRT, Aborted.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
[Current thread is 1 (Thread 0x7f7749ea3700 (LWP 31258))]

(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007f775132342a in __GI_abort () at abort.c:89
#2  0x000000000045939f in pan_ic (func=0x7f77439fb811 &quot;VCL&quot;, file=0x7f77439fb74c &quot;&quot;, line=0,
    cond=0x7f7740098130 &quot;PANIC: deliberately!&quot;, kind=VAS_VCL) at cache/cache_panic.c:839
#3  0x0000000000518cb1 in VAS_Fail (func=0x7f77439fb811 &quot;VCL&quot;, file=0x7f77439fb74c &quot;&quot;, line=0,
    cond=0x7f7740098130 &quot;PANIC: deliberately!&quot;, kind=VAS_VCL) at vas.c:51
#4  0x00007f77439fa6e9 in vmod_panic (ctx=0x7f7749ea2068, str=0x7f7749ea2018) at vmod_vtc.c:109
#5  0x00007f77449fa5b8 in VGC_function_vcl_recv (ctx=0x7f7749ea2068) at vgc.c:1957
#6  0x0000000000491261 in vcl_call_method (wrk=0x7f7749ea2dd0, req=0x7f7740096020, bo=0x0,
    specific=0x0, method=2, func=0x7f77449fa550 &lt;VGC_function_vcl_recv&gt;) at cache/cache_vrt_vcl.c:462
#7  0x0000000000493025 in VCL_recv_method (vcl=0x7f775083f340, wrk=0x7f7749ea2dd0, req=0x7f7740096020,
    bo=0x0, specific=0x0) at ../../include/tbl/vcl_returns.h:192
#8  0x0000000000462979 in cnt_recv (wrk=0x7f7749ea2dd0, req=0x7f7740096020) at cache/cache_req_fsm.c:880
#9  0x0000000000461553 in CNT_Request (req=0x7f7740096020) at ../../include/tbl/steps.h:36
#10 0x00000000004a7fc6 in HTTP1_Session (wrk=0x7f7749ea2dd0, req=0x7f7740096020)
    at http1/cache_http1_fsm.c:417
#11 0x00000000004a72c3 in http1_req (wrk=0x7f7749ea2dd0, arg=0x7f7740096020)
    at http1/cache_http1_fsm.c:86
#12 0x0000000000496bb6 in Pool_Work_Thread (pp=0x7f774980e140, wrk=0x7f7749ea2dd0)
    at cache/cache_wrk.c:406
#13 0x00000000004963e3 in WRK_Thread (qp=0x7f774980e140, stacksize=57344, thread_workspace=2048)
    at cache/cache_wrk.c:144
#14 0x000000000049610b in pool_thread (priv=0x7f774880ec80) at cache/cache_wrk.c:439
#15 0x00007f77516954a4 in start_thread (arg=0x7f7749ea3700) at pthread_create.c:456
#16 0x00007f77513d7d0f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:97
</pre></div>
</div>
</section>
<section id="varnish-gives-me-guru-meditation">
<h2>Varnish gives me Guru meditation<a class="headerlink" href="#varnish-gives-me-guru-meditation" title="Link to this heading">¶</a></h2>
<p>First find the relevant log entries in <a class="reference internal" href="../reference/varnishlog.html#varnishlog-1"><span class="std std-ref">varnishlog</span></a>. That will
probably give you a clue. Since <a class="reference internal" href="../reference/varnishlog.html#varnishlog-1"><span class="std std-ref">varnishlog</span></a> logs a lot of
data it might be hard to track the entries down. You can set
<a class="reference internal" href="../reference/varnishlog.html#varnishlog-1"><span class="std std-ref">varnishlog</span></a> to log all your 503 errors by issuing the
following command:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ varnishlog -q &#39;RespStatus == 503&#39; -g request
</pre></div>
</div>
<p>If the error happened just a short time ago the transaction might
still be in the shared memory log segment. To get <a class="reference internal" href="../reference/varnishlog.html#varnishlog-1"><span class="std std-ref">varnishlog</span></a>
to process the whole shared memory log just add the ‘-d’ parameter:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span>$ varnishlog -d -q &#39;RespStatus == 503&#39; -g request
</pre></div>
</div>
<p>Please see the <a class="reference internal" href="../reference/vsl-query.html#vsl-query-7"><span class="std std-ref">vsl-query</span></a> and <a class="reference internal" href="../reference/varnishlog.html#varnishlog-1"><span class="std std-ref">varnishlog</span></a> man pages
for elaborations on further filtering capabilities and explanation of
the various options.</p>
</section>
<section id="varnish-doesn-t-cache">
<h2>Varnish doesn’t cache<a class="headerlink" href="#varnish-doesn-t-cache" title="Link to this heading">¶</a></h2>
<p>See <a class="reference internal" href="increasing-your-hitrate.html#users-guide-increasing-your-hitrate"><span class="std std-ref">Achieving a high hitrate</span></a>.</p>
</section>
</section>

<div class="clearer"></div>
          </div>
        </div>
      </div>
      <div class="sphinxsidebar" role="navigation" aria-label="main navigation">
        <div class="sphinxsidebarwrapper">
  <div>
    <h3><a href="../index.html">Table of Contents</a></h3>
    <ul>
<li><a class="reference internal" href="#">Troubleshooting Varnish</a><ul>
<li><a class="reference internal" href="#when-varnish-won-t-start">When Varnish won’t start</a></li>
<li><a class="reference internal" href="#varnish-is-crashing-panics">Varnish is crashing - panics</a></li>
<li><a class="reference internal" href="#varnish-is-crashing-stack-overflows">Varnish is crashing - stack overflows</a></li>
<li><a class="reference internal" href="#varnish-is-crashing-segfaults">Varnish is crashing - segfaults</a></li>
<li><a class="reference internal" href="#varnish-gives-me-guru-meditation">Varnish gives me Guru meditation</a></li>
<li><a class="reference internal" href="#varnish-doesn-t-cache">Varnish doesn’t cache</a></li>
</ul>
</li>
</ul>

</div>
  <div>
    <h4>Previous topic</h4>
    <p class="topless"><a href="esi.html"
                          title="previous chapter">Content composition with Edge Side Includes</a></p>
  </div>
  <div>
    <h4>Next topic</h4>
    <p class="topless"><a href="../reference/index.html"
                          title="next chapter">The Varnish Reference Manual</a></p>
  </div>
  <div role="note" aria-label="source link">
    <h3>This Page</h3>
    <ul class="this-page-menu">
      <li><a href="../_sources/users-guide/troubleshooting.rst.txt"
            rel="nofollow">Show Source</a></li>
    </ul>
   </div>
<div id="searchbox" style="display: none" role="search">
  <h3 id="searchlabel">Quick search</h3>
    <div class="searchformwrapper">
    <form class="search" action="../search.html" method="get">
      <input type="text" name="q" aria-labelledby="searchlabel" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
      <input type="submit" value="Go" />
    </form>
    </div>
</div>
<script>document.getElementById('searchbox').style.display = "block"</script>
        </div>
      </div>
      <div class="clearer"></div>
    </div>
    <div class="related" role="navigation" aria-label="related navigation">
      <h3>Navigation</h3>
      <ul>
        <li class="right" style="margin-right: 10px">
          <a href="../genindex.html" title="General Index"
             >index</a></li>
        <li class="right" >
          <a href="../reference/index.html" title="The Varnish Reference Manual"
             >next</a> |</li>
        <li class="right" >
          <a href="esi.html" title="Content composition with Edge Side Includes"
             >previous</a> |</li>
        <li class="nav-item nav-item-0"><a href="../index.html">Varnish version 7.5.0 documentation</a> &#187;</li>
          <li class="nav-item nav-item-1"><a href="index.html" >The Varnish Users Guide</a> &#187;</li>
        <li class="nav-item nav-item-this"><a href="">Troubleshooting Varnish</a></li> 
      </ul>
    </div>
    <div class="footer" role="contentinfo">
    &#169; Copyright 2010-2014, Varnish Software AS.
      Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> 7.2.6.
    </div>
  </body>
</html>

❌ Cancel

🗂️ Enhanced PHP File Manager

📊 Server Information

🔧 Quick Actions

✏️ Editing: troubleshooting.html

📤 Upload File

📁 Create Folder

📄 Create File

✏️ Rename Item

🗑️ Delete Confirmation