Many HTTP 304 responses results in fewer GET requests












1














I have a Django development server hosting a web-page that real-time (ish) displays information gathered from numerous servers I watch over. This web-page is still in development, so I am currently using the built-in web host provided with Django, started on an Ubuntu host with:



python3 manage.py runserver IP:Port



On the same ubuntu host there is a python script continuously reaching out to the monitored servers and formatting the responses into a .html file which the client reloads within a <div> every minute. The general functionality of the page the client accesses is as follows:



<div id="status" style="width:100%; height: 1000"></div>
<script>
$('#status').load("{% static 'alerts/status.html' %}");
setInterval(function() {
$('#status').load("{% static 'alerts/status.html' %}");
}, 60000);
</script>


...so the page loads the status.html file within the division on page-load, and then reloads it every minute. This has been working great, however, I have noticed looking at the Django log, that if status.html has not changed after ten status 304 (Not Modified) responses, the time waited between requests begins to roll-off. That is to say, instead of waiting 1 minute, it waits 2 minutes, then 5 minutes, and so on (roughly, I forget the actual rate of roll-off).



Now the issue I'm facing is that my server went down over the weekend (unrelated), but the display screen I had the web-page up on stayed active, so it rolled off so much that it seems it has completely broken, refusing to download the latest status.html, even when I force Chrome to reload everything and not use the cache (ctrl + R or shift + F5).



I tried researching this roll-off but couldn't find any information on it. I assume this is something built into Google Chrome (the browser I'm using) to save bandwidth when the page is not changing but my status page is a couple kilobytes at most and the 304 responses are already saving the little bandwidth that is so if there's a way to completely disable this roll-off for production that would be ideal.



In any case, any information on why I'm seeing this behavior / where it's coming from would be much appreciated as I can't seem to find any documentation on it. The closest thing I found was from Google's developer documentation on caching here. It mentions the ability to define maximum-age and no-cache behavior, so I could force the client to redownload status.html every minute, but this seems messy. While that would work in my specific scenario given status.html is a couple kilobytes at most, just disabling this roll-off behavior would do the trick and would keep unnecessary bandwidth down.










share|improve this question





























    1














    I have a Django development server hosting a web-page that real-time (ish) displays information gathered from numerous servers I watch over. This web-page is still in development, so I am currently using the built-in web host provided with Django, started on an Ubuntu host with:



    python3 manage.py runserver IP:Port



    On the same ubuntu host there is a python script continuously reaching out to the monitored servers and formatting the responses into a .html file which the client reloads within a <div> every minute. The general functionality of the page the client accesses is as follows:



    <div id="status" style="width:100%; height: 1000"></div>
    <script>
    $('#status').load("{% static 'alerts/status.html' %}");
    setInterval(function() {
    $('#status').load("{% static 'alerts/status.html' %}");
    }, 60000);
    </script>


    ...so the page loads the status.html file within the division on page-load, and then reloads it every minute. This has been working great, however, I have noticed looking at the Django log, that if status.html has not changed after ten status 304 (Not Modified) responses, the time waited between requests begins to roll-off. That is to say, instead of waiting 1 minute, it waits 2 minutes, then 5 minutes, and so on (roughly, I forget the actual rate of roll-off).



    Now the issue I'm facing is that my server went down over the weekend (unrelated), but the display screen I had the web-page up on stayed active, so it rolled off so much that it seems it has completely broken, refusing to download the latest status.html, even when I force Chrome to reload everything and not use the cache (ctrl + R or shift + F5).



    I tried researching this roll-off but couldn't find any information on it. I assume this is something built into Google Chrome (the browser I'm using) to save bandwidth when the page is not changing but my status page is a couple kilobytes at most and the 304 responses are already saving the little bandwidth that is so if there's a way to completely disable this roll-off for production that would be ideal.



    In any case, any information on why I'm seeing this behavior / where it's coming from would be much appreciated as I can't seem to find any documentation on it. The closest thing I found was from Google's developer documentation on caching here. It mentions the ability to define maximum-age and no-cache behavior, so I could force the client to redownload status.html every minute, but this seems messy. While that would work in my specific scenario given status.html is a couple kilobytes at most, just disabling this roll-off behavior would do the trick and would keep unnecessary bandwidth down.










    share|improve this question



























      1












      1








      1







      I have a Django development server hosting a web-page that real-time (ish) displays information gathered from numerous servers I watch over. This web-page is still in development, so I am currently using the built-in web host provided with Django, started on an Ubuntu host with:



      python3 manage.py runserver IP:Port



      On the same ubuntu host there is a python script continuously reaching out to the monitored servers and formatting the responses into a .html file which the client reloads within a <div> every minute. The general functionality of the page the client accesses is as follows:



      <div id="status" style="width:100%; height: 1000"></div>
      <script>
      $('#status').load("{% static 'alerts/status.html' %}");
      setInterval(function() {
      $('#status').load("{% static 'alerts/status.html' %}");
      }, 60000);
      </script>


      ...so the page loads the status.html file within the division on page-load, and then reloads it every minute. This has been working great, however, I have noticed looking at the Django log, that if status.html has not changed after ten status 304 (Not Modified) responses, the time waited between requests begins to roll-off. That is to say, instead of waiting 1 minute, it waits 2 minutes, then 5 minutes, and so on (roughly, I forget the actual rate of roll-off).



      Now the issue I'm facing is that my server went down over the weekend (unrelated), but the display screen I had the web-page up on stayed active, so it rolled off so much that it seems it has completely broken, refusing to download the latest status.html, even when I force Chrome to reload everything and not use the cache (ctrl + R or shift + F5).



      I tried researching this roll-off but couldn't find any information on it. I assume this is something built into Google Chrome (the browser I'm using) to save bandwidth when the page is not changing but my status page is a couple kilobytes at most and the 304 responses are already saving the little bandwidth that is so if there's a way to completely disable this roll-off for production that would be ideal.



      In any case, any information on why I'm seeing this behavior / where it's coming from would be much appreciated as I can't seem to find any documentation on it. The closest thing I found was from Google's developer documentation on caching here. It mentions the ability to define maximum-age and no-cache behavior, so I could force the client to redownload status.html every minute, but this seems messy. While that would work in my specific scenario given status.html is a couple kilobytes at most, just disabling this roll-off behavior would do the trick and would keep unnecessary bandwidth down.










      share|improve this question















      I have a Django development server hosting a web-page that real-time (ish) displays information gathered from numerous servers I watch over. This web-page is still in development, so I am currently using the built-in web host provided with Django, started on an Ubuntu host with:



      python3 manage.py runserver IP:Port



      On the same ubuntu host there is a python script continuously reaching out to the monitored servers and formatting the responses into a .html file which the client reloads within a <div> every minute. The general functionality of the page the client accesses is as follows:



      <div id="status" style="width:100%; height: 1000"></div>
      <script>
      $('#status').load("{% static 'alerts/status.html' %}");
      setInterval(function() {
      $('#status').load("{% static 'alerts/status.html' %}");
      }, 60000);
      </script>


      ...so the page loads the status.html file within the division on page-load, and then reloads it every minute. This has been working great, however, I have noticed looking at the Django log, that if status.html has not changed after ten status 304 (Not Modified) responses, the time waited between requests begins to roll-off. That is to say, instead of waiting 1 minute, it waits 2 minutes, then 5 minutes, and so on (roughly, I forget the actual rate of roll-off).



      Now the issue I'm facing is that my server went down over the weekend (unrelated), but the display screen I had the web-page up on stayed active, so it rolled off so much that it seems it has completely broken, refusing to download the latest status.html, even when I force Chrome to reload everything and not use the cache (ctrl + R or shift + F5).



      I tried researching this roll-off but couldn't find any information on it. I assume this is something built into Google Chrome (the browser I'm using) to save bandwidth when the page is not changing but my status page is a couple kilobytes at most and the 304 responses are already saving the little bandwidth that is so if there's a way to completely disable this roll-off for production that would be ideal.



      In any case, any information on why I'm seeing this behavior / where it's coming from would be much appreciated as I can't seem to find any documentation on it. The closest thing I found was from Google's developer documentation on caching here. It mentions the ability to define maximum-age and no-cache behavior, so I could force the client to redownload status.html every minute, but this seems messy. While that would work in my specific scenario given status.html is a couple kilobytes at most, just disabling this roll-off behavior would do the trick and would keep unnecessary bandwidth down.







      django google-chrome django-staticfiles http-caching






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Nov 13 at 6:19









      Kevin Christopher Henry

      22.5k46361




      22.5k46361










      asked Nov 12 at 20:19









      Eric

      446




      446
























          2 Answers
          2






          active

          oldest

          votes


















          1














          The problem here is that the response for status.html doesn't have an explicit cache expiration header. In the absence of such a header, the browser is free to use its own algorithm (such as the roll-off you're seeing) to choose an expiration time. From RFC 7234:




          Since origin servers do not always provide explicit expiration times, a cache MAY assign a heuristic expiration time when an explicit time is not specified.... This specification does not provide specific algorithms.




          So the solution is straightforward: assign an explicit cache expiration time.



          Implementing this solution is unfortunately not trivial using Django's staticfiles app. A better default for this app would be to not cache the results at all, but that solution was deferred pending a merger with whitenoise.



          Solutions include using a different server (like nginx); using a different app (like whitenoise); or using static views directly rather than the staticfiles app (see this question for a few approaches).






          share|improve this answer





















          • Thanks for the complete explanation. I decided, instead of fighting with Django and risking messing up my other views, to just disable the cache completely there with $.ajaxSetup ({cache: false}); in the <script> in my question. My view is only a few kilobytes pulled once a minute so it's not worth the struggle to save bandwidth on. If I (or someone else) has this issue with a higher-bandwidth scenario the suggested solutions are very helpful though so I'm accepting this as the answer.
            – Eric
            Nov 13 at 18:28





















          0














          Try this:



          // Page reload every 60 seconds
          setInterval(function(){
          location.reload();
          }, 60000);





          share|improve this answer





















          • I am having difficulties getting this to use the cache. I have reduced the update time to 2s for testing and added 'Expires' = '10' as a header to the django view for this page - I expected this to result in 5 status 304s before another status 200 but instead it is downloading status.html every time. Do you know what else I'm missing? I have also tried using reload(false) to be sure it shouldn't use cache.
            – Eric
            Nov 12 at 23:23










          • Also - ideally I would only have to re-download status.html, this as-is downloads the whole page each time (so .css and .js files that aren't changing are now being re-downloaded when they don't have to). Is there any way to restrict this to the <div> in question?
            – Eric
            Nov 12 at 23:25











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53269502%2fmany-http-304-responses-results-in-fewer-get-requests%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          2 Answers
          2






          active

          oldest

          votes








          2 Answers
          2






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          1














          The problem here is that the response for status.html doesn't have an explicit cache expiration header. In the absence of such a header, the browser is free to use its own algorithm (such as the roll-off you're seeing) to choose an expiration time. From RFC 7234:




          Since origin servers do not always provide explicit expiration times, a cache MAY assign a heuristic expiration time when an explicit time is not specified.... This specification does not provide specific algorithms.




          So the solution is straightforward: assign an explicit cache expiration time.



          Implementing this solution is unfortunately not trivial using Django's staticfiles app. A better default for this app would be to not cache the results at all, but that solution was deferred pending a merger with whitenoise.



          Solutions include using a different server (like nginx); using a different app (like whitenoise); or using static views directly rather than the staticfiles app (see this question for a few approaches).






          share|improve this answer





















          • Thanks for the complete explanation. I decided, instead of fighting with Django and risking messing up my other views, to just disable the cache completely there with $.ajaxSetup ({cache: false}); in the <script> in my question. My view is only a few kilobytes pulled once a minute so it's not worth the struggle to save bandwidth on. If I (or someone else) has this issue with a higher-bandwidth scenario the suggested solutions are very helpful though so I'm accepting this as the answer.
            – Eric
            Nov 13 at 18:28


















          1














          The problem here is that the response for status.html doesn't have an explicit cache expiration header. In the absence of such a header, the browser is free to use its own algorithm (such as the roll-off you're seeing) to choose an expiration time. From RFC 7234:




          Since origin servers do not always provide explicit expiration times, a cache MAY assign a heuristic expiration time when an explicit time is not specified.... This specification does not provide specific algorithms.




          So the solution is straightforward: assign an explicit cache expiration time.



          Implementing this solution is unfortunately not trivial using Django's staticfiles app. A better default for this app would be to not cache the results at all, but that solution was deferred pending a merger with whitenoise.



          Solutions include using a different server (like nginx); using a different app (like whitenoise); or using static views directly rather than the staticfiles app (see this question for a few approaches).






          share|improve this answer





















          • Thanks for the complete explanation. I decided, instead of fighting with Django and risking messing up my other views, to just disable the cache completely there with $.ajaxSetup ({cache: false}); in the <script> in my question. My view is only a few kilobytes pulled once a minute so it's not worth the struggle to save bandwidth on. If I (or someone else) has this issue with a higher-bandwidth scenario the suggested solutions are very helpful though so I'm accepting this as the answer.
            – Eric
            Nov 13 at 18:28
















          1












          1








          1






          The problem here is that the response for status.html doesn't have an explicit cache expiration header. In the absence of such a header, the browser is free to use its own algorithm (such as the roll-off you're seeing) to choose an expiration time. From RFC 7234:




          Since origin servers do not always provide explicit expiration times, a cache MAY assign a heuristic expiration time when an explicit time is not specified.... This specification does not provide specific algorithms.




          So the solution is straightforward: assign an explicit cache expiration time.



          Implementing this solution is unfortunately not trivial using Django's staticfiles app. A better default for this app would be to not cache the results at all, but that solution was deferred pending a merger with whitenoise.



          Solutions include using a different server (like nginx); using a different app (like whitenoise); or using static views directly rather than the staticfiles app (see this question for a few approaches).






          share|improve this answer












          The problem here is that the response for status.html doesn't have an explicit cache expiration header. In the absence of such a header, the browser is free to use its own algorithm (such as the roll-off you're seeing) to choose an expiration time. From RFC 7234:




          Since origin servers do not always provide explicit expiration times, a cache MAY assign a heuristic expiration time when an explicit time is not specified.... This specification does not provide specific algorithms.




          So the solution is straightforward: assign an explicit cache expiration time.



          Implementing this solution is unfortunately not trivial using Django's staticfiles app. A better default for this app would be to not cache the results at all, but that solution was deferred pending a merger with whitenoise.



          Solutions include using a different server (like nginx); using a different app (like whitenoise); or using static views directly rather than the staticfiles app (see this question for a few approaches).







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Nov 13 at 6:13









          Kevin Christopher Henry

          22.5k46361




          22.5k46361












          • Thanks for the complete explanation. I decided, instead of fighting with Django and risking messing up my other views, to just disable the cache completely there with $.ajaxSetup ({cache: false}); in the <script> in my question. My view is only a few kilobytes pulled once a minute so it's not worth the struggle to save bandwidth on. If I (or someone else) has this issue with a higher-bandwidth scenario the suggested solutions are very helpful though so I'm accepting this as the answer.
            – Eric
            Nov 13 at 18:28




















          • Thanks for the complete explanation. I decided, instead of fighting with Django and risking messing up my other views, to just disable the cache completely there with $.ajaxSetup ({cache: false}); in the <script> in my question. My view is only a few kilobytes pulled once a minute so it's not worth the struggle to save bandwidth on. If I (or someone else) has this issue with a higher-bandwidth scenario the suggested solutions are very helpful though so I'm accepting this as the answer.
            – Eric
            Nov 13 at 18:28


















          Thanks for the complete explanation. I decided, instead of fighting with Django and risking messing up my other views, to just disable the cache completely there with $.ajaxSetup ({cache: false}); in the <script> in my question. My view is only a few kilobytes pulled once a minute so it's not worth the struggle to save bandwidth on. If I (or someone else) has this issue with a higher-bandwidth scenario the suggested solutions are very helpful though so I'm accepting this as the answer.
          – Eric
          Nov 13 at 18:28






          Thanks for the complete explanation. I decided, instead of fighting with Django and risking messing up my other views, to just disable the cache completely there with $.ajaxSetup ({cache: false}); in the <script> in my question. My view is only a few kilobytes pulled once a minute so it's not worth the struggle to save bandwidth on. If I (or someone else) has this issue with a higher-bandwidth scenario the suggested solutions are very helpful though so I'm accepting this as the answer.
          – Eric
          Nov 13 at 18:28















          0














          Try this:



          // Page reload every 60 seconds
          setInterval(function(){
          location.reload();
          }, 60000);





          share|improve this answer





















          • I am having difficulties getting this to use the cache. I have reduced the update time to 2s for testing and added 'Expires' = '10' as a header to the django view for this page - I expected this to result in 5 status 304s before another status 200 but instead it is downloading status.html every time. Do you know what else I'm missing? I have also tried using reload(false) to be sure it shouldn't use cache.
            – Eric
            Nov 12 at 23:23










          • Also - ideally I would only have to re-download status.html, this as-is downloads the whole page each time (so .css and .js files that aren't changing are now being re-downloaded when they don't have to). Is there any way to restrict this to the <div> in question?
            – Eric
            Nov 12 at 23:25
















          0














          Try this:



          // Page reload every 60 seconds
          setInterval(function(){
          location.reload();
          }, 60000);





          share|improve this answer





















          • I am having difficulties getting this to use the cache. I have reduced the update time to 2s for testing and added 'Expires' = '10' as a header to the django view for this page - I expected this to result in 5 status 304s before another status 200 but instead it is downloading status.html every time. Do you know what else I'm missing? I have also tried using reload(false) to be sure it shouldn't use cache.
            – Eric
            Nov 12 at 23:23










          • Also - ideally I would only have to re-download status.html, this as-is downloads the whole page each time (so .css and .js files that aren't changing are now being re-downloaded when they don't have to). Is there any way to restrict this to the <div> in question?
            – Eric
            Nov 12 at 23:25














          0












          0








          0






          Try this:



          // Page reload every 60 seconds
          setInterval(function(){
          location.reload();
          }, 60000);





          share|improve this answer












          Try this:



          // Page reload every 60 seconds
          setInterval(function(){
          location.reload();
          }, 60000);






          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Nov 12 at 20:26









          Scott Skiles

          5431723




          5431723












          • I am having difficulties getting this to use the cache. I have reduced the update time to 2s for testing and added 'Expires' = '10' as a header to the django view for this page - I expected this to result in 5 status 304s before another status 200 but instead it is downloading status.html every time. Do you know what else I'm missing? I have also tried using reload(false) to be sure it shouldn't use cache.
            – Eric
            Nov 12 at 23:23










          • Also - ideally I would only have to re-download status.html, this as-is downloads the whole page each time (so .css and .js files that aren't changing are now being re-downloaded when they don't have to). Is there any way to restrict this to the <div> in question?
            – Eric
            Nov 12 at 23:25


















          • I am having difficulties getting this to use the cache. I have reduced the update time to 2s for testing and added 'Expires' = '10' as a header to the django view for this page - I expected this to result in 5 status 304s before another status 200 but instead it is downloading status.html every time. Do you know what else I'm missing? I have also tried using reload(false) to be sure it shouldn't use cache.
            – Eric
            Nov 12 at 23:23










          • Also - ideally I would only have to re-download status.html, this as-is downloads the whole page each time (so .css and .js files that aren't changing are now being re-downloaded when they don't have to). Is there any way to restrict this to the <div> in question?
            – Eric
            Nov 12 at 23:25
















          I am having difficulties getting this to use the cache. I have reduced the update time to 2s for testing and added 'Expires' = '10' as a header to the django view for this page - I expected this to result in 5 status 304s before another status 200 but instead it is downloading status.html every time. Do you know what else I'm missing? I have also tried using reload(false) to be sure it shouldn't use cache.
          – Eric
          Nov 12 at 23:23




          I am having difficulties getting this to use the cache. I have reduced the update time to 2s for testing and added 'Expires' = '10' as a header to the django view for this page - I expected this to result in 5 status 304s before another status 200 but instead it is downloading status.html every time. Do you know what else I'm missing? I have also tried using reload(false) to be sure it shouldn't use cache.
          – Eric
          Nov 12 at 23:23












          Also - ideally I would only have to re-download status.html, this as-is downloads the whole page each time (so .css and .js files that aren't changing are now being re-downloaded when they don't have to). Is there any way to restrict this to the <div> in question?
          – Eric
          Nov 12 at 23:25




          Also - ideally I would only have to re-download status.html, this as-is downloads the whole page each time (so .css and .js files that aren't changing are now being re-downloaded when they don't have to). Is there any way to restrict this to the <div> in question?
          – Eric
          Nov 12 at 23:25


















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.





          Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


          Please pay close attention to the following guidance:


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53269502%2fmany-http-304-responses-results-in-fewer-get-requests%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Bressuire

          Vorschmack

          Quarantine