Vfrdtyky

I am designing a ReST API which follows the basic CRUD pattern.

My API can receive a request to update a resource which may take a short time to process. Ideally I would like to inform clients that a new version is about to be available and that there is some uncertainty over when the version I have cached actually expires.

So the process I intend to use something like this (improvements welcome):

client: GET /some/item

myapi:  200 OK

        last-modified: time-stamp-of-v1

        etag: some-hash-relating-to-v1-of-my-item-in-this-format

        content: json or whatever

         data/for/some/item/v1... 



client: PUT /some/item

        if-match: some-hash-relating-to-v1-of-my-item-in-this-format

        content: json or whatever

         data/for/some/item/v2... 



myapi:  202 ACCEPTED,   

        content: json or whatever

         time-accepted: time-stamp-after-v1-but-before-v2

         your item will be at /some/item

         here is a URI /some/taskid to track progress

while upload is pending:

client: GET /some/item

myapi:  200 OK

        some/item ... 

        last-modified: time-stamp-of-v1

        etag: some-hash-relating-to-v1-of-my-item-in-this-format

>>>>    expires: time-stamp-after-v1-but-before-v2 <<<

>>>>    warning: 110 Response is stale    <<<<

        content: json or whatever

         data/for/some/item/v1... 



client: GET /some/task/id

myapi:  200 OK

        content: json or whatever

         time-accepted: time-stamp-after-v1-but-before-v2

         your item will be at /some/item

         status/of/upload/v2... 

after task completed:

client: GET /some/item

myapi:  200 OKAY

        some/item/v2 ... 

        last-modified: time-stamp-of-v2

        etag: some-hash-relating-to-v2-of-my-item-in-this-format

        content: json or whatever

         data/for/some/item/v2... 



client: GET /some/task/id

myapi:  303 SEE OTHER

         look-here: /some/item

If you are a proxy and know know your content is stale you can put "warning: 110 - response is stale" in the header.
However, in this case the data is not actually invalid yet.
I would like to say that I can guarantee it is valid up until the time I received and passed on the upload request (time-stamp-after-v1-but-before-v2 or later as if I am in contact with the upload server). It hasn't really expired at the time I receive the upload request. I just expect its going to.
(In fact if the request fails it might not be updated at all).

Now the default choice is just to serve the old content and let the client catch up on its own. This has high latency. If possible, I would like to do better.

For example, if the client knows the document is about to expire it could poll more often or it could try to upgrade the connection to a web-socket and get sent an update the moment I get it (would that still count as ReST?)

There is another case where using expired data must be avoided at all costs. For that scenario I think I want to tell the client that the resource is temporarily unavailable. Using the warning and expires fields as I have above seems correct there. Though it might be better to send a 503 with a suitable retry-after header.

So the question is: how should I reply to a GET while the upload of a new version is pending?

In anticipation of answers along the lines of use a messaging framework like AMQP or zeroMQ instead for low latency, I should point out this API is acting as a AMQP gateway/proxy for clients unwilling to use AMQP directly. Information on using webhooks or websockets would be still be interesting.

Some related useful content is:

• How to proper design a restful API to invalidate a cache?


• https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html


• HTTP status code for temporarily unavailable pages


• 
  http://www.albertoleal.me/posts/how-to-prevent-race-conditions-in-restful-apis.html
  (the etag prevents races from simultaneously uploads)

Tl;Dr;

While upload is pending send:

client: GET /some/item

myapi:  200 OK

        some/item ... 

        last-modified: time-stamp-of-v1

        etag: some-hash-relating-to-v1-of-my-item-in-this-format

        expires: time-stamp-after-v1-but-before-v2

        stale-while-revalidate: 100

        warning: 110 Response is stale

        content: json or whatever

         data/for/some/item/v1... 

At first sight it looks like using Warning is not correct. See https://tools.ietf.org/html/rfc7234#section-5.5.0

In this case the server is acting as a proxy (though not an HTTP proxy).
It is not disconnected from AMQP and "A proxy MUST NOT send stale responses" unless it is disconnected.
This is annoying as it looked like the right thing to do here.

4.2.4. Serving Stale Responses

A "stale" response is one that either has explicit expiry
information or is allowed to have heuristic expiry calculated, but
is not fresh according to the calculations in Section 4.2.

A cache MUST NOT generate a stale response if it is prohibited by
an explicit in-protocol directive (e.g., by a "no-store" or
"no-cache" cache directive, a "must-revalidate"
cache-response-directive, or an applicable "s-maxage" or
"proxy-revalidate" cache-response-directive; see Section 5.2.2).

> A cache MUST NOT send stale responses unless it is disconnected

(i.e., it cannot contact the origin server or otherwise find a

forward path) or doing so is explicitly allowed (e.g., by the

max-stale request directive; see Section 5.2.1).

A cache SHOULD generate a Warning header field with the 110
warn-code (see Section 5.5.1) in stale responses. Likewise, a
cache SHOULD generate a 112 warn-code (see Section 5.5.3) in stale
responses if the cache is disconnected.

A cache SHOULD NOT generate a new Warning header field when

forwarding a response that does not have an Age header field, even if
the response is already stale. A cache need not validate a response

that merely became stale in transit.

Also

4.4. Invalidation

Because unsafe request methods (Section 4.2.1 of [RFC7231]) such as
PUT, POST or DELETE have the potential for changing state on the

origin server, intervening caches can use them to keep their contents
up to date.

> A cache MUST invalidate the effective Request URI (Section 5.5 of

[RFC7230]) as well as the URI(s) in the Location and Content-Location
response header fields (if present) when a non-error status code is

received in response to an unsafe request method.

However a warning is required if stale-while-revalidate is used (see https://tools.ietf.org/html/rfc5861)

3. 
   

   The stale-while-revalidate Cache-Control Extension

   
   
   

   When present in an HTTP response, the stale-while-revalidate Cache-
   Control extension indicates that caches MAY serve the response in
   
   which it appears after it becomes stale, up to the indicated number
   
   of seconds.

   
   
   

   stale-while-revalidate = "stale-while-revalidate" "=" delta-seconds

   
   
   

   If a cached response is served stale due to the presence of this
   
   extension, the cache SHOULD attempt to revalidate it while still
   
   serving stale responses (i.e., without blocking).

   
   

I thought this was unclear so I submitted an errata. This was rejected (though at the time of writing its still showing as reported) on the grounds that the cache control extensions in rfc5861 override the MUST NOT in rfc7234 ("doing so is explicitly allowed" see above).

It is okay to use expires but its not very helpful as it doesn't imply anything.

5.3. Expires

The "Expires" header field gives the date/time after which the

response is considered stale. See Section 4.2 for further discussion
of the freshness model.

> The presence of an Expires field does not imply that the original

resource will change or cease to exist at, before, or after that

time.