Vfrdtyky

I'm issuing an S3 CopyObject (aka PUT copy) from a source bucket that is different than the destination bucket. I'm wondering if it's safe to delete the source after CopyObject has returned OK to the REST client. By "safe", I mean that the destination object will eventually show up, and that it will initially contain all of the data available at the time the copy was issued.

Corruption in the copy is perhaps unlikely (given that most operations are atomic), but the two operations cancelling each other might be possible. I wish the docs were slightly more fleshed out wrt to eventual consistency.

(in my scenario, nothing's writing to the source nor destination key in the interim. and the same client does the copy and delete).

e.g. synchronous pseudocode:

try:



    # make sure this is a create. read-after-create consistency

    my_tmpname = "new_tempfile" + uuid4()

    s3_put(data, "s3://my-bucket1/" + my_tmpname)



    ...



    # copy it to its final location

    s3_copy("s3://my-bucket1/new_tempfile", "s3://my-bucket2/final_location")

finally:

    # Cleanup temp file.

    #

    # Can this delete interfere with the copy in flight?

    # e.g. Should one wait a few seconds/minutes?

    # e.g. Should one ensure that the target exists before deleting source?

    s3_delete("s3://my-bucket/new_tempfile")

Yes, it is perfectly safe to delete an object immediately after a successful copy operation using that object as the source object, because copy operations are not asynchronous.

The copy request does not return until the operation has either succeeded or failed.

To help better ensure data durability, Amazon S3 PUT and PUT Object copy operations synchronously store your data across multiple facilities before returning SUCCESS.

https://docs.aws.amazon.com/AmazonS3/latest/dev/DataDurability.html

The consistency model in S3 is related only to the visibility of objects, not the durability of their storage.

Amazon S3 features read-after-write consistency for PUTs and the copy case you exemplified:

Amazon S3 Data Consistency Model

Amazon S3 provides read-after-write consistency for PUTS of new
objects in your S3 bucket in all regions with one caveat. The caveat
is that if you make a HEAD or GET request to the key name (to find if
the object exists) before creating the object, Amazon S3 provides
eventual consistency for read-after-write.

Check the documentation for consistency of the other operations.

https://docs.aws.amazon.com/AmazonS3/latest/dev/Introduction.html