admin/3engines_doc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

link icon replaced

Browse Source
This commit is contained in:
govardhan
2025-06-19 14:09:10 +05:30
parent 60adbde60c
commit 172f8e2b34
158 changed files with 996 additions and 996 deletions
Download Patch File Download Diff File Expand all files Collapse all files

68
docs/s3/S3-bucket-object-versioning-on-CloudFerro-Cloud.html.md
View File

@ -1,4 +1,4 @@
S3 bucket object versioning on CloudFerro Cloud[](#s3-bucket-object-versioning-on-brand-name "Permalink to this headline")
S3 bucket object versioning on CloudFerro Cloud[🔗](#s3-bucket-object-versioning-on-brand-name "Permalink to this headline")
===========================================================================================================================
S3 bucket versioning allows you to keep different versions of the file stored on object storage. Here are some typical use cases:
@ -16,7 +16,7 @@ In this article, you will learn how to
> * download different versions of files and
> * set up automatic removal of previous versions.
Prerequisites[](#prerequisites "Permalink to this headline")
Prerequisites[🔗](#prerequisites "Permalink to this headline")
-------------------------------------------------------------
No. 1 **Account**
@ -48,7 +48,7 @@ No. 5 **Terminology: container vs. bucket**
In this article, both “container” and “bucket” represent the same category of resources hosted on CloudFerro Cloud cloud. The former term is more often used by the Horizon dashboard and the latter term is more often used by AWS CLI.
What We Are Going To Cover[](#what-we-are-going-to-cover "Permalink to this headline")
What We Are Going To Cover[🔗](#what-we-are-going-to-cover "Permalink to this headline")
---------------------------------------------------------------------------------------
> * Configuring and testing AWS CLI
@ -83,12 +83,12 @@ What We Are Going To Cover[](#what-we-are-going-to-cover "Permalink to this h
> > + Bucket on which versioning has never been enabled
> > + Suspending of versioning
Configuring and testing AWS CLI[](#configuring-and-testing-aws-cli "Permalink to this headline")
Configuring and testing AWS CLI[🔗](#configuring-and-testing-aws-cli "Permalink to this headline")
-------------------------------------------------------------------------------------------------
Now follows how to configure AWS CLI for the first time; if it has been configured before, you might need to adjust the configuration according to the needs of this article.
### Step 1: Configure AWS CLI[](#step-1-configure-aws-cli "Permalink to this headline")
### Step 1: Configure AWS CLI[🔗](#step-1-configure-aws-cli "Permalink to this headline")
To configure AWS CLI, create a folder called **.aws** in your home directory:
@ -137,7 +137,7 @@ Save the file and exit the text editor..
The commands we provide in this article will have the appropriate endpoint already included, via the **--endpoint-url** parameter, and all you need to do is to select the command for the cloud that you are using.
### Step 2: Verify that AWS CLI is working[](#step-2-verify-that-aws-cli-is-working "Permalink to this headline")
### Step 2: Verify that AWS CLI is working[🔗](#step-2-verify-that-aws-cli-is-working "Permalink to this headline")
Execute command **list-buckets** to list buckets:
@ -198,7 +198,7 @@ Note
In this article, colors have been added to make JSON more legible. AWS CLI typically does not output colored text.
Assigning bucket names to shell variables[](#assigning-bucket-names-to-shell-variables "Permalink to this headline")
Assigning bucket names to shell variables[🔗](#assigning-bucket-names-to-shell-variables "Permalink to this headline")
---------------------------------------------------------------------------------------------------------------------
To differentiate between different buckets used in various examples of this article, we will use the following shell variables:
@ -244,7 +244,7 @@ aws s3api create-bucket \
```
### Making sure that bucket names are unique[](#making-sure-that-bucket-names-are-unique "Permalink to this headline")
### Making sure that bucket names are unique[🔗](#making-sure-that-bucket-names-are-unique "Permalink to this headline")
If **single tenancy** is enabled on the cloud you are using, the name of your bucket needs to be unique for the entire cloud. Buckets called **versioning-test**, **examplebucket** etc. may well already exist.
If that is the case, the output from **create-bucket** command will look like this:
@ -295,7 +295,7 @@ Reusing the existing buckets
Avoid using the existing buckets
: Go through the article without previous baggage, using a “clean slate” approach. This is what you would normally do when using the article for the very first time.
Creating a bucket without versioning[](#creating-a-bucket-without-versioning "Permalink to this headline")
Creating a bucket without versioning[🔗](#creating-a-bucket-without-versioning "Permalink to this headline")
-----------------------------------------------------------------------------------------------------------
Command **create-bucket** will create a bucket under your chosen name (variable **$bucket\_name1**).
@ -332,7 +332,7 @@ aws s3api create-bucket \
The output of this command should be empty if everything went well.
Enabling versioning on a bucket[](#enabling-versioning-on-a-bucket "Permalink to this headline")
Enabling versioning on a bucket[🔗](#enabling-versioning-on-a-bucket "Permalink to this headline")
-------------------------------------------------------------------------------------------------
To enable versioning on the bucket **$bucket\_name1**, use command **put-bucket-versioning**:
@ -377,7 +377,7 @@ On Amazon Web Services, the presence of parameter **MFADelete** increases securi
The output of this command should also be empty.
Uploading file[](#uploading-file "Permalink to this headline")
Uploading file[🔗](#uploading-file "Permalink to this headline")
---------------------------------------------------------------
Let’s say that we upload a file to the root directory of our container. Let the name of the file be **something.txt** and let it have the following content: **This is version 1**.
@ -446,7 +446,7 @@ This upload created the first version of the file. The ID of this version is **w
The output also provides an **ETag** key, which is a hash of the object you uploaded.
S3 paths[](#s3-paths "Permalink to this headline")
S3 paths[🔗](#s3-paths "Permalink to this headline")
---------------------------------------------------
The parameter **--key** from **put-object** command may also be used in other commands to reference an already uploaded bucket in the container.
@ -543,7 +543,7 @@ aws s3api put-object \
```
Uploading another version of a file[](#uploading-another-version-of-a-file "Permalink to this headline")
Uploading another version of a file[🔗](#uploading-another-version-of-a-file "Permalink to this headline")
---------------------------------------------------------------------------------------------------------
Let us now return to **$bucket\_name1**
@ -680,10 +680,10 @@ Owner
In the example above, the bucket still contains only one file - **something.txt**. This upload overwrote it with a new version, but the previous version is still present.
Listing available versions of a file[](#listing-available-versions-of-a-file "Permalink to this headline")
Listing available versions of a file[🔗](#listing-available-versions-of-a-file "Permalink to this headline")
-----------------------------------------------------------------------------------------------------------
### Example 1: One file, two versions[](#example-1-one-file-two-versions "Permalink to this headline")
### Example 1: One file, two versions[🔗](#example-1-one-file-two-versions "Permalink to this headline")
To list the available versions of files in this bucket, use **list-object-versions**:
@ -756,7 +756,7 @@ The output could look like this:
It contains two versions we created previously. Each has its own ID, which is the value of parameter **VersionId**:
Table 5 Key vs. VersionId[](#id1 "Permalink to this table")
Table 5 Key vs. VersionId[🔗](#id1 "Permalink to this table")
| | |
| --- | --- |
@ -766,7 +766,7 @@ Table 5 Key vs. VersionId[](#id1 "Permalink to this table")
Both of them are tied to the same file called **something.txt**.
### Example 2: Multiple files, multiple versions[](#example-2-multiple-files-multiple-versions "Permalink to this headline")
### Example 2: Multiple files, multiple versions[🔗](#example-2-multiple-files-multiple-versions "Permalink to this headline")
Let us now consider an alternative situation in which we have two files, and one of them has two versions.
@ -820,7 +820,7 @@ The output of **list-object-versions** could then look like this:
```
Table 6 Key vs. VersionId[](#id2 "Permalink to this table")
Table 6 Key vs. VersionId[🔗](#id2 "Permalink to this table")
| | |
| --- | --- |
@ -831,7 +831,7 @@ Table 6 Key vs. VersionId[](#id2 "Permalink to this table")
File **something1.txt** has one version, while file **something2.txt** has two versions..
Downloading a chosen version of the file[](#downloading-a-chosen-version-of-the-file "Permalink to this headline")
Downloading a chosen version of the file[🔗](#downloading-a-chosen-version-of-the-file "Permalink to this headline")
-------------------------------------------------------------------------------------------------------------------
Let us return to **$bucket\_name1**.
@ -914,7 +914,7 @@ Displaying its contents with the **cat** command reveals that it is indeed the f
![s3-bucket-versioning-05_creodias.png](../_images/s3-bucket-versioning-05_creodias.png)
Deleting objects on version-enabled buckets[](#deleting-objects-on-version-enabled-buckets "Permalink to this headline")
Deleting objects on version-enabled buckets[🔗](#deleting-objects-on-version-enabled-buckets "Permalink to this headline")
-------------------------------------------------------------------------------------------------------------------------
AWS CLI includes command **delete-object** which is used to delete files stored on buckets. It behaves differently depending on the circumstances:
@ -930,7 +930,7 @@ The version to be deleted is specified
Here are the examples for version-enabled buckets.
### Setting up a deletion marker[](#setting-up-a-deletion-marker "Permalink to this headline")
### Setting up a deletion marker[🔗](#setting-up-a-deletion-marker "Permalink to this headline")
The command to delete files from buckets is **delete-object**.
@ -1120,7 +1120,7 @@ Within the Horizon dashboard, the file is also “invisible”:
Even though the file cannot be seen, the size of the bucket is still displayed correctly - 36 bytes. Each stored version of each file adds to the total size.
### Removing the deletion marker[](#removing-the-deletion-marker "Permalink to this headline")
### Removing the deletion marker[🔗](#removing-the-deletion-marker "Permalink to this headline")
To restore the visibility of a file, delete its deletion marker by issuing command **delete-object** and specify the **VersionID** of the deletion marker:
@ -1310,7 +1310,7 @@ The file should now also be visible in Horizon again:
That on this screenshot, the visible file has size 18 bytes, whereas the total size of this bucket is 36 bytes. This is because the total size includes both stored versions of the file.
### Permanently removing files from version-enabled bucket[](#permanently-removing-files-from-version-enabled-bucket "Permalink to this headline")
### Permanently removing files from version-enabled bucket[🔗](#permanently-removing-files-from-version-enabled-bucket "Permalink to this headline")
You can delete versions of file stored in the bucket just like you can delete the previously mentioned delete marker.
@ -1510,7 +1510,7 @@ we will see that there are no files or versions there:
```
Using lifecycle policy to configure automatic deletion of previous versions of files[](#using-lifecycle-policy-to-configure-automatic-deletion-of-previous-versions-of-files "Permalink to this headline")
Using lifecycle policy to configure automatic deletion of previous versions of files[🔗](#using-lifecycle-policy-to-configure-automatic-deletion-of-previous-versions-of-files "Permalink to this headline")
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
“Noncurrent version” is any version of a file which is not the latest. In this section, we will cover how to configure automatic deletion of these versions after a specified amount of days.
@ -1519,7 +1519,7 @@ For this purpose, we will use function called “lifecycle policy”.
This example covers configuring automatic removal of noncurrent versions of a file 1 day after a newer version of the same file has been uploaded.
### Preparing the testing environment[](#preparing-the-testing-environment "Permalink to this headline")
### Preparing the testing environment[🔗](#preparing-the-testing-environment "Permalink to this headline")
For testing, create bucket whose name is stored in variable **$bucket\_name3** and enable versioning:
@ -1837,7 +1837,7 @@ confirms that one of the files has two versions while the other only has one ver
Contrary to other versions of files stored here, the first version of file **mycode.py** has **false** under the key **IsLatest**. This shows that this is not the latest version of that file.
### Setting up automatic removal of previous versions[](#setting-up-automatic-removal-of-previous-versions "Permalink to this headline")
### Setting up automatic removal of previous versions[🔗](#setting-up-automatic-removal-of-previous-versions "Permalink to this headline")
The lifecycle policy is written in JSON. Create file named **noncurrent-policy.json** in your current working directory (doesn’t have to be the location of the file which contains your login credentials) and enter the following code into it:
@ -2022,7 +2022,7 @@ reveals that the version of file **mycode.py** which is not the latest was delet
```
### Deleting lifecycle policy[](#deleting-lifecycle-policy "Permalink to this headline")
### Deleting lifecycle policy[🔗](#deleting-lifecycle-policy "Permalink to this headline")
Command **delete-bucket-lifecycle** deletes bucket lifecycle policy. This is how to do it on bucket **$bucket\_name3**.
@ -2099,12 +2099,12 @@ argument of type 'NoneType' is not iterable
The policy should now no longer apply.
Suspending versioning[](#suspending-versioning "Permalink to this headline")
Suspending versioning[🔗](#suspending-versioning "Permalink to this headline")
-----------------------------------------------------------------------------
If you no longer want to store multiple versions of files, you can suspend the versioning.
### Bucket on which versioning has never been enabled[](#bucket-on-which-versioning-has-never-been-enabled "Permalink to this headline")
### Bucket on which versioning has never been enabled[🔗](#bucket-on-which-versioning-has-never-been-enabled "Permalink to this headline")
To better understand how it works, let us start with a bucket in which versioning has never been enabled in the first place.
@ -2112,7 +2112,7 @@ On such a bucket, every file will only have one version which has one and the sa
If you upload another file under the same name, its **VersionID** will also be **null**, and will replace the previously uploaded file.
#### Example[](#example "Permalink to this headline")
#### Example[🔗](#example "Permalink to this headline")
For this example, we will create bucket **$bucket\_name4** on which versioning has never been enabled.
@ -2148,7 +2148,7 @@ aws s3api create-bucket \
Buckets can, of course, contain files of various type. For the sake of this example, suppose that the bucket contains the following three files of various types:
Table 7 File vs. the editor[](#id3 "Permalink to this table")
Table 7 File vs. the editor[🔗](#id3 "Permalink to this table")
| | |
| --- | --- |
@ -2464,7 +2464,7 @@ we should get the output like this:
Once again, there are three files, each with exactly one version stored. The file **script.sh** was overwritten during our upload - its parameters **Size**, **ETag** and **LastModified** (timestamp of last modification) have changed.
### Suspending of versioning[](#suspending-of-versioning "Permalink to this headline")
### Suspending of versioning[🔗](#suspending-of-versioning "Permalink to this headline")
When you suspend the versioning, your bucket will start behaving similarly to a bucket on which versioning has never been enabled. All files uploaded from that moment on will have **null** as their **VersionId**.
@ -2551,7 +2551,7 @@ Upload a few files to this bucket with **put-object** command. Make sure that at
In this example, our bucket contains the following files:
Table 8 Key vs. VersionId[](#id4 "Permalink to this table")
Table 8 Key vs. VersionId[🔗](#id4 "Permalink to this table")
| | |
| --- | --- |
@ -2980,7 +2980,7 @@ After this upload, we list versions one more time and get the following output:
Once again, there is only one version which has **null** as its ID - the upload overwrote the previous version. The date of last modification (**LastModified**) has changed. Its previous value was **2024-09-16T11:31:01.968Z** and now it is **2024-09-16T11:34:25.528Z**
What To Do Next[](#what-to-do-next "Permalink to this headline")
What To Do Next[🔗](#what-to-do-next "Permalink to this headline")
-----------------------------------------------------------------
AWS CLI is not the only available way of interacting with object storage. Other ways include: