techsupport@sorcim.com

Amazon S3 Duplicate Finder

Note: This article has been updated to reflect the changes introduced in Cloud Duplicate Finder v4.0 in May 2026.

Amazon Simple Storage Service (S3) is an Internet-based data storage facility provided by Amazon, Inc. It is considered as one of the fastest, most economic and flexible Internet cloud storing platforms, and a major competitor to OneDrive, Google Drive and Dropbox.

Amazon provides a wide variety of storage plans that cater to corporates as well as individuals. It allows data to be accessed privately with signed URLs, while public files are managed by the Access Control Policy (ACP). It has a simple and easy to use interface.

With Amazon S3, you can store any type of data including documents, files, pictures, videos, and much more. You can access these files at any time from any device. The only prerequisite is Internet connectivity.

Amazon S3 employs a unique technique to prevent accidental data loss. This is achieved by replicating data across different AWS regions to prevent accidental data loss caused by human error.

Buckets & Containers

In Amazon S3, data is stored inside multiple buckets which mainly act as containers for data storage. You can store a single file of up to 5 TB in each bucket. A bucket can contain an unlimited number of objects, which are retrieved with a developer assigned key. Data can be easily downloaded by providing the appropriate read/ access permissions.

There are numerous benefits of creating buckets. Some of which are as follows:

  • A bucket helps you identify a user account which is responsible for data storage
  • Buckets are customizable. They can be created in specific regions
  • Buckets provide access control, usage reporting among other features

Multi-Factor Authentication

The most common use of multi-factor authentication is to prevent unauthorized users from accessing your S3 cloud data. Another benefit of multi-factor authentication is that it prevents accidental deletion of your data. Whenever you attempt to delete something stored in a multi-factor authentication-enabled bucket, you will be prompted to confirm your identity. A 6-digit verification code will be sent to the device which you have registered for authentication. Amazon S3 also allows you to restore an earlier version of a file.

Safety & Privacy

Amazon’s online service ensures that your data is safe and secure from unauthorized access and hackers. Only you and anyone you have given permission to access your data will be able to read and download it. Of course, you can revoke these permissions at any time.

Batch Copying

Amazon S3 supports batch copying of thousands of objects within a few minutes. You can bulk copy objects, change tag sets, execute the all powerful AWS Lambda functions. You can select your desired task from the menu bar or simply use the inventory report field to create a custom list.

Other Benefits of Amazon S3

Some other benefits of using Amazon S3 are given below:

  • You can scatter your data across different buckets
  • The user interface is mainly focused to work with different Internet Development Toolkits such as REST and SOAP
  • You can also use official REST APIs and AWS SDKs for added advantage
  • Amazon S3 Glacier allows you to restore archived objects anytime

The Procedure

Step # 1: Create An Account

First and foremost, you need to create an Amazon S3 account. Here is how you can do that:

  1. Go to the website https://aws.amazon.com/s3/.
  2. Click the Get started with S3 button.
  3. Fill-up the required information and proceed with the sign-up.

Step # 2: Create A New Bucket

Let us create a new bucket to store unlimited objects. You can create as many buckets as you wish.

  1. Log-in to your Amazon S3 Console.
  2. Click the Create bucket button.
  3. Type a name for your new bucket in the Bucket name textbox.
  4. Click Create and the new bucket will be automatically created.

Step # 3: Add New Objects to the Bucket

Once you have successfully created a bucket, it is time to add new objects.

  1. Select your preferred bucket from the list and click the Upload button.
  2. You will be informed that your bucket is empty. Simply click the Add files button.
  3. Select a file that you want to upload and click Open.
  4. Click Upload to launch the file upload.

Step # 4: View & Download Objects

Here is how you can view and download your information:

  1. Select the bucket and click the checkbox just before the file you want to download.
  2. A menu will open up on the right-hand of the screen, select the Download button.
  3. The file download will begin momentarily.

It is important to regularly check Amazon S3 for duplicates. Unwanted versions of files can take up precious network storage space which is no less than a digital asset. This storage space can otherwise be used to store meaningful files that matter.

With a robust scan using Cloud Duplicate Finder, an online duplicate finder service, you can find and get rid of duplicates in no time. After finishing up, you will be amazed by the amount of storage space you will reclaim.

Step # 5: Get Your Cloud Duplicate Finder Account

  1. Open the Cloud Duplicate Finder website.
  2. Click the Create an Account button.
  3. Type your Full name, Email ID, confirm your password and select the checkbox that says I agree to the Terms of Service.
  4. Click the Sign up button and look for a confirmation email that will be instantly sent to your registered email ID.
  5. Click on the confirmation link to authenticate your account registration process.

Note: No software download/ installation of any kind is required. You can access the Cloud Duplicate Finder website straight from your browser on any compatible device such as Windows PC, MacBook Android tablets, iPhone and so on.

Step # 6: Integrate Your S3 Account & Scan It

  1. Login to Cloud Duplicate Finder.
  2. Click on the S3 tab, followed by the Add New S3 Account button.
  3. You will have to enter the Bucket Name, Access Key ID and Secret Access Key.

Add new S3 account form with bucket name and access keys

  1. Now choose between All Files or Custom.
  2. The All Files option will scan everything irrespective of its type, while Custom Files will allow you to select specific file filters to further customize the scan depending on your needs. You can choose to scan among documents, images, music, videos, archives and other file types.
  3. Once you are done, click Find Duplicates and the scan will start automatically.

Step # 7: Automatically Fix Duplicates

  1. The scan report will show up the number of duplicates found, duplicate groups count, and the amount of storage space wasted by these duplicate files.
  2. Click on OK.
  3. You can individually select duplicates of your choice, or use the automatic selection feature by clicking on the Select Duplicates button, and then choosing a selection criteria of your choice.
  4. The selection button allows you to choose between retaining the oldest or newest file in each duplicate group.
  5. Once you have selected the files, click the Select Action button and then choose between Move to trash and Move to folders.

Cloud Duplicate Finder – Key Features

Some key features of Cloud Duplicate Finder are as under:

  • Integrate different cloud service accounts such as OneDrive, Google Drive, Dropbox, Amazon S3 & Box Cloud
  • Cloud Duplicate Finder supports all file formats
  • Your files are compared by their content, and not by their names. This is the most secure way to find duplicates accurately!
  • No files will be deleted without your consent. It is up to you whether to delete duplicate files or move them to a folder you opt for
  • CDF uses official APIs from the concerned cloud service provider for data safety & security. This ensures no unauthorized access ever!
  • The user retains full control over their data at all times, and they can easily revoke the permissions granted to CDF whenever they want
  • Your data is scanned without the need of downloading even a single file to your PC!
  • CDF uses a 256 bit IAS algorithm to encrypt and decrypt user’s information. Your data is completely safe against hackers and spoofers
  • The Custom scan option enables you to reduce the overall scan time and get faster results by limiting the file types you want to scan