Presigned URLs in AWS S3: Secure File Upload and Download

baki

Amazon Simple Storage Service (S3) is an object storage service offered by Amazon Web Services (AWS). It allows users to store and retrieve data from anywhere on the web. While you can manage S3 objects directly through the AWS Management Console and the AWS SDKs with dedicated permissions, there are situations where you need to securely grant temporary access to your S3 objects without exposing your AWS credentials. This is where presigned URLs come into play.

What is a Presigned URL ?

A presigned URL is a temporary link that provides time-limited access to a specific object stored in an S3 bucket with specific permission. This URL is generated by an authorized user or application and allows anyone with the URL to download or upload the object during the specified time period, without needing their own AWS credentials. After the expiration time, the URL becomes invalid and a new presigned has to be created if needed.

The credentials used by the presigned URL are those of the AWS user who generated the URL.

The presigned URL can be used multiple times as long as it hasn’t expired yet.

When creating a presigned URL, you have to first provide your security credentials and then specify the following:

  • The Amazon S3 bucket
  • The object key:
    • The name of the object in the bucket if you’re downloading
    • The name of the file if you’re uploading
  • A HTTP method
    • GET for downloading objects
    • PUT for uploading objects
  • An expiration time interval (can be in minutes or hours)

Time Expiration of presigned URLs

Presigned URLs are designed to be temporary and have built-in expiration times, enhancing security and access control. These expiration times are specified when you generate the URLs and can be customized to suit your use case.

The expiration time limit varies according to the method used to generate the presigned URL:

  • AWS console: The expiration time can be set between 1 minute and 12 hours.
  • AWS CLI or AWS SDKs: The expiration time can be set as high as 7 days

So far, we were assuming that permanent access tokens were used to generate the presigned URLs. However, if a temporary token is used, then the URL expires when the token expires, even if the expiration time of the presigned URL is still valid.

It’s important to note that Amazon S3 checks the expiration date and time of the presigned URL at the time of the HTTP request. This behavior leads to two cases:

  • If a client begins to upload a large file immediately before the expiration time, the upload continues even after the expiration time.
  • Suppose a client begins to upload a large file immediately before the expiration time. During the upload, an error occurs, and the upload is interrupted. Meanwhile, the expiration time was exceeded. If the client at this time tries to upload again, then the upload will fail because the presigned URL is no longer valid.

Uses Cases Where We Can Use The Presigned URLs

Presigned URLs can be used in various scenarios where you want to grant temporary, secure access to your S3 objects without exposing your AWS credentials. Here are some common use cases, with examples:

  1. Private File Downloads:
    You want to keep the contents of your S3 bucket private, but you do want to permit authorized people to download files from it. An example would be photo-sharing software that allows users to view their own photographs that are kept in an S3 bucket. When a user asks to download a picture, the program creates a Presigned URL for that particular image and gives it to them. For a brief period of time, the user may download the picture from this URL.
  2. File Uploads:
    You need to allow users to upload files directly to your S3 bucket without going through your servers while still maintaining control over access. An example would be a website that allows users to upload videos. Users can easily upload their films to the specified place in your bucket by using the platform’s presigned URL that is generated with a PUT operation for the S3 bucket.
  3. Secure Data Sharing:
    For a limited duration, you want to safely exchange files or data with clients or partners outside of your company. An e-commerce business, for instance, must communicate purchase orders or invoices to suppliers. They create presigned URLs for individual objects that contain the invoices, rather than disclosing AWS credentials. Suppliers are given the URLs so they can download the invoices within the specified time frame.
  4. Temporary Access for IoT Devices:
    You have IoT devices that need to upload data to an S3 bucket but do not require long-term access credentials. Weather sensors in remote areas, for example, send data to an S3 bucket. To protect the procedure, you construct presigned URLs for each sensor, which allows them to upload data on a regular basis. To prevent illegal access, these URLs expire after a certain time.
  5. Content Distribution:
    You want to serve private material to consumers while assuring security via a content delivery network (CDN).As an example, a video streaming service stores its video assets on S3. When a user wants to see a private movie, the application generates a presigned URL for the video object and sets it up to function with a CDN. This ensures rapid and secure information delivery while preserving access control.

Creating Presigned URLs

Presigned URLs are generated based on your AWS credentials and the permissions you have. You can create them in multiple ways:

  1. AWS Management Console: You can manually generate Presigned URLs in the AWS Management Console. Navigate to your S3 bucket, select the object, choose “Actions,” and then “Generate presigned URL.” Specify the expiration time, HTTP method (GET or PUT), and click “Generate.”
  2. AWS SDKs and CLI: You can programmatically create Presigned URLs using AWS SDKs (e.g., Java SDK) or AWS Command Line Interface (CLI) commands. This approach allows for automation and integration into your applications.

Creating a Presigned URL Using the AWS Console

Let’s start by generating a Presigned URL for downloading an object from an S3 bucket using the AWS Management Console:

  1. Log into the AWS Management Console: Go to https://aws.amazon.com/ and sign in with your AWS credentials.
  2. Access Amazon S3: Navigate to the Amazon S3 service from the AWS Management Console.
  3. Select Your Bucket: Click on the S3 bucket where your object is located.
  4. Select Your Object: Click on the specific object you want to generate a Presigned URL for.
  5. Generate Presigned URL: In the object details page, click on “Actions” and choose “Share with a presigned URL.”
    bucket images
  6. Set expiration: Define the time (in hours or minutes) for which the presigned URL will be valid.
  7. Generate: Click the “Create presigned URL” button, and the presigned URL will be copied automatically to your clipboard. You will see a button to copy the presigned URL if you need to copy it again.
    Presigned url
  8. Use the URL: You can now copy and share the presigned URL with authorized users. Anyone with the URL can access the object within the specified time frame.

Using the AWS console, it is only possible to create a presigned URL to download a file, not to upload one. Using the AWS SDK, it is possible to create presigned URLs for both actions: Upload and Download.

Creating a Presigned URL using the AWS Java SDK

Preparing Your Environment

Before you can generate Presigned URLs using the AWS SDK v2 for Java, you need to set up your development environment and include the necessary dependencies in your project.

  1. Setting up AWS SDK v2: You’ll need to include the AWS SDK v2 in your project. You can do this using a build tool like Maven or Gradle. Here’s how to include the SDK using Maven:
<project>
 <properties>
   <aws.java.sdk.version>2.X.X</aws.java.sdk.version> <!-- Replace with the latest version -->
 </properties>
 <dependencies>
  <dependency>
     <groupId>software.amazon.awssdk</groupId>
     <artifactId>s3</artifactId>
  </dependency>
 </dependencies>
 <dependencyManagement>
   <dependencies>
     <dependency>
       <groupId>software.amazon.awssdk</groupId>
       <artifactId>bom</artifactId>
       <version>${aws.java.sdk.version}</version>
       <type>pom</type>
       <scope>import</scope>
     </dependency>
   </dependencies>
 </dependencyManagement>
</project>

Although you can specify the version number for each component, you don’t need to because you already declared the SDK version in the dependencyManagement section using the bill of materials artifact. To load a different version of the S3 module, specify a version number for its dependency.

  1. AWS Credentials: Ensure that you have AWS credentials configured on your development machine. This can be done by setting up the AWS CLI or using environment variables, IAM roles, or AWS configuration files.
  2. S3 Bucket and Object: You should have an S3 bucket and object (file) that you want to generate a presigned URL. Make sure you have the necessary permissions to access these resources.

Creating a Presigned URL for File Download

To generate a presigned URL for downloading an S3 object, you can use the PresignedRequest API provided by the AWS SDK v2 for Java.

Below is an example of how to create a presigned URL for file download:

import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.model.GetObjectRequest;
import software.amazon.awssdk.services.s3.model.S3Exception;
import software.amazon.awssdk.services.s3.presigner.S3Presigner;
import software.amazon.awssdk.services.s3.presigner.model.GetObjectPresignRequest;
import software.amazon.awssdk.services.s3.presigner.model.PresignedGetObjectRequest;

import java.time.Duration;


public class S3PresignedUrlGenerator {

    public static void main(String[] args) {
        Region region = Region.EU_WEST_1;

        // Define the S3 bucket and object key
        String bucketName = "images-presigned";// Replace with your bucket name
        String objectKey = "computer.jpg"; // Replace with your object key

        ProfileCredentialsProvider credentialsProvider = ProfileCredentialsProvider.create();


        try (S3Presigner presigner = S3Presigner.builder()
                .region(region)
                .credentialsProvider(credentialsProvider)
                .build()) {
            String presignedUrl = getPresignedUrl(presigner, bucketName, objectKey, duration);

            // Print the generated presigned URL
            System.out.println("presigned URL for file download: " + presignedUrl);
        }

    }

    public static String getPresignedUrl(S3Presigner presigner, String bucketName, String keyName) {

        try {
            GetObjectRequest getObjectRequest = GetObjectRequest.builder()
                    .bucket(bucketName)
                    .key(keyName)
                    .build();

            GetObjectPresignRequest getObjectPresignRequest = GetObjectPresignRequest.builder()
                    .signatureDuration(Duration.ofMinutes(60)) // duration after which the presigned url expires
                    .getObjectRequest(getObjectRequest)
                    .build();

            PresignedGetObjectRequest request = presigner.presignGetObject(getObjectPresignRequest);
            return request.url().toString();
        } catch (S3Exception e) {
            e.getStackTrace();
        }
        return null;
    }
}

In this example, we:

  • Specify the S3 bucket name and object key for the file you want to generate a presigned URL for.
  • Initialize an S3Presigner using AWS SDK v2.
  • Create a GetObjectRequest with bucketName and key provided.
  • Create a GetObjectPresignRequest using the GetObjectRequest already created and set the signature duration after which the presigned url expires
  • Generate the presigned URL for file download using the presignGetObject method.

The output was:

https://images-presigned.s3.eu-west-1.amazonaws.com/computer.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20231009T034909Z&X-Amz-SignedHeaders=host&X-Amz-Expires=3600&X-Amz-Credential=AKIAQJW2HATY27ZESOKN%2F20231009%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Signature=c8508c50d9dbc0da6bc8075a8e0e4310559fe068bfad41abe4c030c572374bd2

In the URL we can see the expiration defined when we created the presigned URL. The X-Amz-Expires parameter contains the duration of our presigned URL which was 1 hour (3600 seconds).

Generating a Presigned URL for File Upload

To generate a presigned URL for allowing users to upload a file to an S3 bucket, you can use a similar approach. Here’s an example:

import software.amazon.awssdk.auth.credentials.ProfileCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.awssdk.services.s3.model.S3Exception;
import software.amazon.awssdk.services.s3.presigner.S3Presigner;
import software.amazon.awssdk.services.s3.presigner.model.PresignedPutObjectRequest;
import software.amazon.awssdk.services.s3.presigner.model.PutObjectPresignRequest;

import java.time.Duration;

public class S3PresignedUrlUploader {

    public static void main(String[] args) {
        Region region = Region.EU_WEST_1;

        // Define the S3 bucket and object key
        String bucketName = "images-presigned";// Replace with your bucket name
        String objectKey = "mouse.jpg"; // Replace with your object key

        // Set the duration after which the presigned url expires
        Duration duration = Duration.ofMinutes(10);

        ProfileCredentialsProvider credentialsProvider = ProfileCredentialsProvider.create();


        try (S3Presigner presigner = S3Presigner.builder()
                .region(region)
                .credentialsProvider(credentialsProvider)
                .build()) {
            String presignedUrl = getPresignedUrl(presigner, bucketName, objectKey, duration);

            // Print the generated presigned URL
            System.out.println("presigned URL for file upload: " + presignedUrl);
        }

    }

    public static String getPresignedUrl(S3Presigner presigner, String bucketName, String keyName, Duration duration ) {

        try {
            PutObjectRequest putObjectRequest = PutObjectRequest.builder()
                    .bucket(bucketName)
                    .key(keyName)
                    .build();

            PutObjectPresignRequest putObjectPresignRequest = PutObjectPresignRequest.builder()
                    .signatureDuration(duration)
                    .putObjectRequest(putObjectRequest)
                    .build();

            PresignedPutObjectRequest presignedPutObjectRequest = presigner.presignPutObject(putObjectPresignRequest);
            return presignedPutObjectRequest.url().toString();
        } catch (S3Exception e) {
            e.getStackTrace();
        }
        return null;
    }
}

In this example, the key difference is that we change the requests to PutObjectRequest and PutObjectPresignRequest to indicate that this presigned URL is for file upload. The expiration time and other aspects remain similar to the file download example.

The output was:

https://images-presigned.s3.eu-west-1.amazonaws.com/mouse.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20231007T235622Z&X-Amz-SignedHeaders=host&X-Amz-Expires=600&X-Amz-Credential=AKIAQJW2HATY27ZESOKN%2F20231007%2Feu-west-1%2Fs3%2Faws4_request&X-Amz-Signature=3cfd5cde94984d3d904bb9fcd05ee2327d83c76e6c0643feb00ed2fb8b96a30e

As mentioned above, we can see that the parameter X-Amz-Expires contains the presigned URL duration which was set to 10 minutes (600 seconds)

Conclusion

Presigned URLs in Amazon S3 offer a secure and flexible way to grant temporary access to your S3 objects. Whether you need to allow users to download private files or upload data to private buckets, Presigned URLs provide a secure and time-limited solution. By specifying an expiration time and defining the allowed operations (e.g., GET, PUT), you can tailor the level of access and the duration of that access according to your specific use case.

Related Posts