Simple Storage Service (S3)

S3 is a cloud storage solution used primarily for assets—documents, images, files—versus data. While similar to Google Drive, S3 is a more freeform storage service commonly used for enterprise applications, and is typically used conjunction with other AWS services, like DynamoDB.

This topic assumes a working knowledge of the service and covers the basics of how to use S3’s features within Skuid, including:

  • Buckets are where data is stored within the S3 cloud.
  • Data—which may or may not be individual files—is stored within objects.
  • Each object has a unique identifier known as a key.
  • Buckets have a flat data structure, meaning there is no directory tree.

For more general information, see Amazon’s Introduction to Amazon S3.

Configuration

First configure an authentication provider using your AWS security credentials. After doing so, configure a data source—choosing Amazon S3 as the data source type.

AWS Access Permissions

When using AWS APIs, it is best practice to utilize strict and well-defined IAM policies so users can only perform actions they have explicit access to.

Listed below are the permissions this data source type requires to properly function. Use this list to better define your IAM roles.

  • listBuckets
  • listObjects
  • deleteObject
  • putObject
  • getObject

Using the AWS S3 data source

The S3 data source type has two model entities: Bucket and Object.

Bucket

This model entity represents the “buckets” where data objects are stored. It only has two, self-explanatory fields:

  • Name
  • Creation Date

This entity’s data cannot be edited, and it cannot accept any conditions.

One common use for the Buckets entity would be to create a row action that activates and sets the Object entity’s Bucket model condition.

Object

Objects are the individual data objects within an S3 bucket. They can be standard files, but they can also contain any data value. For more information, see AWS’s Working with Amazon S3 objects.

Fields

A model using the Object entity will always include in these two fields:

There are several other fields available. Some of the most useful include:

  • Object Name: Generated by Skuid for convenience, this field displays the text that follows the final forward slash in an object’s key.

    For example, an object with the key firstfolder/secondfolder/object1 would have an object name of object1.

  • Storage Class: The AWS storage class for that piece of data.

  • Is Folder: Returns whether or not an object is seen by S3 as a “folder,” meaning its key ends with a forward slash. For more information, see Amazon’s Working with Folders.

Conditions

This entity has are two configurable conditions—available by default—one of which is required:

  • Bucket (required): Restricts which objects are loaded based on the selected bucket.

  • FolderPath: Related to the S3 concept of folders, this is restricts data based on the “folder path” of the key field.

    For example, a condition with the value “firstfolder/secondfolder/” would return objects with the following keys:

    • firstfolder/secondfolder/object1, firstfolder/secondfolder/object2, etc.

Uploading files

Since S3 is a data storage service, Skuid’s File Upload component can be used to upload files to an S3 bucket. These uploads will work directly, or through the Platform’s proxy.

  • File Storage Location: S3 Direct Upload
    • This default value simply means files will be placed within the bucket of the Parent Model. This value the cannot be changed.
  • Label: Enter a label to insert custom text for the upload button. If you leave this field blank, the upload button will be labeled “Upload.”
  • File Name: Use this field to rename any files uploaded through this component. Anything entered here will be set as the file’s key.
    • This field is compatible with merge syntax.
    • If this field is blank, files will retain their current file name as their S3 key.
    • Using a dot within this field will cause the file’s extension/file type to be overwritten.
      • If used to rename “spreadsheet.xlsx” to ”budget,” the resulting filename would be “budget.xlsx.”)
      • If used to rename “spreadsheet.xlsx” to ”budget.csv,” the resulting file would be “budget.csv.” )
  • Snippet to Run on Completion: Enter the name of any JavaScript snippets you wish to run once a file has been uploaded to S3.
  • ACL of File: Determines the Access Control List for files uploaded through this component.

In some instances, you may wish to to upload files to an S3 bucket not represented by a model. In this case, set the Parent Model property to None, and several new properties will appear:

  • Data Source Type: Select AmazonS3.
  • Data Source: Select the data source connected to the appropriate S3 account.
  • S3 Bucket Name: Enter a specific bucket name or utilize merge syntax to access a field on a model.
    • In some instances, your bucket name can be contained within a separate data source—such as a DynamoDB table. Utilize merge syntax to have your File Upload component dynamically access and utilize that data.
  • S3 Folder Path: Related to the S3 concept of folders, enter a folder path to be prepended to the file name.

Warning

Uploading a file with a key identical to an existing S3 object will overwrite that object.

Data source actions

The S3 data source type has two data source actions, both of which facilitate file downloads. This action uses Skuid’s generated Object Name field to download the correctly-titled file—meaning the key’s prefixes will not be a part of the file name.

  • Download Row Object: Designed to be used as a row action, this action type downloads the object represented by the row in context.
    • Model of Objects: Select the model pointing to the S3 objects
  • Download Target Object: A more flexible downloading action, this does not need a specific model or row for context. Instead, there are two properties that allow for admins to specify which object should be downloaded by this action. The target object can be set explicitly with strings or dynamically through the use of merge syntax in both of this action’s properties.
    • Key of Object
    • Bucket that holds Object

Troubleshooting

  • When trying to select a model entity, I receive “An internal server error occurred.”

    The IAM user associated with your AWS access credentials within your Skuid authentication provider probably does not have the appropriate permissions for the service you are accessing. Return to the AWS console and ensure your IAM policy settings are correct.

   If your policy settings are correct, recreate your authentication provider with a new access key.

  • “Internal Server Error: An internal server error occurred.”
    • If you attempt to load an S3 model that does not have data at page load, then all models will fail to load.
    • Make sure you have set the Bucket condition for any Object models.