You can connect Crisp with Amazon S3, to sync your distributor and retailer data securely to the cloud, then integrate that data into your internal systems.
Note: You need your AWS cloud region and bucket name on hand to set up this connection.
Crisp provides two authentication options:
- AWS Access Key authentication uses a secret access key and an access key ID, which are long-term credentials.
- AWS STS using OIDC WebIdentity - recommended, uses short-term credentials.
To create the connection, complete the following tasks (you can select a link to jump to a specific section):
- Prepare for the connection (choose Access Key or OIDC WebIdentity authentication method)
- Set up the connection in Crisp
- Access your data in Amazon S3
To prepare for the connection:
Create an IAM policy
Both authentication methods require an IAM policy that grants Crisp access to your bucket.
If you are creating a new policy, Crisp recommends creating a JSON policy using the instructions here: AWS Documentation > Creating policies on the JSON tab and copying/pasting in the following sample JSON that grants Crisp the minimum access needed for the connection.
Note: Replace the example text with your bucket name.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:ListBucket",
"s3:DeleteObject"
],
"Resource": [
"arn:aws:s3:::<YOUR BUCKET NAME HERE>/*",
"arn:aws:s3:::<YOUR BUCKET NAME HERE>"
]
}
]
}No matter how you create the policy or if you attach an existing one, the Crisp connection (at minimum) requires the policy to include the following access level actions. For more on creating policies, see AWS Documentation > Creating IAM policies.
- Write: PutObject and DeleteObject
- Read: GetObject
- List: ListBucket
Choose an Authentication Method
AWS Access Key authentication
As a security best practice, we recommend creating a dedicated user for the Crisp connection, so you can manage the level of access the user has. To create the user, log in to AWS, then follow the process in AWS Documentation > Creating an IAM user (console). Make sure the user has the following configuration options set:
- In Step 4 of the user creation process, make sure to select Access key - Programmatic access.
- In Step 6, attach or create an Identity and Access Management (IAM) policy that grants the user the appropriate level of access in accordance with your organization's policies. Your organization may have may have an existing policy that you want to attach to the user.
AWS Security Token Service (OIDC WebIdentity)
This guide is based on the official AWS documentation.
Create an OpenID Connect identity provider for Google
This is needed to exchange the authentication tokens between Google (Crisp) and AWS (Customer).
In AWS Console:
- Visit IAM page, then Identity providers from the left sidebar
- Select Add Provider and provide the following information:
* Provider type: OpenId Connect
* Provider URL: https://accounts.google.com
* Audience: 116254621267726279882
ℹ️ if you already have an Identity Provider for accounts.google.com, add the 116254621267726279882 audience to it.
Create a role with access to your S3 bucket for the Google Identity Provider
- Visit IAM page, then Roles from the left sidebar
- Select Create Role, then Custom trust policy
-
Paste the following:
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::YOUR_AWS_ACCOUNT_ID:oidc-provider/accounts.google.com" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { "accounts.google.com:sub": "116254621267726279882", "accounts.google.com:oaud": "sts.amazonaws.com", "accounts.google.com:aud": "116254621267726279882" } } } ] } - Update YOUR_AWS_ACCOUNT_ID with your AWS account ID (found in the top right corner of the AWS Console) and press Next.
- Find the permission IAM policy you have created earlier and attach it to the role and press Next.
- Provide a role name, optional description or tags and press Create role.
- Copy the role ARN, you will need to apply it to the connector configuration in Crisp. The ARN looks like this: arn:aws:iam::YOUR_AWS_ACCOUNT_ID:role/YOUR_ROLE_NAME.
To set up the Amazon S3 connection in Crisp:
- Log into your Crisp account.
- From the main menu on the left side of the screen, select Connectors.
-
Select the Add connector button.
- Select the Data Destinations tab, then select the AWS S3 connector tile.
-
If you want, update the default name of the connector. If you're going to set up multiple data feeds to this destination, we recommend using something unique.
-
In the Bucket field, provide the AWS bucket name to which you want to output data, then select the AWS cloud region where the bucket is stored.
- Depending on your authentication choice, provide your AWS Access Key ID and AWS Access Key, or for the OIDC WebIdentity option, the Role ARN in the appropriate fields.
These keys allow Crisp to access your bucket for file storage. - In the Export Format section, select an output format (e.g., CSV), then choose whether you want the files to be delivered compressed or not.
- If you want, fill out the next three optional fields to further customize how you receive the data files.
-
In the Connector source data section, select the Select button.
The Select source data screen appears. -
Select the retailer(s) for which you want to export data (e.g., UNFI), then choose which data tables and columns to include in the export. For detailed instructions on making your data feed selections, see Selecting Destination Data Sources.
Hint: You can select the book icon to the right of a table name to see its documentation. -
To complete your selections, select the Accept button.
The window closes and you return to the connector setup screen. -
If you want to include a column that identifies which source connector the data came from, select the Include Source Connector Configuration ID option.
Hint: This is useful if you have multiple connectors for the same retailer, such as one for U.S. sales and one for Canadian sales.
-
Select the Save button.
The connection is saved and you return to the Connectors screen. You can check the status of the connection by clicking the new AWS S3 tile. On this screen, there are two alerts to validate a good connection. Under Connection diagnostics, Crisp alerts you if your storage container and write access are working properly.
After login and the connection to AWS S3 is validated, any unexpected error with the connections are logged and sent to Crisp engineering for resolution.
Accessing Data in Amazon S3
Unlike storing files on your computer’s hard drive, S3 enables data storage within a specific AWS cloud region. From the defined cloud region (such as AWS East), S3 data is then stored in a bucket. A bucket is the basic container that holds data and may not be nested like files on your computer are typically. Once the data is synced in S3, you will see a folder per table selected. Each folder contains the name of the table and the timestamp the data was synced. Within those folders are the exported data files for use. Exported Crisp data includes the most recent reports plus up to 2 years' worth of backfill data.
To locate a Crisp data object:
- Log in to Amazon S3 as the user you used to set up the Crisp connection.
- Select the AWS region.
- Select the storage bucket you specified during connector set up.
- Select the file(s) you want to use.