Overview Sections:
What is BigQuery?
Crisp BigQuery Connection Configuration
Accessing Data in BigQuery
Working with Crisp Data within BigQuery
Crisp Data
Data Schema
Data Lifecycle
What is BigQuery?
BigQuery is a scalable, dynamic and powerful enterprise-ready cloud data warehouse that enables robust yet simplified computing features, such as business intelligence. BigQuery is a serverless, scalable and distributed analysis engine that allows for fast SQL queries to load anywhere you need to run data analysis.
BigQuery is a cloud service that offers enterprises the ability to scale their platform use both up and down to meet demand, which results in a balance of performance and cost. This delivers a flexible solution capable of a near-unlimited number of concurrent workloads in near-real-time. BigQuery also enables secure data sharing internally and externally.
Crisp BigQuery Connection Configuration
Crisp offers data exports to your BigQuery account, allowing you to capture retailer and distributor data to apply the analytics needed to succeed with your customers.
To get started, you'll need access to your Google Cloud Platform account and have your BigQuery Cloud Region information handy. For a guide on creating the connection in Crisp, please refer to the connector configuration steps.
Accessing Data in BigQuery
Log into your Google Cloud Platform account, and open the BigQuery dashboard. Next, click the “Add Data” button in the top left corner of the screen. Select "Pin a project" and then "Enter project name." Type in “crisp-platform-shared" and click "Pin."
You should now see "crisp-platform-shared" in the list of projects in the sidebar.
Click to expand the dataset and view the available tables you selected during configuration.
Working with Crisp Data within BigQuery
To work with the Crisp data, click “Editor” to the right of the “Explorer” section. You can then insert your query and select "RUN." You'll also have the option to save and/or schedule the query to auto-run per your chosen parameters.
Crisp Data
Currently, Crisp allows you to export multiple source tables to your designated BigQuery project. To do this, you'll select your desired data tables from your account's inbound data connections.
Exported Crisp data includes the most recent reports published by the portal plus backfill data. Crisp backfills data up to the previous two years to date.
Data Schema
Data schema details vary between data table sources. Crisp provides users specific data schema information within our Knowledge Base documentation. Here, you can find up-to-date information on exported column attributes.
Data Lifecycle
Upon completion of the BigQuery connector configuration, Crisp initiates a data load into the BigQuery project within 10 minutes to export the first dataset that includes backfill data. Overall timing to backfill data for the first time depends on the number of tables requested. However, it should not take more than a day. Once the initial data set is loaded, recurring tasks are run on a schedule thereafter to update the data as it becomes available to Crisp from the retailer or distributor portal.
Since inbound data syncs are scheduled by the data provider, data availability varies based on the portal and ranges from daily, weekly to monthly reports. Crisp ingests the data to normalize and exports to users within six hours of the data availability.
When deleting the connector configuration, Crisp deletes the dataset as well.