Search

Welcome to Datagrid
Welcome to Datagrid
Google Cloud Storage

Google Cloud Storage

Google Cloud Storage is a managed service for storing unstructured data, allowing you to store any amount of data and retrieve it as often as you like. By connecting Google Cloud Storage to Datagrid, you can seamlessly integrate your cloud storage data with other business-critical information, enabling enhanced reporting, analysis, and decision-making.

1. How-to

1. Prerequisites

To configure the Google Cloud Storage connector, follow these steps:

  • A Google Cloud account with the necessary permissions to access Cloud Storage.
  • A Cloud Storage bucket with the data you want to import.
  • Google Cloud Storage JSON API enabled for your project.
  • Service account credentials with the Storage Object Viewer role (or higher) to access the bucket.
  • Select the data you want to import into Datagrid

2. Connect

Creating a dataset from the Google Cloud Storage connector involves selecting the specific data you want to import:

  1. Connect Google Cloud Storage App: a. Click on the "+ Create” Button on the top left of the screen. b. Select the "Connect Apps" item. c. Search for the Google Cloud Storage connector from the list. d. Enter your Google Cloud Storage bucket details, including the bucket name and service account credentials. e. Click on the “Next” button.
  2. Pick your Data: a. Pick the Google Cloud Storage data you want to include in your dataset (e.g., objects, files). b. Click on the “Start First Import” Button to start syncing your Google Cloud Storage dataset.

3. Set Up a Schedule:

Scheduling regular data pulls ensures your Datagrid datasets remain up-to-date with the latest information from Google Cloud Storage:

  1. Navigate to Google Cloud Storage Dataset: a. Go to the left side panel and locate and click on the Google Cloud Storage dataset you created.
  2. Schedule Settings: a. Click on the “...” on the top right of the dataset. b. Click on “Edit Pipeline” to edit your connector's name. c. Click the “Schedule” button on the right, beside the “Import Configuration” button.
  3. Configure Schedule: a. Set the desired frequency for data pulls (e.g., daily, weekly, monthly). b. Specify the time of day for the data pull to occur. c. Specify downtime if needed – when the sync should not happen. d. Click the “Update” button to update the new configuration.

4. Data Access

  • Buckets
  • Objects
  • Folders
  • ObjectAccessControls
  • DefaultObjectAccessControls
  • ManagedFolders
  • HMAC Keys

5. Use Cases

  • Backups and Archives: Use Cloud Storage for backup, archives, and recovery. Cloud Storage's nearline storage provides fast, low-cost, highly durable storage for data accessed less than once a month, reducing the cost of backups and archives while still retaining immediate access.
  • Media Content Storage and Delivery: Stream audio or video directly to apps or websites with Cloud Storage’s geo-redundant capabilities. Geo-redundant storage with the highest level of availability and performance is ideal for low-latency, high-QPS content serving to users distributed across geographic regions.
  • Data Lakes and Big Data Analytics: Develop and deploy data pipelines and storage to analyze large amounts of data. Cloud Storage offers high availability and performance while being strongly consistent, giving you confidence and accuracy in analytics workloads.
  • Machine Learning and AI: Launch a Google-recommended, preconfigured solution that uses generative AI to quickly extract text and summarize large documents stored in Cloud Storage.
  • Website Hosting: Configure a Cloud Storage bucket to host a static website for a domain you own.

6. FAQ

Q: What is Cloud Storage? A: Cloud Storage is a managed service for storing unstructured data. You can store any amount of data and retrieve it as often as you like.

Q: What type of data can I import from Google Cloud Storage? A: You can import objects, files, and metadata associated with your stored data.

Q: How do I ensure secure connections to my Cloud Storage bucket? A: Use service account credentials with appropriate permissions and configure access control policies to restrict access to your data.

7. Support & Additional Resources