Browse Integrations

BigQuery (Google Cloud)

Fully managed data storage solution that is a part of Google Cloud platform

Google Cloud BidQuery is a fully managed data storage solution that is a part of Google Cloud platform and is provided to customers based on PaaS (Platform as a Service) model. BidQuery supports ANSI SQL querying and has built-in machine learning, business intelligence and geospatial analysis capabilities. 

Thanks to our flexible input APIs, Clarify can be easily connected to Google Cloud BigQuery, providing a cost-efficient and simple way of getting value from the time series data collected by this service.

Clarify makes it easy to visualize time series data from your IoT network, access it on web and mobile devices, as well as sharing data to other people. With Clarify, your organization can focus on key business goals instead of configuring dashboards or developing expensive custom solutions.

About Google Cloud Platform 

Google Cloud Platform (GCP) is a cloud computing services suite by Google that includes various tools for cloud management, networking, computing, data storage, Big Data analytics and other purposes. The platform also provides serverless computing environments, infrastructure as a service, and platform as a service.

Services that are part of GCP are relying on the same infrastructure that is used by Google’s internal end-user services, such as the Google search engine, Gmail, YouTube, etc. Google Cloud Platform is part of Google Cloud enterprise offering, which also includes Google Workspace (formerly G Suite), enterprise versions of Android and Chrome OS, as well as APIs for machine learning and enterprise mapping services.

Google Cloud Platform was started in 2008 from App Engine, a platform for the development and hosting of web applications in the cloud. App Engine became generally available in 2011. Over the years, Google added multiple other cloud services to the platform. 

The platform consists of multiple physical assets, such as computers and hard disk drives, and virtual resources, such as virtual machines (VMs), that are contained in Google's data centers in different parts of the world. 

In total, Google Cloud Platform incorporates over 100 different services, divided into a number of categories, including AI and machine learning, API management, cloud computing, containers, data analytics, databases, developer tools, healthcare and lifesciences, hybrid and multi cloud, IoT, management tools, networking, migration, media and gaming, storage, security, and others. 

About Google Cloud BidQuery 

BigQuery was developed as a fully managed enterprise data warehouse solution that allows businesses to collect, store and analyze data with built-in features including machine learning, geospatial analysis, and business intelligence. 

BidQuery supports ANSI SQL querying and uses a scalable, distributed analysis engine for high-speed query response. The solution also separates the compute engine that analyzes user data from storage technology. Users are able to store and analyze their data within BigQuery or use BigQuery to assess the data in third-party databases. Federated queries let users read data from external sources while streaming supports continuous data updates.

BigQuery’s analytics features include business intelligence, ad hoc analysis, geospatial analytics, and machine learning. The solution also provides centralized management of data with Identity and Access Management (IAM), the access model used throughout Google Cloud. GCP’s best practices provide a number of data security customization options that can include traditional perimeter security or more complex and granular defense-in-depth approach.

BigQuery ML component allows users to build and operationalize ML models on planet-scale structured or semi-structured data directly inside BigQuery using SQL Users can export BigQuery ML models for online prediction into Vertex AI, a Jupyter-based fully managed enterprise-ready environment for data scientists that is art of GCP, or their own serving layers.

BigQuery BI Engine is an in-memory analysis service built into BigQuery that enables users to analyze large and complex datasets interactively with sub-second query response time and high concurrency. BI Engine natively integrates with Google’s Data Studio using BI Engine single node and natively accelerates any other business intelligence tools using BI Engine SQL interface.

BigQuery Omni component is an analytics solution that allows users to analyze data across clouds such as AWS and Azure. BigQuery GIS adds the support for geospatial analysis into BigQuery, allowing users to augment analytics workflows with location intelligence. 

BigQuery supports a number of interfaces, including Google Cloud Console interface and the BigQuery command-line tool. Developers and data scientists can use client libraries with familiar programming languages including Python, Java, JavaScript, and Go, as well as BigQuery's REST API and RPC API to transform and manage data. ODBC and JDBC drivers provide interaction with existing applications including third-party tools and utilities.

BigQuery’s data storage 

BigQuery stores data using a columnar storage format called Capacitor.that is optimized for analytical queries. The data is presented in tables, rows and columns, and provides full support for database transaction semantics (ACID). For high availability, BigQuery storage is automatically replicated across multiple locations.

The proprietary format is used because it can evolve in tandem with the query engine, which takes advantage of deep knowledge of the data layout to optimize query execution. BigQuery uses query access patterns to determine the optimal number of physical shards and how they are encoded.

The data is physically stored on Google's distributed file system, called Colossus, which ensures durability by using erasure encoding to store redundant chunks of the data on multiple physical disks. 

Users can run BigQuery queries on data outside of BigQuery storage, such as data stored in Cloud Storage, Google Drive, or Bigtable, by using federated data sources. These sources, however, are not optimized for BigQuery operations.

Why integrate with Clarify?

  • Search - let people on your team easily find data across siloed systems - always available on their phone or in the browser
  • Visualize - See trends and combine data to visually explore
  • Explore - See statistics and seamlessly move from millisecond detail to years worth of data
  • Export - Combine data sources and easily export to Excel for further data work
  • Annotate/label - Add meaning and context to your data by labeling both points and periods in time, to build training sets for your AI/ML efforts
  • Contextualize - Ensure naming, labeling, and metadata is of high quality
  • Cloud Storage - one safe space for your raw time series data built for cost-efficient speed and performance at scale
  • Integrate - break down your data silos and build one source of truth
  • Mobile apps for iOS and Android - make data easily available for your team, whenever and wherever they are. Fuel a remote work culture with live data.
  • Collaboration - share visualizations and discuss data with comments, threads and the activity feed. Add files for even more context.
Documentation

Resources

Homepage