BigQuery Omni: Major News for Google Cloud Customers and Partners

By Avi Ben Ezra Friday, July 31, 2020

The Google Cloud Next 2020 virtual event brought to light some big news: the release of BigQuery Omni, a multi-cloud extension of BigQuery that could run in Amazon Web Services (AWS) and Microsoft Azure.

Launched in 2010, BigQuery is a serverless data warehouse accessible on Google Cloud. It was one of the first managed cloud services from Google, after only App Engine, and it’s known as the most widespread and well-liked cloud data warehouse platform in the public cloud. More than that, it’s one of the most rapidly expanding services in the Google Cloud product collection.

BigQuery can load data from numerous sources, including BigTable, Cloud Storage, Google Drive, and Cloud SQL. Furthermore, it supports well-known data formats such as Avro, CSV, JSON, Apache ORC, and Apache Parquet.

The data warehouse has continually progressed to meet consumer needs, making it the base for fashioning data-driven and analytics-based applications running in Google Cloud. Consumers have the power to execute standard SQL requests integrated with high-level machine learning algorithms to complete forecasts and categorizations based on current datasets. The Connected Sheets feature enables Google Sheets to become the front-end to examine billions of rows of data saved in BigQuery.

With the acquisition of Looker, Google supplements BigQuery with analytics and visualization tools to construct strong dashboards. This mixture of BigQuery and Looker is a feasible substitute for Amazon Redshift with Amazon QuickSight and Azure Synapse Analytics with Microsoft Power BI.

What Is BigQuery Omni, and Why Is it Significant?

The extension brings with it a subsection of BigQuery capabilities to AWS and Azure. It allows consumers to use the well-known user experience and API of BigQuery without obviously shifting the data to Google Cloud.

Consumers have standardized their data warehouse on BigQuery because of its functioning and quickness. Regardless of workloads being run on alternative cloud environments, they funnel the data to Google Cloud, which is ultimately transferred into BigQuery for analysis.

The transference of data from other environments into Google Cloud is costly. AWS charges $0.09/GB, and Azure charges $0.087/GB for outbound data transfer. This means that an ecommerce portal functioning on AWS but feeding the data to Google Cloud has enormous costs when it comes to data transfer.

Separately from the outbound data transfer costs, there is the issue of dormancy when moving data between cloud platforms. There’s a gap for consumers while the data is being transferred into Google Cloud and then loading it into BigQuery before the analysis is performed. It is these two issues that BigQuery Omni tackles – data transfer cost and dormancy.

What this fundamentally does is bring compute closer to data rather than moving the data to compute. This means that AWS and Azure customers can instantaneously use a BigQuery instance running within the same region, availability zone, and virtual network as their workloads.

With BigQuery Omni, consumers can use an Amazon S3 bucket or an Azure Storage container to assemble the data and telemetry to absorb and analyze data minus the effort to transfer the data outside of their environment. The query results from BigQuery Omni can be saved in the local storage account with zero cross-cloud transference.

Looker will remain the visualization tool for BigQuery Omni, as Looker has always supported AWS and Azure.

What This Means for Google

BigQuery Omni offers flexibility, cost benefits, and speed - while guaranteeing the fulfillment of data locality and data sovereignty policies. More significantly, it provides options for cloud data warehouse and analysis tools to consumers.

BigQuery Omni supports Google in being positioned as the brains of multi-cloud operations. Decreasing the friction of working in a multi-cloud environment aids in lessening the advantage maintained by the market leaders in the cloud space. This is what BigQuery does by automating the multi-cloud data aggregation process.

However, it is important to note that Google still has some catching up to do as it seized 6% of the global cloud infrastructure services spend in the first quarter of 2020. This is behind both Microsoft Azure’s 17% share and AWS’ 32% share, as released by Canalys.

About the Author

Avi Ben Ezra

Avi Ben Ezra is the Chief Technology Officer (CTO) and Cofounder of SnatchBot and SnatchApp (Snatch Group Limited). He leads the Group’s long-term technology vision and is responsible for running all facets of the tech business which includes being the architect of the platforms and UI interfaces.

Related Articles