Google recently announced two minor price changes for BigQuery later this year. Users of BigQuery (BQ) in conjunction with Google Cloud Storage (GCS) have received one or two emails regarding a price modification. As those emails look pretty similar, I am writing this blog post to explain the differences and how you can minimize the impact on your cloud bill.
What are the new Charges?
The first change will be Effective on October 21, 2024:
Google has started charging network costs for data transfer to other regions with two exceptions. Transfers between BQ hosted in the EU multi-region and europe-west4, and, the US multi-region and us-central1 are still free. These multi-regions store their data (most of the time) in the associated regions, so there isn’t really a transfer of data here.
For example, if you clone or copy from EU multi-region to europe-west2 (London) you get charged for the network cost.
The SKU’s will be covered by:
- Network Data Transfer GCP Multi-region within {{REGION}}
- Network Data Transfer GCP Replication within {{REGION}}
More info about the cost can be found on the pricing page.
The second change will be effective on November 1, 2024:
Google is starting to charge for read and write operations performed by BigQuery, such as reading data from an external table stored in GCS. In the past, there were no charges.
This means that the bucket owner will be charged for:
- Retrieval fees for Coldline, Nearline, and Archive storage.
- Inter-region network data transfer fees for reading data stored in a Cloud Storage bucket from a BigQuery dataset in a different location/regions.
How to minimize the impact on your cloud bill
Likewise, it is essential to understand that BQ multi-region is considered a different location type than GCS multi-region. This means that you need to be mindful of egress charges between BQ and other services, especially when using multi-region.
To validate this, you could check your billing data to see if you have SKUs for different locations, as this is an indication of this happening.
To minimize inter-region data transfer charges, store data in a Cloud Storage bucket local to the destination BigQuery dataset if you import data from remote Cloud Storage locations. For example:
- Move to single regions as they are best suited for most workloads integrated with other Google Cloud services:
If you are reading data into BigQuery US multi-region, consider placing your Cloud Storage bucket in us-central1.
- If you are reading data into BigQuery us-west1, consider placing your Cloud Storage bucket in us-west1.
- Additionally, if you want to co-locate an existing Cloud Storage bucket to your BigQuery region, you can use the gcloud storage command or Storage Transfer Service to move your bucket.
To minimize retrieval fees, consider
- Enabling Autoclass on the bucket
- Migrate data from Nearline, Coldline, or Archive to Standard storage.
The following SKUs cover these charges:
- Archive Data Retrieval
- Coldline Data Retrieval
- Nearline Data Retrieval
DoiT Cloud Navigator makes it easy to monitor the impact of this change.
DoiT offers the DoiT Cloud Navigator, with Cloud Navigator you can easily understand, optimize, and control your cloud costs. You can use pre-built or custom dashboards to get more grip on the cost.
If you are a DoiT customer, go to:
https://console.doit.com/ → analytics → Reports
Create a new report:
- Add “Service Filter:
- Select Google Cloud Storage and BigQuery.
- Group the data by SKU.
- Run the report!
Screenshot Cloud Navigator
You will be able to see the relevant SKUs (see above for details) and understand the impact on your bill (after the changes are effective).
Need our guidance or help?
Visit doit.com/services and learn how we can help today!