How Huq built a powerful data platform to deliver near real-time location data at scale

January 2021

DoiT helped Huq migrate to Google Cloud and optimize costs while scaling data processing capabilities

50%: cost reduction
4x: data volume increase

“We're a compact technical team, which means we're able to move very fast and rapidly respond to changes. But when you're adopting new technologies, you need to leverage cutting-edge expertise. The key reasons we chose DoiT International were the quality of communication and the team's deep technical know-how.”
Isambard Poulson
Co-founder and CTO, Huq

The Challenge

Huq's customers required increasingly fast turnaround times and more regular access to high-frequency data. The company needed to adapt its data pipeline to respond to customer demands in near real time while maintaining flexibility for customers with varying technical needs.

The Solution

DoiT helped Huq migrate its core data processing pipeline from AWS to Google Cloud using Apache Beam and Google Dataflow. The solution included workflow orchestration with Cloud Composer, data storage with Cloud Storage and BigQuery, and cost optimization through reOptimize CMP and superQuery IDE.

Results

Cut cloud costs by 50% while data volumes quadrupled from 200 million to 800 million data points daily
Built scalable architecture capable of processing almost one billion geo-location records per day
Achieved ability to scale data processing 100-fold without stack changes

Meet Huq

Huq measures real-world factors that affect businesses to help them gain a holistic view of their marketplace. The company produces economic and societal benchmarks using mobility data that serves as a proxy for footfall in retail, transport, town centers, and workplaces. This anonymized resource incorporates high volumes of rich, descriptive data that enables complex analysis of consumer behavior.

Finding the right solutions based on a tailored approach

Isambard turned to DoiT in May 2020 on recommendation from another client. DoiT impressed Huq with its tailored approach, taking time to understand requirements rather than pushing existing products. The onboarding process was smooth and the available tools were highly compelling for Huq's needs.

Building a scalable data pipeline

DoiT helped Huq complete its migration to Google Cloud, building a data pipeline that rapidly ingests, processes, analyzes, and exports almost one billion geo-location records daily. The solution uses Google Cloud Composer for workflow orchestration, Cloud Storage for data loading, Dataflow for enrichment, and BigQuery for client access.

Staying on top of costs with optimization tools

DoiT provided reOptimize CMP for enhanced visibility into cloud usage and cost optimization. Huq also gained access to superQuery IDE, an AI-powered development environment for BigQuery that enables more efficient querying and flexible customer response.

Dramatic cost savings while scaling

With DoiT's help, Huq cut its cloud bill in half while data needs quadrupled. The company now processes 800 million data points daily compared to 200 million previously, with costs falling by 50%. The new stack can scale data processing 100-fold without changes.

See how DoiT helps cloud teams control spend

Explore how DoiT Cloud Intelligence helps teams improve visibility, governance, and unit economics across cloud environments.

Visit DoiT Talk to our team

Story overview

What they say

DoiT was a true partner, not a vendor. They helped us understand the problem, refine the vision, and build something production-ready far faster than we could have on our own. Their expertise, responsiveness, and commitment made all the difference.

Dr. Anish Kapur, Founder & CEO, Promptly

DoiT's Customer Success and Forward Deployed Engineering teams work very closely with us. The regular sessions with our CSM keep us focused on the right priorities, and the FDEs provide the deep technical guidance we need to validate decisions and optimize our environment. That combination has been genuinely valuable for us.

Alexander Lundberg Santos, Platform Engineer at Extenda Retail

When we started working with DoiT, we deployed Flexsave to save time and reduce complexity. We still use it today. But what really stands out is the expert support. Having someone to collaborate with on deep cloud cost topics, someone who really understands the nuances, is incredibly valuable.

Jesper Terkelsen, CTO at Monta

DoiT's Cloud Accelerator turned our AI idea into a shipped product, saving at least three months of development and delivering reliable, explainable insights our customers trust.

Scott Desgrosseilliers, CEO and co-founder, Wicked Reports

DaySmart

Working with DoiT gave us the speed of a startup and the rigor of an enterprise R&D lab. We launched an AI feature in ninety days with zero engineering lift, unified our cloud visibility across multiple products and built a foundation to scale AI responsibly.

Tim Green, SVP of AI, DaySmart

Our AWS environment had grown rapidly, and complexity was outpacing control. DoiT helped us bring structure and clarity, aligning financial and technical decisions through a shared foundation.

Maurizio Trezzi, Director of IT, Vivaticket

DoiT gives us contextual insight into what's happening in our cloud environment. That helps us ensure we're meeting customer needs while managing our costs. It's that central pane of glass that lets engineering and finance work together, powered by the technology and backed by DoiT's people.

Matt Warner, Co-Founder and CEO at Blumira

For anyone considering DoiT: do it. The tools pay for themselves. We've reduced costs, improved visibility, and built a platform that can scale globally with control and confidence.

Matej Serbec, CTO, TradeLocker

DoiT didn't just give us a product – they gave us clarity. Cloud Diagrams has helped our teams understand what we're running, why we're running it, and where we can improve. The real-time visibility is a game changer for engineering, leadership, and everything in between.

Donnie Green, VP of Platform Engineering, Bloomerang

Sentric Music

With maybe 5-10 minutes, Ava had guided me right down into the low-level detail of the services that we were running in our compute environment, named the exact instances that were starting to chew through our money, produced a report for me that showed that this was almost a direct correlation to the cost growth that we were seeing, which meant that we were able to get on top of it pretty much immediately.

Stephen Roughley, Head of Engineering at Sentric Music

Atadex

With DoiT Cloud Intelligence™ managing our procurement and DoiT's engineers acting as an extension of my team, my costs are down, my efficiency is up, and my profitability is up

Mitch Bernet, Founder and CEO at Atadex

Skyral

A key takeaway for me working with DoiT is that the level of professionalism is extremely high. They're an obvious choice if you're looking for a partner to help implement best practices in your cloud environment and reduce cost.

Oliver Young, Senior Technical Manager at Skyral

We knew WhatsApp was the perfect channel for global remittances, but building secure, real-time conversations across languages was a massive challenge. DoiT helped us break through. They didn't just advise, they helped us architect and launch a GenAI solution that's 100x more powerful than what we had. We're now in a completely different league.

Jaime Gonzalez-Gasque, CEO, Reimagination Technologies

Your cloud bill shouldn't be a mystery

Let us show you what ships this week.

Book a demo