Into The Source: Data lake planning, migration, and delivery through cloud native consulting

October 2024

Digital agency leverages DoiT's cloud expertise for complex data lake migration, achieving significant cost savings and operational efficiency.

27%: Decrease in cloud spend

“DoiT's support has enhanced our operational capabilities, cost efficiency, time savings, and team development, giving us a significant competitive advantage.”
Tim Vogt
Lead Developer, Into The Source

The Challenge

Into The Source needed to build a highly complex data lake project for client Vadain, requiring real-time visibility and predictable scale. The challenge involved supporting larger, complex programs with confidence and governance, developing a bespoke cloud platform capable of handling vast data amounts and processing thousands of orders per week.

The Solution

DoiT provided expert Cloud Reliability Engineers for technical support and optimization. The solution included data lake planning and migration using AWS Database Migration Service, immersion sessions and upskilling workshops, and implementation of DoiT Cloud Intelligence for ongoing optimization and insights.

Results

Achieved 27% reduction in cloud spending through DoiT Cloud Intelligence optimization
Reclaimed ~75 minutes per support ticket (51 hours YTD), freeing engineering capacity for higher-value work
Successfully migrated complex data lake with real-time visibility and scalable AWS infrastructure
Empowered development team with AWS knowledge and tools for ongoing maintenance and optimization

Meet Into The Source

Into The Source is a digital agency that helps companies, brands, and organizations grow by delivering full-service software solutions. It leverages creativity, data, knowledge, and expertise to create bespoke and user-centric approaches to meet complex client requirements. The company prides itself on building data-driven digital solutions, believing that delivering powerful digital experiences leads to customer loyalty. Its client base ranges from sports clubs and hardware stores to construction and retail organizations, providing custom solutions for business analysis, UX/UI design, SEO, software development, and integration.

Data Lake Planning and Migration

DoiT began with an initial assessment and planning phase, understanding platform needs and aligning data lake goals with stakeholders. In collaboration with Into The Source, DoiT developed an architectural diagram for the data lake with clear runbooks and guardrails. The team identified key components including data ingestion, storage, processing, and analytics while highlighting risk mitigation strategies. Using AWS Database Migration Service, DoiT assisted with migrating large amounts of data from the data warehouse, ensuring on-premise compatibility through meticulously planned phases over several months.

Expert Training and Upskilling

DoiT Cloud Engineers conducted workshops and helped bridge specific knowledge gaps within the team. This proved crucial in equipping Into The Source's development team with necessary tools for ongoing fixes and proactive AWS optimizations. The training empowered internal teams and provided long-term tools to achieve client goals and objectives for similar future projects.

Cloud Intelligence for Optimization

DoiT Cloud Intelligence provided Into The Source with automated solutions and easy-to-implement recommendations. The platform enabled the team to optimize pricing, eliminate waste and redundancy, and better align requested resources to actual usage. This gave the company better insights into costs and optimizations, helping identify areas of potential overspending or inefficiency across both individual developers and business operations.

Measurable Results and Impact

The support from DoiT empowered Into The Source's developers, enhancing AWS knowledge and capability to manage complex data lake processes with confidence. The team reclaimed approximately 75 minutes per support ticket, freeing engineering capacity for higher-value work. Since implementing DoiT Cloud Intelligence, Into The Source achieved a 27% reduction in cloud spending, with savings strategically reinvested into new projects and innovations for further growth and development.

Future Roadmap

The data lake project continues with DoiT supporting Into The Source's journey to secure ISO accreditation through clear roadmaps and governance guardrails. This involves strengthening security posture and compliance on AWS, with prioritized roadmapping to determine the best strategy for achieving ISO objectives. The ongoing partnership positions Into The Source for continued growth and competitive advantage.

See how DoiT helps cloud teams control spend

Explore how DoiT Cloud Intelligence helps teams improve visibility, governance, and unit economics across cloud environments.

Visit DoiT Talk to our team

Story overview

What they say

DoiT was a true partner, not a vendor. They helped us understand the problem, refine the vision, and build something production-ready far faster than we could have on our own. Their expertise, responsiveness, and commitment made all the difference.

Dr. Anish Kapur, Founder & CEO, Promptly

DoiT's Customer Success and Forward Deployed Engineering teams work very closely with us. The regular sessions with our CSM keep us focused on the right priorities, and the FDEs provide the deep technical guidance we need to validate decisions and optimize our environment. That combination has been genuinely valuable for us.

Alexander Lundberg Santos, Platform Engineer at Extenda Retail

When we started working with DoiT, we deployed Flexsave to save time and reduce complexity. We still use it today. But what really stands out is the expert support. Having someone to collaborate with on deep cloud cost topics, someone who really understands the nuances, is incredibly valuable.

Jesper Terkelsen, CTO at Monta

DoiT's Cloud Accelerator turned our AI idea into a shipped product, saving at least three months of development and delivering reliable, explainable insights our customers trust.

Scott Desgrosseilliers, CEO and co-founder, Wicked Reports

DaySmart

Working with DoiT gave us the speed of a startup and the rigor of an enterprise R&D lab. We launched an AI feature in ninety days with zero engineering lift, unified our cloud visibility across multiple products and built a foundation to scale AI responsibly.

Tim Green, SVP of AI, DaySmart

Our AWS environment had grown rapidly, and complexity was outpacing control. DoiT helped us bring structure and clarity, aligning financial and technical decisions through a shared foundation.

Maurizio Trezzi, Director of IT, Vivaticket

DoiT gives us contextual insight into what's happening in our cloud environment. That helps us ensure we're meeting customer needs while managing our costs. It's that central pane of glass that lets engineering and finance work together, powered by the technology and backed by DoiT's people.

Matt Warner, Co-Founder and CEO at Blumira

For anyone considering DoiT: do it. The tools pay for themselves. We've reduced costs, improved visibility, and built a platform that can scale globally with control and confidence.

Matej Serbec, CTO, TradeLocker

DoiT didn't just give us a product – they gave us clarity. Cloud Diagrams has helped our teams understand what we're running, why we're running it, and where we can improve. The real-time visibility is a game changer for engineering, leadership, and everything in between.

Donnie Green, VP of Platform Engineering, Bloomerang

Sentric Music

With maybe 5-10 minutes, Ava had guided me right down into the low-level detail of the services that we were running in our compute environment, named the exact instances that were starting to chew through our money, produced a report for me that showed that this was almost a direct correlation to the cost growth that we were seeing, which meant that we were able to get on top of it pretty much immediately.

Stephen Roughley, Head of Engineering at Sentric Music

Atadex

With DoiT Cloud Intelligence™ managing our procurement and DoiT's engineers acting as an extension of my team, my costs are down, my efficiency is up, and my profitability is up

Mitch Bernet, Founder and CEO at Atadex

Skyral

A key takeaway for me working with DoiT is that the level of professionalism is extremely high. They're an obvious choice if you're looking for a partner to help implement best practices in your cloud environment and reduce cost.

Oliver Young, Senior Technical Manager at Skyral

We knew WhatsApp was the perfect channel for global remittances, but building secure, real-time conversations across languages was a massive challenge. DoiT helped us break through. They didn't just advise, they helped us architect and launch a GenAI solution that's 100x more powerful than what we had. We're now in a completely different league.

Jaime Gonzalez-Gasque, CEO, Reimagination Technologies

Your cloud bill shouldn't be a mystery

Let us show you what ships this week.

Book a demo