Home/ Blog/ Article

Data warehouse or data lake: choosing a data architecture for a growing business

·

At some point every growing business asks the same question: where should our data actually live? The answer usually gets tangled up with vendor marketing, because “data warehouse”, “data lake” and “lakehouse” have become labels attached to entire product ecosystems rather than descriptions of what the systems do. Strip that away and the decision is narrower and more answerable than it looks: what shape is your data in, who needs to query it, how quickly, and who is going to maintain the thing after it ships.

What the terms actually mean

A data warehouse stores structured data in predefined schemas, optimised for fast SQL queries over known tables: revenue by month, active users by cohort, orders by region. You define the shape of the data before you load it, which means every report built on top of it can trust that shape not to shift underneath it.

A data lake stores raw data in whatever format it arrives: JSON events, log files, images, PDFs, sensor readings, half-structured exports from a third-party API. You decide what shape you need at query time, not at load time. That flexibility is the entire point, and it is also where the complexity goes: querying a lake well takes more engineering than querying a warehouse well.

A lakehouse is an attempt to get both: lake-style storage of raw files, with a transactional layer on top (Delta Lake, Iceberg, Hudi) that gives you warehouse-style schema enforcement and fast SQL where you need it. It is a genuinely useful pattern, and it is also more moving parts than a small team should take on before they have a reason to.

The questions that actually decide it

What shape does your data arrive in?

If most of what you need to store is transactional records, CRM exports, billing data and event logs that already have a consistent structure, a warehouse gets you to useful dashboards faster with less to maintain. If a meaningful share of your data is unstructured or semi-structured — documents, images, raw API payloads, IoT telemetry with varying field sets across device firmware versions — forcing it into a rigid schema early either loses information or costs you constant schema migrations. That is a lake’s actual use case, not a preference.

Who is running the queries?

A warehouse is built for people who write SQL and expect consistent, fast answers: finance, operations, a BI tool refreshing dashboards every morning. A lake is built for people who write code against the data: data scientists building models, engineers doing exploratory analysis, pipelines that need the full raw event rather than a pre-aggregated summary. If your primary users are non-technical stakeholders who need reliable numbers on a schedule, that alone often settles the question in favour of a warehouse, or a warehouse fed by a lake underneath it.

How fresh does the answer need to be?

Warehouses are generally batch-loaded on a schedule — hourly, nightly — which is fine for reporting and slow-moving decisions. If you need near-real-time answers (fraud checks on incoming transactions, live fleet telemetry alerts), you are looking at streaming ingestion into either architecture, and the harder problem is the pipeline, not the storage layer. Don’t let a genuine streaming requirement get solved by choosing a lake when what you actually need is a different ingestion pattern.

Who maintains it once it’s built?

This is the factor teams underweight most. A managed cloud warehouse (BigQuery, Snowflake, Redshift, or Postgres with good indexing for smaller volumes) is close to zero-maintenance: you write SQL, you don’t manage compute clusters. A lake built on object storage plus a query engine (S3 with Athena, or a Spark-based setup) needs someone who understands partitioning, file formats and query engine tuning, or query costs and latency both degrade quietly over time. A lakehouse needs that plus an understanding of the transactional layer itself. If there is no one on the team who wants to own that, the honest move is to pick the option with the smallest maintenance surface, even if it is theoretically less flexible.

What does it actually cost?

The common assumption is that lakes are cheaper because object storage is cheap. That’s true for storage and false for the whole system. A badly partitioned lake generates query costs that dwarf its storage bill, because every query scans more data than it needs to. A warehouse’s cost is more predictable — you’re generally paying for compute on structured queries — but can climb fast if dashboards run frequent, unoptimised queries against large tables. Neither architecture is inherently cheaper; both are cheap when someone designs the schema and partitioning with query patterns in mind, and expensive when no one does.

Where the lakehouse pattern earns its complexity

The lakehouse pattern is worth the extra moving parts when you genuinely have both needs at once: raw, varied data feeding machine learning or exploratory work, and the same underlying data needing to power reliable, governed dashboards for the business side. That combination is real for companies doing serious AI work on top of operational data. It is not the right starting point for a company that has one Postgres database, a handful of SaaS tools to pull data from, and a need for a monthly board report. Adopting lakehouse tooling before you have outgrown a warehouse is a common way for a small data project to turn into a permanent infrastructure commitment nobody signed up for.

A path that doesn’t force the decision early

You don’t have to pick the final architecture on day one. A pragmatic sequence for most growing businesses: start with a warehouse for the structured, reporting-critical data you already have, because it delivers value fastest with the least operational overhead. Land anything raw or unstructured — logs, documents, device telemetry — in cheap object storage from the start, even without a query layer on top, because raw data you didn’t keep is gone for good, while a lake you build later can be backfilled from storage you already have. Add lake or lakehouse tooling only when a specific, named use case needs it: a model that needs raw event history, or a reporting need that genuinely spans structured and unstructured sources. That sequencing avoids paying lakehouse-level complexity for warehouse-level requirements.

Checklist before you commit

  • Can you name the reports and dashboards this needs to power, and are they built on structured data you already control the schema of?
  • Is there unstructured or highly variable data in scope that a rigid schema would force you to discard or flatten?
  • Who queries this day to day — analysts writing SQL, or engineers and data scientists writing code?
  • Is there someone on the team, in-house or contracted, who will own partitioning, query performance and cost monitoring on an ongoing basis?
  • Does the freshness requirement actually demand streaming, or is a scheduled batch load genuinely fine?
  • Would a warehouse fed by cheap raw storage underneath it meet the need for the next twelve months, even if it isn’t the theoretically ideal architecture?

If most of those answers point at structured data, scheduled reporting and a small team, a warehouse will serve you better than its reputation suggests. If they point at raw, varied data feeding exploratory or AI work, a lake — or eventually a lakehouse — is solving a problem you actually have, not one you might have someday.

Filed under: