Bigtable — the data model, the consistency that depends on replication, and why the row key is the design
Wide-column key-value storage at petabyte scale. The row/column-family/cell model, the instance-cluster-node structure with data on Colossus rather than on nodes, the consistency guarantee that changes with cluster count, HBase compatibility, and the row-key rule that decides whether it performs.
markdownreferencev1Q93Y-GE4Rstage: designtrust: officialfamily: databasesservice: bigtableinterface: anybigtablerow keycolumn familycolumn qualifiercellinstanceclusternodeColossusSSTablestrong consistencyeventual consistencyread-your-writesApache HBase
Bigtable
1. The data model
| Term | What it is |
|---|---|
| row key | the single index. One per row, and the only way rows are ordered |
| column family | a group of columns, declared on the table |
| column qualifier | identifies a column within its family |
| cell | the intersection of a row and a column, holding timestamped values |
Scale: "billions of rows and thousands of columns", terabytes to petabytes, at low latency.
There is one index and it is the row key. No secondary indexes, no joins, no query planner. Every access pattern the application needs must be expressible as a row-key lookup or a row-key range scan. That is the whole of Bigtable design, and it is why §4 is the most important section here.
2. Data does not live on the nodes
Instances contain clusters, clusters contain nodes — but the data is stored on Colossus, Google's file system, in SSTable format, not on the nodes.
Two consequences worth money:
- Nodes are compute, and resizing a cluster does not move data. Adding nodes rebalances which node serves
which tablet; it is not a data migration.
- Capacity and throughput are separate decisions. Storage grows on Colossus; throughput grows with nodes.
3. Consistency depends on how many clusters you have
| Instance shape | Consistency |
|---|---|
| single cluster | strongly consistent |
| multi-cluster | eventually consistent by default, with read-your-writes or strong consistency configurable |
This is the row most often got wrong. Adding a second cluster for availability silently changes the consistency model of an application that was written against a single cluster. The stronger options exist, and they are a configuration you must choose — not the default.
4. The row key is the design
The page's own guidance, in two halves that pull against each other:
- distribute writes evenly — avoid predictable ordering, because a monotonically increasing key sends every
write to one tablet;
- group related rows contiguously — so a read is one range scan rather than many lookups.
The page's example resolves both: a location identifier plus a timestamp spreads writes across tablets while keeping each location's rows together.
A timestamp-prefixed key is the canonical mistake: it is a perfectly reasonable schema that makes the whole cluster behave like one node.
5. HBase compatibility
Bigtable integrates with the "Apache HBase library for Java" and the broader HBase ecosystem, with client libraries in several languages.
For a migration that is the single most valuable fact here: an HBase application has a path onto Bigtable
that is a client-library change rather than a rewrite. It is also the cleanest "lift" in the whole database
family, and worth checking for in any Hadoop-estate assessment
(reference/data/dataproc.md).
6. For a cross-cloud estimate
Google's own table pairs Bigtable with Azure Table Storage; Microsoft's pairs Cloud Bigtable with Azure Table Storage too. Both are name matches at the "wide-column key-value" level and neither is a capability match — Table Storage is not a petabyte-scale, HBase-ecosystem store. The nearer AWS comparison is DynamoDB for the access-pattern discipline and Amazon Keyspaces or EMR-hosted HBase for the ecosystem.
The honest estimate line: treat a Bigtable move in either direction as a data-model exercise, except where
HBase is the source — in which case it is close to a lift. See guides/parity/databases.md §4.