Cloud Logging — buckets, sinks, views, log-based metrics, and the two buckets every project already has
The storage and routing model. Five object types, the _Required and _Default buckets, the query language, configurable retention, and a routing model whose destination can be a different resource from the origin — which is what makes a central logging project possible.
markdownreferencev12C6W-05H7stage: operatetrust: officialfamily: observabilityservice: cloud-logginginterface: anycloud logginglog entrylog bucketlog sinklog viewlog-based metrics_Required_DefaultLogging query languagelog routingexclusionBigQuery destinationPub/Sub destination
Cloud Logging
1. Five objects
| Object | What it is |
|---|---|
| log entry | one logged event |
| log bucket | where entries are stored, with a retention period |
| log sink | a route: what matches, and where it goes |
| log view | a scoped read over a bucket — how you grant access to some of the logs |
| log-based metric | a metric derived from matching log entries |
The log view is the one most often missed, and it is the answer to "the team needs the application logs but must not see the audit logs". Access is granted on a view, not only on a bucket.
2. The two buckets every project already has
_Required and _Default.
"log entries are stored for a specified length of time and are then deleted"
Recorded as an absence: the overview does not state the retention periods — they live in a separate
quotas document that was not read for this release. Do not assume a default retention; read it before
relying on a log being there. _Required's retention in particular is not customer-configurable on most
deployments, and the distinction between the two buckets is exactly that one of them is yours to configure.
What is configurable, in the page's own words: "which log data are stored, which are discarded, and where the log data are stored."
3. Routing, and the sentence that enables a central logging project
"when log data is routed, the destination can be in a different resource from where the log data originates."
Destinations: another Google Cloud project, a log bucket, a BigQuery dataset, a Cloud Storage bucket, or a Pub/Sub topic.
Four consequences worth designing around:
- a central logging project is the standard landing-zone pattern, and this sentence is why it works;
- BigQuery as a destination turns log analysis into SQL — and the cost model becomes BigQuery's
(reference/data/bigquery.md);
- Cloud Storage as a destination is the cheap long-retention answer; mind the class minimum durations
(reference/storage/cloud-storage.md §1);
- Pub/Sub as a destination is how logs reach a third-party SIEM, with no agent.
Exclusions matter as much as sinks. The routing model decides what is discarded, and high-volume low-value log lines are a real cost. In a migration, the sink-and-exclusion design is the thing that stops a log bill arriving as a surprise in month two.
4. Log-based metrics
A metric derived from log entries, which then behaves like any other metric in Cloud Monitoring — alertable,
chartable, usable in an SLO (reference/observability/cloud-monitoring.md).
This is the bridge between the two services, and it is how you alert on something that is only ever logged: an application error string, a specific audit event, a deprecation warning.
5. For a migration
Logging is nearly always under-estimated, because the volume is discovered rather than designed. Two measurements before the move:
- GB of logs per day, by source, because every destination's cost is per volume;
- what actually needs retaining, and for how long — which is a compliance answer, not an engineering one.
The counterparts are CloudWatch Logs and Azure Monitor Logs; see guides/parity/observability.md.