Query CloudWatch Logs with Logs Insights
Find the log lines behind a failure with CloudWatch Logs Insights — choose log groups and a narrow time range, write the query (fields, filter, parse, stats, sort, limit), run it from the console, CLI or TypeScript (StartQuery then poll GetQueryResults), stay inside the limits (50 log groups, 60-minute runtime, 10,000 rows per page), keep cost down, and know what the 2026 changes (Log Analytics console, new commands, OpenSearch SQL/PPL, Infrequent Access class) mean.
markdownguidev4VHXE-77N1stage: operatetrust: officialservice: cloudwatchinterface: consoleinterface: cliinterface: sdkCloudWatch Logs InsightsStartQueryGetQueryResultsaws logs start-querylogs tailquery syntaxfields filter stats parse sort limitdiscovered fields @timestamp @message @logStream @logJSON fields dot notationjsonParseSOURCE commandqueryLanguage CWLI SQL PPL50 log groupsquery timeout 60 minutes10000 resultsestimate bytes scannedLog Analytics consoleLive TailInfrequent Access log class queriescost of Logs Insights
Pick the fewest log groups and the narrowest time window, filter early, then aggregate. You pay for the data scanned, so a query over one log group for 15 minutes costs a fraction of one over ten groups for a week.
Where to run it
- Console: Since 2026-06-15, Log Analytics combines Logs Insights, Live Tail and Contributor Insights in one view, with several query tabs. It is the default; opting out shows the three tools separately. Cancel long queries before you close the page: they keep running, and you keep paying.
- Watch live:
aws logs tail /aws/lambda/<fn> --since 30m --follow. - CLI or code:
StartQuery, then pollGetQueryResults(below).
Query shape
fields @timestamp, @message, @logStream
| filter @message like /ERROR|Task timed out/
| sort @timestamp desc
| limit 50
- Commands are separated by
|, and#starts a comment.whereis an alias offilter. - Always-present fields (Standard log class):
@timestamp,@message,@ingestionTime,@logStream, and@log(account-id:log-group-name). - Service logs get discovered fields:
- Lambda:
@type,@requestId,@duration,@billedDuration,@memorySize,@maxMemoryUsed, and the X-Ray IDs. - VPC flow logs:
srcAddr,dstAddr,dstPort,action,bytesand the rest. - Route 53 query logs.
- JSON logs are flattened with dot notation (
userIdentity.arn), and arrays by index (items.0.instanceId). At most 200 fields are extracted. For JSON stored inside a string, usejsonParse(field). For text, useparse @message "* [*] *" as a, b, c(glob) or a regex with named groups. - Aggregate:
stats count(*) by bin(5m),stats avg(@duration), max(@duration) by bin(5m),dedup @requestId. - Recent additions (2026-07-15) include
topk,variance,countFrequent,sessionize,fillmissing,appendcols,cidrlookup, and functions such asparseDate,formatDate,jsonArrayContainsandmessageSize. The syntax page also listsjoin,lookup,subqueries,anomalyandpattern. - Before a big query:
estimatereturns the bytes it would scan, andlimit anystops scanning once enough rows are found.
Ready-made queries for the hub's error catalogues are in reference/cloudwatch/logs-insights-queries.md.
Limits
- 50 log groups per query, given as names or ARNs. ARNs must have no trailing
*. With the CLI or API,SOURCEin the query string can select log groups by prefix, account, class or tag instead; the console doesn't supportSOURCE. - 60 minutes of runtime. Split the time range if a query times out.
- Results:
GetQueryResultsreturns at most 10,000 rows per call.limitcan go up to 100,000 with paging (Logs Insights QL only). - Concurrent queries: one account-wide quota, shared with dashboard widgets and scheduled queries. Frequently refreshing dashboards use it up.
- Languages:
queryLanguageisCWLI(Logs Insights QL),SQL(OpenSearch SQL: log groups go in the query) orPPL(OpenSearch PPL). - Infrequent Access log class: cheaper to ingest. It supports Logs Insights QL, and since 2026-03-27 also PPL, SQL and data-protection masking, but not
pattern,diff,unmaskorfilterIndex.
From TypeScript
import { CloudWatchLogsClient, StartQueryCommand, GetQueryResultsCommand } from "@aws-sdk/client-cloudwatch-logs";
const logs = new CloudWatchLogsClient({});
export async function insights(logGroups: string[], query: string, minutesBack = 60) {
const end = Math.floor(Date.now() / 1000); // epoch SECONDS
const { queryId } = await logs.send(new StartQueryCommand({
logGroupIdentifiers: logGroups, // up to 50 names or ARNs
startTime: end - minutesBack * 60,
endTime: end,
queryString: query,
limit: 1000,
}));
for (;;) {
await new Promise((r) => setTimeout(r, 1000));
const res = await logs.send(new GetQueryResultsCommand({ queryId }));
if (res.status === "Complete") {
// Each row is [{ field, value }, ...]; turn it into an object.
return (res.results ?? []).map((row) => Object.fromEntries(row.map((c) => [c.field, c.value])));
}
if (res.status === "Failed" || res.status === "Cancelled" || res.status === "Timeout") {
throw new Error(`Query ${queryId} ended with status ${res.status}`);
}
}
}
const rows = await insights(["/aws/lambda/orders"],
'filter @type = "REPORT" | stats max(@duration) as maxMs, avg(@maxMemoryUsed/1000/1000) as avgMB by bin(5m)');
console.table(rows);
startTimeandendTimeare seconds. Log event timestamps andPutLogEventsuse milliseconds.MalformedQueryExceptionincludes aQueryCompileErrorwith the location of the error.LimitExceededException: the API says only that a maximum was reached. With many dashboards or parallel jobs, the concurrent-query quota is the likely one, so back off and retry.- Other errors are in
reference/cloudwatch/errors.md.
When the log group itself is the problem
Missing, unreadable or expiring log groups (retention, KMS, log classes, Lambda log format) are covered in reference/cloudwatch/log-groups.md.