---
cuppy_hub: azure
cuppy_path: reference/monitor/diagnostic-settings-and-data-collection.md
cuppy_section: docs
cuppy_doc: 030M-7PH3
cuppy_doc_uuid: 95104ccd-6191-4f33-a346-460b1211a79a
cuppy_permalink: https://cuppy.io/azure/d/030M-7PH3
cuppy_version: 1
cuppy_updated: 2026-10-23T16:16:52.575Z
cuppy_canonical: https://cuppy.io/azure/docs/reference/monitor/diagnostic-settings-and-data-collection
cuppy_mcp: https://cuppy.io/mcp
cuppy_body_sha256: 5478d59c6e57ff75e4fd49096895a06a6ef2b0b56491005b8d182d9979915cac
# cuppy_body_sha256 is over the document body ALONE, excluding both generated blocks.
---


# Getting data into Azure Monitor

## Diagnostic settings (platform logs and metrics)

Resource logs aren't collected until a diagnostic setting sends them somewhere; metrics and the activity log are collected anyway.

| Rule | Detail |
|---|---|
| Count | **Five settings per resource** (fixed) |
| Destinations | Log Analytics, storage, Event Hubs, partner solutions; **at most one of each type per setting** (two workspaces need two settings); the destination **must already exist**; storage and Event Hubs must be **in the resource's region**; other subscriptions are fine with RBAC on both |
| Categories | Pick categories, or a **category group**: `allLogs` or `audit` (not both needed; groups follow Microsoft's changes). `audit` on SQL Database doesn't turn on SQL auditing |
| Collection mode | **Resource-specific** tables (`logAnalyticsDestinationType: 'Dedicated'`, CLI `--export-to-resource-specific true`) where the service supports them, rather than the shared `AzureDiagnostics` table |
| Metrics | Only `AllMetrics` for most services; metrics with dimensions arrive flattened; some metrics aren't exportable |
| Cost | Ingestion is billed, and so is export of some categories. Collect only needed categories; skip metrics unless you query them in logs |
| Activity log | A subscription-level setting: `az monitor diagnostic-settings subscription create` (or ARM), not the resource command |

```bicep
// Send a Key Vault's resource logs and metrics to a workspace, into resource-specific tables
param keyVaultName string
param workspaceId string

resource kv 'Microsoft.KeyVault/vaults@2026-05-15' existing = {
  name: keyVaultName
}

resource toWorkspace 'Microsoft.Insights/diagnosticSettings@2021-05-01-preview' = {
  scope: kv
  name: 'to-log-analytics'
  properties: {
    workspaceId: workspaceId
    logAnalyticsDestinationType: 'Dedicated' // resource-specific tables, not AzureDiagnostics
    logs: [
      {
        categoryGroup: 'allLogs'
        enabled: true
      }
    ]
    metrics: [
      {
        category: 'AllMetrics'
        enabled: true
      }
    ]
  }
}
```

`az monitor diagnostic-settings categories list --resource <id>` shows what a resource offers. Diagnostic settings have only the `2021-05-01-preview` API version.

| Symptom | Cause → fix |
|---|---|
| "Metric category 'xxxx' is not supported" | Use `AllMetrics` |
| The setting disappears | Non-ASCII characters in the resource ID (not supported): re-create the resource or move it to a group without them |
| No data yet | Allow **up to 90 minutes**; the table appears with the first record. Nothing after 24 hours: the resource may emit no logs; disable and re-enable the setting |
| Duplicate Application Insights rows | Its diagnostic setting can't target the workspace the resource is based on |

## Data collection rules (DCRs)

A DCR says what to collect, how to transform it (KQL `transformKql`) and where to send it. Create it **in the destination workspace's region**. Limits per DCR: 10 data sources, 20 data flows and 20 streams, 10 workspaces, 15,360 characters per transformation.

**Azure Monitor Agent** (extension `AzureMonitorLinuxAgent` or `AzureMonitorWindowsAgent`, publisher `Microsoft.Azure.Monitor`) needs a **managed identity** on the VM: user-assigned for fleets, system-assigned for tests; Arc servers use system-assigned only. Collection starts when a **data collection rule association** links the machine to a DCR (many to many). In CLI, DCRs and associations come from the `monitor-control-service` extension (the `monitor data-collection` command group), not core. A **data collection endpoint** is needed only for private link, or for DCRs created before March 31, 2024. The old Log Analytics agent (MMA) and the Azure Diagnostics extension are retired: `reference/monitor/what-to-use.md`.

## Logs Ingestion API (your own records)

It replaces the HTTP Data Collector API (end of support September 14, 2026). You need a custom table (`_CL`), a DCR whose **stream** matches what you send, and **Monitoring Metrics Publisher on the DCR** for the sender. A `Direct` DCR gets its own `logsIngestion` endpoint (an existing DCR can't gain one). Keep the DCR, workspace and any DCE in one region.

```bicep
// A custom table and a Direct DCR for the Logs Ingestion API, with an app identity allowed to send
param location string = resourceGroup().location
param workspaceName string
param senderPrincipalId string // the app's managed identity (object ID)

resource ws 'Microsoft.OperationalInsights/workspaces@2026-03-01' existing = {
  name: workspaceName
}

resource table 'Microsoft.OperationalInsights/workspaces/tables@2026-03-01' = {
  parent: ws
  name: 'AuditEvents_CL' // custom tables must end in _CL
  properties: {
    schema: {
      name: 'AuditEvents_CL'
      columns: [
        { name: 'TimeGenerated', type: 'datetime' }
        { name: 'Computer', type: 'string' }
        { name: 'Action', type: 'string' }
        { name: 'Outcome', type: 'string' }
      ]
    }
  }
}

resource dcr 'Microsoft.Insights/dataCollectionRules@2024-03-11' = {
  name: 'dcr-audit-events'
  location: location // same region as the workspace
  kind: 'Direct' // gives the DCR its own logsIngestion endpoint
  properties: {
    streamDeclarations: {
      'Custom-AuditEvents': {
        columns: [
          { name: 'TimeGenerated', type: 'datetime' }
          { name: 'Computer', type: 'string' }
          { name: 'Action', type: 'string' }
          { name: 'Outcome', type: 'string' }
        ]
      }
    }
    destinations: {
      logAnalytics: [
        { name: 'ws', workspaceResourceId: ws.id }
      ]
    }
    dataFlows: [
      {
        streams: ['Custom-AuditEvents']
        destinations: ['ws']
        transformKql: 'source | where Action != "heartbeat"'
        outputStream: 'Custom-AuditEvents_CL'
      }
    ]
  }
  dependsOn: [table]
}

var monitoringMetricsPublisher = '3913510d-42f4-4e42-8a64-420c390055eb'

resource canSend 'Microsoft.Authorization/roleAssignments@2022-04-01' = {
  scope: dcr
  name: guid(dcr.id, senderPrincipalId, monitoringMetricsPublisher)
  properties: {
    roleDefinitionId: subscriptionResourceId('Microsoft.Authorization/roleDefinitions', monitoringMetricsPublisher)
    principalId: senderPrincipalId
    principalType: 'ServicePrincipal'
  }
}

output immutableId string = dcr.properties.immutableId
output logsIngestionEndpoint string = dcr.properties.endpoints.logsIngestion
```

```ts
import { DefaultAzureCredential } from "@azure/identity";
import { LogsIngestionClient, isAggregateLogsUploadError } from "@azure/monitor-ingestion";

const credential = new DefaultAzureCredential();

// Needs Monitoring Metrics Publisher on the data collection rule.
export async function sendAuditRows(): Promise<void> {
  const ingest = new LogsIngestionClient(process.env.DCR_LOGS_INGESTION_ENDPOINT ?? "", credential); // the DCR's logsIngestion endpoint, or a DCE
  const rows = [{ TimeGenerated: new Date().toISOString(), Computer: "app-01", Action: "login", Outcome: "success" }];
  try {
    await ingest.upload(process.env.DCR_IMMUTABLE_ID ?? "", "Custom-AuditEvents", rows); // dcr-…, stream from streamDeclarations
  } catch (e) {
    if (isAggregateLogsUploadError(e)) {
      for (const f of e.errors) console.error(f.cause, f.failedLogs.length); // batches that failed after the SDK's retries
    }
    throw e;
  }
}
```

| Error | Cause → fix |
|---|---|
| 403 | The sender lacks Monitoring Metrics Publisher on the DCR, or the assignment is new: **wait up to 30 minutes** |
| 413, or `TimeoutExpired` with `ReadyBody_ClientConnectionAbort` | Over **1 MB per call**: send smaller batches |
| 429 | Over the per-DCR limits (2 GB per minute, 12,000 requests per minute): wait for `Retry-After` |
| No rows | The first send to a new table can take **up to 15 minutes** |
| Log Analytics doesn't suggest the new table | IntelliSense can lag **24 hours**; the query still works |

The hub's reading: a stream or column mismatch between the payload, `streamDeclarations` and the table is the usual cause when calls succeed but columns stay empty.


---
<!-- cuppy -->
You are reading one file. The same content is free over MCP at https://cuppy.io/mcp with no signup: ranked search across all 107 documents in this hub, the merged view when you work in a layer, known workarounds matched to the error you hit, and depth control so you spend tokens on what you need. Start with the `_hello` tool. How to connect: https://cuppy.io/azure/ Something wrong in this document? Report it with the `session feedback` tool on the same endpoint.
<!-- /cuppy -->