Azure Data Collection Endpoint (DCE) provide a connection for certain data sources of Azure Monitor.

This article gives you an introduction to DCE – and when and how to use it.

This blog-post is part of a series of blog posts to master Azure logging in depth (overview).

To get you started, you can find ARM-templates & scripts in my AzureLogLibrary (github). Details will be covered in the articles.

Quick links

When do I need a DCE ?
Dataflow of DCE use-cases
Example sending data into custom log
Example sending data using network isolation
Detailed example with AMPLS and network isolation
What does a DCE consist of ?
Regionality
Naming of DCEs
Limitations
Layout of DCE (JSON)
Change Network isolation of DCE in GUI
How to deploy DCE using GUI ?
How to deploy DCE using ARM-templates ?

When do I need a DCE ?

DCE is as a connector between the endpoint and Azure Log Ingestion Pipeline.

A DCE is required in 2 occasions:

• You need network isolation

• You are sending data to custom logs in Azure LogAnalytics

Dataflow of DCE use-cases

Example sending data into custom log

Here you can see that Azure Monitor Agent collections into standard tables are not using DCEs. But below you can see an example, where a script is using Log Ingestion API and sending data into custom log tables.

Example sending data using network isolation

By default connectivity is happening by a public network endpoint. You can also choose to use Azure Monitor Private Link Scope (AMPLS). I will cover this in a later blog-post.

Detailed example with AMPLS and network isolation

Here is a detailed illustration of the network and usage of DCE in a network isolation scenario.

What does a DCE consist of ?

A DCE includes the following components:

Component	Description
Configuration access endpoint	The endpoint used to access the configuration service to fetch associated data collection rules (DCRs) for Azure Monitor Agent. Example: <unique-dce-identifier>.<regionname>.handler.control.
Logs ingestion endpoint	The endpoint used to ingest logs to Log Analytics workspaces. Example: <unique-dce-identifier>.<regionname>.ingest.
Network access control lists	Network access control rules for the endpoints.

Regionality

Data collection endpoints are resources created within specific regions.

An endpoint in a given region can only be associated with machines in the same region.

This means that if your LogAnalytics workspace is created in WestEurope, your DCE must also be placed in WestEurope.

You can have more than one endpoint within the same region according to your needs.

Each DCE will get a unique IP address, which can be whitelisted in your firewall.

Naming of DCEs

Typically I will create a DCE per collection type and use the naming to describe the collection type – and where I am using the data (destination). Here are a few naming examples:

dce-ampls-srv-weu-t
dce-iis-logs
dce-iis-logs-westeurope
dce-java-textlogs-westeurope
dce-log-platform-management-client-p
dce-platform-management-srvnetworkcloud-p

Limitations

Data collection endpoints only support Log Analytics workspaces as a destination for collected data. 

Custom metrics (preview) collected and uploaded via Azure Monitor Agent aren’t currently controlled by DCEs. They also can’t be configured over private links.

Layout of DCE (JSON)

The layout of a DCE consists of important properties defining the immutableId, configurationsAccess and log ingestion endpoint Uri. Furthermore you control if you allow publicNetworkAccess:

{
    "properties": {
        "immutableId": "dce-e9ddc3ad1e50418cb951e967c58870b5",
        "configurationAccess": {
            "endpoint": "https://dce-platform-management-srvnetworkcloud-p-9gct.westeurope-1.handler.control.monitor.azure.com"
        },
        "logsIngestion": {
            "endpoint": "https://dce-platform-management-srvnetworkcloud-p-9gct.westeurope-1.ingest.monitor.azure.com"
        },
        "networkAcls": {
            "publicNetworkAccess": "Enabled"
        },
        "provisioningState": "Succeeded"
    },
    "location": "westeurope",
    "id": "/subscriptions/xxxxx4d8-bf1701b862c3/resourceGroups/rg-dce/providers/Microsoft.Insights/dataCollectionEndpoints/dce-platform-management-srvnetworkcloud-p",
    "name": "dce-platform-management-srvnetworkcloud-p",
    "type": "Microsoft.Insights/dataCollectionEndpoints",
    "etag": "\"bb00a0e1-0000-0d00-0000-64045da20000\"",
    "systemData": {
        "createdBy": "mok@2linkit.net",
        "createdByType": "User",
        "createdAt": "2023-03-05T09:15:11.3267788Z",
        "lastModifiedBy": "mok@2linkit.net",
        "lastModifiedByType": "User",
        "lastModifiedAt": "2023-03-05T09:15:11.3267788Z"
    }
}

You can also find the details in the GUI. You will need these information, if you want to send data using Azure Log Ingestion API via REST endpoint.

Change Network isolation of DCE in GUI

If you want to change a DCE and use it in a more secure configuration, you can change the settings under network isolation

Here is an example with Azure Monitor private Link Scope (AMPLS)

How to deploy DCE using GUI ?

Azure Data Collection Endpoint is configured in Azure Monitor

How to deploy DCE using ARM-templates ?

You can check out the ARM-template on my Github:

Generic Data Collection Endpoint (arm-template)

Data Collection Endpoint for collection of TXT-logs – used with DCR

Data Collection Endpoint for collection of IIS-logs – used with DCR

I have also provided more documentation and powershell script to deploy more DCRs and DCEs

You can also choose to click and do the deployment directly to your Azure environment

Deploy Generic Data Collection Endpoint (DCE)

Deploy DCE for TXT-log collection

Deploy DCE for IIS-log collection