I am really passioned about the logging capabilities in M365 Defender and Azure with the power to bring data back from clients, servers, cloud and 3rd party systems – and getting cool valuable information out of the data – besides of course for security hunting.
As outlined in the last section of this intro/overview, I have prepared a series of blog posts to master Azure logging in depth. I will be releasing 19 blog-posts as shown below.
You will learn about Azure Log Ingestion Pipeline, Azure Data Collection Rules, Data Collection Endpoints, Azure LogAnalytics custom table (v2), Azure Monitor Agent, Azure Monitor Private Link Scope, migration from v1 to v2, etc.
To get you started, I have also prepared templates & scripts in my AzureLogLibrary (github) – with focus on the new capabilities.
Happy reading 🙂
Understanding Azure logging, DCRs, DCEs, AMA & transformations in depth
Collecting data using Azure Monitor Agent (AMA)
Collecting Security events using Azure Monitor Agent
Collecting System & Application events using Azure Monitor Agent
Collecting Performance data using Azure Monitor Agent, VMInsights and ServiceMap
Collecting IIS logs using Azure Monitor Agent
Collecting text logs using Azure Monitor Agent
Collecting Syslogs using Azure Monitor Agent
Transformations of data
If you are sending data using HTTP Data Collector API (REST) today, you should continue reading, as this API will be deprecated, as part of the transition to Log ingestion API using Azure Data Collection Rules, Azure Pipeline, Azure LogAnalytics custom tables (v2).
As you can see from the illustrations above more components (DCR, DCR, Pipeline, Schema) are added, which also increases the complexity.
I have built a Powershell module, AzLogDcrIngestPS which will ease the steps, if you want to send any data to Azure LogAnalytics custom logs (v2) – using the new features of Azure Log Ingestion Pipeline, Azure Data Colection Rules & Log Ingestion API.
You can find AzLogDcrIngestPS in Powershell Gallery
My Powershell module AzLogIngestPS for Azure Log Ingestion API
AzLogDcrIngestPS – your helper to send data via Azure Pipeline, Azure Log Ingestion API & Azure Data Collection Rules into Azure LogAnalytics table
AzLogDcrIngestPS – how to do data manipulation before sending data via Azure Pipeline, Log ingestion API & Azure Data Collection Rules into Azure LogAnalytics ?
AzLogDcrInstPS – how to do transformation of your data using Azure Data Collection Rules into Azure LogAnalytics?
AzLogDcrIngestPS – tips & tricks sending data via Azure Pipeline, Azure Log Ingestion API, Azure Data Collections into Azure LogAnalytics
Introducing “Log-hub” – sending your data through intermediate hub into Azure Pipeline, Log Ingestion API, Azure LogAnalytics (github)
ClientInspector – showcase for AzLogIngestPS and Azure Log Ingestion
ClientInspector – a cool showcase to demonstrate Log ingestion API, Azure Log Ingestion Pipeline, Azure Data Collection Rules and my new Powershell module AzLogDcrIngestPS
ClientInspector-DeploymentKit – How can I get started with ClientInspector so I can play with Azure Pipeline, Azure Data Collection Rules and Azure LogAnalytics (Gibhub)?
Lastly I would like to thank the product teams in Seattle, USA and Israel, who are delivering rock star products every day.
Especially, I would like to give big credits to a few people, who I have worked together with on building AzLogDcrIngestPS Powershell module and my daily work with the Azure log & viewing capabilities:
|Ivan Varnitski||Program Manager – Azure Pipeline|
|Evgeny Ternovsky||Program Manager – Azure Pipeline|
|Nick Kiest||Program Manager – Azure Data Collection Rules|
|Oren Salzberg||Program Manager – Azure LogAnalytics|
|Guy Wild||Technical Writer – Azure LogAnalytics|
|John Gardner||Program Manager – Azure Workbooks|
|Shikha Jain||Program Manager – Azure Workbooks|
|Ingo Bringemeier||Principal Program Manager – Azure Monitor|
|Shayoni Seth||Program Manager – Azure Monitor Agent|
|Jeff Wolford||Program Manager – Azure Monitor Agent|
|Xema Pathak||Program Manager – Azure VMInsight (integration to Azure Monitor Agent)|
11 thoughts on “Master Azure Logging in depth”