Do you want to automate alert rules including creating new alert rules and update existing – with checks every x hours so your environment is continuously updated to help detect new threats?
Do you want to keep your Sentinel Alert Rules updated with latest Alert Rules templates from Microsoft so queries, entity-support gets updated and new features come into play?
Do you want to support new rule types as they get released ?
Maybe you want to have a default alert rule action like ‘Send mail‘ linked to your alert rules in case you want to use a logic app to distribute the emails depending on who should receive the alert.
If yes, keep reading and get inspired with the possibilities and a few challenges 🙂
Solution objective
As mentioned, it is crucial to stay updated with Sentinel Alert Rules, in order to detect new threats or critical vulnerabilities like Log4J. I recommend to check for new alert rules every 4-6 hours, so Sentinel is maximum 4-6 hours behind detecting new threats.
Microsoft, 3rd party vendors or community will release new or updated alert rules as soon as a threat can be detected similar when support for detecting these threats were released very fast:
- Vulnerable Machines related to log4j CVE-2021-44228
- AV detections related to Ukraine threats
- KNOTWEED AV Detection
- Dev-0270 WMIC Discovery
- HAFNIUM New UM Service Child Process
By implementing an automation for alert rules, you will be continuously having an updated SIEM environment being able to detect new use-cases.
Ways you can do this ?
Microsoft are providing multiple ways to achieve this including:
- Powershell modules and REST api, which can be accessed though a Powershell script running as part of your automation
- Deploying custom content through your Github or Azure DevOps repository. This solution is currently in public preview and I urge you to try it out (link)
This blog will focus on the script-method, as it can be used by companies of all sizes without deep knowledge of DevOps. This is a great ‘accelerator’ to get up and running – and stay running – in a fast way.
I will add an article on deploying custom content through Github and Azure DevOps later.
Script overview
- Script will use templates, part of Sentinel (see list below). Currently there is no support for content hub.
- Script should run every x hours to check for changes.
- Script will check if alert rules templates have changed by Microsoft or other 3rd parties, part of the defined scope.
- Add/create
- In case a new alert rule has been released, it will be created.
- Update
- In case an existing alert rule has been updated, it will be updated (overwritten using 100% settings from alert rule template)
- Remove
- In case of scope changes (excluded), script can automatically remove any orphaned alert rules that was used prior, where the connector now should be excluded. This parameter is configurable.
- In case of duplets based on displayname, script can remove these. This parameter is configurable.
- Add/create
- Script can automatically add a default action to each alert rules, for example a logic app that will send an email. This is configurable using relevant parameters.
- Script will create issue-logfiles in case some alert rules cannot be created.
- This is typically caused by column, tables or normalization which is missing.
- You can then fix these issues. Next time it will retry any pending alert rules.
To get started, please look in the section ‘How to Get Started’.
You can download the script from my Github
IMPORTANT REMARKS
I have taken the assumption, that most of us wants a standard configuration, which potentially can be updated with a high frequency as new versions gets released.
Therefore if you have made customizations to your alert rules, they will be overwritten by the latest configuration from the templates, when the update runs.
If an alert rule has been disabled, it will remain disabled even though alert rule will be updated.
Solution can easily be enhanced in various ways adding more processes including test & approval, exporting alert rule before updating, include-checks, etc. I might be adding these later.Filtering / Scope definition Challenge
To my knowledge, it is currently not possible to retrieve a complete list of the connected connectors using REST API and Powershell, so I am using a workaround outlined below.
Of course, Microsoft will fix this, adding more features to the API every month. Latest API is from last month (2022-12-01-preview).
I will keep an eye on this link and update script accordingly.
Goal is to retrieve the list shown below with the 17 connected connectors.
# Method 1 (PS) - Get-AzSentinelDataConnector
Get-AzSentinelDataConnector -ResourceGroupName $global:MainLogAnalyticsWorkspaceResourceGroup -workspaceName $global:MainLogAnalyticsWorkspaceName
In my case, it returns 6 connectors - but I have 17 connected connectors.# Method 2 (REST API preview 2022-12-01-preview)
$baseUri = "/subscriptions/$($global:MainLogAnalyticsWorkspaceSubId)/resourceGroups/$($global:MainLogAnalyticsWorkspaceResourceGroup)/providers/Microsoft.OperationalInsights/workspaces/$($global:MainLogAnalyticsWorkspaceName)"
$connectedDataConnectorsUri = "$baseUri/providers/Microsoft.SecurityInsights/dataConnectors/?api-version=2022-12-01-preview"
$ConnectorsInUse = (Invoke-AzRestMethod -Path $connectedDataConnectorsUri -Method GET).Content | ConvertFrom-Json
$ConnectorsInUseList = $ConnectorsInUse.value
In my environment, it returns 11 connectors - but I have 17 connected connectors.Temporary Workaround to define connectors in scope
Until supported by API, I have chosen a fairly simple approach, where I define a list of connectors, which are excluded.
Someone might think, why are you not using include. The reason for that is, that I have seen connector-renames.
As part of the variables, you define a list of data-connectors, which are NOT part of the scope of the required data connectors.
I have made the assumption, that I am only excluding alert rules, which are having a explicit requirement for a data connector. I have decided to enable the alert rules, which are having more than 1 data connector, where one of them are on the exclude list. Many alert rules are doing lookups in multiple tables, including 1 which is excluded. These are included.
Example below where script will exclude all alert rules being dependent on one of the below data connectors.
$global:Sentinel_DataConnectors_ExcludeAlertRules = @(
"AIVectraStream"
"AWS"
"AWSS3"
"Barracuda"
"CEF"
"CheckPoint"
"CiscoASA"
"CiscoUmbrellaDataConnector"
"Corelight"
"Dynamics365"
"F5"
"Fortinet"
"GCPDNSDataConnector"
"InfobloxNIOS"
"IoT"
"MicrosoftSysmonForLinux"
"NXLogDnsLogs"
"PaloAltoNetworks"
"ProofpointPOD"
"PulseConnectSecure"
"QualysVulnerabilityManagement"
"SquidProxy"
"Syslog"
"ThreatIntelligence"
"ThreatIntelligenceTaxii"
"TrendMicro"
"WAF"
"Zscaler"
)
The complete list of standard connectors per Jan 9, 2023.
AIVectraStream
AWS
AWSS3
AzureActiveDirectory
AzureActiveDirectoryIdentityProtection
AzureActivity
AzureAdvancedThreatProtection
AzureFirewall
AzureKeyVault
AzureMonitor(IIS)
AzureMonitor(VMInsights)
AzureMonitor(WireData)
AzureNSG
AzureSecurityCenter
Barracuda
BehaviorAnalytics
CEF
CheckPoint
CiscoASA
CiscoUmbrellaDataConnector
Corelight
DNS
Dynamics365
F5
Fortinet
GCPDNSDataConnector
InfobloxNIOS
IoT
MicrosoftCloudAppSecurity
MicrosoftDefenderAdvancedThreatProtection
MicrosoftSysmonForLinux
MicrosoftThreatProtection
NXLogDnsLogs
Office365
OfficeATP
OfficeIRM
PaloAltoNetworks
ProofpointPOD
PulseConnectSecure
QualysVulnerabilityManagement
SecurityEvents
SquidProxy
Syslog
ThreatIntelligence
ThreatIntelligenceTaxii
TrendMicro
WAF
WindowsFirewall
WindowsForwardedEvents
WindowsSecurityEvents
ZscalerAlert Rules with no requirement for a data connector
Right now, there are 54 alert rules in the templates with no requirement for a data connector. Personally I think that some of them are mistakes, which I have escalated to the product-team.
I have made support to also add these alert rules, even though some of them might be of the grid of what you need.
Suspicious link sharing pattern
External User Access Enabled
User login from different countries within 3 hours (Uses Authentication Normalization)
SUNBURST and SUPERNOVA backdoor hashes (Normalized File Events)
Base64 encoded Windows process command-lines (Normalized Process Events)
Unusual Anomaly
Sign-ins from IPs that attempt sign-ins to disabled accounts (Uses Authentication Normalization)
Potential re-named sdelete usage (ASIM Version)
Azure DevOps PAT used with Browser.
Vulnerable Machines related to OMIGOD CVE-2021-38647
Exchange Server Suspicious File Downloads.
GitHub Activites from a New Country
Zoom E2E Encryption Disabled
HAFNIUM Suspicious UM Service Error
AppServices AV Scan Failure
Sdelete deployed via GPO and run recursively (ASIM Version)
Trust Monitor Event
Azure DevOps Build Variable Modified by New User.
Dev-0228 File Path Hashes November 2021 (ASIM Version)
NOBELIUM - suspicious rundll32.exe execution of vbscript (Normalized Process Events)
AppServices AV Scan with Infected Files
HAFNIUM Suspicious File Downloads.
Azure DevOps Retention Reduced
Potential Password Spray Attack (Uses Authentication Normalization)
NRT GitHub Two Factor Auth Disable
NRT Azure DevOps Audit Stream Disabled
New PA, PCA, or PCAS added to Azure DevOps
Malware in the recycle bin (Normalized Process Events)
Azure DevOps Service Connection Addition/Abuse - Historic allow list
Azure DevOps Variable Secret Not Secured
Azure DevOps Service Connection Abuse
Probable AdFind Recon Tool Usage (Normalized Process Events)
Advanced Multistage Attack Detection
SUNBURST suspicious SolarWinds child processes (Normalized Process Events)
Azure DevOps Personal Access Token (PAT) misuse
Azure DevOps Pipeline modified by a new user.
Azure DevOps Audit Stream Disabled
User joining Zoom meeting from suspicious timezone
Potential Fodhelper UAC Bypass (ASIM Version)
Azure DevOps Agent Pool Created Then Deleted
Brute force attack against user credentials (Uses Authentication Normalization)
New Agent Added to Pool by New User or Added to a New OS Type.
Vulnerable Machines related to log4j CVE-2021-44228
External Upstream Source Added to Azure DevOps Feed
Azure DevOps Pull Request Policy Bypassing - Historic allow list
Azure DevOps New Extension Added
Azure DevOps Administrator Group Monitoring
Users searching for VIP user activity
OMI Vulnerability Exploitation
Missing Domain Controller Heartbeat
GitHub Two Factor Auth Disable
Azure DevOps Pipeline Created and Deleted on the Same Day
Wazuh - Large Number of Web errors from an IP
GitHub Security Vulnerability in RepositoryHow can I build a list of possible data connectors to make my exclude list ?
In case you want to adopt the script and want to keep the exclude-list up-to-date, you can run the below REST API command to list all possible data connectors. Then you can adjust you exclude list.
$baseUri = "/subscriptions/$($global:MainLogAnalyticsWorkspaceSubId)/resourceGroups/$($global:MainLogAnalyticsWorkspaceResourceGroup)/providers/Microsoft.OperationalInsights/workspaces/$($global:MainLogAnalyticsWorkspaceName)"
$Uri = "$baseUri/providers/Microsoft.SecurityInsights/alertRuleTemplates/?api-version=2022-12-01-preview"
$AllAlertRuleFromTemplatesApi = (Invoke-AzRestMethod -Path $Uri -Method GET).Content | ConvertFrom-Json
$AllAlertRuleFromTemplates = $AllAlertRuleFromTemplatesApi.value
$CompleteConnectorList = $AllAlertRuleFromTemplates.properties.RequiredDataConnectors.ConnectorId | Sort-Object -Unique
Write-Output "Complete list of Data Connectors defined in alert rules templates"
$CompleteConnectorListHow can I get started ?
You can download the script from my Github
If you want to get familiar with the script, I propose that you use your test-environment, where you can run the script to test its functionality.
Consider to setup a test-environment with a basic Sentinel environment and some basic connectors.
Connectivity to Azure
For demo purpose, I have just added a simple Connect-AzAccount in the script.
Remember to change this into using for example connection using certificate and Azure App registration – or alternative approach. I will not be covering this topic, but there are many great samples of this on the internet.
Connect-AzAccount -CertificateThumbprint $global:HighPriv_Modern_CertificateThumbprint_Azure -TenantId $global:AzureTenantId -Application $global:HighPriv_Modern_ApplicationID_Azure
Variables
Before running the script, please adjust It is rather simple to get the script configured, as you just have to define the below variables:
###############################################################
# LogAnalytics Workspaces
###############################################################
$global:MainLogAnalyticsWorkspaceName = "log-srvnetworkcloud-p"
$global:MainLogAnalyticsWorkspaceSubId = "xxxxxxxxxxc6-43fb-94d8-bf1701b862c3"
$global:MainLogAnalyticsWorkspaceResourceGroup = "rg-logworkspaces"
###############################################################
# Sentinel
###############################################################
$global:Sentinel_DataConnectors_ExcludeAlertRules = @(
"AIVectraStream"
"AWS"
"AWSS3"
"Barracuda"
"CEF"
"CheckPoint"
"CiscoASA"
"CiscoUmbrellaDataConnector"
"Corelight"
"Dynamics365"
"F5"
"Fortinet"
"GCPDNSDataConnector"
"InfobloxNIOS"
"IoT"
"MicrosoftSysmonForLinux"
"NXLogDnsLogs"
"PaloAltoNetworks"
"ProofpointPOD"
"PulseConnectSecure"
"QualysVulnerabilityManagement"
"SquidProxy"
"Syslog"
"ThreatIntelligence"
"ThreatIntelligenceTaxii"
"TrendMicro"
"WAF"
"Zscaler"
)
# Sentinel Alert Management
$global:Sentinel_DeleteExcludedAlertRulesFromTemplateIfFound = $false
$global:Sentinel_DeleteDupletAlertsRulesIfFound = $false
$global:Sentinel_CreateUpdateAlertRulesWithNoDataConnectorReq = $true
# Sentinel Alert Rule Action (default)
$global:SentinelAlertingEnableLogicAppAction = $true
$global:SentinelAlertingForceSetExistingRules = $true
$global:SentinelAlertingLogicAppActionName = "SendEmail"
$global:SentinelAlertingLogicAppActionRG = "AzureRG3-Management-WestEurope"
$global:SentinelAlertingLogicAppActionTriggerName = "When_a_response_to_an_Azure_Sentinel_alert_is_triggered"
# Sentinel Alert Rule management logging
$global:Sentinel_Issues_List = "D:\SRIPTS\OUTPUT\SENTINEL_AlertRules_Issues_List.txt"
$global:Sentinel_Issues_Detailed= "D:\SCRIPTS\OUTPUT\SENTINEL_AlertRules_Issues_Detailed.txt"
Excellent blog and very detailed.
Thanks a lot!
Excellent post, Morten! Thank you for sharing.
I found that now both the Get-AzSentinelDataConnector and the dataConnectors API operation return the correct number of connectors. An additional filter to exclude kind=AzureSecurityCenter is required.