Hey @baserock, thanks for bringing this issue up. Can you attach a sample redacted event?

Of the event I tuned out? or the events I want to retain?
In json?

Sample event I am tuning out:
{ "_index": ".ds-logs-o365.audit-[customer]-2022.08.08-000011", "_id": "8bbdad98-2adc-4cd3-b391-183302870900", "_version": 1, "_score": 0, "_source": { "agent": { "name": "[customer]-lxc-ubuntu-collector", "id": "[agent_ID]", "ephemeral_id": "f13dd8c1-162d-4e9e-b212-3cf40786de86", "type": "filebeat", "version": "8.2.1" }, "elastic_agent": { "id": "[agent_ID]", "version": "8.2.1", "snapshot": false }, "source": { "geo": { "continent_name": "Oceania", "region_iso_code": "AU-NSW", "city_name": "Sydney", "country_iso_code": "AU", "country_name": "Australia", "region_name": "New South Wales", "location": { "lon": 151.2006, "lat": -33.8715 } }, "as": { "number": 4764, "organization": { "name": "Aussie Broadband" } }, "ip": "[Ext_IP]" }, "tags": [ "forwarded", "o365-audit" ], "network": { "type": "ipv4" }, "o365": { "audit": { "AzureActiveDirectoryEventType": "1", "ObjectId": "Unknown", "ResultStatus": "Success", "UserKey": "[USR_Key]", "ActorIpAddress": "[Ext_IP]", "ErrorNumber": "50074", "ExtendedProperties": { "ResultStatusDetail": "Success", "RequestType": "SAS:BeginAuth" }, "IntraSystemId": "8bbdad98-2adc-4cd3-b391-183302870900", "Target": [ { "Type": 0, "ID": "Unknown" } ], "RecordType": "15", "Version": "1", "SupportTicketId": "", "UserId": "Not Available", "TargetContextId": "[ID]", "Actor": [ { "Type": 0, "ID": "[USR_Key]" } ], "LogonError": "UserStrongAuthClientAuthNRequiredInterrupt", "CreationTime": "2022-08-29T23:59:58", "InterSystemsId": "11269262-aa50-4f48-a6f7-c7cc316e970d", "DeviceProperties": [ { "Value": "Windows 10", "Name": "OS" }, { "Value": "Edge", "Name": "BrowserType" }, { "Value": "False", "Name": "IsCompliantAndManaged" } ], "ApplicationId": "a85cf173-4192-42f8-81fa-777a763e6e2c", "UserType": "4", "ActorContextId": "[ID]" } }, "input": { "type": "o365audit" }, "@timestamp": "2022-08-29T23:59:58.000Z", "ecs": { "version": "8.2.0" }, "related": { "ip": [ "[Ext_IP]" ] }, "data_stream": { "namespace": "[customer]", "type": "logs", "dataset": "o365.audit" }, "organization": { "id": "[ID]" }, "host": { "id": "[ID]" }, "client": { "address": "[Ext_IP]", "ip": "[Ext_IP]" }, "event": { "agent_id_status": "verified", "ingested": "2022-08-30T00:10:02Z", "code": "AzureActiveDirectoryStsLogon", "provider": "AzureActiveDirectory", "kind": "event", "action": "UserLoginFailed", "id": "8bbdad98-2adc-4cd3-b391-183302870900", "type": [ "info", "start", "access" ], "category": [ "web", "authentication" ], "dataset": "o365.audit", "outcome": "success" }, "user": { "id": "Not Available" }, "user_agent": { "original": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; WebView/3.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.19044", "os": { "name": "Windows", "version": "10", "full": "Windows 10" }, "name": "Edge", "device": { "name": "Other" }, "version": "18.19044" } }, "fields": { "o365.audit.SupportTicketId": [ "" ], "elastic_agent.version": [ "8.2.1" ], "event.category": [ "web", "authentication" ], "o365.audit.UserId": [ "Not Available" ], "o365.audit.ApplicationId": [ "a85cf173-4192-42f8-81fa-777a763e6e2c" ], "user_agent.original.text": [ "Mozilla/5.0 (Windows NT 10.0; Win64; x64; WebView/3.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.19044" ], "o365.audit.DeviceProperties.Name": [ "OS", "BrowserType", "IsCompliantAndManaged" ], "user_agent.os.version": [ "10" ], "client.address": [ "[Ext_IP]" ], "o365.audit.TargetContextId": [ "[ID]" ], "source.geo.region_name": [ "New South Wales" ], "source.ip": [ "[Ext_IP]" ], "agent.name": [ "[customer]-lxc-ubuntu-collector" ], "user_agent.version": [ "18.19044" ], "source.geo.region_iso_code": [ "AU-NSW" ], "event.agent_id_status": [ "verified" ], "event.kind": [ "event" ], "o365.audit.Actor.Type": [ 0 ], "source.geo.city_name": [ "Sydney" ], "event.outcome": [ "success" ], "user_agent.original": [ "Mozilla/5.0 (Windows NT 10.0; Win64; x64; WebView/3.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36 Edge/18.19044" ], "user.id": [ "Not Available" ], "o365.audit.ExtendedProperties.ResultStatusDetail": [ "Success" ], "input.type": [ "o365audit" ], "user_agent.name": [ "Edge" ], "client.ip": [ "[Ext_IP]" ], "data_stream.type": [ "logs" ], "o365.audit.ObjectId": [ "Unknown" ], "tags": [ "forwarded", "o365-audit" ], "event.provider": [ "AzureActiveDirectory" ], "event.code": [ "AzureActiveDirectoryStsLogon" ], "agent.id": [ "[agent_ID]" ], "o365.audit.AzureActiveDirectoryEventType": [ "1" ], "ecs.version": [ "8.2.0" ], "o365.audit.RecordType": [ "15" ], "organization.id": [ "[ID]" ], "agent.version": [ "8.2.1" ], "source.as.number": [ 4764 ], "o365.audit.ActorContextId": [ "[ID]" ], "o365.audit.LogonError": [ "UserStrongAuthClientAuthNRequiredInterrupt" ], "o365.audit.ErrorNumber": [ "50074" ], "o365.audit.CreationTime": [ "2022-08-29T23:59:58" ], "user_agent.os.full": [ "Windows 10" ], "source.geo.location": [ { "coordinates": [ 151.2006, -33.8715 ], "type": "Point" } ], "user_agent.os.name.text": [ "Windows" ], "o365.audit.UserKey": [ "[USR_Key]" ], "user_agent.os.name": [ "Windows" ], "o365.audit.Version": [ "1" ], "agent.type": [ "filebeat" ], "event.module": [ "o365" ], "related.ip": [ "[Ext_IP]" ], "source.geo.country_iso_code": [ "AU" ], "elastic_agent.snapshot": [ false ], "o365.audit.InterSystemsId": [ "11269262-aa50-4f48-a6f7-c7cc316e970d" ], "host.id": [ "[ID]" ], "network.type": [ "ipv4" ], "source.as.organization.name.text": [ "Aussie Broadband" ], "o365.audit.Target.Type": [ 0 ], "elastic_agent.id": [ "[agent_ID]" ], "data_stream.namespace": [ "[customer]" ], "o365.audit.IntraSystemId": [ "8bbdad98-2adc-4cd3-b391-183302870900" ], "o365.audit.ActorIpAddress": [ "[Ext_IP]" ], "source.as.organization.name": [ "Aussie Broadband" ], "source.geo.continent_name": [ "Oceania" ], "o365.audit.ExtendedProperties.RequestType": [ "SAS:BeginAuth" ], "o365.audit.UserType": [ "4" ], "o365.audit.Target.ID": [ "Unknown" ], "user_agent.os.full.text": [ "Windows 10" ], "event.ingested": [ "2022-08-30T00:10:02.000Z" ], "o365.audit.ResultStatus": [ "Success" ], "event.action": [ "UserLoginFailed" ], "@timestamp": [ "2022-08-29T23:59:58.000Z" ], "event.type": [ "info", "start", "access" ], "data_stream.dataset": [ "o365.audit" ], "agent.ephemeral_id": [ "f13dd8c1-162d-4e9e-b212-3cf40786de86" ], "o365.audit.DeviceProperties.Value": [ "Windows 10", "Edge", "False" ], "user_agent.device.name": [ "Other" ], "source.geo.country_name": [ "Australia" ], "event.id": [ "8bbdad98-2adc-4cd3-b391-183302870900" ], "event.dataset": [ "o365.audit" ], "o365.audit.Actor.ID": [ "[USR_Key]" ] } }

Please note that username does not exist in this event and the threshold aggregation on field 'user.id' has a value of "not_available".

This event is unhelpful from a purely username brute force attempt rule. I haven't figured out if the absence of username is a logging error or an event type/source discrepancy (from Microsoft) and I honestly can't waste more time differentiating it.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Hello,
we are seing the same kinds of alerts where user.id: "Not available" and these logs are not usable. I have not figured out why o365 is logging these events without any user.id either. @baserock did you ever figure these out?

I am however seeing what seems like the events are acutally connected to specific user, but for some reason o365 does not log this user.id. The o365.audit data does contain the same ID for all the logs,

Event with user.id: 
"Actor": [
        {
            "ID": "d65777ba-fd17-43a4-8d32-404a24619f81",
            "Type": 0
        },
        {
            "ID": "[email protected]",
            "Type": 5
        }
    ],

Event with user.id="Not available":

"Actor": [
       {
           "ID": "d65777ba-fd17-43a4-8d32-404a24619f81",
           "Type": 0
       }
   ]

Would be great to know what these numerous logs with user.id "Not available" are and what is causing them and also how to exclude them from the Elastic alerts.

HI, it seems that with the new ES|QL version of the rule the case of the user "Not available" it is solved