This is the fourth post from Google Cloud Principal Security Strategist John Stoner as part of his deep-dive "New to Chronicle" series, which helps propel forward security teams either new to SIEM or replacing their SIEM with Chronicle.
So far, our new to Chronicle series has introduced UDM and built a detection on a single event. From there, we moved into correlating across multiple events. Today, we are going to introduce another section of YARA-L called outcome.
The outcome section, as mentioned previously, is not a mandatory section of a YARA-L rule.
Wait wait! Don’t turn the page!
Just because it isn’t a mandatory section doesn’t mean that it is something that should be ignored. In fact, the outcome section provides the detection engineer an opportunity to provide additional contextual information to the analyst when a rule fires. How so? Read on to learn more!
Today, the outcome section can have up to 10 outcome variables defined in it. I know, you are thinking, only 10 fields? Keep in mind that is in addition to all of the other fields that are part of UDM that describe the event(s) and any associated context fields. Not only that, but depending on the type of rule you have written, these fields can be arrays of values extracted from individual events, but I am getting ahead of myself. We will come back to that one in a bit.
Let’s look at an example where we can use the outcome section in a single event rule. Our rule will alert when events are identified as malicious based on a file scan. Admittedly, the event criteria isn’t super exciting, but it serves to create a solid foundation. To trigger this rule, we are looking for events that are SCAN_FILE and SOFTWARE_MALICIOUS. The remainder of the fields in the events section serve as placeholder variables.
events: $event.metadata.event_type = "SCAN_FILE" $event.security_result.category = "SOFTWARE_MALICIOUS" $event.security_result.action = $secAction $event.security_result.threat_name = $threatname $event.security_result.rule_name = $rulename $event.target.file.full_path = $targetfilepath
The outcome section will contain values that we want to provide with our detection to add further context to the detection. risk_score is a special field whose value will be written to the Enterprise Insights page, if utilized. We could calculate risk_score as an integer for a single event rule, $risk_score = 50, but we might want to be a bit more nuanced in calculating our risk.
In our example, we are looking at the values within the security_result.action field to drive our risk score.
Our risk score will start at 100 but we will reduce it if we see an action of quarantine or block. We can use if then else statements to accomplish this.
outcome: $risk_score = max(100 - if($secAction = "QUARANTINE", 50, 0) - if($secAction = "BLOCK", 70, 0)) $action = array_distinct($secAction) $threat = array_distinct($threatname) $rule_detection = array_distinct($rulename) $file_path = $targetfilepath $tlp = "amber"
Notice that we are using the max function in front of our risk score calculation. Because the field security_result.action is a repeated field, we need an aggregation value at the start of the calculation. The tool tip in the rule editor provides a pop-up to remind us in case we forget, like I did.
The remainder of the fields we are creating use the placeholder variables from the events section.
Notice that the action, threat and rule_detection fields have the array_distinct function in front of the variable names. These are also repeated fields. If the associated event has multiple values in it, we won’t get repeated values, just distinct values. Finally, file_path and tlp fields are created, but notice there is no aggregate function. That’s because the $targetfilepath is not representing a repeated field and amber is a constant, so there is no need for an aggregation function for those fields in a single event rule. Notice I said single event rule, that will change when we look at multi-event rules!!!
Hey John, do we need to use placeholder variables for outcomes?
Good question, no we do not need them though they provide a way to streamline those longer event names and can be handy when building more complex rules. We could take each one of those event fields and map them to the outcomes and the rule would still compile and fire. For example, $threat = array_distinct($event.security_result.threat_name) would work without the placeholder value of $threatname. At that point, we would not even need the following line in the event section of the rule; $event.security_result.threat_name = $threatname. However, it is important to point out that we cannot conjure up fields and their associated values to include in the outcome that are not already defined with event variables in the event section of the rule.
When we test our rule, we can see we returned three detections. Notice the risk_score is varied based upon the value in the action field.
Let’s take this a step further! Perhaps we don’t care about the blocked file being alerted on or more broadly, we don’t care about events with a lower risk score being alerted on. We could use fields that are created in the outcome section and apply conditions to these values to improve the fidelity of the rule. Yes, in this specific example, we could have excluded blocks by inserting $event.security_result.action != "BLOCK" back in the events section of the rule, but bear with me. We could take our event criteria and then only trigger our rule if the risk_score is greater than 49, which would essentially suppress the BLOCK event above.
condition: $event and $risk_score > 49
Another method to use outcome fields in this same rule would be to use arrays.contains to look inside of the action field that we created for the value QUARANTINE.
condition: $event and arrays.contains($action, "QUARANTINE")
These two conditions, in this case, return the same two detections. The point of this is not to make it harder to develop our rule, if there is a specific condition that can be eliminated in the event section of the rule, by all means use it, but these examples are highlighting how the condition section can be leveraged to further refine the rules that we want to fire.
Outcome values can also be passed along to other platforms, like a SOAR. In my Siemplify instance, I can write the risk score of an event into a case.
The use of outcomes with multiple events is very similar. The biggest difference is that because there are multiple events being used in each rule, every outcome must have an aggregate function associated with it. In this example, we are attempting to detect a password spray where many different user names are being attempted over a broad period of time.
events: $event.metadata.event_type = "USER_LOGIN" $event.metadata.vendor_name = "Microsoft" $event.principal.hostname = $targetHost $event.target.user.userid = $targetUser $event.security_result.category = "AUTH_VIOLATION" $event.security_result.action = "BLOCK"
match: $targetHost over 60m
In the outcome section, we can use the aggregate functions count_distinct and count to generate metrics on the number of unique usernames attempted versus the number of events that attempted to login with a username. We can also use the array_distinct and array functions to generate an output of the values in the placeholder variable $targetUser. It is important to note that both array functions will hold up to 25 values in them, but the ability to download a csv with all of the supporting fields and values exists as well.
outcome: $risk_score = max(50) $target_user_distinct_count = count_distinct($targetUser) $target_user_count = count($targetUser) $target_user_distinct = array_distinct($targetUser) $target_user = array($targetUser) $tlp = array_distinct("amber")
condition: #targetUser > 10 and $target_user_count > 600 }
Notice in the condition section that we used the count of the target_user from the events section of the rule and the count from the target_user_count field in the outcome section to bound our rule.
One final thought regarding outcomes…Keep in mind we are creating new field/value pairs with outcomes as well as defining risk scores as well. It would be a wise idea to develop a standard nomenclature for defining these fields. Risk scores should have a standardized range, whether that is 0-100 or 0-10,000 is up to you, but develop a standard and stick with it. Similarly, outcome fields should follow standards so that rules don’t have outcome fields like $target_user, $targetuser and $array_target_user that all contain the same type of data in them.
I hope this provides you with a better appreciation for the outcome section of YARA-L and how this capability can be used to provide additional context for rules that are fired, but also provide a means to further refine rule conditions to create higher fidelity alerts. Outcomes can work with single and multiple-event rules and support a number of different types of aggregate functions. If outcomes are not part of your rules today, it may be time to consider adding them!
Until next time…