The foundations of threat hunting

Threat hunting is a term often used by vendors and analyst agencies in their publications. One that caught our eye was IBM’s definition, which reads, “Threat hunting is a proactive approach to identifying previously unknown or ongoing, unpatched threats to an organization’s network and IT infrastructure.” And while there is no shortage of information on the Internet about tools and techniques for hunting a specific threat… What is missing are general ideas, an introduction to data analysis in the service of IT security, or simply a starting point for applying detailed methods and advanced tools.

In this article, we discuss a few areas from which we can start our threat hunting activities. A good hypothesis about an attack is only the first step in starting an organized threat hunt. It’s a good idea to start by distinguishing between types of threat hunting:

1. Structured ones are a more formal exploration of the tactics used by attackers, particularly by looking at the specific techniques and behavior patterns they employ. They are called structural threat hunts because each is built around a central hypothesis about specific attackers and their associated tactics, techniques and procedures (TTPs). This claim usually takes the form of a falsifiable, formalized assertion that is based on external sources of threat information.

2. Unstructured-these usually take the form of free-flowing ad hoc claims that are primarily based on data from internal log sources. Hunters opportunistically dig through logs and use simple data manipulation techniques such as pivot table searches or other methods used by analysts, and often rely primarily on investigative methodologies such as the principle of least view to catch anomalies in the data.

Knowing which type of hunt to engage in at any given time allows for an efficient response and thwarting attacks. Further helping are the concepts of data analytics, which, combined with the security context, are the next step to success.

Here are the issues to look out for when analyzing data during a threat hunt:

QUANTITY/VOLUME: look at the numerical elements of the data to give yourself a starting point. The first numerical element to look at is the amount of data analyzed in each area, the number of occurrences of each value….

An example of such a hunt would be firewall logs. Looking at all the high ports used, for example, and look for those with more connections.

FREQUENCY: Is this suspicious domain observed at about the same time every day? Do you see any noticeable pattern? Or an anomaly?

By checking the details of questionable calls made on a regular basis, you may, for example, discover G2 communications or ongoing exfiltration of small amounts of data (known as Drip-leakage)

GROUPING: useful, especially when you have access to detailed data. Allows you to look for outliers in the details. For example, if you have data from programs installed on systems, look for those that don’t have a vendor designation.

Of particular interest is grouping by commands that users have run in a terminal or command line. Look for non-administrators running “interesting” commands such as whoami, net use, ipconfig or nmap.

STACK COUNTING: one of the most effective methods for hunting for unstructured threats. For example, you can:

• put all program execution data (Windows Event ID 4688) from a department or organization on the stack. Then you can look for process names that are not known, or that are not considered “authorized” by the company.

• stack the most common passwords for existing programs. Look for hashes that may stand out, or compare them to lists of known good hashes for those programs.

• reveal names that are similar but different, especially for common process names. Look for processes that may use characters other than Roman numerals to try to hide themselves.

• analyze AV logs for both cleaning and quarantining files. Check where the cleanup and quarantine are taking place. A file cleaned in the “My Downloads” folder is much less worrisome than one in „system32.”

•  count occurrences of common file names, such as Calc.exe. Is the file where it should be? If not, it may be time to dive into the data to find out why.

• count things like network metadata to look for patterns. For example, analysts can stack user agent strings and look for suspiciously short or long ones. Or even ones that have small variations, like a period at the end.

PROCESS BEHAVIOR: One of the easiest structural hunts that analysts can do is to look at the parent-child relationships in individual processes and the entities assigned to them. For example, should Microsoft Word really be calling PowerShell? Why is svchost being called by a process other than services.exe? Analysts can also look for executing processes from locations that look abnormal.

USER BEHAVIORS: Another great method that can be used for structured hunting is to look at user behavior. For example, an analyst can:

• Look at users added to privileged groups (look for IDs 4728, 4732, 4756). Are the people making these changes authorized to do so? Are the accounts being added normal?

• Search for users logging in with alternative credentials (ID 4648). Is this behavior consistent with an administrator raising credentials? Or something more insidious?

• Check users running script interpreters. The most popular of these interpreters are cscript and wscript, and regular users are unlikely to use them.

• Look for executables such as Rundll32. Although these executables serve a legitimate function, the user in his daily work will not use them. So if you notice a user calling them, it may be time to dig deeper.

• Look for activity around scheduled tasks. This could mean suspicious task creation, but also users turning them off and on again.

TERMINAL ACTIVITY: One of the best sources of information about an adversary during or after a breach is terminal activity. Terminals such as PowerShell, cmd.exe, BASH and others should be centrally stored. This data can provide hunters with many hunting options. Hunters can look for:

• administrators using unsafe procedures, such as passing passwords explicitly. Hunters can look for commands containing the “-p” argument or the more eponymic strings “pass” or “password.” This type of activity should be flagged for tracking.

• users passing commands using an encoded command parameter.

• other suspicious PowerShell and BASH activity, such as outbound calls (PowerShell is one of the threat actor’s best friends, and that’s because it’s very versatile)

But terminal activity is not limited to looking for dangerous routines. Another useful hunt can be to analyze arguments and switches passed to commands or executable programs. Some arguments (like “securlsa::”) can often alert the hunter to the presence of a tool – like mimikatz – even if its name has been changed.

ENTROPY: in simplest terms, is a measure of the randomness of any attribute. Entropy can be complicated to understand, but once mastered, it can be very useful for detecting ordinary activity in the environment. For example, hunters can look at the entropy of file or process names across a system to detect names that don’t seem right. Hunters can also look at the entropy of domain names, looking for domain-generated algorithms (DGAs).

SUSPICIOUS SIGNS: Some hunts don’t fit into any category, but if they are visible, they still set off alarms. A great example is looking through AV logs and looking for suspicious detections. What do we mean by suspicious detections?

• Events related to quarantining or cleaning offensive security tools. These are tools that the user would have to download on their own – such as generic mimikatz – and are less likely to contain malware.

• Connections directly to the IP without an associated DNS query. While this may be common behavior for technical people, it is much less common in finance.

• Standard HTTP connections going through non-standard ports. This can reveal a lot of interesting activity, from network proxies to command-and-control malware.

Finding inspiration for structured and unstructured hunts can sometimes be a challenge. Regardless of the methods used, it is important to remember to maintain the hygiene of the activity chain. This will allow you to find additional value from correlating and grouping attack phase data by cyber kill chain.