Reliable data from the Windows Kernel?

Last year I learned about a product called Carbon Black, before the merger with Bit9. Carbon Black looks to be a great host-data collection tool from what I've seen. Unfortunately, in enterprise environments it's often difficult to get approval to purchase and deploy new tools, leaving administrators to be resourceful with the tools they already have. If you're one of these admins, you've probably arrived at my same quandary of wishing you could replicate the functionality of these awesome new tools. Being that most enterprise environments are Windows based, I'm going to attempt the minimalist approach and leverage tools packaged within that OS.

While Windows Vista was probably the least popular OS Microsoft has released, it introduced a myriad enhancements that might just prove useful. Firstly, the introduction of PowerShell; if you're not learning, loving, and using it everyday you're more of a noob than me. Secondly, Microsoft merged Event Tracing for Windows (ETW) and the classic Windows Event Log into what's now referred to as simply Windows Events. Along with the new event reporting model were the built-in methods of consuming event data via the trace function added to NetShell (netsh.exe) and user-defined data collector sets in the Performance Monitor (Perfmon.exe) application. I'll start with the NetShell example.

Using netsh in PowerShell, we can easily display the list of all currently registered data providers from which we're able to consume event data using the following command:

netsh trace show providers

More specifically, though, we're interested in what providers are delivering data directly from the kernel; so let's do a little Windows grep-fu and shorten the list.

netsh trace show providers | Select-String kernel

There seem to be quite a few kernel data providers, according to this output. However, there is one special one that is the true kernel source; in this list it's referred to as "Windows Kernel Trace", but elsewhere it may be referred to as the "NT Kernel Logger". Netsh will give amplifying information about this specific provider's keywords if we drill down another layer:

netsh trace show provider "Windows Kernel Trace"

But since this is the special case of event data coming directly from the NT Kernel, if you try to consume it using the netsh trace function, it errors out as seen below. 

netsh trace start provider="Windows Kernel Trace"

The only built-in way to consume events directly from the NT Kernel is through the Perfomance Monitor application I mentioned earlier. In Windows 8, hit the Windows key, start typing Perfmon, and hit enter when the application populates in the search pane. Once the application loads, expand the data collector sets, right-click on "User Defind", hover over the "New" sub-menu, and click "Data Collector Set".

You should now be in a creation wizard. Give your data collector set a name > select the Create Manually (Advanced) radio button > Next

At the next prompt, check the Event trace data box > Next

At the next prompt, click Add

Windows will populate a list of all registered data providers for you to choose from. We're going to pick the Windows Kernel Trace provider for this demo. Once that's done, click the Edit... button. Here you can see the same list of keywords that were shown by netsh, and as you select them you'll see that their respective values are added together next to the Manual radio button option.

Click OK > Finish to exit the wizard and you'll see your newly created data collector set. You can either highlight the new data collector set and click the green play arrow that appears, or right-click on it and select  Start. You'll only want to run it for a few seconds as the kernel outputs a large amount of data in real-time. Once you stop the collector you'll see that Perfmon automatically lists the output directory. We'll navigate there for the next step.

You can double-click on the .etl file generated by Perfmon to view the output in the Windows Performance Analyzer application.

You could alternatively use the built-in Trace Report (tracerpt.exe) application to convert the .etl file into a CSV for further analysis. Using the tracerpt.exe -h command in PowerShell will list its usage options.

The combination of options to convert the .etl to .csv are listed below, but you can see in the picture that the .etl file is much compressed compared to the CSV it generated.

tracerpt.exe .\DataCollector02.etl -o dump.csv -of CSV

Looking at the CSV in Excel isn't too terrible at all, if you know what you're looking at.

However, the ultimate goal is to use PowerShell to deploy this kernel-data-collection mechanism to several hundred machines that process the output in real-time based on signatures and only send back small bits of relevant data to a central server. I'll follow up with more details on that process as it unfolds.