macOS vs. Windows – What kernels tell you about security events: Part 1

By
Yamin Tian

How would you compare the Windows and macOS operating systems? In what ways are they similar? Why do they each take different approaches to solving the same problem? For the last 19 years I've developed security software for Windows. Recently, I’ve started implementing similar features on macOS. Since then, people have asked me questions like this. The more experience I gained on these two operating systems, the more I realized they’re very different. Even when the problems they set out to solve are common, their paths differ. In this two-part series, I will focus on the operating system events supported by Windows and macOS, and explain each operating system's approach.

Overview

In Part 1, we’ll discuss process events. In Part 2, we’ll discuss file and network events. These events are crucial for endpoint security analysis. Through multiple perspectives in this post, I hope to:

  • Help you better understand the general differences between macOS and Windows on the system level
  • Help those who want to expand their experiences onto another platform be better prepared and adapted to differences that might seem to be shocking from their past experiences
  • Help cybersecurity professionals enrich their toolsets for creating innovative security products

This post is based on some of my own experiences. It does not intend to be an exhaustive list.

Open vs. closed

Microsoft Windows has traditionally been open. It runs on hardware from different vendors and allows third-party developers to write applications on top of it. macOS is part of Apple’s end-to-end integrated ecosystem, which is quite the opposite of being “open.” This article does not try to take sides on the decades-long “open vs. closed” debate. Apart from end-to-end user experience, developers and researchers usually care about other things. 

The abundance of APIs and documents provided by Windows really enabled developers to do virtually anything in the cyber world. While on the macOS side, the beauty of simplicity reminds people that Apple takes lots of things as their own responsibility instead of sharing them with third-party developers. If you find something from Apple difficult to use, it’s perhaps because you’re looking at something Apple feels is their own backyard. Take kernel mode debugging or kernel core dumps as examples. On the upside though, macOS developers can indulge themselves in the open source Darwin kernel, whereas Windows’ kernel is closed source and yet Windows developers feel spoiled with an abundance of documentation.

Microkernel vs. monolithic kernel

Figure 1: Structure of monolithic and microkernel-based operating systems

Modern macOS and Microsoft Windows both have hybrid kernels, which are neither a pure microkernel, nor monolithic. However, one cannot infer that the two kernels are similarly architectured based on the same kernel type — they actually came from different places.

Microsoft Windows, in the Windows 9x days, used a monolithic kernel. macOS XNU consists of several core building blocks, one of which came from the Mach kernel that carries the genes of a microkernel. Even though both operating systems have evolved into today’s “hybrid” form, their early influences still manifest and affect developers’ daily lives. For example, Apple announced in WWDC 2019 it’s going to deprecate kernel extensions (kext) support for third-party developers starting with macOS 10.16. This means they are kicking third-party developers out of the kernel. For some, this may seem shocking. However, thinking from a microkernel point of view, this change sounds just like a logical next step.

Over the years, Microsoft created a new Windows Driver Framework. It also moved lots of traditional kernel mode driver support into user mode, encouraging people to implement drivers in user mode. However, caveats exist. For example, in Windows Filtering Platform, even though most of the functionalities can be done in user mode, some of the crucial functionalities still have to be done in the kernel. 

Backward compatibility vs. complete rewrite

Some people have doubts about Apple's WWDC 2019 kext deprecation announcement, as it means that backward compatibility is broken for any third-party software that includes kexts. People will have to rewrite things in user mode from scratch. It’s understandable that some may want Apple to delay or cancel the deprecation. Regardless, deprecation is happening as planned. Because it is a closed ecosystem and Apple retains end-to-end control, backward compatibility is never on its menu and deprecation has been a consistent part of macOS releases. Check out the “Deprecation” sections in a few of the past release notes: Mojave, Catalina, and the coming release of Big Sur.

This means, as a macOS developer, you may frequently find yourself rewriting everything from scratch — even if your software is well-architected and implemented. Windows, on the other hand, has been keeping its promise of backward compatibility. You may find applications created for Windows 3.1 still run on the latest Windows 10. Microsoft does occasionally attempt to deprecate things. For example, it has been trying to deprecate support for legacy file system drivers ever since the introduction of its replacement —  file system minifilter — in Windows XP SP2 on August 25, 2004. But, as of today, it’s still unable to do so. However, this doesn’t mean legacy file system drivers remain a viable choice for third-party developers.

Process events

Windows and macOS organize process events very differently. Windows provides a single kernel callback, set by PsSetCreateProcessNotifyRoutine/Ex, to trace all process creations and terminations. macOS process events are mainly retrieved from two sources:

  • An executable file’s vnode operations
  • fork() system call activities

Windows

PsSetCreateProcessNotifyRoutineEx allows developers to register a callback with Windows. Later on when a process is created or exits, Windows will notify the caller by invoking the callback. The callback can be used for both prevention and notification. The parameters passed to the callback contain a rich set of information useful for process events, shown in the following figure:

Figure 2: Screenshot of a debug session that shows PsSetCreateProcessNotifyRoutineEx parameters

To learn how to use PsSetCreateProcessNotifyRoutineEx, take a look at Microsoft’s ObCallback  Registration Driver sample code. 

It’s also worth mentioning the context of a process. On Windows, when the callback registered by PsSetCreateProcessNotifyRoutineEx is invoked, it’s in the context of the creating process for process creations. For process terminations, it’s in the context of the terminating process (see the document). Inside the Windows kernel, most of the time you can find out the actor process by simply calling PsGetCurrentProcessId, because the kernel service usually shares context with the actor process. However, in the macOS kernel (for reasons), developers are advised to avoid calling proc_self in the kernel to find out the actor process. This is easier to understand from a microkernel’s point of view, because most of the system services are delegated to other processes on a microkernel system. Particularly, when kernel extensions are replaced by system extensions (which runs in user mode) in macOS 10.16, the same advice should be strictly followed.

macOS

Kauth/MACF

Kauth is a Kernel Programming Interface (KPI) provided by macOS to allow third parties — primarily security software developers — to intercept select operations.1 As the definition suggests, it can only be used in kernel mode. There’s another KPI, called Mandatory Access Control Framework (MACF), which is way more powerful than Kauth, but it’s Apple’s private KPI and not “open” for third-party developers. With the kext deprecation announcement, both Kauth and MACF won’t be available for third-party developers, starting with macOS 10.16. I briefly describe Kauth here to present a continuity view for you to get an idea of macOS's evolution:

Kauth defines “scopes,” which Apple describes as “areas of interest” for authorization within the kernel.1 A couple scopes in Kauth provide means for third-party software to get notified when a process is launched.

  • Vnode scope - Vnode scope is capable of preventing a process from being launched. By calling KPI kauth_listen_scope(KAUTH_SCOPE_VNODE,...), a developer can register a vnode scope listener callback. When an operation on a vnode is about to happen, macOS will invoke the callback. The callback can then block or allow the operation. Vnode operations are not limited to executions. There are other types of vnode operations as well, such as open/close/read/write/etc. By checking KAUTH_VNODE_EXECUTE flag against the vnode operation (action), we know a file is executed — which roughly translates to “a process is launched” in Windows language.
  • File operation scope - Similar to vnode, file operation scope is able to report operations on a file. The KAUTH_FILEOP_EXEC flag in the action parameter indicates that a file is being executed. However, file operation scope works only in listening mode. It can not be used to prevent a file from being executed inline.

Endpoint Security system extensions

macOS Endpoint Security (ES) is a kind of System Extension which runs in user mode, and it’s the replacement for Kauth. In the new ES framework, event types are better organized in one enum definition es_event_type_t defined in XCode’s SDK header file:

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/EndpointSecurity/ESTypes.h

You can see that process events are just a subset of ES events (more discussion about ES events can be found in the next File Events section). The few types of ES events pertaining to process events are: 

  • ES_EVENT_TYPE_AUTH_EXEC
    •
  • ES_EVENT_TYPE_NOTIFY_EXEC
    •
  • ES_EVENT_TYPE_NOTIFY_FORK
    •
  • ES_EVENT_TYPE_NOTIFY_EXIT

Note that if you find “AUTH” in the event type, it means it can be used for inline prevention. The ones with “NOTIFY” are just for notifications and can not be used for inline prevention.

Compared to the Windows callback set by PsSetCreateProcessNotifyRoutineEx, the information we can get from the Kauth scope listener callback is quite limited. For example, it doesn’t provide the command line and environment arguments information for a process launch, which security vendors consider crucial. However, Kauth’s successor — Endpoint Security — is indeed an enhancement of Kauth in many ways. For example, the long-waited command line and environment arguments information is provided now by ES, along with some cool new information:

  • Ancestry
  • Code-signing information

The macOS ES framework puts this information in a message envelope es_message_t and passes it to third-party ES extensions via the registered event handler block. We'll talk about es_message_t more in the "File events" section of Part 2. In the message envelope, you can find the information pertaining to process events, such as: es_event_exec_t, es_process_t. They’re defined in ESMessage.h, which is under the same folder as ESTypes.h

Here are some example process events collected by macOS ProcessMonitor using the new ES framework: 

{ 
   "event":"ES_EVENT_TYPE_NOTIFY_EXEC", 
   "timestamp":"2020-03-23 23:24:50 +0000", 
   "process": 
   { 
      "pid":30663, 
      "path":"/usr/libexec/xpcproxy", 
      "uid":0, 
      "arguments":["xpcproxy","com.apple.cvmsCompAgent_x86_64"], 
      "ppid":1, 
      "ancestors":[1], 
      "signing info": 
      { 
         "csFlags":603998721, 
         "signatureIdentifier":"com.apple.xpc.proxy", 
         "cdHash":"8F8FF2D627C5731043FED9495312FBC72F54889", 
         "isPlatformBinary":1 
      } 
   } 
} 
{ 
   "event":"ES_EVENT_TYPE_NOTIFY_FORK", 
   "timestamp":"2020-03-21 05:47:44 +0000", 
   "process": 
   { 
      "pid":28758, 
      "path":"/System/Library/CoreServices/logind", 
      "uid":0, 
      "arguments":[], 
      "ppid":120, 
      "ancestors":[120,1], 
      "signing info": 
      { 
         "csFlags":603998721, 
         "signatureIdentifier":"com.apple.logind", 
         "cdHash":"2EBF2C2991F47E1C578ECD707E06D76634ED8D0", 
         "isPlatformBinary":1 
      } 
   } 
} 
{ 
   "event":"ES_EVENT_TYPE_NOTIFY_EXIT", 
   "timestamp":"2020-03-21 05:47:44 +0000", 
   "process": 
   { 
      "pid":28758, 
      "path":"/System/Library/CoreServices/logind", 
      "uid":0,"arguments":[], 
      "ppid":120, 
      "ancestors":[120,1], 
      "signing info": 
      { 
         "csFlags":603998721, 
         "signatureIdentifier":"com.apple.logind", 
         "cdHash":"2EBF2C2991F47E1C578ECD707E06D76634ED8D0", 
         "isPlatformBinary":1 
      }, 
      "exit code":0 
   } 
}

For more hands-on experience, take a look at my colleague Will Yu’s Mac system extensions for threat detection: Part 3 blog post and download his sample code SimpleEndpoint. Alternatively, you can download a macOS ProcessMonitor to play around with and explore its source code here.

We’ll continue the discussion about File Events and Network Events in macOS vs. Windows – What the kernels tell you about security events: Part 2

References

  1. Levin, Jonathan. *OS Internals, Volume III: Security & Insecurity. Edison, NJ: Technologeeks.com, 2016.


Appendix

es_event_type_t defined in ESTypes.h, as of Xcode 11.5 (11E608c)

/**
* The valid event types recognized by EndpointSecurity
*
* @discussion When a program subscribes to and receives an AUTH-related event, it must
respond
* with an appropriate result indicating whether or not the operation should be allowed to
continue.
* The valid API options are:
* - es_respond_auth_result
* - es_respond_flags_result
*
* Currently, only ES_EVENT_TYPE_AUTH_OPEN must use es_respond_flags_result. All other
AUTH events
* must use es_respond_auth_result.
*/
typedef enum {
// The following events are available beginning in macOS 10.15
ES_EVENT_TYPE_AUTH_EXEC
, ES_EVENT_TYPE_AUTH_OPEN
, ES_EVENT_TYPE_AUTH_KEXTLOAD
, ES_EVENT_TYPE_AUTH_MMAP
, ES_EVENT_TYPE_AUTH_MPROTECT
, ES_EVENT_TYPE_AUTH_MOUNT
, ES_EVENT_TYPE_AUTH_RENAME
, ES_EVENT_TYPE_AUTH_SIGNAL
, ES_EVENT_TYPE_AUTH_UNLINK
, ES_EVENT_TYPE_NOTIFY_EXEC
, ES_EVENT_TYPE_NOTIFY_OPEN
, ES_EVENT_TYPE_NOTIFY_FORK
, ES_EVENT_TYPE_NOTIFY_CLOSE
, ES_EVENT_TYPE_NOTIFY_CREATE
, ES_EVENT_TYPE_NOTIFY_EXCHANGEDATA

, ES_EVENT_TYPE_NOTIFY_EXIT
, ES_EVENT_TYPE_NOTIFY_GET_TASK
, ES_EVENT_TYPE_NOTIFY_KEXTLOAD
, ES_EVENT_TYPE_NOTIFY_KEXTUNLOAD
, ES_EVENT_TYPE_NOTIFY_LINK
, ES_EVENT_TYPE_NOTIFY_MMAP
, ES_EVENT_TYPE_NOTIFY_MPROTECT
, ES_EVENT_TYPE_NOTIFY_MOUNT
, ES_EVENT_TYPE_NOTIFY_UNMOUNT
, ES_EVENT_TYPE_NOTIFY_IOKIT_OPEN
, ES_EVENT_TYPE_NOTIFY_RENAME
, ES_EVENT_TYPE_NOTIFY_SETATTRLIST
, ES_EVENT_TYPE_NOTIFY_SETEXTATTR
, ES_EVENT_TYPE_NOTIFY_SETFLAGS
, ES_EVENT_TYPE_NOTIFY_SETMODE
, ES_EVENT_TYPE_NOTIFY_SETOWNER
, ES_EVENT_TYPE_NOTIFY_SIGNAL
, ES_EVENT_TYPE_NOTIFY_UNLINK
, ES_EVENT_TYPE_NOTIFY_WRITE
, ES_EVENT_TYPE_AUTH_FILE_PROVIDER_MATERIALIZE
, ES_EVENT_TYPE_NOTIFY_FILE_PROVIDER_MATERIALIZE
, ES_EVENT_TYPE_AUTH_FILE_PROVIDER_UPDATE
, ES_EVENT_TYPE_NOTIFY_FILE_PROVIDER_UPDATE
, ES_EVENT_TYPE_AUTH_READLINK
, ES_EVENT_TYPE_NOTIFY_READLINK
, ES_EVENT_TYPE_AUTH_TRUNCATE
, ES_EVENT_TYPE_NOTIFY_TRUNCATE
, ES_EVENT_TYPE_AUTH_LINK
, ES_EVENT_TYPE_NOTIFY_LOOKUP
, ES_EVENT_TYPE_AUTH_CREATE
, ES_EVENT_TYPE_AUTH_SETATTRLIST
, ES_EVENT_TYPE_AUTH_SETEXTATTR
, ES_EVENT_TYPE_AUTH_SETFLAGS
, ES_EVENT_TYPE_AUTH_SETMODE
, ES_EVENT_TYPE_AUTH_SETOWNER
// The following events are available beginning in macOS 10.15.1
, ES_EVENT_TYPE_AUTH_CHDIR
, ES_EVENT_TYPE_NOTIFY_CHDIR

, ES_EVENT_TYPE_AUTH_GETATTRLIST
, ES_EVENT_TYPE_NOTIFY_GETATTRLIST
, ES_EVENT_TYPE_NOTIFY_STAT
, ES_EVENT_TYPE_NOTIFY_ACCESS
, ES_EVENT_TYPE_AUTH_CHROOT
, ES_EVENT_TYPE_NOTIFY_CHROOT
, ES_EVENT_TYPE_AUTH_UTIMES
, ES_EVENT_TYPE_NOTIFY_UTIMES
, ES_EVENT_TYPE_AUTH_CLONE
, ES_EVENT_TYPE_NOTIFY_CLONE
, ES_EVENT_TYPE_NOTIFY_FCNTL
, ES_EVENT_TYPE_AUTH_GETEXTATTR
, ES_EVENT_TYPE_NOTIFY_GETEXTATTR
, ES_EVENT_TYPE_AUTH_LISTEXTATTR
, ES_EVENT_TYPE_NOTIFY_LISTEXTATTR
, ES_EVENT_TYPE_AUTH_READDIR
, ES_EVENT_TYPE_NOTIFY_READDIR
, ES_EVENT_TYPE_AUTH_DELETEEXTATTR
, ES_EVENT_TYPE_NOTIFY_DELETEEXTATTR
, ES_EVENT_TYPE_AUTH_FSGETPATH
, ES_EVENT_TYPE_NOTIFY_FSGETPATH
, ES_EVENT_TYPE_NOTIFY_DUP
, ES_EVENT_TYPE_AUTH_SETTIME
, ES_EVENT_TYPE_NOTIFY_SETTIME
, ES_EVENT_TYPE_NOTIFY_UIPC_BIND
, ES_EVENT_TYPE_AUTH_UIPC_BIND
, ES_EVENT_TYPE_NOTIFY_UIPC_CONNECT
, ES_EVENT_TYPE_AUTH_UIPC_CONNECT
, ES_EVENT_TYPE_AUTH_EXCHANGEDATA
, ES_EVENT_TYPE_AUTH_SETACL
, ES_EVENT_TYPE_NOTIFY_SETACL
// The following events are available beginning in macOS 10.15.4
, ES_EVENT_TYPE_NOTIFY_PTY_GRANT
, ES_EVENT_TYPE_NOTIFY_PTY_CLOSE
, ES_EVENT_TYPE_AUTH_PROC_CHECK
, ES_EVENT_TYPE_NOTIFY_PROC_CHECK
, ES_EVENT_TYPE_AUTH_GET_TASK
// ES_EVENT_TYPE_LAST is not a valid event type but a convenience

// value for operating on the range of defined event types.
// This value may change between releases and was available
// beginning in macOS 10.15
, ES_EVENT_TYPE_LAST
} es_event_type_t;

es_event_exec_t defined in ESMessage.h, as of Xcode 11.5 (11E608c)

/**
* @brief Execute a new process
*
* @field target The new process that is being executed
* @field args Contains the executable and environment arguments (see note)
* @field script Script being executed by interpreter. This field is only valid if a script was
* executed directly and not as an argument to the interpreter (e.g. `./foo.sh` not `/bin/sh
./foo.sh`)
* Field available only iff message version >= 2.
*
* @note Process arguments and environment variables are packed, use the following
* API functions to operate on this field:
* `es_exec_env`, `es_exec_arg`, `es_exec_env_count`, and `es_exec_arg_count`
*
* @note Fields related to code signing in `target` represent kernel state for the process at the
* point in time the exec has completed, but the binary has not started running yet. Because
code
* pages are not validated until they are paged in, this means that modifications to code pages
* would not have been detected yet at this point. For a more thorough explanation, please see
the
* documentation for `es_process_t`.
*
* @note There are two `es_process_t` fields that are represented in an `es_message_t` that
contains
* an `es_event_exec_t`. The `es_process_t` within the `es_message_t` struct (named
"process")

* contains information about the program that calls execve(2) (or posix_spawn(2)). This
information
* is gathered prior to the program being replaced. The other `es_process_t`, within the
* `es_event_exec_t` struct (named "target"), contains information about the program after the
image
* has been replaced by execve(2) (or posix_spawn(2)). This means that both `es_process_t`
structs
* refer to the same process, but not necessarily the same program. Also, note that the
* `audit_token_t` structs contained in the two different `es_process_t` structs will not be
* identical: the pidversion field will be updated, and the UID/GID values may be different if the
* new program had setuid/setgid permission bits set.
*/
typedef struct {
es_process_t * _Nonnull target;
es_token_t args;
union {
uint8_t reserved[64];
struct {
es_file_t * _Nullable script; /* field available iff message version >= 2 */
};
};
} es_event_exec_t;

es_process_t defined in ESMessage.h, as of Xcode 11.5 (11E608c)

/**
* @brief Information related to a process. This is used both for describing processes that
* instigated an event (e.g. in the case of the `es_message_t` `process` field, or are targets
* of an event (e.g. for exec events this describes the new process being executed, for signal
* events this describes the process that will receive the signal).
*
* @discussion
* - Values such as PID, UID, GID, etc. can be extracted from the audit token via API in libbsm.h.
* - Clients should take caution when processing events where `is_es_client` is true. If multiple
ES

* clients exist, actions taken by one client could trigger additional actions by the other client,
* causing a potentially infinite cycle.
* - Fields related to code signing in the target `es_process_t` reflect the state of the process
* at the time the message is generated. In the specific case of exec, this is after the exec
* completed in the kernel, but before any code in the process has started executing. At that
* point, XNU has validated the signature itself and has verified that the CDHash is correct in
* that the hash of all the individual page hashes in the Code Directory matches the signed
CDHash,
* essentially verifying the signature was not tampered with. However, individual page hashes
are
* not verified by XNU until the corresponding pages are paged in once they are accessed while
the
* binary executes. It is not until the individual pages are mapped that XNU determines if a
* binary has been tampered with and will update the code signing flags accordingly.
* EndpointSecurity provides clients the current state of the CS flags in the `codesigning_flags`
* member of the `es_process_t` struct. The CS_VALID bit in the `codesigning_flags` means
that
* everything the kernel has validated up to that point in time was valid, but not that there has
* been a full validation of all the pages in the executable file. If page content has been
* tampered with in the executable, we won't know until that page is paged in. At that time,
the
* process will have its CS_VALID bit removed and, if CS_KILL is set, the process will be killed,
* preventing any tampered code to be executed. CS_KILL is generally set for platform binaries
and
* for binaries having opted into the hardened runtime. An ES client wishing to detect
tampered
* code before it is paged in, for example already at exec time, can use the Security framework
to
* do so, but should be cautious of the potentially significant performance cost of doing so.
The
* EndpointSecurity subsystem itself has no role in verifying the validity of code signatures.
* - The `tty` member will be NULL if the process does not have an associated TTY.
*/
typedef struct {
audit_token_t audit_token;
pid_t ppid; //Note: This field tracks the current parent pid
pid_t original_ppid; //Note: This field stays constant even in the event a process is
reparented

pid_t group_id;
pid_t session_id;
uint32_t codesigning_flags; //Note: The values for these flags can be found in the
include file `cs_blobs.h` (`#include <kern/cs_blobs.h>`)
bool is_platform_binary;
bool is_es_client; //indicates this process has the Endpoint Security entitlement
uint8_t cdhash[CS_CDHASH_LEN];
es_string_token_t signing_id;
es_string_token_t team_id;
es_file_t * _Nonnull executable;
es_file_t * _Nullable tty; /* field available iff message version >= 2. Note: */
} es_process_t;