Techwiki:IPIs

This document describes the kernel mechanism to send and process IPIs.

IPIs on x64

Freeze IPIs

On x64 freeze IPIs are NMIs. This has the advantage, that the IPI will be received, even when the target processor has temporarily disabled interrupts. Freeze IPIs therefore do not take part in the mechanism for generic IPIs.

Generic IPIs

On x64 generic IPIs are handled using a mailbox system: Each processors PRCB has an array of REQUEST_MAILBOX structures, each corresponding to one sender processor. Up to Windows Vista this array is hardcoded to 64 processors, starting with Windows 7 the array is dynamically allocated. Each REQUEST_MAILBOX contains a RequestSummary field, which encodes the request (code, original data address, size), a KREQUEST_PACKET structure that contains the parameters for a generic call. Up to Windows Vista the KREQUEST_PACKET is in a union that is overlapped with an array of 7 pointers. Starting with Windows 7 it contains an additional Next field to form a linked list. Starting with Windows 8 there are 2 additional node related fields.

struct _KREQUEST_PACKET
{
    VOID* CurrentPacket[3];                                                 //0x0
    VOID (*WorkerRoutine)(VOID* arg1, VOID* arg2, VOID* arg3, VOID* arg4);  //0x18
};

#if (NTDDI_VERSION >= NTDDI_WIN7)
struct _REQUEST_MAILBOX
{
    struct _REQUEST_MAILBOX* Next;                                          //0x0
#if (NTDDI_VERSION >= NTDDI_WIN10)
    ULONGLONG RequestSummary;                                               //0x8
#else
    LONGLONG RequestSummary;                                                //0x8
#endif
    struct _KREQUEST_PACKET RequestPacket;                                  //0x10
#if (NTDDI_VERSION >= NTDDI_WIN8)
    volatile LONG* NodeTargetCountAddr;                                     //0x30
    volatile LONG NodeTargetCount;                                          //0x38
#endif
}; 
#else
struct _REQUEST_MAILBOX
{
    LONGLONG RequestSummary;                                                //0x0
    union
    {
        struct _KREQUEST_PACKET RequestPacket;                              //0x8
        VOID* Virtual[7];                                                   //0x8
    };
};

struct _KPRCB
{
    // ...
#if (NTDDI_VERSION >= NTDDI_WIN7)
    struct _REQUEST_MAILBOX* Mailbox; // Head of listed link of entries in use
    // ...
    struct _REQUEST_MAILBOX RequestMailbox[1]; // dynamically sized array
#else
    struct _REQUEST_MAILBOX RequestMailbox[64];
    volatile KAFFINITY SenderSummary; // Bitmask specifying the entries in use
    // ...
#endif
};
#endif

Windows XP/2003/Vista

Up to Windows Vista the maximum number of processors is 64. The PRCB contains an array of 64 REQUEST_MAILBOX strcutures, each corresponding to one IPI sender. An additional SenderSummary field, which is a KAFFINITY, lists the slots that are in use, i.e. the sending processors, who have sent an IPI, which still needs to be processed. The sender of an IPI first checks, whether the bit in the SenderSummary field, that corresponds to the set member of the sending processor is set, and if it is, spins until it is free. Only then the sender populates the REQUEST_MAILBOX slot and finally sends the IPI interrupt to the target processor.

Windows 7+

Starting with Windows 7, the maximum number of processors is dynamic. The PRCB contains a dynamically sized array of REQUEST_MAILBOX structures at the end of the PRCB. Again, each entry corresponds to the processor index of the sender. There is no RequestSummary field though, but instead each entry is pushed atomically on a list with the Mailbox field in the PRCB being the head of the currently active entries.