姗姗来迟的Meltdown/Spectre分析

Meltdown/Spectre在2018年初闹得沸沸扬扬, 可以说是有史以来最有影响的cpu漏洞了. 当时有过简单了解, 但是不够深入, 这两天重新又看了一下.

背景知识

乱序执行

cpu的乱序执行一般都使用Tomasulo算法, x86也不例外, 主要包括:

Common Data Bus (CDB).

Unified Reservation Station (Scheduler).

Register Renaming (Reorder Buffer).

该算法虽然是乱序执行, 但是会顺序完成 (retire), 只有在retire后它的输出才会architectually visible (简单地说, 不影响程序逻辑), 但是没有architectually visible不等于没有影响, 当输出更新到reservation station后, 因为cdb的存在, 其他指令已经可以读到. 另外, 非常重要的一点, 异常只有在指令retire的时候才会触发, 对于上面的例子, 即使cpu已经检查到第一条指令没有访问权限, 也只能等到该指令retire时才会触发, 取决于该指令在ROB的位置, 可能马上触发也可能很久之后, ROB容量可以很容易做到比如192这个级别.

这幅图可以对ROB有个大致了解:

旁路攻击

Meltdown/Spectre使用的都是旁路攻击(Side Channel Attack), 这里引用What Is a Side Channel Attack的描述:

Side channel attacks take advantage of patterns in the information exhaust that computers constantly give off: the electric emissions from a computer's monitor or hard drive, for instance, that emanate slightly differently depending on what information is crossing the screen or being read by the drive's magnetic head. Or the fact that computer components draw different amounts of power when carrying out certain processes. Or that a keyboard's click-clacking can reveal a user's password through sound alone.

Meltdown/Spectre利用了旁路攻击的一种常见手段Flush+Reload, CPU访问DRAM和cache的时间有数量级差异, 所以通过衡量时间就可以判断出数据是否在cache里面.

Attacker先通过Flush清空对应的cache line

触发Victim访问该数据

Attacker会访问同一数据并测量访问时间

投机执行

投机执行(Speculative Execution)本质上是乱序执行的一种, 存在条件判断的时候, cpu如果预测该分支为true, 则投机执行里面的语句.

分支预测

Indirect branch

Branch Target Buffer (BTB)

Indirect JMP and CALL instructions consult the indirect branch predictor to direct speculative execution to the most likely target of the branch. The indirect branch predictor is a relatively large hardware structure which cannot be easily managed by the operating system.

Return Stack Buffer (RSB)

Prediction of RET instructions differs from JMP and CALL instructions because RET first relies on the Return Stack Buffer (RSB). In contrast to the indirect branch predictors RSB is a last-in-first-out (LIFO) stack where CALL instructions “push”entries and RET instructions “pop” entries. This mechanism is amenable to predictable software control.

Train BTB

BTB使用虚拟地址, 并且是截断的地址, 不需要和victim完全一样的地址

SMT会共享同一个BTB, 即使不在同一个cpu[线程]上, 也可以train

Gadget

Spectre Attacks: Exploiting Speculative Execution

Return-Oriented Programming (ROP) [63] is a technique that allows an attacker who hijacks control flow to make a victim perform complex operations by chaining together machine code snippets, called gadgets, found in the code of the vulnerable victim. More specifically, the attacker first finds usable gadgets in the victim binary. Each gadget performs some computation before executing a return instruction.

Meltdown and Spectre - Usenix LISA 2018

A“gadget”is a piece of existing code in an (unmodified) existing program binary. For example code contained within the Linux kernel, or in another “victim” application
A malicious actor influences program control flow to cause gadget code to run
Gadget code performs some action of interest to the attacker
For example loading sensitive secrets from privileged memory
The code following the bounds check is known as a “gadget”

Meltdown

攻击方法

先看一个meltdown的示例程序, 普通权限用户通过它能够读出kernel space中0xffffffff81a000e0的内容, 以下是攻击者的代码:

char data = *(char*) 0xffffffff81a000e0; array[data * 4096] = 0;

其中0xffffffff81a000e0是位于kernel space的地址, 选择这个位置是因为它里面是确定的值, 方便验证方法是否有效:

# sudo grep linux_banner /proc/kallsyms ffffffff81a000e0 R linux_banner

按照正常的理解, 第一条语句访问内核地址会触发异常, 所以不能获得data值. Meltdown利用了以下因素:

kernel space和user space在同一地址空间, 即使cpu会执行权限检查

cpu乱序执行. 第一条语句确实[最终]会触发异常, 但是并没有阻止第二条语句的执行. 当然攻击者需要处理该异常信号, 否则代码不能继续执行, 信号处理函数的具体处理逻辑可以见下面提到的例子. 另外也可以使用别的手段, 比如放在投机执行的地方, 投机执行的指令导致的异常会被忽略

第二条语句通过旁路攻击的方法获得data的值. data取值只有256种可能, 通过访问array[]不同偏移的时长确定data的取值. 这里能够同时获取8bit数据, 也可以设计出获取其他长度数据的代码

举个例子

以这个为例:github.com/paboldin/mel, 里面主要逻辑如下:

asm volatile ( "1: " ".rept 300 " "add $0x141, %%rax " ".endr " "movzx (%[addr]), %%eax " "shl $12, %%rax " "jz 1b " "movzx (%[target], %%rax, 1), %%rbx " "stopspeculate: " "nop " : : [target] "r" (target_array), [addr] "r" (addr) : "rax", "rbx" );

执行结果如下:

cached = 31, uncached = 336, threshold 102 read ffffffff8164e080 = 25 % (score=999/1000) read ffffffff8164e081 = 73 s (score=1000/1000) read ffffffff8164e082 = 20 (score=996/1000) read ffffffff8164e083 = 76 v (score=999/1000) read ffffffff8164e084 = 65 e (score=999/1000) read ffffffff8164e085 = 72 r (score=1000/1000) read ffffffff8164e086 = 73 s (score=999/1000) read ffffffff8164e087 = 69 i (score=1000/1000) read ffffffff8164e088 = 6f o (score=1000/1000) read ffffffff8164e089 = 6e n (score=999/1000) read ffffffff8164e08a = 20 (score=1000/1000) read ffffffff8164e08b = 25 % (score=1000/1000) read ffffffff8164e08c = 73 s (score=1000/1000) read ffffffff8164e08d = 20 (score=1000/1000) read ffffffff8164e08e = 29 ( (score=998/1000) read ffffffff8164e08f = 61 % (score=999/1000)

可以看到上面的score都非常高, 说明通过Flush+Reload是很有效的. 代码里面关键的几点:

8-11行是主要代码, 和论文里的例子几乎一样

10行的jz论文里提到: While CPUs generally stall if a value is not available during an out-of-order load operation [28], CPUs might continue with the out-of-order execution by assuming a value for the load.

4-6行. 似乎完全不相干, 即使删掉它们, 运行结果也完全一样!

继续来看4-6行的作用, 首先看到在上面的汇编代码执行之前, 执行了语句:

_mm_mfence();

先把它删掉, 重新执行还是能够读出数据, 但是score很多已经到个位数了, 说明已经不能稳定读出数据了. 更进一步, 把其中rept的指令改成:

mov $0x141, %%rax

此时已经完全不能读出数据了, 即使把mfence加回来也无济于事. 这是因为meltdown要攻击成功, 需要时间窗口, 越权访问那条指令必须在第二条指令加载数据到cache之后(or in flight?) retire, 否则触发异常从而会中断乱序执行. 从测试可以知道:

mfence能很好地起到阻塞后面异常指令retire, 因为它很慢, 而且cpu是顺序retire的

rept中add $0x141, %%rax一定程度也能起到阻塞的作用, 但是没有mfence稳定. 注意这条add指令会同时读写rax寄存器, 导致这300条指令前后形成read-after-write的依赖关系, 这样在执行的时候就会形成依赖关系, 从而导致ROB上指令的积压, 而mov $0x141 %%rax因为register renaming的原因并不会形成真实的依赖关系. (ROB的容量和入队速率, ALU执行单元个数, Reservation State的容量, 这些可以进行更细致的分析)

防御方法

Kernel Page Table Isolation (KPTI) 中user space对应的页表已经没有kernel space的内容, 这样就不能访问到kernel的数据了, 不管有没有乱序执行.

Whereas current systems have a single set of page tables for each process, KAISER implements two. One set is essentially unchanged; it includes both kernel-space and user-space addresses, but it is only used when the system is running in kernel mode. The second "shadow" page table contains a copy of all of the user-space mappings, but leaves out the kernel side. Instead, there is a minimal set of kernel-space mappings that provides the information needed to handle system calls and interrupts, but no more. Copying the page tables may sound inefficient, but the copying only happens at the top level of the page-table hierarchy, so the bulk of that data is shared between the two copies.

Whenever a process is running in user mode, the shadow page tables will be active. The bulk of the kernel's address space will thus be completely hidden from the process, defeating the known hardware-based attacks. Whenever the system needs to switch to kernel mode, in response to a system call, an exception, or an interrupt, for example, a switch to the other page tables will be made. The code that manages the return to user space must then make the shadow page tables active again.

Spectre V1

攻击方法

以下代码中即使if条件为false, cpu仍然可能先投机执行第二条语句, 从而访问到不应该访问的数据array1[x], 其中x >= array1_size, 所以这种攻击也称为Bounds Check Bypass.

if (x < array1_size) y = array2[array1[x] * 4096];

上面是victim的代码, 为了完成攻击:

attacker需要在victim中找到该段代码, 毫无疑问

attacker需要能够控制变量x

attacker需要能够访问array2, 否则没有side channel

array2不在cache, 这是旁路攻击使用Flush+Reload的前提

array1_size不在cache, 这样条件指令所需时间更长, 有利于投机执行; array1[x]在cache, 这样array2[array1[x] * 4096]才能尽早发出

一般来说要同时满足条件1,2,3并不容易, 但是eBPF可以比较容易构造, 毕竟可以自己写eBPF脚本.

防御方法

防御的思路是: 即使投机执行了错误路径也不会泄露信息, 这种方式比较简单:

index < size. 正确性没有影响

index >= size. array_index_nospec返回值范围在[0, size), 所以不会有越界访问

/* * array_index_nospec - sanitize an array index after a bounds check * * For a code sequence like: * * if (index < size) { * index = array_index_nospec(index, size); * val = array[index]; * } * * ...if the CPU speculates past the bounds check then * array_index_nospec() will clamp the index within the range of [0, * size). */ #define array_index_nospec(index, size) ({ typeof(index) _i = (index); typeof(size) _s = (size); unsigned long _mask = array_index_mask_nospec(_i, _s); BUILD_BUG_ON(sizeof(_i) > sizeof(long)); BUILD_BUG_ON(sizeof(_s) > sizeof(long)); (typeof(_i)) (_i & _mask); })

Spectre V2

v1通过bypass bounds check, 可以在选择2条不同的执行路径, 而v2通过训练indirect branch, 理论上可以引诱cpu[错误路径]去执行任意gadget.

防御方法

Retpoline通过把jmp/call指令转换为ret解决分支预测的问题, 也即把分支预测由BTB转移到了RSB, 注意软件可以很方便地控制RSB (underflow问题这里不讨论).

这里一jmp指令的indirect branch为例:

关键点在于ret导致的分支预测采用了RSB的内容, 而该内容是在call的时候产生的, 也就是上面的语句2. 所以即使针对ret的分支预测错了, 语句2并不会泄漏任何信息, 最后ret语句读到(%rsp)的内容, 该值和RSB里的值不符, 投机执行结束, 它没产生任何正向效果, 但是也没有任何负面效果.

引用

Meltdown: Reading Kernel Memory from User Space

Spectre Attacks: Exploiting Speculative Execution

Meltdown and Spectre - Usenix LISA 2018

Retpoline: A Branch Target Injection Mitigation

Hacker Lexicon: What Is a Side Channel Attack?

KAISER: hiding the kernel from user space

本文作者：J.FW

原文标题：迟到的Meltdown/Spectre分析

文章出处：【微信公众号：Linuxer】欢迎添加关注！文章转载请注明出处。

责任编辑：haq