Defensive programming

In the absence of the two types of countermeasures above, a security-conscious programmer that writes his own programs and accepts a performance penalty can clear the buffer before freeing memory with a function such ascudaMemset.

If the end-user can not modify the program, he should erase the GPU memory when finishing an execution on a GPU. This countermeasure seems trivial, nevertheless its practical implementation can be difficult due to the compli-cated memory hierarchy present in GPUs (e.g., access mechanisms depend on the type of memory). A standalone CUDA program that cleans the memory would allocate the maximum amount of memory, and then overwrite it (e.g., with zeros). However, this solution relies on the CUDA memory manager, which does not guarantee the allocation of the whole memory. Portions of memory risk not to be properly erased because of fragmentation issues. We built an experiment to illustrate this. We run a first CUDA program for some time, then we stop it to run a second CUDA program that cleans the memory.

We finally dump the memory via PRAMIN to access the whole memory. We recovered a portion of the memory that was not cleaned by the second CUDA program, demonstrating clear limitations of this countermeasure.

A practical solution for NVIDIA Tesla GPUs that benefit from ECC memory is to enable ECC and reload the driver, or to reset the GPU when ECC is enabled.

As we saw in our experiments Section 5.5.1, these sequences of actions clear the memory.

5.8 Related work

Using the CUDA framework, Di Pietro et al. [DLV13] showed that GPU ar-chitectures are vulnerable to information leakage, mainly due to memory isolation issues. The leakage affects different memory spaces in GPU: global memory, shared memory, and registers. Di Pietro et al. also showed that current implementations of AES cipher that leverage GPUs allow recovering both plaintext and encryption key in GPU global memory. Bress et al. [BKS13]

considered using these vulnerabilities to perform forensic investigations. Nev-ertheless, they noted that we cannot guarantee that calls to the CUDA API do not modify the memory. These two articles began to pave the way of GPU security. However, they did not evaluate information leakage by GPUs in the context of virtualization that is characteristic of cloud computing. In contrast, we analyzed several approaches to access the GPU memory, experimenting with different drivers and GPGPU runtimes, as well as the PCI configuration space. We also extended these findings to virtualized environments.

Subsequent work of Lee et al. [LKKK14] showed further the applicability of this type of information leakage by recovering textures rendered by the GPU for web browsers. This attack allows inferring which web pages are visited by a victim. Danisevskis et al. [DPS13] showed that mobile GPUs, which are becoming increasingly more common, and in particular their DMA capabilities, can also be abused for malware delivery.

5.9 Conclusions and perspectives

In this chapter, we evaluated the confidentiality issues that are caused by the recent advent of GPU virtualization. Our experiments in native and virtual-ized environments showed that the driver, operating system, hypervisor and the GPU card itself do not implement any security related memory cleanup measure. As a result, we observed information leakage from one user to another, and in particular from one virtual machine to another in a virtual-ized environment. Amazon EC2 seems to implement proper GPU memory cleaning at the provisioning of an instance; we could thus not confirm any information leakage from one Amazon instance to another. However, because of the general lack of GPU memory zeroing, we cannot generally exclude the existence of data leakage in cloud computing environments.

The rise of GPGPU increases the attack surface and urges programmers and industry to handle GPU memory with the same care as main memory. For this matter, industry should include GPU memory cleaning in its best practices.

We provided a set of recommendations for proper memory cleanup at the various layers involved in GPU virtualization (application, driver, hypervisor).

The most recent trend in GPU virtualization is the shift from sequential sharing of a GPU card to simultaneous sharing between several tenants. In-deed, NVIDIA launched a new series of GPUs called GRID that handle virtu-alization in hardware with a dedicated hypervisor. In this context, memory isolation is even more challenging, and new issues arise such as covert and side channels. This effectively allows the other attacker model we described in Section 5.2,i.e., theparallel attacker. In context of native environments, inte-grated GPUs are another unexplored research topic. Being manufactured on the same die as the CPU, they must fit on a smaller surface than a discrete GPU and are thus less efficient. Moreover, they do not have any dedicated memory and therefore they use the system memory. Yet, the latest range of integrated GPUs increased in performance, and they are now much more widespread.

The consequences of this integration to the CPU in terms of security will be explored in a future work.

6

Conclusions and future directions

Contents

6.1 Contributions . . . . 85 6.2 Future directions . . . . 86 Information leakage due to shared hardware has recently gained a new impetus with the appearance of cloud computing environments. Indeed, these environments heavily use virtualization to co-locate several virtual machines, that can be owned by different customers, on the same physical machine. This thesis focuses on attacks due to memory isolation issues as well as covert and side channels performed at the microarchitectural level. Indeed, main memory and some microarchitectural elements are shared between virtual machines and accessible without physical access. However, these attacks are highly dependent on the hardware, which evolves constantly. Our work was thus motivated by the following question: How do the recent evolutions impact information leakage due to hardware sharing?

6.1 Contributions

In this thesis, we explored the impact of evolutions in recent hardware in terms of information leakage on shared hardware and applied our findings to virtualized environments.

Exploiting hardware features to build a low-requirement covert channel Cross-core cache attacks target the last-level cache that is shared by all CPU cores. Previous attacks on the last-level cache relied on shared memory in order to evict cache lines. However, disabling shared memory is sufficient to circumvent these attacks, and this countermeasure is already in production in some cloud environments. We demonstrated a cross-core covert channel without using any shared memory, exploiting the shared and inclusive prop-erties of the last-level cache. We evaluated this covert channel in native and virtualized environments.

Reverse-engineering cache internals to improve attacks The function that maps a physical address to a last-level cache slice is undocumented on Intel CPUs. This renders attacks either slow, difficult or even impossible to perform, because of the loss of precision in the cache collisions. We built an automatic and generic method for reverse-engineering the complex addressing func-tion. We evaluated our method and retrieved the function for a wide range of processors, encompassing different microarchitectures and different num-bers of cores. We demonstrated the broad applicability of these findings by accelerating our cache covert channel and adapting Rowhammer, a DRAM vulnerability, in sandboxed JavaScript.

Investigating setups causing information leakage on GPU memory GPUs have been designed to provide maximum performance. They have not been designed with security in mind, nor to support concurrent access. Yet, they are increasingly used in cloud computing setups. We documented the security implications of the different GPU virtualization techniques. We systematically experimented and analyzed the behavior of GPU global memory in the case of direct device assignment. We found information leakage in scenarios such as an attacker launching a virtual machine after a victim’s virtual machine using the same GPU. With these results, we clearly underlined bypasses of the isolation mechanisms of virtualization. We proposed countermeasures to be implemented by cloud providers and end users.