Zero and forget -- caveats of zeroing memory in C

Published October 29, 2012, by .

Last updated November 11, 2012.

()

In C programs, it’s often desirable to “wipe” memory, for example after having handled cryptographic material. The popular way of doing this is with the standard library’s memset function.

However, there is an important caveat to consider when doing this: an optimizing compiler may just drop your call to memset entirely, leaving your sensitive data in memory despite your best intentions.

This post will explain the problem with using memset to zero memory, present a simple solution, and demonstrate how to use it easily.

An experiment

Consider a simple example. In a first file, foo.c, we declare an array, initialize its contents, and print it out:

#include <stdio.h>
int main(void) {
    unsigned char a[3];
    a[0] = 1;
    a[1] = 2;
    a[2] = 3;
    printf("%u %u %u\n", a[0], a[1], a[2]);
    return 0;
}

If we compile this to assembler, without compiler optimizations (i.e. with -O0), the output is predictable. Now consider a second program, bar.c, identical to the above but with a call to memset to zero the array a before returning:

#include <stdio.h>
#include <string.h>
int main(void) {
    unsigned char a[3];
    a[0] = 1;
    a[1] = 2;
    a[2] = 3;
    printf("%u %u %u\n", a[0], a[1], a[2]);
    memset(a, 0, 3);
    return 0;
}

Note that here I’ll be using the Clang compiler:

$ export CC=clang

Compiling both files to assembler without optimization, and comparing, we get:

$ $CC -O0 -S foo.c
$ $CC -O0 -S bar.c
$ diff -u foo.s bar.s
--- foo.s	2012-10-14 19:06:12.000000000 -0400
+++ bar.s	2012-10-14 19:06:07.000000000 -0400
@@ -23,9 +23,12 @@
 	movzbl	-5(%rbp), %ecx
 	movb	$0, %al
 	callq	_printf
-	movl	$0, %ecx
+	movl	$0, %esi
+	movabsq	$3, %rdx
+	leaq	-7(%rbp), %rdi
 	movl	%eax, -12(%rbp)         ## 4-byte Spill
-	movl	%ecx, %eax
+	callq	_memset
+	movl	$0, %eax
 	addq	$16, %rsp
 	popq	%rbp
 	ret

The call to memset is obviously present in the second and not the first program, as intended. However, compiling with optimizations (-O3), the output of the two programs is now identical:

$ $CC -O3 -S foo.c
$ $CC -O3 -S bar.c
$ diff -u foo.s bar.s
$ echo $?
0

So the compiler is ignoring the call to memset entirely. If the array a had contained something sensitive that the developer had intended to get rid of (like cryptographic keying material), they’ve made a bad assumption.

The problem is that assigning a value to a variable, and then never using it again, is (correctly) interpreted by the compiler as a waste of time, and optimized out.

Below I describe a simple solution to the problem, and a slightly optimized version.

A trivial solution

The solution is to use the zeroed memory as an rvalue somewhere after having zeroed it. So, for example, simply replace a statement like:

memset(a, 0, n);

where n is the count of bytes to be written, with:

a[0] = 0;
for (i = 1; i < n; i++) {
    a[i] = a[i - 1];
}

Here, each byte depends on the previous in the array, even if they all end up with the same value. This loop will not be optimized out.

(Note: the third parameter of memset is the number of bytes to write, not the number of elements in the array.1)

But of course, this solution could be much slower than memset, which may be implemented in assembler with hardware-specific optimizations. So, we will instead attempt to write an optimized, yet portable version of the above.

The memzero solution

Let memzero have the exact same declaration as memset:

void *memzero(void *, int, size_t);

If we were to use the previous trivial implementation, its definition would look like so:

void *memzero(void *mem, int c, size_t n) {
    size_t i;
    (void) c;
    assert(n > 0);
    mem[0] = 0;
    for (i = 1; i < n; i++) {
        mem[i] = mem[i - 1];
    }
    return mem;
}

But for performance reasons perhaps we should write more than one byte at a time… We could use type punning to write 8 octets at a time, like so:

size_t j;
uint64_t *q;
uint64_t qzero = 0;
if (n >= 8) {
    q = mem;
    q[0] = qzero;
    for (j = 1; j < n/8; j++) {
        q[j] = q[j-1];
    }
}

That should be about 8 times faster for large inputs. The remaining bytes past the last multiple of 8 would be zeroed one at a time, just as before. Thus, using these two approaches together, we get the following definition:

void *memzero(void *mem, int c, size_t n)
{
    size_t i, j;
    uint64_t *q;
    uint64_t qzero = 0;
    uint8_t *b;
    uint8_t bzero = 0;
    assert(mem != NULL);
    assert(n > 0);
    (void) c;	    
    i = 0;
#if defined(__LP64__)
    if (n >= 8) {
        q = mem;
        q += i;
        q[0] = qzero;
        for (j = 1; j < n/8; j++) {
            q[j] = q[j-1];
        }
        i += j*8;
    }
#endif
    if (i >= n) {
        return mem;
    }
    b = mem;
    b += i;
    b[0] = bzero;
    for (j = 1; j < n-i; j++) {
        b[j] = b[j-1];
    }
    return mem;
}

Note that the type punning is only actually useful on systems where memory addresses are 64 bits wide, hence we include that code conditionally for environments with the LP64 data model, which incudes most Unix-like systems.

You can also use memzero by simply dropping the files memzero.c and memzero.h into your project. These files are made available as free software, at http://code.google.com/p/memzero/.

Using memzero

At this point you may have wondered why we kept the same function prototype as memset? The reason is that while I use memzero in my code, others don’t — but by using free software, I have to take into account everyone else’s programming flaws. Secondly, if you would like to use memzero, you probably don’t want to do any manual patching, so an automated approach is desirable.

One can use the Coccinelle tool to automatically transform source code to use memzero wherever memset had been used to zero memory.2 To do so, use this simple semantic patch:

@a@
@@
#include <string.h>

@b depends on a@
expression x, n;
@@
- memset(x, 0, n);
+ memzero(x, 0, n);
... when != x

@c@
@@
#include "memzero.h"

@d depends on b && !c@
@@
#include <string.h>
+#include "memzero.h"

There are four rules in this semantic patch: a to d. The first rule matches the inclusion of the string.h header file — which is where memset is declared. The second rule, which depends on a match to the first, matches all instances of memset which have a second parameter of 0, and after which no further reference to its first argument is made; all such instances are replaced by memzero. The third rule, matches the inclusion of the memzero.h header file; and the final rule simply includes the memzero.h header file if it hadn’t already been included.

This semantic patch is distributed with the other memzero files as the file memzero.cocci.

For example, try it on the file bar.c:

$ spatch --sp-file memzero.cocci bar.c
...
@@ -1,11 +1,12 @@
 #include <stdio.h>
 #include <string.h>
+#include "memzero.h"
 int main(void) {
     unsigned char a[3];
     a[0] = 1;
     a[1] = 2;
     a[2] = 3;
     printf("%u %u %u\n", a[0], a[1], a[2]);
-    memset(a, 0, 3);
+    memzero(a, 0, 3);
     return 0;
 }

Now try adding the line a[0] = a[0]; after the call to memset and run spatch again.

This script is quite nice to use, especially on large codebases. As an exercise, try running spatch with the memzero.cocci semantic patch on the latest version of OpenSSL to see how many bugs you find:

$ spatch --sp-file memzero.cocci --dir openssl-1.0.1c \
| tee openssl-1.0.1c-memzero.patch
$ grep -v ^--- openssl-1.0.1c-memzero.patch | grep ^- | wc -l

Enjoy, be careful with memory in C, and always question your assumptions!

Leave a comment below if you enjoyed reading or have any questions.

Update: Thanks to all the helpful comments on Hacker News and below, I’ve made a few improvements to memzero, as you can see from the Git commits. Thanks again to everyone who took the time to critique — I’ve learned quite a bit about memory (especially alginment issues) from the discussion.


  1. This is another, separate mistake developers often make. Do not confuse number of bytes with number of elements in an array. For example, to overwrite an 8-element array of 64-bit types, memset(a, 0, 8) will only overwrite the first element.

  2. For an introduction to Coccinelle and semantic patching, see: http://lwn.net/Articles/315686/

← Return to blog index