[PATCH] memset 0 in obscure is optimized away by compiler

Tito farmatito at tiscali.it
Wed Apr 16 16:47:22 UTC 2014


Hi,
while reading some interesting stuff about memset being optimized
away by compilers if the variable is not read after the memset call
I recalled there was something similar in libbb/obscure.c file:

static int string_checker(const char *p1, const char *p2)
{
	int size, i;
	/* check string */
	int ret = string_checker_helper(p1, p2);
	/* make our own copy */
	char *p = xstrdup(p1);

	/* reverse string */
	i = size = strlen(p1);
	while (--i >= 0) {
		*p++ = p1[i];
	}
	p -= size; /* restore pointer */

	/* check reversed string */
	ret |= string_checker_helper(p, p2);

	/* clean up */
	memset(p, 0, size);

	free(p);

	return ret;
}

I've tried to find out if memset is really optimized away in this case
with some test code that I've compiled with :

gcc -O0  -S  test.c

and then with  -O1 -O2 -O3 -Os.
At a first glance the memset call is there only with the -O0 flag.
I've then tried the libbb nuke_str call instead ans it seems to 
survive compiler optimization, therefore I propose the following
patch to obscure.c:

--- libbb/obscure.c.orig	2013-06-02 13:56:34.000000000 +0200
+++ libbb/obscure.c	2014-04-16 18:20:39.783664194 +0200
@@ -75,8 +75,9 @@
 	/* check reversed string */
 	ret |= string_checker_helper(p, p2);
 
-	/* clean up */
-	memset(p, 0, size);
+	/* clean up, don't use memset as it is optimized away by compiler */
+	/*memset(p, 0, size);*/
+	nuke_str(p);
 	free(p);
 
 	return ret;


As my understanding of the assembler code generated with gcc -S
is _VERY_ limited and due to the fact that it was tested on a different
handcrafted test file I'm not 100% sure the same happens in busybox
but I suspect it so more experienced programmers should take a look
at it.

Ciao,
Tito


-------------- next part --------------
A non-text attachment was scrubbed...
Name: obscure.patch
Type: text/x-patch
Size: 491 bytes
Desc: not available
URL: <http://lists.busybox.net/pipermail/busybox/attachments/20140416/236ddf3d/attachment.bin>


More information about the busybox mailing list