YaK:: WebLog #535 Topic : 2006-06-13 02.10.36 matt : making a test list to drive features [Changes]   [Calendar]   [Search]   [Index]   [PhotoTags]   
  [Back to weblog: pretention]  
[mega_changes]
[photos]

making a test list to drive features

I have delegated code-related things in my BlackHat class to an old friend. I am going to help drive the development with tests, and here is what I came up with at the dog park on Sunday.


As some people know, I'm giving a 2-day class at BlackHat at the end of July with my friend Luis . I have the outline of the class plotted and now I need to come up with a test list to drive the class discussion as if we were talking about an implementation of binary static analysis.

I learned about the concept of a test list from Kent Beck's TDD book . There were a couple of gems in that book, like the "Red-Green-Refactor" pattern and a few others. I'm still not sure how I feel about it as a book everyone should read, though. I prefer other books mentioned in previous posts for that. The test list is great, because it is easy to get caught up in testing lots of things not directly related to the functionality you are driving. The test list is a great way to brain storm creatively and not have it screw up your ability to get quick customer feedback. On sunday, I went to the dog park with Baxter and I started a test list while he was socializing and checking his pee-mail.

Most exploitable bugs have to do with out-of-bounds (OOB) memory access. The easiest example of this to discuss and analyze look like this:

void *buffer = malloc(16);

There are several pointer dereference cases to test just with this one simple allocation. This doesn't even include any branch cases.

buffer[16] = 0;    //  BUG, x >= 16
buffer[15] = 0;    //   OK, x <  0 && x <  16
buffer[-1] = 0;    //  BUG, x <  0
buffer[0]  = 0;    //   OK, x >  0 && x <  16

// t is tainted data -- data that came from a user-supplied source.

buffer[16] = t;    // XBUG, x >= 16
buffer[15] = t;    //   OK, x <  0 && x <  16
buffer[-1] = t;    // XBUG, x <  0
buffer[0]  = t;    //   OK, x >  0 && x <  16

Codifying these in tests would be easy, and they are ordered for optimal TDD implementation. They document a couple of things: First, OOB accesses aren't just beyond but also before the array. Second, don't report in-bounds cases even if they involve tainted data. (Some analysis tools lose on both of these points, which is quite shameful.)

Some people might say "But what about stack emulation, PE/ELF header parsing, branch instructions, my third nipple, non-contiguous functions, conjoined twin myslexia, etc??!$%?!?%???". These are the people who don't know how to start simple and let a design emerge, and probably are into Big Design Up Front (BDUF). More power to them, but I've learned the hard way that starting simple/narrow and generalizing/refactoring by letting the tests drive things in the best way to come up with a clean, elegant, well-tested, and functional program that your users love because it is always moving forward, and never backward. It's also makes everything around it (business or otherwise) work more smoothly thanks to a lower maintenance profile, with the cost of change (optimization, feature adds, and bug fixing) being more predictable.

Discussion:

showing all 0 messages    

(No messages)

>
Post a new message:

   

(unless otherwise marked) Copyright 2002-2014 YakPeople. All rights reserved.
(last modified 2006-07-03)       [Login]
(No back references.)