Build-it, Break-it, Fix-it Contesting Secure Development By: Andrew ruef, Michael hicks, James parker, Dave Levin, Michelle L. Mazurek, Piotr Mardziel

What is Build It, Break It, Fix It?

A contest that is hosted on the builditbreakit website that would push programmers to develop secure clean code

Why have a contest?

  • Hackers like to show off
  • Wanted a real world experience that emulated what developers might experience
  • No competition was out that focused on both building and breaking software
  • Wanted a contest where contestants could write software and hack their own software

What would contest need?

  • Need a lot of data on software security issues
  • Want to have the ability for two or three researchers to run it at a university and multiple people could participate
  • Need to be FUN!

What might this contest look like?

  1. Build It - Make the software during a two week period. Can use any programming language
  2. Break It - Distribute code from Build It phase to different teams and have other teams find bugs in code submissions to gain points
  3. Fix It - Ten day period where teams get to fix their code that was broken during Break It phase

How do you motivate people to write clean code?

  • Features give you extra points during Build It phase
  • Security will lose you fewer points in the Break It phase
  • Breakers want to look at all different problems that there might be with the software
  • Want to find major security property bugs not low level bugs

Build It scoring

  • Correctness and Performance

Break It scoring

  • Bugs Found - More points awarded for Security, Crashes, Correctness

Fix-It scoring

  • Teams are given points based on their ability to fix coding errors

Fall 2015 Example - Secure Communications over network channel

Fall 2015 - 48 teams composed of 122 participants

Spring 2015 and Fall 2015 Results

Survey Says!

Survey Questions Asked to Participants

Ship Scores - Build It Results

  • Teams who programmed in C or C++ performed on average 121 and 92 points better than those who programmed in dynamically typed or statically typed languages
  • Due to having more optional features
  • MOOC teams performed 119 points better
  • Each additional line of code in a submission correlated with a drop of .03 points

Break It Results

  • More team members meant they found more bugs
  • 430 additional points per team member
  • More bugs found in Fall 2015 project on security because of bank server authorization

Fix It Results

  • So many bugs coders did not even try to fix problems

What are the big take away's from this?

  • Having more team members correlated with a higher break it score, but also did not help the builder scores
  • The best Built It submission's utilized C/C++ and submissions coded in a statically-typed language were less likely to have a security flaw
  • If a team was a good Break It team they were also a good Build It team
  • Using an advanced technique like fuzzing did not have an impact on break it score

Conclusion

  • Build It, Break It, Fix It, is a relatively new security contest that emphasized building secure software not just breaking it and finding bugs
  • Prize packages and automated testing mean it's possible to run on a large scale across the internet
  • Data gathered from contest help to measure relationships between software development and security
Created By
Peter Schwartz
Appreciate

Made with Adobe Slate

Make your words and images move.

Get Slate

Report Abuse

If you feel that this video content violates the Adobe Terms of Use, you may report this content by filling out this quick form.

To report a Copyright Violation, please follow Section 17 in the Terms of Use.