Why have a contest?
- Hackers like to show off
- Wanted a real world experience that emulated what developers might experience
- No competition was out that focused on both building and breaking software
- Wanted a contest where contestants could write software and hack their own software
What would contest need?
- Need a lot of data on software security issues
- Want to have the ability for two or three researchers to run it at a university and multiple people could participate
- Need to be FUN!
What might this contest look like?
- Build It - Make the software during a two week period. Can use any programming language
- Break It - Distribute code from Build It phase to different teams and have other teams find bugs in code submissions to gain points
- Fix It - Ten day period where teams get to fix their code that was broken during Break It phase
How do you motivate people to write clean code?
- Features give you extra points during Build It phase
- Security will lose you fewer points in the Break It phase
- Breakers want to look at all different problems that there might be with the software
- Want to find major security property bugs not low level bugs
Spring 2015 and Fall 2015 Results
Ship Scores - Build It Results
- Teams who programmed in C or C++ performed on average 121 and 92 points better than those who programmed in dynamically typed or statically typed languages
- Due to having more optional features
- MOOC teams performed 119 points better
- Each additional line of code in a submission correlated with a drop of .03 points
Break It Results
- More team members meant they found more bugs
- 430 additional points per team member
- More bugs found in Fall 2015 project on security because of bank server authorization
Fix It Results
- So many bugs coders did not even try to fix problems
- Build It, Break It, Fix It, is a relatively new security contest that emphasized building secure software not just breaking it and finding bugs
- Prize packages and automated testing mean it's possible to run on a large scale across the internet
- Data gathered from contest help to measure relationships between software development and security