Pros vs Joes and HackWest
This past week was our first HackWest conference. On Wednesday and Thursday, there were a number of all-day events, some of which went both days. Friday, we had the keynotes and talks throughout the day, along with villages and other awesome stuff. One of those events on Wednesday and Thursday was a Pros vs Joes Capture-the-Flag competition. If you have never done one of these, I would recommend doing it. There’s something to learn for everyone, and quite likely even something to teach. I will mention that it is a good thing if you know how to manage blood pressure, because it gets very intense and stressful.
The first day, there are three teams: two blue teams and one red team. From hackwest.org, we were given this scenario:
You are just hired, along with the rest of your team for a typical corporate entity, with a typical IT infrastructure (a few web sites, a shopping cart, some databases, a CMS, phones, email, domain controller, etc.), and typical corporate needs (get users added/removed, grant access to drives or FTP, keep the web presence online, make sure email and phones are working, etc.). Why has the *entire* team been hired today? Because the last team wasn’t able to get the hackers out. It’s been going on and on, email and phones haven’t been reliable, the web sites are being hacked, data has been exfiltrated. Corporate management had a complete lack of faith in the security team’s ability to solve it, so they cleaned house. This is where you come in. The hackers are already in. You’re already behind the 8-ball. You need to get them out now, and keep them out, and keep all the corporate distractions at rest.
I was on one of the blue teams. We named ourselves “Marv Was Here.” We were given most of our credentials, and a host list that wasn’t quite complete. We had to run nmap to discover the additional hosts and get into them if we could. We had to lock down our servers and patch them. Uptime for the services was scored, as well. So if a red teamer got into one of our servers and shut down a service, we lost points. They would also place a beacon on the server. This let the scorebot know that they had pwned the server. If a beacon appeared on the scoreboard, there was no indication of which server it was on. We were working to keep our servers and services up and keep the hackers out. They were trying to get in, break our stuff, and set up beacons. Yep, intense and stressful.
The rules we were given are all listed out here.
The second day, the red team split in half. Each half joined a blue team. Each team was then allowed to attack and defend, where we were only defending on the first day. And, the red teamers already knew how to exploit the servers. This made it doubly-critical to harden and patch the servers twice as fast as it was done the first day.
Here is the final score from the second day:
See all that red? That’s from what is called “Scorched Earth,” where essentially anything goes. Make the other teams’ servers unusable. Delete critical OS files. Change their passwords. As the entire infrastructure was on ESXi, we discovered each others’ VMWare passwords and just kept shutting down servers. This is essentially when it ended.
In anticipation of how intense it would be, I wrote a set of scripts for the Linux servers. My thought was that we could automate some things. I had to make them as portable as possible, as I did not know what kind of servers we would be working with.
First, we wanted to allow only one user to ssh into the box We didn’t want to allow passwords, only ssh keys. We wanted only one user allowed to ssh into the box. The first script created the user and added that user to the appropriate sudo group. It then added our public ssh keys to that user’s authorized_keys file.
The next script hardened and restarted SSH.
The next couple of scripts set up and added rules to iptables. Huge shout out to bashNinja for his help with the firewall rules script!
Finally, the last scripts set up package repos and refreshed package information so that we could patch the servers quickly.
The feedback I got from our team was that these scripts were very helpful in getting our stuff locked down quickly to allow us to focus on other tasks.
I am releasing the set of scripts we used in hopes that it may be helpful for anyone else doing a Pros vs Joes competition.
They are available on github: https://github.com/4ndronicus/pros-vs-joes