CCDC Is The “Real World” And Here’s Why

Anyone who knows me, knows that I strongly believe Collegiate Cyber Defense Competition (CCDC) is literally the best extracurricular activities students interested in cybersecurity can participate in. This belief is rooted in own personal experience – from my time as a student at the Rochester Institute of Technology, I competed in CCDC multiple times, including at the national level in 2011. Every year since, I’ve returned as a volunteer red teamer, both regionally and nationally. As of this writing, that totals 21 CCDC events.

Many outsiders would frame CCDC as “just a game” and not realistic. By outsiders, I mean people who’ve never competed in CCDC, and therefore cannot understand how CCDC may or may not be applicable to the real world.

Let’s look into why this argument is fake news by analyzing the competition as both a blue teamer and a red teamer.

Blue Team Perspective

I started my blue team path by barely making my school’s team. I was assigned to manage an inconsequential Slackware Linux desktop, a distribution I had never touched prior to walking into the room. That position molded me into a team leader who helped shape the composition of our blue team for years to come. I wanted to hoist the Alamo Cup over my head.

And to that end, I did think of it as a game. How can I get more points than other teams? Given all the various ways CCDC is scored, answering this question in any meaningful way takes a strategy. You have to prioritize – are you going to focus on injects or uptime? What about filing compromise reports? I spent hundreds of hours with CCDC teammates John O’Connor and Alex Shagla-McKotch, iterating over these details.

We concluded that a really good blue team needs three things to be successful: experience, skill, and flexibility.

You can’t buy experience. It must be earned through sweat and tears and many sleepless nights. We even coined an expletive phrase to describe the moment when you sit in your blue team chair for the first time at CCDC – “s**ting your pants”.

And anyone who’s blue teamed before knows EXACTLY what I’m talking about. You can have a perfect GPA and know the kernel like it was your girlfriend, but when you sit down at “T0” and the competition begins, a wave of panic will hit you like a brick to the face. It happened to me and everyone I ever saw in that position. How long it lasts is irrelevant – everyone always comes to sooner or later. So if red team was starting at the same time, how do you ensure your team doesn’t freeze? Experience. I take a lot of inspiration from our veteran colleagues and their service. I’ll let the Marine’s say it best: “Improvise, Adapt and Overcome.”

We would prioritize seat assignments first, by experience, and second, by skill. When you figure out mitigations for the fact rookie players are going to need a moment to get their head in the game, you start to understand the first lesson of why blue teaming at CCDC is the real world. Zynga, who recruited me directly from CCDC, brought me onto their incident response team. Almost immediately, this experience of having to handle myself under pressure gave me an obvious leg up compared to my peers. Even if the technology was foreign, I had a much faster reaction time and generally thought with more clarity. Classrooms and certifications and Twitter can’t teach this.

So what about skill? Where does that play into it? Literally everywhere. People have often (wrongfully) asserted that CCDC is more of a game of systems administration than security. This fallacy shows how in the dark the individuals who makes these assertions may be. I’ve worked both incident response and red team positions for the better part of a decade, and I can tell you one thing with certainty:

Your capacity to operate and manage infrastructure (systems administration) is directly proportional to your effectiveness in your security engineering position. In July 2010, I worked with Karen Evans of the Center for Strategic & International Studies on a publication titled “A Human Capital Crisis in Cybersecurity“. In that paper, we showed that skills in areas like systems administration and IT were critically important to a cybersecurity professional operating at an expert level.

Six years later, I spoke at the White House on this same topic. Not only did I maintain the same belief, I now had the experience and wisdom to confirm that the skills you develop participating in CCDC are as real as it gets. Beyond the obvious soft skills you learn participating in CCDC (technical writing, crisis communication, project management, etc.), there’s vast areas of knowledge you get better in. (Shameless plug) Next month, I’m speaking at the Rocky Mountain Information Security Conference on my experience building out Uber’s Incident Response Team. In that presentation, I talk about what I’d learned to look for when evaluating candidates compared to how most people assessed skill. Ordinarily, most candidates were examined in the following areas (or some combination of):

• Security Engineering (Defense)
• Malware Knowledge (Offense)
• Reverse Engineering
• Forensics

I argue that while those are important, you should also vet candidates in these areas:

• Software Engineering
• Systems Administration
• Systems Architecture
• Scalability & Resilience (SRE)
• Automation
• Logging
• Deep knowledge of:
  • Operating Systems
  • Networking
  • Databases
• UI/UX Design
• Data Science
• Technical Writing
• Time Management
• Project Management
• Security Policy & Compliance

I’ve bolded the topics that directly apply to blue teaming at CCDC. So we’ve shown that experience and skill are fundamentally critical to both CCDC and the real world. So let’s wrap up why blue teaming is literally the real world with the final component: flexibility.

Remember that little old Slackware Desktop I said I got assigned to for my very first competition? There was no better teacher for flexibility than getting assigned to this insanely old, completely broken, Slackware linux machine and told to “figure it out.”Legitimate security personnel at some of the worlds biggest companies deal with this quandary on a daily basis, regardless of budget or technology. You need to be prepared for the unknown. This is applicable to everything from your team’s overall strategy, to what variants of BSD you might encounter, to the subtle but important differences between PostgreSQL and MySQL. Almost all of the knowledge I bring to work everyday is a direct result of being able to learn on the fly and use the power of associative memory to rapidly deconstruct complex systems, thereby translating needs into solutions.

Red Team Perspective

I’ve done red teaming for two major technology companies, as well as worked at Lares, one of the world’s most renowned red teams. Every year, I’ve been able to transpose my experience red teaming CCDC with my experience in the real world. Not only is CCDC absolutely real world from an attacker perspective, in fact, I’d argue that most professional red teamers are actually less realistic than the CCDC red team! I will explain.

Let’s compare and contrast the TTPs of the following groups: nation state actors, cyber criminals, CCDC red teams, professional red teamers.

	Nation States	Cyber Criminals	CCDC Red Teams	“Real” Red Teams
Compromise systems in ways that could impact a business.	YES	YES	YES	NO
Compromise collateral targets and use those positions against one another.	YES	YES	YES	NO
Steal large volumes of data – not just a record or two for “confirmation”.	YES	YES	YES	NO
Can decide arbitrarily to corrupt or destroy data in a material way.	YES	YES	YES	NO
Required to follow all laws, regulations, and policies.	NO	NO	NO	YES
Required to conform to a set amount of effort and time.	NO	NO	NO	YES

Notice a pattern? There are simply limits to what you can do as a professional red teamer that you aren’t beholden to in the other three groups. Malicious actors around the world are something most security professionals have never experienced in a controlled environment. This is why CCDC is actually more real world than basic penetration tests. The blue teamers get to experience a determined and sophisticated adversary that disregards collateral damage and adjacent victims.

And as a red teamer, that is incredibly valuable. After CCDC last year, I wrote at length about the philosophy that guides good red teams and the “Attacker’s Mindset”. In that article, I describe the mission of a professional red team:

“A red team’s mission is to simulate threats to identify previously unknown and unmitigated risk. Through a unique blend of adversarial engineering and standard penetration testing methodologies, red team exercises are meant to challenge your organization’s belief in the strength of their security. By emulating a wide array of threat actors, red teams bring perspective uniquely suited to the iterative development models commonly practiced in information security programs.”

This captures the essence of how a penetration tester is different from a red teamer. The most valuable skill you bring to the table during a red team is to act like an evil villain, without actually being one. Hollywood sets a phenomenal example in how this works in life. Consider Bane and Tom Hardy, the actor who played him. His performance will put chills in your blood. A tribute to the apex villain. But do you think that when the cameras stopped rolling, Tom continued to break spines and blow up Heinz Field? Absolutely not! He’s capable of this because he’s acting. He has developed a skill known as method acting to be able to be the bad guy on camera without carrying the monster off screen with him. This type of emulation takes thousands of hours of practice and a few lessoned learned to get right. But it’s immensely important to do so. Without being able to maliciously hack, you can never simulate a malicious hacker. If only there was a place red teamers could sharpen their skills in a safe and beneficial manner!

In all seriousness, what are the options? Capture the Flags (CTFs) too often are just computer science puzzles. Training courses and certifications? You mean a basic exercise in knowledge regurgitation, no sign of the skill, experience, or flexibility? Gross. Myself and many of my veteran red teamers use CCDC quite literally as a dojo for developing our professional skills in an ethical and community building way. Not only do we get to practice strategies, tactics, and techniques that will make us better at our day jobs, we get to give back to the community by providing blue teams the adversary they deserve to face.

So let’s get into some of the details behind what goes into red teaming CCDC.

Tooling and the software development done in preparation are the biggest differentiator between CCDC red teamers who succeed and fail. One of the reasons I’ve stayed away from big name conferences the last few years is that I got sick of sharing drinks with “lab hackers”. A lab hacker is someone who spends an exorbitant amount of time and effort in a controlled setting, then makes sweeping, hyperbolic comments about how that work is applicable everywhere in the world. Then, more times than not, when it’s looked at by others, it’s uncovered what little to no merrit the comment had in the first place. Where as the lab hacker puts their effort and focus on getting accepted to a conference, we focus on weaponization. This means that 90%+ of the lab hacker’s work is not ready for the field.

So CCDC is a great way to test yourself and your tools. You’re walking into an environment that you have no visibility into – will your tools work at gametime or will you make excuses about how you can’t apt install some dependency or the victim doesn’t have “.NET”. Let me play you sad little song on my little violin.

But what about stealth? Clearly we must trade stealth for speed and that’s unrealistic right? Wrong. Look at some of the major worm that’s hit in the last decade or so:

• Conficker
• Mirari
• RPCDCOM Worm
• WannaCry

Would you say any of those were “quiet” or “stealthy”? Not even a little bit. They slashed and burned around the world without resistance. Does the bubonic plague need to be stealthy to be effective? Hell no and neither do your adversaries. In the “real world”, competent red teams (and of course the attackers they model) use what works, whether that’s default creds, phishing, unpatched vulns, and yes, even 0day (something that has been produced during a CCDC event in fact). Also, if you don’t know this already, not all phishing involves email.

I’ve spent years developing my CCDC red team toolbox for nothing but the purpose of CCDC. So when Vault7 hits and describes an exploit that you independently stumbled upon and weaponized specifically for using in CCDC, you start to take notice how that toolbox stacks up to quite literal nation state actors. In all seriousness, the only open source or commercial security tools I used at all this year were nmap and Paragon VMDK Mounter. As the maintainer of the CCDC Red Team’s GitHub, I have the stats to prove it. In the last 2-3 years, we’ve had well over 1000 commits made 30 different volunteers. Much of that code is inspired by field work all of us have done. That code transcends over 12 different languages targeting virtually every major operating system, including historic versions. It’s gotten to the point where a group of us actually shared a house for the week prior to Nationals and wrote code the entire time. (Stay tuned for another blog post in the coming days detailing what the infrastructure behind all of this is and why you are unable to simply “block” us.)

This massive undertaking is mutually beneficial. Blue teams get the simulated experience they can expect from real world threats and we can sharpen and refine our skills in a morally, ethically, and legally sound way. In another post I wrote for work, I wrote:

“The majority of Uber’s security response team volunteers on the CCDC Red Team, providing us an opportunity to continuously improve our ability to defend Uber’s infrastructure.”

And don’t take it from me, other companies also do the same. In fact, it’s highly competitive. At the national level, outside of the core red team there is only 10 volunteer spots and 10 sponsor spots. To earn a volunteer slot, you must apply directly. This process will take only a very select few of the total applicants and happens new again every year. I encourage you to apply. We love to red team with people who put in the blood, sweat, and tears to better themselves and their tradecraft.

Conclusion

As you read above, there are many aspects of both blue teaming and red teaming in a competition like CCDC that prepare students and volunteers alike in ways nothing else can. If CCDC was so detached from the real world, that could not happen. I’ve seen it happen for many students and, in my particular case, this real-world exposure changed the course of my life and my career.

I’d like to tell share with you where my attacker’s mindset comes from. I cut my teeth in security as a pre-teen cracking, hacking, and botting video games like Runescape and Diablo II. By the time I was 14, I had dropped out of high school and spent the next four years of my life running the streets with gang bangers and dope dealers. After turning 18 with mounting legal issues, I decided it was time to grow up. I had found a small community college in San Francisco that offered an associates degree in network security. I had no idea the skills I learned growing up on the streets could be used for good. Most people would repress such painful experiences, but I’ve been able to take those memories and wisdom and apply it to being a white hat security professional. Over time, I’ve met other professionals with similar stories. Hands down, their some of the most qualified cybersecurity professionals I know. Experience is without question the best patch for human deficiencies.

This unique experience is a scar I cherish. It gave me a serious leg up in our field. People can spend their entire careers being nerds chasing the skills to emulate criminals, and here I was, the criminal turned college freshman who needed to learn to become a nerd. Having given up the mask in exchange for the cape, there has been no better conduit for personal growth and development than CCDC. Blue team or red team, it doesn’t matter. The skills derived through that experience is quite literally what makes me good at my job. There are people with more certifications, more degrees, more retweets, but when push comes to shove, when I sit down at that keyboard, my CCDC experiences give me a far greater advantage than any piece of paper you have.

It doesn’t get any more real than that.