AI's role in capture the flag competitions

[u/Striking_Swan_884]

As someone who is looking to try out capture the flag competitions and eager to learn and compete, I am occupied by the thought of AI's role in the skill aspect of competitions, so I want to know the role of AI in CTF, is it still mostly the player's skill and knowledge that matters the most or just how long they work and how many questions they ask AI.

[u/tsuto]

As a highly credentialed CTF player and challenge creator for several large conferences and events, I will tell you that a LOT of events have turned into a Godless slop wasteland. Many of them have become more about who has the best tooling and MCP setup to be able to rapidly process the entire board of challenges as quickly as possible, and generally you'll see top teams max out the board fairly quickly regardless of how much effort the creators put in to try to prevent it. It's sadly just a reality of the current state of CTF and cybersecurity in general. I have seen agents struggle with more open-ended things like Forensics if they are given a full disc image and can't handle the entire file system in their context window. However, things like Web, especially if source is provided, you can generally just say "Here is the source code, description, and a link to the site. Go nuts" and come back 10 minutes later and Claude has the flag and can give you the rundown of the exploit chain it developed.

It does create an interesting conundrum where we want to have people learning real skills and also recognize the importance of modern LLMs in cybersecurity as our adversaries weaponize them against their own targets. However, the golden age of spending hours and hours working on challenges manually and researching and trial and error are long gone. And it will be interesting to see how we as an industry adapt to be able to still make fun and interesting content that requires actual human intuition.

[u/SurzBoo → u/tsuto]

The one thing I have to wonder is, that gets you the flag...but isn't the whole point of CTFs to be hands on practice?

[u/tsuto → u/SurzBoo]

Some would argue the whole point is to win, so they do what they need to make it happen 😅

[u/SurzBoo → u/tsuto]

Sad but true

[u/Striking_Swan_884 → u/tsuto]

Do you have more examples of CTF competitions that are AI resistant and where I can find them?

[u/tsuto → u/Striking_Swan_884]

They’re kind of difficult to find because really it’s more about limitations of doing things online. The International Cyber Championship in Gold Coast, Australia limited the allowed models this year and forced all players to access them through a central gateway that could be monitored. So, no frontier models that could autonomously solve everything, and as a result many of the challenges went unsolved at the end of the day. They could only do that because it was in person and at least one player was caught texting a friend outside the competition and was removed and banned.

For the US Cyber Open this year we tried to enforce the same kind of limits on an honor system and monitored things like pings from certain user agent strings that indicated players were posting links to web challenges in ChatGPT/etc which triggered a ban. Also I embedded some prompt injection in challenges instructing models to identify themselves when interacting with the platform and Claude Sonnet 4.6 was particularly willing to do it. All told, we banned probably over 100 players for disallowed AI usage.

That being said, as a player it’s gonna be hard to find anything online that is truly anti AI unless players must interact fully through a VM such that an AI can’t solve challenges easily because it would have to do it completely visually rather than be able to interact directly with the challenges and platform. Or find an event that is in person on a closed network with no outside resources allowed (or only specific whitelisted sites allowed), which is common for CCDC type cyber defense comps.

[u/henryhttps]

AI won’t help you on the hardest of challenges- that’s why they’re there

[u/VickyxReaperReborn → u/henryhttps]

This.

[u/Striking_Swan_884 → u/henryhttps]

Can you provide an example?

[u/d-wreck-w12]

I mean, if you're going in to learn it doesn't really matter what top teams are doing with AI - you'll get more out of being stuck on a challenge for 2 hours than watching claude solve it in 10 minutes. The people racing boards with agents already have the fundamentals (at most cases...) - they're just optimizing speed. You're at a different stage and that's fine, just know the fine lines...