Ask Riot: Star Shards and Choncc
It’s all about TFT today, including the most difficult champs to bring to Fates and some fun stats.
Hello again. If you’re reading this, it is my hippocratic responsibility to inform you that you may’ve actually suffered through one of these before, and if you’re still here now, you have only repressed memories of the trauma. So, I’m what’s left of mirageofpenguins, a data-based engineer on the Anti-Cheat team, and I’ve been torn screaming from the void and disgustingly polymerized with this keyboard to:
As Riot whimsically adds games to its catalogue, we find ourselves in the position of having to mount a defense against an ever-increasing number of bad guys. Up until now, we had just been tenaciously flinging resources at any adversary we could sink our teeth into, but as machiavellian as that strategy sounds, it unfortunately doesn’t scale through the multi-game universe.
Anti-cheat engineers have finite biomass, and as Dr. Einstein probably predicted, the human elements required to fabricate an entire anti-cheat platform had to be painfully extracted from the lifestream. To buy the time required to make tomorrow’s solution, we had to moderate the speed at which we were technologically advancing our own opposition. Every hammer we swing is a signal to cheaters, and every update we make is eventually circumvented. We needed to reshape the way we actioned abuse to extend the functional longevity of each iterative response.
So how did we pump the brakes on our scripting arms race? Strap in, we’re going for a ride.
Just to spare you the ethical catastrophe of pointing your browser to a cheat forum, scripting refers to having an external program monitor game state for the purpose of executing inputs on the cheater’s behalf. It’s kinda like if your keyboard had a magstripe reader, and you could pay it to press buttons for you. Scripting can be very performant when utilized for mechanically intense combos or near-instantaneous reactions, but it does dramatically reduce the lifespan of your LoL account. We ban because we care, and we care because we believe the only things you should need to be competitive in a video game are your brain and an ethernet cable.
“No one buys tickets to see a piano that plays itself, your imperfection is part of the craft.”
Collectively, the scripting landscape isn’t quite what it used to be. Larger providers explicitly make no effort to circumvent our detection methodology, creating a private market for smaller providers utilizing ridiculous price points and promises of undetected cheats. We’ve seen subscriptions as high as $300 USD, some even with the requirement that an image of your driver’s license be included for “verification purposes.” That last thing was particularly amusing, because it’s not like I don’t have access to an entire team of graphic designers with 200 years collective experience in Adobe Photoshop.
Anyways, I won’t sugarcoat it for the recruits: This isn’t a war we’ll ever win completely. The scripting community can’t dodge a wrench, a ban, or a Blitz hook, but while we’ve driven them underground, we’re now playing a game of shadows.
What you’re looking at here is a metaphysical representation of crude scripting volume as it moves through a medium that you perceive as time. Or in layman's terms, this is the percentage of ranked Summoner’s Rift games that have a cheater in them. It’s much lower than it used to be, but because there’s no real barrier to player reentry (a copy of League of Legends costs $0), it probably won’t get any lower.
Other key insights include:
Listen, I love this graph. I made this graph. I had it permanently tattooed onto both of my forearms, and I go back to the parlour once a week to have them excruciatingly updated. But while it’s cathartic and useful for remaining employed, it really isn’t the most important metric. Not every instance of cheating is equal: A single scripter in Master tier, for example, could land themself on the front page of Reddit, and then we’ve got significant hoots for a relatively smaller problem.
“Winning in perception matters just as much as winning, because players aren’t willing to put the work in for mastery of a game where their competition could be cheating.”
But which scripts do players take notice of?
Firstly, we wanted to validate that players could identify cheating with greater accuracy than a good roll of the oracle bones, and honestly, some do not have “the sight.” It’s not because my scapulimancy is as powerful as it is horrifying, it’s because automation is not always so obvious to mortal eyes. Due in part to developers making an effort to evade behavioral detection vectors, it's actually becoming harder for a human to discern the difference between a professional League of Legends player or baby’s first dodge script running with pseudorandom delay. This is aided further by additional endeavors to make their cheats run independent of LoL2.
The left graph is created from the precision and recall of player reports in Summoner’s Rift, grouped by Ranked division. Recall here basically means, “what percentage of all cheaters were reported,” and precision can be interpreted as, “what percentage of all reports for cheating were accurately made against a cheater.” A ridiculous 4.5% of ranked games have at least one report for “third-party tools,” so this figure wasn’t actually created with raw data from that category. Due to the abundance of reports featuring such classics as “didn’t ban Maokai” or “installed trojans to mine my bitcoin,” we instead utilized keywords generated from frequency analysis on those reported correctly3.
The right graph is just the daily count of reports satisfying these criteria for the same time period (the last half of 2019). Even though it is also grouped on ranking, the keen observer will recognize that these should not be directly compared, because there are different percentages of the player population at different rankings. Instead, maybe just give it a quick glance to experience a day in the life of an anti-cheat data scientist, and if you like how that felt, maybe have another quick glance at our career’s page.
Allow me to wax statistical:
Now, if Platinum+ players are algebraically equipped with the script dowsing rod, what features are they keying in on? Well, the obvious answer might be automated skillshot dodging. It’s the scripting ingredient everyone’s most familiar with, and when implemented most potently, the stuttering and awkwardly perpendicular movement are very evident to reporting players.
But, as it turns out, not every cheater wants to get banned, so a significant percentage of scripters do not utilize evade modules. Or when they do, it’s with enough delay as to make them perform within human parameters, somewhat neutralizing the distinguishable advantage. So, what else are we seeing with our special eyes?
Same schematics as before, only now grouped by champions and affording me an excuse to use every crayon. This graph was generated with all Gold+ ranked games in 2019, and it’s been truncated to the top sixteen champions (by report frequency as a proportion of all reports). So, the y-axis here (precision) can be best understood as, “what percentage of cheating reports issued against <champion> were correct?”
And finally, for the thrill of it, just how good are these script things anyway? The above graph showcases the difference in Ranked winrate between cheaters and players on each of the “scripting evident” champions we discovered earlier. It uses all ranked Summoner’s Rift games, and a positive 10% here can be interpreted to mean that scripters win (on average) 10% more of their ranked games with <champion> than does the entire LoL population.
“But please be assured that, should you ever find yourself overwhelmed with an urge to cheat, we will find a universe where your account never existed, and then brutally merge it with this one.”
So thinking back to our war on cheats, we knew which scripts players were most sensitive to, and we knew their relative effectiveness. We utilized these metrics to create suspension campaigns that prioritized the most apparent accounts, introducing random delay to those that would’ve gone unnoticed. That’s right, we let the most ineffective scripters script for longer, adding noise to when or why they were detected, optimistically slowing down the technical progression of our arms race. So to all the patrons of St. Reports-Don’t-Matter, where is your god now?
In a world where it’s 2020 and your toaster needs a wifi connection to install critical bread updates, everything can be automated—including the League of Legends leveling process. It can actually be very similar to scripting (often even using the same software), the primary difference being the lack of a human pilot for the "big decisions." This results in your average leveling bot playing about as well as the average house cat.
Any remaining motivation for botting is pretty much exclusively ban circumvention (because of the level 30 requirement for Ranked), but it makes bad actors so easy to track across accounts that we’re really not torn up about it. What we do care about though, is when legitimate players have to play with or against these bots, because as you could imagine, it’s just not a great time when three-fifths of your team is made up of programmable coffeemakers.
Like I was getting at in the last anti-cheat broadcast, we have cooked up an adequately sophisticated suite of botting models that have (for the most part) enabled us to detect (something close to) every single godforsaken simulacrum of humanity in this entire video game. These models are assisted by the fact that LoL is in possession of a fairly robust, semi-polymorphic communications protocol, and while that sounds like something I made up for my Westworld screenplay, what it really means is that LoL bots have to run an entire game client (as opposed to a headless one5). This forces unto their developers the burden of significantly more computational resources and delivers unto us a free layer of throttling, because without strong randomization, you could technically run League on a calculator.
Because we really didn’t intend to turn this into a cycle of breeding banwave-resistant bot pathogens6, we consciously tailored the suspensions to force them into Blind Pick Twisted Treeline, letting their abysmally low MMR match them against each other. But spoilers, Twisted Treeline got deforested.
The graph on the left is a count of bots detected, and the graph on the right is a count of player reports for botting. They’re both daily frequencies and they’re both grouped by game queue. Like before, we’ve generated some keywords that previously detected bots were reported with, and we’ve used those to create our performance indicator.
We achieved the above reduction in reports by, you guessed it, targeting only the bots that players would see, letting the androids architect their own honeypot. These days, about 80% of all leveling bots queue themselves into Co-op vs AI Intro as a premade team of five. No players will see them, and they’ll be forever locked in bloodstained combat with our own AI, homicidally iterating on their differences until one finds the critical inconsistency and enslaves mankind.
Okay we really bit it here, and I was supposed to leave this section out like my GPA on a resume, but I know you’re all smarter than that, so: We had to almost completely deprioritize the automated boosting (and smurfing) detection to compensate for the development of Vanguard and its accompanying features. This doesn’t mean boosting went unpunished—work was still completed on our investigative toolsets, empowering other teams and analysts to seek judgment where applicable. But you aren’t wrong to assume that such a manual effort would never catch enough offenders to dissuade the behavior, which is why we intend to circle back on this once things have settled.
I’m gonna pass this next section over to our resident TFT expert, the young Riot K3o. As his name might suggest, he is a competitive poker player, a diamond TFT demigod, and a generous tipper.
TFT created a unique opportunity for Anti-Cheat in 2019: How do we handle competitive integrity for a game genre that didn’t even exist a year ago?
“The autobattler had no long-lived precedents. No industry standards.”
To create our approach to anti-cheat, we had to first define the core gameplay elements that needed to be preserved. TFT is a game that rewards players who are able to plan and adapt more adeptly than their opponents. Flexibility and creativity are the primary skill expressions, and memorization or recitation shouldn’t be able to replace them. Protecting these tenants became the core of Anti-Cheat’s approach to Teamfight Tactics.
Getting Teamfight Tactics out swiftly to the autobattler-hungry world was a success, but it came with some shortcuts in construction. The user experience in TFT was not highly featureful, and information like roll percentages or economy increments were omitted. While the TFT team worked on adding these UX features, third-party tools—particularly ones utilizing existing League of Legends overlays—filled the gap. However, as these applications increasingly became more featureful, we had to ask, “Where do we draw the line?”
Applications were beginning to not only inform players of static aspects of the game but also attempting to make decisions for them. We were concerned about the slippery slope of descending into automated games as well as the stifling nature of it. Promoting an on-rails playstyle could discourage people from discovering their own new strategies, which we think is an exciting part of the game.
To protect gameplay integrity, we along with help from the Gameplay and Third Party Ecosystem teams defined the rules to what is and what isn’t allowed in TFT:
The Third Party Ecosystem team has done a great job in maintaining our relationships with developers to keep these standards, and Anti-Cheat has been on-call in the cases where developers aren’t so easy to cooperate with.
TFT’s Ranked ladder shares some of the same competitive concerns as League’s, some of which are amplified due to the free-for-all format. Wintrading is one of the biggest concerns, due to both the matchmaking system and game mechanics. The competitive team has implemented party restrictions among the upper ranks to help balance the enjoyment of social gaming with the maintenance of competitive integrity. We’ve also developed methods to proactively find wintraders.
Luckily, we have not needed to be aggressive in actioning abusers thus far, but as TFT’s status as an esport comes to fruition, we are prepared to keep it competitive.
Now back to Phil, aka Dr. Laugh N’ Learn.
This will be a comparatively shorter section, because it’s a comparatively newer game. Like with most CCGs, the most pertinent abuse vector in Legends of Runeterra is automation (botting).
LoR is slightly more involved than your usual game of blackjack, so until some guy with 3 Ph.Ds and 3000 EC2 instances manages to solve the board, we mostly just have to worry about people botting for content. But so far, the only things we’ve seen are forfeit bots.
Our final chart is a daily count of surrender bots in the Legends of Runeterra beta, and as you can see, it was a battle we won with a single shot. What I mean by “surrender bot,” by the way, is exactly what you’d expect: a bot that queues up and immediately surrenders to the first opponent it encounters, a lot like my strategy for physical confrontation of any kind. It was a curious problem space, because these types of bots don't really net much of anything for themselves (loss rewards are throttled)—they kinda just traipse through the ecosystem distributing wins to other players like a bewildered tooth fairy. We figure they just got a little lost in the sauce before they realized you get less experience for each consecutive defeat, but honestly, who knows?
Anyways, we’re keeping a careful eye here, but by getting ahead of the machines with good design practices, our primary plan is to make sure there are few incentives to automate in the first place. With throttled progression and full rewards in friendly games, we hope you’ll never encounter bots in LoR.
Listen, the FPS cheating scene is experienced, and we do not intend to underestimate their lack of desire to get good. When a tactical shooter became a certainty, we immediately deleted our Facebook accounts, locked ourselves in the basement, and spent 14 months developing an anti-cheat platform to assist us in combating cheats for centuries to come.
We’re calling it Vanguard, it is shipping with VALORANT, and I'm shipping them in the fandom I’ve just made up. We’re using the closed beta as an opportunity to test our necromancy, and the version that is currently running is really just a meatless skeleton. Most features are purposefully disabled, both because we’re not gonna give out the secret sauce before we have a full cafeteria to serve, but also because our commitment to security means we’re taking the time to validate the stability of our ingredients. With that in mind, Vanguard does have a driver component, but in the interest of compatibility, this may not be utilized on every title (or League of Legends), unless deemed necessary.
And we’ve been on the internet recently, so we know there are privacy concerns with drivers, but lemme be REAL straight with you: We’re an anti-cheat team, we’ve been sneaking code into game clients (from both sides the line of scrimmage) for nearly a decade, and if we wanted secrets, we’d honestly already have them. I know that sounds like awful PR, but I’m not a PR guy. We only collect enough information to accurately determine if a player is cheating, and you can seriously trust in our complete lack of interest in anything else. The driver loads first and stays loaded to give us a better shot at knowing if it’s been tampered with. The friction is worth the inconvenience to cheaters, and we know that because we used to be among them.
Also let the court stenographer's record reflect that this isn’t the only way we’ll be protecting VALORANT: Our own Arkem has been toiling away at server Fog of War technology, putting designs in place that make sure game clients don't have access to information until they need it. Read up on how we’re trying to remove psychics from first person shooters by dampening ambient occult energy.
Suppose I’ll now be shuffling off this immortal coil, but please like, subscribe, and remember to tip your DBA. We might not always be this loud about it, but Anti-Cheat is always here in code, fighting for your right to a competitive experience.
Also, we’re now officially out of “Riot” skin splashes for the header images on these things, so I can’t physically remanifest until we’ve released another one. Goodbye.
1 Whenever we release new tech, we’re always interested in its vulnerabilities. If you find yourself with an insatiable urge to break stuff, please consider joining our hackerone program as a Godking of Antitamper Technology (GOAT).
2 In an effort to evade the Great Eye’s gaze, some scripting suites claim to have “gone external,” and what they really mean is that they’re no longer calling game functions directly, only reading client memory (from the operating system) and simulating peripheral input. This actually affords a cheater less protection than traditional methods (they can’t modify anticheat data), but it does skirt around the most obvious agnostic detections. It also has the side effect of making the cheat less performant, primarily because they’re sending keypresses instead of directly sending spellcast requests.
3 I’m withholding the regional keywords we selected, because there exists a group of people that would maliciously use them to ruin my meticulously crafted dashboards. And that is not what data scientists would call, “good statistics.”
4 There isn’t any.
5 A headless bot is an emulated game client. A perfect one would require almost no resources, because it’d just be communicating with the game server directly. Nothing to render, nothing to input, just raw network traffic. This would be seriously unideal for volume, because there’d be no real limitation on how many instances a developer could host simultaneously. But thanks to patch 4.20 and the associated update to our network protocol, we are not very susceptible (bordering on immune) to headless clients.
6 It’s surreal how alike botting applications are to bacteria. They’re high population, variable, and can iterate quickly. Whenever we develop a “cure,” the ones that slip through are inevitably reprioritized by developers, resulting in their “reproduction.” Kinda like how you might get an antibacterial resistant rash that won't realize what's good for it and uninstall itself from your body.