Notifications are a core part of the experience on Jest.
Every game hosted on Jest can schedule reminders, updates, and re-engagement messages for every player. These messages are constantly being created, updated, canceled, and re-scheduled based on real-time player behavior.
Six months into the launch of Jest, we are already moderating more than 1 million notifications every month. This rapid growth created an unusual challenge for our team - one that simple automation couldn’t solve. To handle it, we built The Referee, an autonomous AI agent, directly into our notification pipeline.
Why notification moderation looks different on Jest
A lot of messaging systems are built around static marketing campaigns. Ours is built around live game activity.
A single game might schedule a reminder when a player joins, cancel it when they come back, then schedule a different one when the player misses a turn. Multiply that across thousands of players and many different games, and you get a massive stream of data that never sleeps.
But volume is only half the challenge. Because we deliver these messages over RCS and SMS, it’s not enough for them to be timely and relevant - they have to be compliant. These channels require a much stricter standard for safety. Specifically, we must ensure content stays clear of restricted categories like SHAFT (Sex, Hate, Alcohol, Firearms, and Tobacco).
In this environment, simple keyword filtering is too blunt; it misses the nuance of gaming slang. Conversely, manual review is a total bottleneck that can’t keep up with the millisecond-pace of game logic. We needed a system that could look at messages as they were being scheduled, make a split-second decision, and stop unsafe content before it ever reached a player’s phone.
How The Referee works
Every notification scheduled on Jest goes through a check by The Referee before it becomes eligible for delivery.
When a game triggers a notification, we store it, run it through the moderation check, and only let it move forward if it passes. If the content is flagged, the notification is “red-carded” and blocked instantly before it can be sent.
The AI does not see this as a generic piece of text. It understands the context: a game notification headed for a sensitive personal channel. By reviewing the full experience - including the main message, title, and call to action - The Referee can operate continuously at scale, processing over 1,000,000 checks a month without slowing the product down.
Continuous learning loop
We tuned The Referee to be conservative by default.
When sending messages over sensitive channels like RCS and SMS, the cost of letting a violation slip through is significantly higher than the cost of blocking a message that turns out to be okay. Because of this, our system is designed to err on the side of caution.
That naturally means we encounter false positives. Sometimes a harmless notification gets flagged because the wording is playful, ambiguous, or unusual outside the specific context of a game. In those cases, the “Ref” was simply too strict on a tackle.
We don’t view these instances as failures; we treat them as feedback.
When a valid notification is blocked, we review the trigger and adjust the moderation layer to better recognize that pattern in the future. Whether it’s refining the prompt, clarifying harmless game slang, or adding new examples to our test set, the system is constantly learning from real traffic.
Our goal is to keep the safety bar high while steadily reducing friction for legitimate notifications. After all, a moderation system that blocks everything is technically safe, but practically useless. The real work is in making The Referee smarter with every “call.”
What this gives us
AI moderation lets us do what was previously impossible: support a high-volume, dynamic notification system without introducing bottlenecks while navigating the strict regulatory frameworks of RCS and SMS.
This gives us more than just scale; it gives us a superior developer experience.
We can empower developers to generate rich, timely notifications while maintaining a proactive safety layer between game logic and delivery. Because we can monitor outcomes, inspect blocked messages, and refine the system as new edge cases appear, we don’t have to choose between speed and safety. We can do both, even at a pace of 1 million notifications per month.