Tips for Building Profanity Filters with AI for Educational Games: Lessons from the Field
Updated: Sep 23
Our Senior developer Erik has put together a few practical tips after building a profanity filter for an educational multiplayer game, worth sharing for anyone doing similar work with AI coding assistants.
Author: Erik Hogan
Sometimes we work with clients that required a profanity filter for the educational multiplayer games we develop with them.
Here is the process we followed for a recent project requiring a profanity filter:
AI models from Claude were used to assist with development, and since the filter was powered by a very long list of profane words, we made sure to configure Claude to hide that file from access. We didn't want Claude to see that file and assume we were creating harmful content.
(Note: Advice online to use '.claudeignore' to lock access to certain files should be ignored, despite blog posts to the contrary .claudeignore is not a real feature of Claude Code and may have been the result of an AI hallucination. Use .claude/settings.json to enforce this.)
We add some tightening on the filter to make it harder to bypass it. While it's impossible to enforce all minor character replacements that could be used to express profanity without perfectly matching the dictionary spelling of the word, we nevertheless asked Claude to audit the code running the filter (not the blacklist) to see if the code could be improved.
The results of the audit were eye opening: it pointed out that various words inside the educational text of the game itself like "Weed" were in fact in the blacklist, despite Claude being banned from opening the file.
Claude was able to do this by running the function code itself, probing it repeatedly with candidate words until it could infer what was on the blacklist. As an example, since it tested the word "Weed" and received "****" back from the function, it knew that the blacklist contained that word.
Some takeaways
Simply blocking Claude access to certain files is not adequate protection in of itself. To be truly safe, in this scenario a dummy blacklist of safe words should have been provided, with the real blacklist held outside the project folder.
However, using a dummy blacklist would not have revealed that words legitimately used by the game were banned by the filter. As such it's difficult to give a hard recommendation here. As long as you're aware of how an AI model may infer content that you intended to hide, use your best judgement to tackle situations like this. Ask your model directly to see how it may respond upon incidentally reading profanity, and make sure that settings files are tightened appropriately.
There's a genuine difference in context between asking Claude to generate harmful content, and profane words being seen incidentally by Claude when scanning a file. While there's not zero risk, Claude did see that the blacklist was a legitimate container for profanity and did not give me any pushback about the content.
Comments