iDevGames Forums
Profanity filtering? - Printable Version

+- iDevGames Forums (http://www.idevgames.com/forums)
+-- Forum: Development Zone (/forum-3.html)
+--- Forum: Game Programming Fundamentals (/forum-7.html)
+--- Thread: Profanity filtering? (/thread-1461.html)



Profanity filtering? - Holmes - Apr 13, 2009 01:19 PM

Hello,

I'm curious if anyone has any experience with profanity filtering in a C/Objective-C context.

I have an Objective-C server application that is running 24 hours a day. People can log with whatever name they'd like, and sometimes they choose to use names that are monstrously offensive. Normally I wouldn't care, but I want my game to be suitable for anybody to play. I don't want to have to replace its 4+ rating with an MA.

I'd really like to be able filter names so that profane portions of usernames are filtered out to *** symbols.

Any ideas?

thanks

- Holmes


Profanity filtering? - ThemsAllTook - Apr 13, 2009 01:35 PM

It's impossible for a computer to do this well. Having a botched profanity filter implementation can seriously hurt the usability of your product, doing much more damage than some users using profanity is likely to. Don't make The Clbuttic Mistake.

A human filter would be the most reliable detection mechanism, but is almost certainly impractical. You could do something like make new user names invisible to everyone else and put them in a queue to be reviewed manually, but that would mean you're tying yourself indefinitely to the task of keeping up with reviewing them.

A lot of console games put of a warning on startup that says something like "game experience can change during online play and may include content not rated by the ESRB". Could you potentially do something similar and still be OK with a 4+ rating? You could pair this with a "no profanity" policy, a complaint mechanism, and occasional review and banning of users violating the policy, and maybe have something workable. Trying to do it purely algorithmically is asking for trouble from both ends (doesn't stop violators, makes non-violators' experience worse).


Profanity filtering? - Holmes - Apr 13, 2009 02:05 PM

ThemsAllTook Wrote:It's impossible for a computer to do this well. Having a botched profanity filter implementation can seriously hurt the usability of your product, doing much more damage than some users using profanity is likely to. Don't make The Clbuttic Mistake.

A human filter would be the most reliable detection mechanism, but is almost certainly impractical. You could do something like make new user names invisible to everyone else and put them in a queue to be reviewed manually, but that would mean you're tying yourself indefinitely to the task of keeping up with reviewing them.

A lot of console games put of a warning on startup that says something like "game experience can change during online play and may include content not rated by the ESRB". Could you potentially do something similar and still be OK with a 4+ rating? You could pair this with a "no profanity" policy, a complaint mechanism, and occasional review and banning of users violating the policy, and maybe have something workable. Trying to do it purely algorithmically is asking for trouble from both ends (doesn't stop violators, makes non-violators' experience worse).

Yeah, you're probably right. The advantage of filtering, however, is that I can do it on the server side. I should just get to work on adding client-side abilities to display warnings sent from the server -- then I can warn users of offensive names and possibly ban repeat offenders.

What I'm deathly afraid of (this is an iPhone game) is Apple logging in to test an update and seeing a user logged in as "S*** my D***" (stars added by me, and yes, people have logged in with this name).


Profanity filtering? - TythosEternal - Apr 13, 2009 03:17 PM

ThemsAllTook Wrote:A lot of console games put of a warning on startup that says something like "game experience can change during online play and may include content not rated by the ESRB".

I would advise considering this as a safety net, and not as your primary means of defense. Here are my two cents:
  1. I would recommend looking into a handful of reliable regular expressions first to capture common variations on the more prolific four-letter words. If you'd like, we could start a collection of good profanity RegEx's here. I think it would be worthwhile.
  2. Second, it might be worth the trouble to implement a minor reporting system in your application that allows other users to inform you of inappropriate names - this is, in my opinion, by far the most reliable method (other people will almost always be logged on and viewing user names when you're not).
  3. Third, scan newly-registered names on a regular basis (it would be impractical to view everything, of course, but just a few moments spent looking over a generated list of names might still be worth it).
  4. Lastly, you can try using the ESRB quote, but again I'd recommend considering this to be a safety net.



Profanity filtering? - Holmes - Apr 17, 2009 01:07 PM

In case anybody needed a reminder of how disgusting human kind can be:

Somebody logged into my game today as "LYNCH N*****". I think you can imagine what he had there instead of *'s. This was while 12 other people were playing. I had to shut down the server until I could program in a method to ban him permanently.


Profanity filtering? - ThemsAllTook - Apr 17, 2009 03:41 PM

Holmes Wrote:In case anybody needed a reminder of how disgusting human kind can be:

Never been to 4chan, I take it?

The internet can sometimes really bring out the worst in people. Since they have a shroud of anonymity to hide behind, they can behave as badly as they want without (in their mind) any consequences. This is really just the reality of things; all we can do is try to find good ways to deal with it.


Profanity filtering? - Holmes - Apr 17, 2009 03:49 PM

ThemsAllTook Wrote:Never been to 4chan, I take it?

The internet can sometimes really bring out the worst in people. Since they have a shroud of anonymity to hide behind, they can behave as badly as they want without (in their mind) any consequences. This is really just the reality of things; all we can do is try to find good ways to deal with it.

Yeah, it's just tough having to police this online system. It requires so much attention it makes me feel like I'm a parent Blink