Recently on CreateDebate, we had a situation where a user, instead of attacking users’ arguments with logic and good rebuttals, decided it would be easier to try to karma bomb people they disagreed with.  They could have responded to specific points the users made, but instead went into the user’s post history and voted down almost 800 arguments of two users.

One of the statistics we use to judge the strength of a debater is the efficiency percent.  We calculate this by dividing your number of positive votes by the total number of votes.  Essentially, it shows if people think your arguments are solid, use good supporting evidence, and are well reasoned.  A recent look at some negatively scored arguments shows personal attacks, opinion without any reasoning behind it, and a few racist remarks.  The vote attack made the victim’s efficiency drop to the point that they looked like they were debating with the logic of a raving fanatic.

Any site that allows open debate or open posting where anyone can easily sign up and start putting their opinion online will inevitably get a few trolls.  Some sites deal with the problem by using moderators who vet every posting before they become visible on a site.  Some sites use moderators who watch and remove troll posts.  A more democratic version of the moderating system is to let everyone become a moderator, which is the system we’ve gone with.  Aside from showing which arguments are the best in a debate, the argument voting system allows people to vote down arguments by the trolls until they’re no longer visible, essentially making everyone a moderator.

The karma bomb we recently saw is a reflection of people using their power as a moderator to personally attack another user.  So in addition to rolling back the massive amount of downvotes, we put in place a system that watches for karma bombs like this one.  If a large number of votes are going to attack or artificially inflate people’s points, the vote system enforces a cool-down period.  The cool-down period should guide people instead to rebut arguments with well-developed arguments.

From watching the growth and evolution of many other websites who try to tackle the problem of trolls, it’s a constant battle between the “griefers” and the community.  Wikipedia, digg, reddit, mixx, facebook, myspace, and plenty of other social sites all have their own system for dealing with the problem, but it the more well developed sites all rely on the community to watch itself for trolls. We’re toying with some ideas from these sites, like how digg lets you see who voted on your arguments, or how twitter allows you to block users from showing up in your version of the site.  At the same time we need protection from trolls, we also don’t want to lose the core democratic debate process, where everyone has a say.

We’re constantly trying to make the debate experience better at CreateDebate, so let us know if the karma bomb protection is as annoying as airport security, and if you think it needs to be tweaked, or if it’s unobtrusive enough that the benefits we all get make it worth the effort.  We’re still thinking through changes to our overall points system, but this system should work in the meantime.  And as always, let us know what you think, either through site messages, or by using the feedback page.