Manip in FML: The Numbers

[FML|red7z7]

Foreward:
Hey all, in this analysis article I will look deeply into the issue of manipulation, which has always been a great discussion throughout the years. While this analysis is not going to be perfect, I think it illuminates the general trends in a way we have never seen before.

Many players, especially the veterans, have claimed that games have become longer, more boring, and full of manipulation compared to the past. For some, it has been the reason for dropping out of FML. Are these claims warranted, or is it far from the truth?

First of all, special thanks to :Germany:FML|Pinball and :Canada:FML|Renaud for uploading so many replays. Without their help I would never have been able to gather these statistics. Also special thanks to Juliusz 'Julas' Gonera for his Warcraft III replay parser (which I believe is also used for ffareplays.com and replayers.net). The site for the parser can be found at http://w3rep.sourceforge.net/.

I scraped 362 FML replays for data. With a whopping total of 34,121 hours of FML gameplay. This includes all season finals, except S9, S11 and S19. Fragments of S7 and S8. And all of S10, and S12-19. Yes, the data could be much more significant if I had complete replay archives for S1-6, but I will make do with what I have and in the future, perhaps can incorporate that data into further analysis. That said, if anyone reading this knows they have archives of the earlier seasons, let this be a call to urge you to post those up, preferably in Pinball's thread http://ffamasters.net/index.php?topic=1816.msg33294;topicseen#new

The Data:

Game lengths

The first set of data we look at is a histogram of the game lengths from all the FML matches. The x axes is bins for game-lengths, and the y-axis is the number of replays that fall into that bin. It appears that most replays are in the 75-minute range, with a tapering long-tail of outliers of very long games.


The second set of data we look at is the trend in game-lengths over the years. Here is the raw data with the x-axis being date and y-axes being game length in minutes. A trend line shown in red. Wow! That's actually a big surprise that the games have tended to get shorter on average over the last 4 years at least. It may not be a huge difference, but it certainly isn't a rise in game length as many players have suggested. I suspect that at least part of this trend is the slow migration away from certain maps that lend themselves to very long games, such as the banning of Gold Rush.


And here is the same data with a bit of smoothing applied so it is easier to see what's going on. Again, it is clear that there is a decrease, about 10-20 minutes in game length on average.

Words per minute

Now, game length is interesting and all, but it does not tell us about the quality of games. Maybe the games are shorter on average, but there is way more chat and manipulation in each game? Let's look at the data. Here is a plot of the trend in words per minute (y-axis) over the years. Well, indeed, there is a slight upward trend, an increase of perhaps as high as 50% words per minute. Considering this is compounded with the trend of shorter games, it is interesting.


Same data but smoothed out for clarity. It does appear to not be terribly significant, especially throughout the last 4 years. There does seem to be a difference between S7/8 and the S10+ a bit.

Sentiment analysis

Well, so the games have tended to be a bit shorter, and there tends to be a slightly higher rate of chat. But what if the chat is just banter, or reasonable in-game discussion. How would we know if it is manipulation/flaming/raging? Well, I came up with a list of the top ~400 most frequent words and then ranked them each on a sentiment value. That way, key words like 'manip'/'manipping' and curse words and insults have a positive score (hot), while key words like 'thanks', and emoticons had a negative value (cool). I then scanned each replay for the frequencies of those words to generate a 'sentiment' score, where high scores indicate more manipulationg and flaming, while lower scores indicate positive chat. Here are the results, with the sentiment score on the y-axis. To me, it looks as if there is very little change at all over the years. There are a few outlier games with extremely high scores, which might be fun to watch as we can expect a lot of rage/manip in those games  . There are actually a few games with overall negative scores, which surprised me a lot. I inspected those replays and discovered that most of them had :Canada:FML|Renaud in them, which makes sense because we generally know Renaud to be a mild-mannered (for the most part) player who uses plenty of emoticons to diffuse the stress in the game!


Same data, smoothed. When it is smoothed, it is actually pretty clear there is a downward trend in manipulation/flaming. Wow! Who would have thought that 


Conclusion:
So, in conclusion, what we have found is 3 things:

• The games have gotten a bit shorter, at least in the last 4 years
• The games have slightly higher words-per-minute, indicating more in-game chat
• There is actually less manipulation and negative-chat in the games than there was in the past

These, at least to me, are very interesting results. I think it highlights the true trends in manipulation/game lengths and is contrary to common belief. I would like to hear from the community what you think about this, or if you see any flaws in the analysis.

[FML|red7z7]

I'm sad the images are cut off on the right side because of these narrow forums. I recommend either opening the images in new windows.

[FML|Mog]

Very cool. When mage and I were casting older replays it seed to back up this analysis, and made me think peoples' reminiscing of "the better old times" was just thinking of some positive moments after (possibly) a very frustrating game. Sometimes those few memories aren't a fair representation of the scene as a whole.

I personally the game quality has significantly increased over time, and while there will always be that super frustrating game with a lot of manip, I don't think that is necessarily true of just our time, but has really always been the case in FFA.

[FML|Renaud]

You can also hold the scrolling ball on your mouse and move to the right

Love it, never though someone would go through all of that!  Make me want to make a complete replay database even more!

I'm guessing that early huge spike is the game on harvest with me/louis/lost/nooblex? :D

@mog: it's not that there is less manip/teaming now, it's that there is more crying!

btw did you only check the players chat?

Great work as always!

[FML|Mog]

That's true Renaud. But whining is manip in a sense, it is just the really obnoxious kind that shouldn't ever work.

[FML|red7z7]

Thanks guys!

You can also hold the scrolling ball on your mouse and move to the right

Love it, never though someone would go through all of that!  Make me want to make a complete replay database even more!

I'm guessing that early huge spike is the game on harvest with me/louis/lost/nooblex? :D

@mog: it's not that there is less manip/teaming now, it's that there is more crying!

btw did you only check the players chat?

Oh wow didnt know about the middle mouse button thing. That's really helpful. I believe the huge spike was your game, yes :P

And yes, I forgot to mention it in the write up, but the parser labels each chat with the mode (to all, to obs, etc) and I filtered out for only player chat.

Indeed, there may be more crying now, although when I gave scores to the frequent words, I just focused on negative or positive terms in general, so it should include crying as negative. Of course it is hard because ideally we would look at phrases rather than individual words, but that is tricky.

[FML|Renaud]

I just realized: we used to allow private chat! That wasn't recorded in the replays!
(Although only a very small number of player used it, so it was probably not that significant)

[FML|WorpeX]

This write up is ridiculous. Awesome job! :) Cool to see these stats!

My only suggestion is to label the axis on the graphs better as the graphs mean nothing to me otherwise. haha

[Meeds]

very good job ! Thanks !

[Vyvanne]

While I don't care for Smash Brothers and vastly prefer Street Fighter or King of Fighters, this guy fucking rules. *sniffs and wipes tears of approval*

http://kotaku.com/mysterious-masked-player-wins-smash-bros-tournament-1659752915?utm_campaign=Socialflow_Kotaku_Facebook&utm_source=Kotaku_Facebook&utm_medium=Socialflow


As one who doesn't care for manip, this man rules. :D

[FML|Mage]

Just went back and re-read all this, and boy it's awesome work Red.  Before I started casting older games I would have said the main difference between now and then was that now players tend to do over the top manipulation.  For instance they're easily winning a game but continue to relentlessly manipulate player 3 to attack player 2 handing player 1 a very easy win that's 45 minutes longer than it should be.

After watching the older games though, it was much worse.  All the players saying it's worse now are wearing rose tinted goggles or my sample size has somehow all been wrong (10 games or so, so it is possible).  Also chat being up does make sense and I think it's great how you did a negative and positive filter.  I know that personally I talk a bunch in FFA just because I'm bored and I like to catch up with players.  Not a bad thing, but it's certainly not manip.

[FML|Renaud]

I think it might depended on the amount of american players (or Lone players?) vs the number of Northrend players.

I know my manip has lowered, but then again most of my manip revolved around the ladder being anonymous, and many players HATING private chatter

My favourite:

Renaud: Sorry I don't private chat
Player 2: what?
Renaud: Oh Player 3 asked to team
Player 3: WAIT WHAT????
Player 2 & 4: LETS KILL HIM

Good times :)