Is rating volatility a bug or a feature? [Forked]

@paisley:
I’ve finally gotten around to read you first reply to this thread :laughing:

Given how long it appeared to be, a surprisingly painless experience – even for a quasi-layman like me, it was pretty understandable.

Well, I don’t know what log-likelihood is, but I’ll take you at your word that it might be a good measure for evaluating a rating system.

Now, I think the best way to perform the test is to implement all of the decent proposals for methodologies and get many different kinds of evaluation. If they are good measures and there’s a clear-cut answer to be found, they should mostly agree, and if they don’t agree, that will tell us something weird is going on. And anyway, the decision will be left to the readers/peer-reviewers as to if any of the systems are good.

So what I mean is: for what it’s worth, if it comes down to me I’ll do my best to understand the method you proposed thoroughly and implement it. (Hopefully people more proficient than me can collaborate though :laughing:)

I believe the point you bring up about the volatility (that it should be impossible to update with 1-game rating periods) was vaguely referenced in this reply and then again in this reply, not much, but it might help you. It would also have been impossible to get specific without seeing the code, which you seem to be working on (assuming that goratings repository is the code used right now).

Before reading, I was actually planning on giving you some advice on how to format and layout a long message to make it more readable, but to be honest the vibe I’m getting in the last few days is that I have alienated most of this forum’s community with my long replies, so I guess my advice is: don’t write long replies? :laughing: