Originally Posted by
DanK
Thanks. This is very helpful.
I'm curious what they are actually doing on their servers. I'd be willing to wager that it is just heavier-duty processing applying already-determined algorithms, not actual machine learning, and that the machine learning was done earlier, on training sets.
As I understand it--and I am certainly no expert--machine learning is fundamentally different from the rater training that Manfred mentioned. In the case of rater training, at least the rater training I have done, the trainer isn't trying to learn what underlies a rating. They are just trying to get raters to come close to matching a criterion. In fact, a standard part of rater training is telling the raters what they should attend to and how they should evaluate it. In the case of machine learning, this is reversed: the humans are using machines to find patterns that underlie positive ratings so that these patterns can be applied to new data. It's an inductive process based on the training sets.
Similar approaches have been used in statistics for many decades. For example, stepwise and other best subsets regressions use empirical fit to determine which predictors are useful and how they should be weighted. Like machine learning, this is subject to chance, so the results are cross-validated, as in Manfred's post. In one of the early efforts to get computers to rate essays, statistical analysis was used to find linguistic patterns that happened to predict human raters' scores. Those factors had nothing to do with actual meaning, but once the model was created, after years of work, the resulting ratings correlated with human ratings roughly as well as the ratings of two humans.