If it looks too good to be true..
Judging from the general thrust of comments on one of the blogs linked in the article, a job as a rater doesn't seem to be much different than the "work from home" horror stories of yesteryear. The ones where poor/desperate people get sucked in to investing time and money in 'home assembling ballpoint pens' or 'stuffing envelopes for marketing companies", only to find they either make no money for a lot of effort or in fact lose money due to up front investment in time that could be spent more productively, purchasing materials or paying for compulsory 'training'.
Prospective raters appear to be required to take an initial simple test based on the instruction manual (probably to weed out those that can't actually read). After this they are then required to take a 140+ question 'test' evaluating actual sites. What is interesting is that a lot of the posters appear to have to "wait" until some test data is available (why aren't they using a bank of standardised tests?), and when they inevitably fail they are either immediately hit with a request to re-take the test with new data (some several times) or after a long period of begging they are suddenly 'allowed' to re-take it (also presumably reflecting that there may be a load of actual work at that time or they have to wait until more becomes available). Of course they don't get paid for any of this testing!
A suspicious person might conclude that most of the 'testing' is unpaid processing of new test data supplied by actual customers in a bid to keep overheads down and profits up. After all they already have a basic ability to weight the candidate's findings based on the short initial test, and since they don't have to pay these chumps they are probably amalgamating the results across a whole bunch of them plus those of them a small number of paid testers whom they have already found to be reliable, classic crowdsourcing with a twist.
There are a few really positive comments that reek of astroturfing, and a few more genuine-looking ones from people who are getting paid, most claiming they book 8 or 9 hours of work a week, but that they have to invest significantly more than that in the research side of the evaluation which they cannot book. One guy claims the going rate is 9 Euros an hour (i.e. about a quid more than UK minimum wage), plus one suspects the 'contractor' is responsible for all tax, NI or other equivalents plus 30-60 days payment terms. Factor in the non-paid research overheads and you'd probably be better off getting a paper round.