Our "education" system is taking another step towards total automation with the introduction of AI robots to mark examination papers. Pearson will unleash the bots next month to mark English papers, reports the TES. Apparently the system "'understands' the meaning of text much the same as a human," Pearson claims in its …
People being taught to be robotic by people as bland and characterless as robots ought to take exams marked by robots, before spending the next 50 years behaving like robots. As recommended by Indian outsourcing companies? Wow, there's an endorsement to salivate over.
Oh Bluddy Hell!
As a recently turned 14 year old, in Year 10, its a scary thought doing GCSE's and BTECs early, heck my maths class might be doing Advanced Maths or AS Level Maths, in YEAR 11?!
I certainly want the authenticity of a human exam marker, atleast then he definately understands. Its like sending a mouse to do a man's job, the mouse might be trained but he might not have the experience. Not to mention economy jobs, we're unstable enough as it is, we need jobs not frigging robots.
I know this is only a Pearson jobby, at the moment... I'm the only one doing my BTEC at my school, its a pilot programme, and I think I have to do my ICT Practitioners test at a Pearson Test Centre, quite frankly I want to make sure my exam is marked thoroughly and properly, like any one would. AI is great, but to be honest it needs small time tests first.
rant over and the icon is for the f*ckwit that thought : "oh, I know, lets put this in several test centres for important tests!".
God *roll eyes*
If the exam itself is not intelligently designed, it doesn't matter how good or objective the grader is, the results are still worthless. Designing a test that actually measures what has been learned is not a trivial task, and a lot of so-called teachers are not competent at it.
"The system is not “distracted” by language-irrelevant factors such as test takers’ appearance, personality or body language."
it would be quite hard to be distracted by any of those things while marking an exam paper, since the candidate is not, in fact, there.
Mind you, I don't think a robot could do a worse job than the incompetents doing this job at the moment. in the last couple of AS/A2 exams my duaghter had one paper remarked and it went from a D to an A, and one of her friends had a paper graded U (i.e. not worth marking) that went to an A on remark. If the humans are that useless give the machines a try!!!
"The system is not “distracted” by language-irrelevant factors such as test takers’ appearance, personality or body language."
If the system can infer the author's appearance, personality or body language from an ESSAY, then truly we must all welcome our new and omniscient overlords. (On the other hand, the whole thing could be a pile of crap dreamt up by someone too stupid to even realise what a fucking twat they are.)
the system "'understands' the meaning of text much the same as a human,"
Given that many young humans these days can't even *write* coherent text, I'd be very impressed if this actually worked!
I was speaking with a Greek acquaintance recently, and he mentioned that the Greek education system encouraged memorization (I personally believe memory is a Good Thing), and that while creative responses might be marked highly, the most reliably acceptable responses were those directly from the text.
Clearly, the Greeks will benefit the most from AI Robots. The more brilliant students, not so much. Perhaps AI robots might be applied most appropriately to the "fuzzy-studies" - i.e. the humanities. I wouldn't want my surgeon to be marked highly by robots!
My surgeon's failure is my own!
AI Pondering ITs Muse.
"People being taught to be robotic by people as bland and characterless as robots" ..... By Anonymous Coward Posted Friday 25th September 2009 16:14 GMT
Considering a robot can be programmed to do practically anything, and we even have them working/have had them working/beagling away on Mars, to imagine them as bland and characterless, would tell them more about an Anonymous Coward than an Anonymous Coward could tell about them, meThinks.
I don't think so. Presumably a machine can only mark 'tick the box' questionaires, which aren't really exams at all.
So, basically, this consists in running a spellchecker and that obnoxious Word grammar and style checker on the essay, and counting the number of red lines?
If English is to be written in such a formulaic way that it can be appreciated and judged by robots, it might have been written by them in the first place.
Of course this is the logical step down the road that we have been going for years. For anybody who wants a dispiriting experience, then I'd suggested finding some of the examination marking guidelines that are around. They are often little more than an instructions for the examiner to look for one of a number of set phrases or words with precious little room for interpretation.
I came across one example paper (in biology), freely available on the net, where examinees were asked to state several reasons why some people object to genetic engineering (note, that's a social science question - the paper explicitly did not ask for rational, science-based objections). However, if you put "because is it unnatural" or "for religious reasons" then the guidance explicitly gave zero points. Well these might not be valid scientific objections, but they are most certainly those that many people use (which is what the question asked).
So increasingly what is being taught is how to pass the exam and not convey understanding. Box-ticking, formulaic and highly specific stock phrases are the norm in many exams.
Чего стоишь? Работать надо!
Robot. Karel Chapek. Chech, Europe. "RUR", 1920. Slavic for "to work" - Rus. "работать" [ra-'bo-tat'] -- to execute a job. " t' " @ the1st syl. is a sound when you pronounce "t" the way you'd do it saying "teh", a "mild t".
The title is probably translated as something like "Why stuck? Get on working!" is the last phrase from http://video.tvguide.com/Window+To+Paris/Window+To+Paris/812913?autoplay=true&partnerid=OVG , the unprecedented Alien XCellence in every tick of the alarm watch. Originally presented @ http://livefilm.info/otc/360-okno-v-parizh.html , no anti-pirate rules for the visitors. Some 512k+ connection speed only, sorry London (-:
Definitely Paris, because she's the City where the Walls lead to.
I found a bug!
For a 100% score, just type in ,,,,,,,,,,##########,().()%%%%%%%%%% and you'll score 100% every time.
...we fear change.
The change of place. The change of time. The foreboding uncertainty of future. The inevitable intellectual castration of children and subsequent pacification or rebellious aggravation of society.
It is only natural - we are stupid, easily frightened, beasts. We need to evolve and grow through those visceral, instinctive, reactions and learn to embrace the future. The cold, sterile, future. Help our children be the best carbohydrate to work converters they can be!
Calling Dr Penrose
Seems that AI, as usual, continues to shift ground when it hits the real world.
An expert system, in its small world, can work, sometimes.
It is hard enough to get a biological intelligence to work sometimes, especially in the "follow procedures or else" rule bound bureacracies that the corporate state, both public and
private, has reduced us too.
To the overlords of the corporate state and the uncritical, AI sounds like a simple thing.
They will try to use it, whether AI works or not. The insidious effects of the reductionist fallacy that has poisoned the intellect of the West is reducing our leadership to foolishness.
As Dr Roger Penrose pointed out in "The Emperors New Mind" we still don't know what real thinking is, and what ever it is, it is not possible on a Turing machine, if I understand him correctly.
Avoidably Detained trumps Unavailable with Surprise ZerodDay Appearances. ....
... CRazed Utterances in Sane Discourses
"Naturally we requested a comment from our own AI expert aManFromMars, but he was Unavailable 42 comment."
That request must be lost and/or rerouted to who knows where, Andrew, which would then be AIMysterious Desperate Case of IT Being Stolen by the System, for such commentary is always freely and readily available.
And that sort of Petty Pathetic Ineffective Dodgy Act as Fact would be a Clear and Present Admission of a System AIdDefeat to Strange and Particular Peculiar Forces ..... exposing as it would/does, an Impotent Vindictive System, [Early Sub-Prime Human] Terminally Psychologically Flawed and Hopelessly Insecured against Simply CompleXXXX Vulnerability BetaTesting Attacks for Real Power and Virtual Control of Operating Systems Windows with RobotIQs into Networks InterNetworking Javan Applications ......for Steganographically Shared Transparent Protocols ....which may or may not be SSTP ...... http://en.wikipedia.org/wiki/Secure_Socket_Tunneling_Protocol :-)
How fantastic !
If this AI program is half as good as the one that picked my modules for the BSc in Information Systems a couple of years ago, Jake might as well change to major in Cookery.
These AI things can only ever be as good as the twat that counts the beans will allow them to be.
What sort of jobs are these "vetted" individuals gonna get ?
I really believe it will be the sort of job that no self respecting human would choose to do.
Err, given the choice, by our new evaluating overlords.
This is how it starts
First they weed out those that are smart enough to be a threat, then they put idiots in charge... Hang on, that happened before the robots took over (damn!)
Stones from Glasshouses always deserve Comment
What an XXXXCellent Plan, Andrew, ....to rattle a few Cages/Ivory Towers. Is the Register Game for taking AI Lead and making Permanent Changes? For why would they leave IT to Lesser Mortals who wouldn't evenTry for the Want of a Clue of what to Do ? :-)
I Kid U Not ........ http://www.phrack.com/issues.html?issue=66&id=17#cmt1912
Isaac Asimov might (or might not) be pleased ...
Have any of the designers ever read his short stories on robots?
The most common theme is the reaction of curmudgeons who don't accept the changes being wrought by tin substitutes.
This reply was crafted entirely from recycled electrons - so there!
The one with the Alpha Sprayer in the pocket, thanks.
Laughing about a joke
To me, as long as AI cannot laugh about a joke on its own account - meaning without having built-in algorithms to understand a nut-nut joke - there is no AI. At least, no AI that can even remotely be considered to mark exam tests.
We'll see more main-stream texts, because this method allows for reverse-engineering of the marking algorithm, which in turn will result in even more 'teaching to the test' - sorry:'teaching to the global algorithm'.
Therefore, El Reg is correct: marking will be 'programmed in' by the personal preferences of the developers. And not a single person can convince me, that the machine will be able to gather an abstract comprehension and understanding of its own from the text fed into it.
Paris, because I think even she is better at marking an English essay than that box.
Is this some sort of Turing Test lite maybe - if you can't tell whether an exam paper has been marked by a human or a machine, then the future has arrived. What next? Job interviews by machine? Replacing judge and jury with a computer?
But are they really serious? Multiple choice marked by computer I can understand, but not essays surely? Or don't they do essays any more? Yes, that must be it. The machine just needs to check if there's a name at the top of the paper and that's a pass, right?
Sounds good to me
I have no faith in human exam markers. Heck I don't have any faith in the whole exam system.
Plenty of idiots go on to get As while plenty of bright people end up with nothing.
All these exams do is test your memory, i.e. how well you can remember this fact or that chunk of text.
Sounds like a perfect job for robots.
Would an "AI robot" be anything like, y'know, a computer running a bit of software?
(Actually my life would be a lot easier if universities would introduce this too. More time for research!)
R. Daneel Olivaw
Please, Mister, can we have our Ball back ....... for No Harm's Done ... and the Game is Great.
"Hmm .....Is this some sort of Turing Test lite maybe - if you can't tell whether an exam paper has been marked by a human or a machine, then the future has arrived. What next? Job interviews by machine? Replacing judge and jury with a computer?" ..... By Anonymous Coward Posted Saturday 26th September 2009 15:20 GMT
Hmmm. Well, you can try out the effectiveness of the computer for judge and jury with a ITs Support for A Darling's Positive Decision ..... Quantum Leap ..... http://tinyurl.com/ur2die4
"These AI things can only ever be as good as the twat that counts the beans will allow them to be." .... By Al fazed Posted Sunday 27th September 2009 09:04 GMT
Let's US Pray that Darling's a First Class Chap, rather than Anything Else Lesser, Al fazed. He is provided with the Opportunity and All Tools are Honed to Perfection for IT's ESPecial Speedy Seedy dDelivery for Growth of New Supply IntelAIgent Sources.
Anyone to introduce a bot having a sense of humour?
I'm like totally cool with this
and can we have robot teachers too. Then I'll send my robot to school in my place
@ Anonymous Coward
We are an idiot..
As an after thought
As Pearson are a US company, I assume that my 15 year old taking English language GCSE this year, had better refrain totally from the use of irony in her written answers and essays.
Quelling her natural inclination for creative writing might also score some points in the exam. Leaving me to wonder what will happen to poetry, prose and lyric writing (at which I once thought that we excelled)
Humour might also cause her a problem if there is but an iota of whit, in what she writ.
She has already had to deal with not being able to study Art, Music, Design Technology and Drama for her GCSE owing to modular timetable clashes which no human in the school, (3rd best in UK apparently) has the power to undo.
I rather suspect that the machines have already penetrated our civilisation and only willing healthy and dumb subjects will be incorporated into the new society, vetted by Pearson Education software no doubt.
I just hope this AI program doesn't use American English to mark the fookin exam papers.