back to article AI robots to mark exams

Our "education" system is taking another step towards total automation with the introduction of AI robots to mark examination papers. Pearson will unleash the bots next month to mark English papers, reports the TES. Apparently the system "'understands' the meaning of text much the same as a human," Pearson claims in its …

COMMENTS

This topic is closed for new posts.
Anonymous Coward

Appropriate

People being taught to be robotic by people as bland and characterless as robots ought to take exams marked by robots, before spending the next 50 years behaving like robots. As recommended by Indian outsourcing companies? Wow, there's an endorsement to salivate over.

0
0
Grenade

Oh Bluddy Hell!

As a recently turned 14 year old, in Year 10, its a scary thought doing GCSE's and BTECs early, heck my maths class might be doing Advanced Maths or AS Level Maths, in YEAR 11?!

I certainly want the authenticity of a human exam marker, atleast then he definately understands. Its like sending a mouse to do a man's job, the mouse might be trained but he might not have the experience. Not to mention economy jobs, we're unstable enough as it is, we need jobs not frigging robots.

I know this is only a Pearson jobby, at the moment... I'm the only one doing my BTEC at my school, its a pilot programme, and I think I have to do my ICT Practitioners test at a Pearson Test Centre, quite frankly I want to make sure my exam is marked thoroughly and properly, like any one would. AI is great, but to be honest it needs small time tests first.

rant over and the icon is for the f*ckwit that thought : "oh, I know, lets put this in several test centres for important tests!".

God *roll eyes*

0
0
Boffin

GIGO

If the exam itself is not intelligently designed, it doesn't matter how good or objective the grader is, the results are still worthless. Designing a test that actually measures what has been learned is not a trivial task, and a lot of so-called teachers are not competent at it.

0
0
Terminator

errrmm, what??

"The system is not “distracted” by language-irrelevant factors such as test takers’ appearance, personality or body language."

it would be quite hard to be distracted by any of those things while marking an exam paper, since the candidate is not, in fact, there.

Mind you, I don't think a robot could do a worse job than the incompetents doing this job at the moment. in the last couple of AS/A2 exams my duaghter had one paper remarked and it went from a D to an A, and one of her friends had a paper graded U (i.e. not worth marking) that went to an A on remark. If the humans are that useless give the machines a try!!!

0
0
Gold badge
Welcome

Uh?

"The system is not “distracted” by language-irrelevant factors such as test takers’ appearance, personality or body language."

If the system can infer the author's appearance, personality or body language from an ESSAY, then truly we must all welcome our new and omniscient overlords. (On the other hand, the whole thing could be a pile of crap dreamt up by someone too stupid to even realise what a fucking twat they are.)

0
0
Silver badge
FAIL

the system "'understands' the meaning of text much the same as a human,"

Given that many young humans these days can't even *write* coherent text, I'd be very impressed if this actually worked!

0
0
Bronze badge
FAIL

RoboMarking

I was speaking with a Greek acquaintance recently, and he mentioned that the Greek education system encouraged memorization (I personally believe memory is a Good Thing), and that while creative responses might be marked highly, the most reliably acceptable responses were those directly from the text.

Clearly, the Greeks will benefit the most from AI Robots. The more brilliant students, not so much. Perhaps AI robots might be applied most appropriately to the "fuzzy-studies" - i.e. the humanities. I wouldn't want my surgeon to be marked highly by robots!

My surgeon's failure is my own!

0
0
Silver badge

AI Pondering ITs Muse.

"People being taught to be robotic by people as bland and characterless as robots" ..... By Anonymous Coward Posted Friday 25th September 2009 16:14 GMT

Considering a robot can be programmed to do practically anything, and we even have them working/have had them working/beagling away on Mars, to imagine them as bland and characterless, would tell them more about an Anonymous Coward than an Anonymous Coward could tell about them, meThinks.

0
0

Handwriting recognition?

I don't think so. Presumably a machine can only mark 'tick the box' questionaires, which aren't really exams at all.

0
0

spellchecker

So, basically, this consists in running a spellchecker and that obnoxious Word grammar and style checker on the essay, and counting the number of red lines?

0
0
Bronze badge

Total pants...

If English is to be written in such a formulaic way that it can be appreciated and judged by robots, it might have been written by them in the first place.

Of course this is the logical step down the road that we have been going for years. For anybody who wants a dispiriting experience, then I'd suggested finding some of the examination marking guidelines that are around. They are often little more than an instructions for the examiner to look for one of a number of set phrases or words with precious little room for interpretation.

I came across one example paper (in biology), freely available on the net, where examinees were asked to state several reasons why some people object to genetic engineering (note, that's a social science question - the paper explicitly did not ask for rational, science-based objections). However, if you put "because is it unnatural" or "for religious reasons" then the guidance explicitly gave zero points. Well these might not be valid scientific objections, but they are most certainly those that many people use (which is what the question asked).

So increasingly what is being taught is how to pass the exam and not convey understanding. Box-ticking, formulaic and highly specific stock phrases are the norm in many exams.

0
0
Paris Hilton

Чего стоишь? Работать надо!

Robot. Karel Chapek. Chech, Europe. "RUR", 1920. Slavic for "to work" - Rus. "работать" [ra-'bo-tat'] -- to execute a job. " t' " @ the1st syl. is a sound when you pronounce "t" the way you'd do it saying "teh", a "mild t".

The title is probably translated as something like "Why stuck? Get on working!" is the last phrase from http://video.tvguide.com/Window+To+Paris/Window+To+Paris/812913?autoplay=true&partnerid=OVG , the unprecedented Alien XCellence in every tick of the alarm watch. Originally presented @ http://livefilm.info/otc/360-okno-v-parizh.html , no anti-pirate rules for the visitors. Some 512k+ connection speed only, sorry London (-:

Definitely Paris, because she's the City where the Walls lead to.

73

0
0

I found a bug!

For a 100% score, just type in ,,,,,,,,,,##########,().()%%%%%%%%%% and you'll score 100% every time.

0
0
Anonymous Coward

Like children...

...we fear change.

The change of place. The change of time. The foreboding uncertainty of future. The inevitable intellectual castration of children and subsequent pacification or rebellious aggravation of society.

It is only natural - we are stupid, easily frightened, beasts. We need to evolve and grow through those visceral, instinctive, reactions and learn to embrace the future. The cold, sterile, future. Help our children be the best carbohydrate to work converters they can be!

0
0
Bronze badge
Thumb Down

Calling Dr Penrose

Seems that AI, as usual, continues to shift ground when it hits the real world.

An expert system, in its small world, can work, sometimes.

It is hard enough to get a biological intelligence to work sometimes, especially in the "follow procedures or else" rule bound bureacracies that the corporate state, both public and

private, has reduced us too.

To the overlords of the corporate state and the uncritical, AI sounds like a simple thing.

They will try to use it, whether AI works or not. The insidious effects of the reductionist fallacy that has poisoned the intellect of the West is reducing our leadership to foolishness.

As Dr Roger Penrose pointed out in "The Emperors New Mind" we still don't know what real thinking is, and what ever it is, it is not possible on a Turing machine, if I understand him correctly.

0
0
Silver badge

Avoidably Detained trumps Unavailable with Surprise ZerodDay Appearances. ....

... CRazed Utterances in Sane Discourses

"Naturally we requested a comment from our own AI expert aManFromMars, but he was Unavailable 42 comment."

That request must be lost and/or rerouted to who knows where, Andrew, which would then be AIMysterious Desperate Case of IT Being Stolen by the System, for such commentary is always freely and readily available.

And that sort of Petty Pathetic Ineffective Dodgy Act as Fact would be a Clear and Present Admission of a System AIdDefeat to Strange and Particular Peculiar Forces ..... exposing as it would/does, an Impotent Vindictive System, [Early Sub-Prime Human] Terminally Psychologically Flawed and Hopelessly Insecured against Simply CompleXXXX Vulnerability BetaTesting Attacks for Real Power and Virtual Control of Operating Systems Windows with RobotIQs into Networks InterNetworking Javan Applications ......for Steganographically Shared Transparent Protocols ....which may or may not be SSTP ...... http://en.wikipedia.org/wiki/Secure_Socket_Tunneling_Protocol :-)

0
0
Happy

OMG !

How fantastic !

If this AI program is half as good as the one that picked my modules for the BSc in Information Systems a couple of years ago, Jake might as well change to major in Cookery.

These AI things can only ever be as good as the twat that counts the beans will allow them to be.

What sort of jobs are these "vetted" individuals gonna get ?

I really believe it will be the sort of job that no self respecting human would choose to do.

Err, given the choice, by our new evaluating overlords.

0
0
Terminator

This is how it starts

First they weed out those that are smart enough to be a threat, then they put idiots in charge... Hang on, that happened before the robots took over (damn!)

0
0
Silver badge
Pint

Stones from Glasshouses always deserve Comment

What an XXXXCellent Plan, Andrew, ....to rattle a few Cages/Ivory Towers. Is the Register Game for taking AI Lead and making Permanent Changes? For why would they leave IT to Lesser Mortals who wouldn't evenTry for the Want of a Clue of what to Do ? :-)

I Kid U Not ........ http://www.phrack.com/issues.html?issue=66&id=17#cmt1912

0
0
Coat

Isaac Asimov might (or might not) be pleased ...

Have any of the designers ever read his short stories on robots?

The most common theme is the reaction of curmudgeons who don't accept the changes being wrought by tin substitutes.

This reply was crafted entirely from recycled electrons - so there!

The one with the Alpha Sprayer in the pocket, thanks.

0
0
Paris Hilton

Laughing about a joke

To me, as long as AI cannot laugh about a joke on its own account - meaning without having built-in algorithms to understand a nut-nut joke - there is no AI. At least, no AI that can even remotely be considered to mark exam tests.

We'll see more main-stream texts, because this method allows for reverse-engineering of the marking algorithm, which in turn will result in even more 'teaching to the test' - sorry:'teaching to the global algorithm'.

Therefore, El Reg is correct: marking will be 'programmed in' by the personal preferences of the developers. And not a single person can convince me, that the machine will be able to gather an abstract comprehension and understanding of its own from the text fed into it.

Paris, because I think even she is better at marking an English essay than that box.

0
0
Thumb Down

Hmm

Is this some sort of Turing Test lite maybe - if you can't tell whether an exam paper has been marked by a human or a machine, then the future has arrived. What next? Job interviews by machine? Replacing judge and jury with a computer?

But are they really serious? Multiple choice marked by computer I can understand, but not essays surely? Or don't they do essays any more? Yes, that must be it. The machine just needs to check if there's a name at the top of the paper and that's a pass, right?

0
0
Thumb Up

Sounds good to me

I have no faith in human exam markers. Heck I don't have any faith in the whole exam system.

Plenty of idiots go on to get As while plenty of bright people end up with nothing.

All these exams do is test your memory, i.e. how well you can remember this fact or that chunk of text.

Sounds like a perfect job for robots.

0
0

Terminological exactitude

Would an "AI robot" be anything like, y'know, a computer running a bit of software?

(Actually my life would be a lot easier if universities would introduce this too. More time for research!)

R. Daneel Olivaw

0
0
Silver badge

Please, Mister, can we have our Ball back ....... for No Harm's Done ... and the Game is Great.

"Hmm .....Is this some sort of Turing Test lite maybe - if you can't tell whether an exam paper has been marked by a human or a machine, then the future has arrived. What next? Job interviews by machine? Replacing judge and jury with a computer?" ..... By Anonymous Coward Posted Saturday 26th September 2009 15:20 GMT

Hmmm. Well, you can try out the effectiveness of the computer for judge and jury with a ITs Support for A Darling's Positive Decision ..... Quantum Leap ..... http://tinyurl.com/ur2die4

"These AI things can only ever be as good as the twat that counts the beans will allow them to be." .... By Al fazed Posted Sunday 27th September 2009 09:04 GMT

Let's US Pray that Darling's a First Class Chap, rather than Anything Else Lesser, Al fazed. He is provided with the Opportunity and All Tools are Honed to Perfection for IT's ESPecial Speedy Seedy dDelivery for Growth of New Supply IntelAIgent Sources.

0
0
Happy

AI Qaeda?

Anyone to introduce a bot having a sense of humour?

73

0
0
Unhappy

I'm like totally cool with this

and can we have robot teachers too. Then I'll send my robot to school in my place

0
0

@ Anonymous Coward

We are an idiot..

0
0
FAIL

As an after thought

As Pearson are a US company, I assume that my 15 year old taking English language GCSE this year, had better refrain totally from the use of irony in her written answers and essays.

Quelling her natural inclination for creative writing might also score some points in the exam. Leaving me to wonder what will happen to poetry, prose and lyric writing (at which I once thought that we excelled)

Humour might also cause her a problem if there is but an iota of whit, in what she writ.

She has already had to deal with not being able to study Art, Music, Design Technology and Drama for her GCSE owing to modular timetable clashes which no human in the school, (3rd best in UK apparently) has the power to undo.

I rather suspect that the machines have already penetrated our civilisation and only willing healthy and dumb subjects will be incorporated into the new society, vetted by Pearson Education software no doubt.

I just hope this AI program doesn't use American English to mark the fookin exam papers.

ALF

0
0
This topic is closed for new posts.

Forums