back to article 'Real' JavaScript benchmark topped by...Microsoft

Douglas Crockford - the man who "discovered" JSON and a senior JavaScript architect at Yahoo! – has released a new benchmark designed to test the "actual" performance of the major web browsers on "real" JavaScript applications. And according to the test, the browser with the speediest JavaScript engine...is not Google Chrome. In …

COMMENTS

This topic is closed for new posts.

Page:

Silver badge
Thumb Down

Consider the source

"A recent paper coauthored by Microsoft researchers...."

'Nuff said. We now know why the paper pegs IE 10 as the fastest brower.

On every JavaScript application I've ever written (which, given that web development is the biggest portion of my job, is a lot) Chrome is noticable faster than IE. As is Firefox. Safari and IE clock in roughly equal in my experience.

17
19
Anonymous Coward

consider the article?

Crockford wasn't involved in the original MS co-authored paper, he merely took its findings regarding benchmarking and used his own js application as a benchmark. I'm involved in a couple of projects which are pretty js heavy and like you generally find that Chrome is markedly faster, closely followed by FF. Crockford's results are interesting with regard to IE10 because it has only just been released as a platform preview. It will be interesting to see how it shapes up at full release - all good from my point of view, healthy competition driving the technology forward.

16
0
FAIL

Consider the test...

"...does not benefit from the faster JavaScript engines because performance of those applications is limited by the DOM (Document Object Model, the browser's crap API)". So a test to see how well javascript performs pretty much *excludes* updating the web page? Any javascript test that does not include how it interacts with the stuff seen on screen is not much good as a web-browser test. Technically (please insert many lawyer-type weasel words here) it is a test of the *javascript* engine, but can hardly be called representative of what a user would experience sat in front of a browser.

9
0
FAIL

Auto-Microsoft bashing failure

It's sad to see that about 50% (of those who vote on posts) failed to comprehend that Douglas Crockford, "senior JavaScript architect at Yahoo!", is nothing to do with Microsoft. Or that the Mircosoft paper wasn't used to provide IE10 benchmarks.

7
3
Headmaster

Crockford not at MS

But for how long?

1
1
Happy

Warms the cockles of my heart to...

… see Google fail so miserably :)

12
14
g e
Silver badge

On a Microsoft report of a Microsoft product?

My what small cockles you have...

6
13
FAIL

Re: g e

"On a Microsoft report of a Microsoft product?"

As stated before, a Microsoft report said IE10 was the fastest, so an independent developer (of JSON none-the-less) took his Javascript program and ran benchmarks on it that were independent of the DOM or browser APIs (likely full of just raw Javascript since it's a javascript debugger and analyzer, rather than looping <div> ID lookups and the like). His results concurred with Microsoft's findings.

10
0
Anonymous Coward

Makes sense

It's DOM manipulation that's always slowed all our JS applications to an absolute crawl in IE. Don't really care about JS performance until they sort their awful rendering engine

9
0
Alert

cool

Can't wait for 2013 now for IE 10! lol

4
1
Badgers

Sounds Like Cheating to Me

Tuning software to a specific benchmark is like studying for a test you have access to - you only bother to learn what you know will be asked. The only valid benchmarking results from running against a benchmark the testees didn't have access to in advance.

Your goal is to have the benchmark results be scalable to the real world and if the testees are allowed to tune specifically to the benchmark this will not be the case unless the coders are volunteering to tune their code to each and every user's actual code also.

4
2
Silver badge

Everybody tunes to the benchmark, because those

are the numbers the tech rags report. My issue with this isn't whether it is tuned to the benchmarks or even whether or not he's an MS shill. It is that he is comparing an unreleased product to those in production.

0
0
Thumb Down

Title schmitle

If we're including IE10 then why aren't the pre-production / nightly versions of the other browsers on the list?

8
0

So Firefox best....

I love misleading article titles and editorials. The article was really saying that with this 'suspect' benchmark Firefox was fasted out of all the currently released browsers.....

Don't think it is fair to compare IE10 as it is only a preview rather than a fully functioning browser with all Microsoft's traditional bloat

1
0
Joke

Are we native yet?

"The approach is similar to what Firefox has done with the TraceMonkey extension to its JavaScript engine. TraceMonkey does not use adaptive compilation, but it works to detect code loops and covert them to assembly code."

Cool. That answers this question:

http://arewenativeyet.com/

0
0

To be Fair

It doesn't matter how they score in these trivial tests, its how they perform in real world tests. So Google have tuned their browser to the most commonly used code, thats only natural. I stand by Chrome as the 'real' fastest.

4
7
Pint

you don't need to hear a fart to

know it stinks.

Crankshaft was inspired by Sun's Java HotSpot performance engine. Here is your hotspot fart.

0
4
Silver badge
Unhappy

Well there's your problem!

"His benchmark is based on JSLint, a JavaScript code quality tool developed by Crockford himself. It's designed to look for problems in JavaScript programs, and it too is a JavaScript program. Unlike existing JavaScript benchmarks, Crockford says, his is more representative of large, well-written JavaScript programs – because JSLint is a large well-written JavaScript program."

I don't buy it.

He says that his benchmark is more "real" because it represents large, well-written JavaScript programs because it's based on his code. I'll let the second assumption (i.e, that his code is well-written) stand because I don't intend to refute it. Instead I intend to use it against him.

Instead, I'll focus on the much larger initial assumption: that "real" JavaScript is large, well-written code. The "large" part may well be true, but if you look at JavaScript in the wild, "well-written" is very debatable.

If he wanted to put forth his benchmark as one valid for testing "proper" JavaScript performance, then we could just look at the second assumption for validity. But the first assumption is so far at odds with the prevailing evidence that I believe that it is Mr. Crockford's responsibility to prove his point.

Show us, Mr. Crockford, with a valid study proving that the majority of JavaScript out there being consumed on the web today is well written. Please. I would love to believe that to be true.

15
2

just The Good Parts

Let's allow your assumption that most Javascript is not well written. However, a crucial point here is whether 'real' must be only empirical or may be aspirational too.

Crockford has done more than perhaps anybody else to work on improving the quality of Javascript in the wild. Including through his release of jslint. Perhaps it is reasonable to allow that Crockford is shooting at where the target will (should) be rather than where it is now. As a developer of extensions for one browser (not IE), who has a well-thumbed copy of The Good Parts on his desk, I am interested when Crockford says IE 10 handles large, well-written code better than other browsers do - even if IE10 may not benefit the majority of badly written code that is out there.

9
1
Anonymous Coward

Agreed

Good Parts is the best programming book I've read. That's partly because I found it eminently readable, and partly because it helped me love javascript and stop trying to treat it like some other language (which seems to be what 95% of us do with javascript, and that's why it's so bad).

So, yeah, I agree. He's not going to be right, just because he's Crockford, but he is going to be someone worth listening to with the cynicism dialled down 2 notches.

Also, Crockford helped me lose three stone one summer.

0
1
Bronze badge
WTF?

@John Dee

Sorry, but about half of 'Good Parts' was a repetition of the first half. Even so, it was a pretty short book.

As for this being a 'Real World Test' that is utter rubbish - there is no such thing, as all real world usage has some interaction with the DOM, which automatically invalidates any attempt to test the JS engine on its own, the best you can do is deliberately test including the DOM, and accept that this result is dependant on the rendering engine.

2
3

nonDOM

"all real world usage has some interaction with the DOM"

You are apparently unaware of nodejs (V8) and couchdb (SpiderMonkey) for instance. Not all JS runs in a browser nowadays.

2
1
Bronze badge
WTF?

@henchan

Well aware of both of them, and of Hyperwave too. Which of them is comparable with IE10?

2
1
Anonymous Coward

No

There is a modicum of repetition. I believe that's called "building on what we learned earlier". There is slightly more than a modicum of repetition in the appendices. I believe that's called "a useful summary of everything we learned earlier". You'll find both of these in most text books worth the cover price. The second half in no way repeats the first half.

As for it being short. I wanted a book about the javascript language. Not the DOM. Not the browser. I didn't want 900 Bible pages of minutiae about the Date Object. We have an internet for all of those.

0
0
Headmaster

∀x(p) ∧ ∃x(!p) == FALSE

all real world usage [of Javascript] has some interaction with the DOM

AND I am well aware of some real world usage of Javascript which has no interaction with the DOM

== FALSE

No wider point here, other than to pick up on your incorrect use of an absolute.

2
0

Alarm bells

They ring in my head when I look at the list of browsers and see the usual stable releases and one ultra-bleeding-age hasn't-even-reached-beta version of a certain company's browser. I wonder what made him choose IE10 while sticking to Chrome 10, instead of Chrome 11 or 12 which have been readily available for longer than the IE developer preview.

Firefox performing well is a surprise, and further cements my opinion that JS performance isn't as big a deal in the real world as it's made out to be. In day-to-day usage both Opera and Chrome feel faster FF4. Maybe it's the more responsive user interface, but I'll stick with what 'feels' fastest regardless of whichever browser is currently waving its e-penis from the top of the benchmark table.

8
1

Performance

Right now, the main thing holding back Firefox is that their Garbage Collection algorithm is slower than Chrome, and introduces pauses when it runs. So for day-to-day performance issues, that is where the difference is likely to be. They are working on this at the moment, and have landed some improvements to the FF5 and FF6 code lines.

All the different benchmarks are equally valid, they are just testing different aspects of JavaScript and different optimisation techniques that are possible.

And yes, real-world JavaScript interacts with the document (DOM) and canvas, so will trigger performance issues there (such as when dynamically modifying the page). Real-world JavaScript also relies on helper libraries such as jQuery and prototype. The prototype library is easy to test as JavaScript only as it does not interact with DOM. With jQuery, you can test some parts of it (e.g. $.each on an array) but not others (DOM manipulation) without it not being a JavaScript test -- that is, the test will be testing a different area of the browser, such as how fast it can draw lines on a HTML5 canvas.

0
0
FAIL

sounds great

so where do i download IE10 for my linux machines and mac? Oh, that's right I can't.......so it is about as much use as a chocolate teapot!

ice.

11
11
Jobs Horns

Re: sounds great

That's ok... calm down, Apple will send you a dictat when you when you can / must download something for your Mac. They can't allow you to make such decisions yourself...

Anyway they probably already installed the IE10 beta "by mistake" ... then Skeletor decided to remove it without your permission while you were sleeping.

2
2
Linux

very droll

However, OP's point still stands - that IE is effectively part of one OS alone, whereas the others are all cross-platform stand-alone browsers. Even if IE10 is the best browser ever, no-one is about to use a different OS just for that. With the others, you don't have to - so a *Huge* minus for IE, then.

I could be wrong, of course - maybe I'll soon be typing "apt-get install IE10-non-free" ......or not

5
2
Badgers

Meh

The guy's claims are bullshit. He's running his parse library against itself. Hardware also still plays a role. On my Atom N280 @ 1.4GHz, Opera 11.10 can run the same parse in ~3.3sec. Consistently. With sixty other tabs open. FF4.0.1 takes ~3.4sec consistently, run separately and with no other tabs open.

The new IE10 core....is this 32bit or 64bit? Because 64bit IE9 is dog-slow. Possibly the speed improvement in IE10 would just come from porting Chakra. He also fails to acknowledge that DOM operations and general rendering are integral to a web application as a whole. A better benchmark for javascript game performance would be measuring the resource consumption of JS Workers threading. In fact, his application doesn't really seem to make proper use of concurrent multithreaded javascript AT ALL. I'll have to look more closely, but I don't think I saw any Float operations either.

A program that runs once and exits, parsing plain text, is a TERRIBLE example of a modern web app. This just further advertises his shitty programming and inflated ego.

5
4

So...

...you think you can come up with better real world tests? Show us.

2
5

Re: So...

The ability to recognize bad tests is not the same thing as the ability to write better ones. Just like how I can recognize a bad book when I read it, but can't necessarily write a better one.

10
0

Yep!

That's exactly what I'm thinking. Even if you despise the coding of sites like Facebook, or take issue with certain workarounds in Google Docs/Reader/GMail etc, the fact remains that THOSE are actual sites which receive a lot of traffic.

This Crockford arrogates himself to the position of claiming that a specialized code-analysis toolkit WHICH RUNS ONCE LOCALLY AND EXITS, should supersede REAL code that uses REAL network connections to fetch actual data and update the page. Apparently he thinks IE10 would make a decent engine for some web application that does your taxes. Unfortunately our demands are often much greater.

2
0
g e
Silver badge

I too have an impressive CV

Although an appropriately large payment will have me say Microsoft rules.

3
5
Gates Halo

Who gives a shit about IE on Linux

Forget market share stats. The truth is consumers and enterprise customers are the biggest users of IE and so the performance of IE matters very much in the real world. Testing IE10 preview is fair, the point I suspect was to show how good next generation JavaScript engines will be.

Most of the reactions on here seem to me to be emotive around the fact that Microsoft might be doing something right for a change with IE. It's time to stop bashing MS for mistakes of two decades ago and start praising them for the things they are starting to do right with IE. The "real world" is about good user experience, not about 100% standards compliance or test results.

6
17
Silver badge

Who mentioned Linux?

#1, testing a not-even-released bleeding-edge product against current stable products is fair how? That gives IE a distinct advantage as IE10 SHOULD have more advanced tech in it.

#2, No one's bashing MS for mistakes made 2 decades ago. They're bashing MS for recent deceptive marketing BS.

#3, Standards compliance is an important part of the user experience in the "real world". If I code my site to be standards compliant then my customers are going to have a good experience in any standards compliant browser (assuming, of course, that my design carries a good experience to begin with) but will not have a good experience in a browser that doesn't follow standards. On the other hand, if I code my site to work in one specific, non-standards-compliant browser then only the people using that browser will have a good experience. But to be fair IE's standards compliance has gotten markedly better in the last few versions.

12
3
Silver badge
FAIL

And...

"The "real world" is about good user experience"

And although I am not a Chrome user, most people that I know that use Chrome feel that it is much faster for them than IE.

Microsoft are only harping on about standards and their latest browser tech because of the competition at the moment. You would quickly see them going back to the mistakes of two decades ago given a different environment.....

4
1
Silver badge

Until

microsoft de-couple IE from their operating systems, the 2 decades of mistakes will carry on

or as someone put it

pwn me once shame on you, pwn me twice shame on me

2
0
Gold badge

Enterprise desktops lag behind

But many enterprise users are on XP with IE6.

So what good is IE9 or 10 when they require Vista and 7.

6
0
Thumb Up

REe: Who gives a shit about IE on Linux

Most linux distros circulation is probably down to it being installed on 19$ switches, netbooks (which then get formatted and replaced by old XP so you can actually use it for something), set top boxes, or apache web servers that get 6 hits a month and are much loved by commodity hosting providers, as the markup is great.

Why people are complaining because Microsoft have not released a beta for linux... oh dear, where to start with this one... so many idiots, so little time...

Linux is useful for speed tests for the UK mirror service though, as a 650 MB download that gets deleted much more quickly.

1
8
Stop

but IE is sold illegally

Funny how it is that Microsoft employees always suggest that their illegal business practices should be ignored and everyone should only look at their technical work.

I have absolutely no intrerest in any company or their products when they count upon illegal practices to force the sale of their products. Period.

If you have a copy of IE, your opinion simiply does not count. Microsoft prevents it. And that includes your opinion of IE.

1
4
Stop

you will purchase IE again and again and again

Microsoft bundled IE to force you to buy it. You have to be real slow in the thinking department not to know that.

And the truth remains: The US Appellate Courts decided that commingling code between the OS and IE was in fact illegal. That has not changed. Microsoft continues that illegal practice despite the bad engineering.

Just how many copies of IE have you had? And how many of them did you purchase?

If you have a copy of IE, Microsoft mandates that your opinion matters not one bit. You will buy it again and again. Illegally. The consumer is not engaged in the illegal practice but Microsoft certainly is. All lawyers at Microsoft know that for fact. Maybe you have not figured it out yet?

That would be pretty slow thinking.

2
2

@Sisk

"testing a not-even-released bleeding-edge product against current stable products is fair how? That gives IE a distinct advantage as IE10 SHOULD have more advanced tech in it."

Or does it put Microsoft at a disadvantage because their product is not yet finished?

0
0

Mixed baggage

On the one hand, I'm just glad there is now a battle for Javascript speed dominance. It was only a few years back that none of the browser manufacturers paid the least bit of attention to it.

That said, it does seem that Crockford's testing contradicts his stated aim. First, he claims that most JS test suites don't test code as you would encounter in the wild. Then, instead of gathering a sample of real-world JS apps, he chooses to only test against his own, well-written codebase. I don't believe the test skewed his results for Microsoft (using a pre-release browser did that), but he did potentially skew his results against real-world JS apps.

Now that Microsoft has relegated itself to only trying to capture a subset of a subset of the browser market, how well IE performs on speed tests has become a lot less interesting.

5
0
Silver badge
Headmaster

Even if you disagree with the guy's methodology some of the usual........

........comments from the usual suspects that imply he is a paid stooge for MS slagging off Chrome show very clearly that they did not bother to read this bit of the article:

"Crockford admits that he expected Chrome to top the list. "My guess is that they overspecialized for specific styles of programming, and that Chrome was tripped up by a real program. There are some very smart people at Google, and I would expect them to rectify this.""

Now you can agree or disagree with what he says but his comments (or his work for that matter) cannot simply be dismissed as paid for by The Great Satan From Redmond.

3
0
Silver badge

Crockford's real-world vs. the real-real-world

Let me cut out Mr. Crockford's bias against DOM from this quote, and get it to the part that matters:

"...most of it does not benefit from the faster JavaScript engines because performance of those applications is limited by the DOM..."

This little gem is spot on, and exactly why his test is nothing like a real-world javascript program. The reality is, from the perspective of working with (X)HTML or XML (what the majority of javascript does) the DOM is how much of it is done. Because of that, a truly realistic representation of JS performance also should test out the specific implementation of the DOM.

He goes on to say:

"My guess is that they overspecialized for specific styles of programming, and that Chrome was tripped up by a real program. There are some very smart people at Google, and I would expect them to rectify this."

Yes, probably "overspecialized" for more DOM intensive styles. You know, like most of the JS that will be encountered in the real-real-world (as opposed to the "real-world" where code analyzers are the majority of the code we run *boggle*).

As impressive as Mr. Crockford's CV may be, he has still missed the forest for the trees. He has created (and I propose we call this) "Yet Another Worthless Browser Benchmark" because of his particular gripes with DOM.

14
0
Stop

His definition of "the real world", then...

...is "one large well-written application"?

oh yeah, right. the real world of javascript is a wild, wild west of code horribly written and mashed together, copy-pasted over and over again from the same stupid "here's how to make letters jump"-style tutorials.

2
0
Alert

@Arctic fox.

Whilst he may or not be a stooge for MS, I notice that we are given a lack of information in the article presented to us. When he talks about Chrome overspecialising for particular types of code, surly they have specialised for the 'most common code'. We are also given no indication of what types of operation's his code does and in what ratios, nor is it mentioned on what basis does he believe that his 'well written' code is indicative of 'real world code'.

No your final comment "Now you can agree or disagree with what he says but his comments (or his work for that matter) cannot simply be dismissed as paid for by The Great Satan From Redmond.", draws my interest, as if IE10 did not top the list or get very high, would anybody of paid any attention to this chaps benchmark?, additionally, I suspect he often works in a corporate environment (ie his customers environment), this his tools are designed with IE in mind.

Now you could call that scepticism with out cause, however there is not enough in the article el'reg provided to tell one way or the other. It would also not be out of character for articles/benchmarks to be more 'beneficial' to one browser over another.

3
0
Silver badge
Headmaster

@Reg Sim

I am not in any way defending this guy's benchmarking or any implicit assumptions he may be operating on. Indeed I am not in any way defending the ranking he has generated either. Several of the postings we see on this thread represent considered criticism of his work and they I have no problem with. I do however point out that the *main* thrust of *some* of the postings is that he *must* be some kind of paid shill for MS - it is that kind of posting where the messenger rather than the message is attacked that I was criticising.

2
0

Page:

This topic is closed for new posts.

Forums