back to article Twitter survives election after Ruby-to-Java move

Micro-blogging site Twitter experienced record traffic as the results of the 2012 US Presidential election were announced on Tuesday night, but the service never faltered despite the increased load – something Twitter engineers credit to the company's move from Ruby to Java for its backend software. According to a blog post by …

COMMENTS

This topic is closed for new posts.

Page:

  1. Charlie Clark Silver badge

    Pity

    I think it was better when there was a built-in bullshit brake!

    As for the speed not sure that Java is so relevant. A Java foundation in theory provides a scalable environment because so much money has been spent by Sun, IBM and others on the JVM but no one has ever really claimed Java to be fast and memory efficient (in comparison say with C). Moving to a more functional paradigm probably brings better IO performance with lots and lots of asynchronous clients.

    1. stanimir

      Re: Pity

      no one has ever really claimed Java to be fast and memory efficient (in comparison say with C)

      Java delivers +-5% of C code speed. Memory efficiency for large projects is quite the same provided the developers don't do it the 'fluffy' way. After the initial memory footprint, there is not much difference between java and C, we are talking large projects, say at least 100k LoC.

      There are lacking areas at the moment - like vector manipulation where the intrinsics to go w/ SSE instruction set are not satisfactory but other than that the JVMs deliver very high quality machine code, plus the benefit of optimize/deoptimize on the fly, i.e. adding/removing/replacing modules w/o restart is godsend.

      1. Tim Parker

        @stanimir Re: Pity

        "Java delivers +-5% of C code speed."

        For what, exactly ? Without any details or context that's a fairly meaningless comment. That said, i've personally not seen anything non-trivial (computationally) that is runs remotely in the same ballpark in Java as C/C++ (disregarding their own faults).

        1. stanimir

          @Tim

          It's the quality of code generated by JVM compared to gcc -o2. In C you can have a structure/object peeling, so the more structure/objects fit the cache (line), in java it's doable but way harder - that would be a major one. The other, I mentioned already, is vectors manipulation - SSE alike is not java forte. Peeling part does require profiling and non-trivial skills but I give it.

          JVM has default profile guided compilation which is not the norm in C (still doable though). Allocation on the stack (in C) is no biggie as "new" in java is just a pointer bump (when elided it's nothing but that's not often enough).

          The other one often mentioned is bounds checking but well-written java code (needs to know when it would be optimized) will have the bound checks removed by the JIT. For instance: I got ~20% perf. increase by adding an extra check in jzlib to show the compiler the loop variable stays in bounds. In C you don't have to do that, although imo automatic bound checks are really, really good feat. Knowing when the JVM can inline the code is probably important as well.

          The human factor would be the main strength of C - the developers are generally more experienced/skilled/talented. Few Java developers have a clue how the cache coherency works.

          However "time-to-market" and the lack of explicit memory managed (especially in concurrent environment) is a clear win for Java.

          This is sort of anecdotal evidence - Azul didn't have native zib lib so they coded in java, the difference was minuscule and when the code was paralleled (having 768 cores), java version was a lot better.

          For large scale projects - java is still much better: easier to debug with good stack traces (no debug builds or anything). You can even get a memory dump in production (still slow but doesn't stop the process or attach a debugger for real, jstack/jmap technically use the debugging interfaces to communicate)

      2. Anonymous Coward
        Anonymous Coward

        Re: Pity

        You haven't used clunky Eclipse then!

  2. spaceyjase
    Thumb Up

    It's an interesting performance choice for sure. There's not much parsing of a tweet's content once submitted but there's some additional overhead to produce something that maps into a bit of XML (e.g. take a look at a tweet's XML when accessed via the API) and to keep track of retweets and conversations. I'd say it's a fairly spectacular goal to achieve, and to keep going to claim higher performance, especially as this is the back-end that has changed all hints at a really well designed and engineered bit of software.

    Quite exciting, this computer magic.

  3. Anonymous Coward
    Anonymous Coward

    So...

    Will their website now need less of the current batch of cycle-guzzling javascript and quite the torrent of data to serve really just "140 characters or less" of content?

    If not, no matter what the engine brand, twitter remains the chelsea tractor of the intarwebz.

  4. Hywel Thomas

    15,000 in.

    The 15,000 writes. People are forgetting the reads. Single tweet, many followers, some using multiple clients perhaps with multiple views is going to generate a lot of traffic *from* their servers. The numbers were given for a sense of the change of scale as Twitter has grown in popularity.

  5. clocKwize
    Facepalm

    You can't bash Ruby for not handling a global service that has clocked 15k tweets in a single second. Ruby is slower than other languages, we know this. Yet we'd all still rather use it and know our systems are slightly slower.

    If you were to build the same system in java/c++/c# it would be faster, hell yes. But it'd also take a LOT longer time to build and would be harder to maintain.

    It's very easy to over-optimize "my app isn't as fast as it can possibly be, lets spend 3 months making it quick" but then there is probably a 95% chance you will never have enough traffic to even notice it being slow. That is a waste of resources.

    and if in the happy event that your site does become so popular its straining under the load, thats GREAT. spend some of the mountains of cash you've just made rebuilding it for what you now know isn't wasted effort.

    Might I point out, is exactly what Twitter have done.

  6. Anonymous Coward
    Anonymous Coward

    RE: Meh

    Then think about the hundreds of thousands of people reading those 15000 tweets on the website and the hundreds of thousands of smart phones constantly polling for new tweets...

    1. Tom 38
      Facepalm

      Re: RE: Meh

      Oh noes, subscribers, events and channels. If only someone working for JP Morgan Chase had written something in 2006 that handles this exact problem.

  7. Anonymous Coward
    Anonymous Coward

    RE: As a PHP and semi-Java developer, I say..

    We don't all want to be left in a world of sql-injection vulnerabilities and untested code. nor do we want to be left in a world of lines like this one:

    public static <T, K> Multimap<K, T> partitionBy(final Iterable<? extends T> inputList, final Function<? super T, ? extends K> partitionFunc) { ...

    1. Richard Wharram

      Re: RE: As a PHP and semi-Java developer, I say..

      You can write bad code that allows for SQL injection in any language.

      Ruby has gone some ways to making it hard to write code that allows it by default although lazy coders could still override it.

  8. Anonymous Coward
    Anonymous Coward

    Hmm

    I wonder what difference it would have made if they had moved away from Ruby to...a re-write from scratch in Ruby.

    I don't think Java has much to offer here and I suspect the chance of throwing the old code away is what paid off regardless if they had redone it in Basic, Perl, or C++.

    "Build one to throw away - because you will" F. Brooks.

  9. Anonymous Coward
    Anonymous Coward

    RE: Hmm

    I agree.

    They now have a clear picture of their architecture to know the best way to do things, which probably wasn't what was there before

  10. Flashy Red
    Stop

    And if they hadn't bothered at all...

    The rest of the world would be more productive without Twitter :-)

  11. Anonymous Coward
    Anonymous Coward

    RE: Re: RE: As a PHP and semi-Java developer, I say..

    Its possible in any language, given enough effort. In ruby things like that are made hard so people don't do them. In PHP it requires you to physically do something to prevent it

  12. Michael C.
    Happy

    Ahh.

    This made me smile. For someone that did Java for years at university and got his first job this year as Ruby developer, I still have pangs for Java.

  13. toadwarrior

    Ruby is one of the slowest dynamic languages iirc. It's a good thing though for a startup to start off with less enterprise solutions. It will be more costly and take longer and you don't know if your product will even be liked by anyone. Rewriting once you've solved how you want to do something is piss easy. The only people that would hate that idea are people stuck in the past thinking everything should be done in some butt ugly c++.

  14. Anonymous Coward
    Anonymous Coward

    Who would have thought leaving a little message the length of a 1990s pager could hold per message was memory intensive... i wonder how sites like youtube even exist.

  15. peyton?
    Paris Hilton

    Would be interesting to know

    If they considered JRuby at any point, and if so what it's drawbacks were. It can certainly be faster and more memory efficient than the stock Ruby interpreter.

  16. Tom Reg

    Go looks good for this

    It has a compiler, but no header files, less junk than ruby, etc. As a long time C, C++, Objective-C, Ruby programmer I look forward to working with it. Google seems to use it a lot.

  17. darwin39

    Debian code shootout results suggest that there isn't a lot of performance difference between Ruby and JRuby - both are a long way behind most other mainstream languages.

Page:

This topic is closed for new posts.

Other stories you might like