147 posts • joined 20 Nov 2009
Yeah, I agree...
The online info on the GADGET site isn't written to be what I'd regard as understandable for the typical techie. Way too much jargon, not much clarity, not great....
Re: O.M.G. Wow
In retrospect, I should probably set up an anonymous account if I'm going to shamelessly tout my own articles.
Re: Links, Please?
Hey John, yeah, I had some problems finding it at first too. Here's a link to the developers: http://www.mpa-garching.mpg.de/gadget/ and here's a link to the Wiki article on GADGET: http://en.wikipedia.org/wiki/GADGET
I should probably hyperlink all of the software references, that's an oversight on my part. I'll do that in the future.
And, for my money? Inspector Gadget should have been much better. Great concept, so-so execution.
Hey, this IS the most comprehensive coverage of student cluster competition I've ever seen. No need to eat a handful of spider webs, at least as far as I'm concerned. The writing literally leaps off the page, goes through my eye holes, and gets into my head - in a good way.
Re: "carrot cake is not a real dessert"
You are obviously very wise, and not just in the pastry and dessert arena.
Hey, I live to use those cliches...in fact, I'm looking for more of them in order to keep from using the same cliches over and over.
As you can tell, I really like these competitions. I'd like folks in technology to see these events much like the general public does college football, basketball, soccer, or whatever.
My strategy for accomplishing this overly ambitious goal is to treat these competitions as much like a major collegiate sport as I can. I want to cover the technical details (which are the "x's and o's" in other sports), and also generate some excitement and add some humor along the way.
If anyone has any good cliches for me to add into my mix, let me know by hitting my name and sending me an email. If they're good, I'll use 'em and even credit you for the contribution.
Hmm....good point, but
My thinking was that a brace of accelerators would be the typical maximum that you'd ever see on a typical system - ie four. Thus a double brace would be eight. But your research has opened my eyes and I'll bow to that definition in future articles.
And stop calling me Shirley
So I'm the only one???
I'm not a creative professional. I run a small industry analyst firm and put up a fair amount of stuff here in El Register. You may have seen some of my videos covering the student cluster competitions and a few webcasts in my Reg HPC Blog.
When I stated doing more audio and video work in 2010, I realized that I needed some serious tools if I wanted my stuff to look and sound better. Looking at what I was doing, I figured I needed a good video editing suite, photo editing, audio processing, and even something I could use to cobble together the odd logo or two. I tried GIMP, Audacity, and some other free tools, but wasn't wild about them. I had a hard time using them, which is mainly a function of me not having enough time to devote to getting up the learning curve.
I then took a hard look at various Adobe offerings, but getting full versions of Premiere, Photoshop, Audition (which was called something else back then), plus Illustrator or maybe AfterEffects, was going to cost me anywhere from $2,000 -$3,000, plus a few hundred bucks every couple of years for updates.
That was a big enough number that I just couldn't justify it. Then CC came along and gave me all of this stuff for $50 a month. Now I have full versions that are always up to date, and I can use them on my desktop, laptop, and on big server - as long as I only run one instance at a time.
Speaking for myself, I've found CC to be one of the best purchases I've made in a long time. I haven't had any problems with the apps or with CC verification - even when I'm on extended business trips. I did have a problem once when my credit card expired, but CC gave me 30 days of use before pulling the plug. I called Adobe customer support, and it was taken care of in about 5 minutes.
Financially, I think it's about a wash in my case. I'm a guy who typically purchases updates when they're available, and I'm really glad to see that there's so much educational material out there. If I'm stuck trying to do something, I seem to always be able to find a Youtube video or tutorial that shows me the way.
I'm using these apps more than I thought I would, primarily Audition and Premiere, with a fair amount of AfterEffects and Photoshop, and a smidgen of Illustrator, Prelude, and Speedgrade.
As much as I like bitching about things in general, CC has been great for me. I think they did it right, and I think this is would be a great delivery model for other ISVs to emulate.
Good point on the price difference, but NVIDIA is a big supporter of these competitions and had been great about giving the kids whatever they need.
Thanks for the info on GeForce and CUDA. I've been working off and on with a PCIe extension box that has GeForce cards in it, trying to get it working with my laptop. So far, no success in getting the laptop to see the box or the cards, but that could be a limitation of Windows 7 or the lousy PCIMIA card I'm using to connect. I'll try connecting it to a desktop motherboard directly via PCIe.
LOL....I've been writing about this school and their cluster competitions for years and you're right!! I didn't see it before. Nice job...
Yeah, should have been 1,000x, not 1,000%....I screwed the pooch on that one....total pooch screw.
I agree, that would be pretty cool. Maybe for GTC15? Hmm...I wonder if I could get the GE guys to give me some footage of the the torture tests they put their ruggedized systems through? That might be a substitute for what you wanted to see....
Re: Very flexible
That's a good point about potentially higher power requirements for general purpose gear vs. special function chips. If anyone is interested, I could talk to Dustin and see how GPUs compare to FPGAs and other custom chips in terms of performance, power, heat, etc. Might make a good webcast.
I think that the environmentals can probably be engineered around, particularly since the general purpose stuff is so much less expensive, it would free up budget for the re-engineering. But one problem I haven't considered is security. Does using general purpose gear make the device more vulnerable to hacking by an enemy? I would imagine it would, so what steps are taken to keep it secure?
The 'big deal' part of this is that we have computers that are fast enough to output the various waves, modulate them correctly, etc., then received the input, sort it, interpret it, and route it to the right place. All of this on the fly and at the same time - with a computer that's small enough to fit in a small compartment on a plane. That's pretty cool, isn't it?
Yeah, I know what you mean. I've wondered about that stuff too, whether any system could withstand the kind of punishment it would receive on a real battlefield. But, damn, they ruggedize the hell out of their gear. You can see a bit of it in this video I shot at GTC13 http://www.theregister.co.uk/2013/04/10/ge_ruggedised_systems/
They also pay a lot of attention to thermals. I don't know if you can see this in the video, but the metal casings around their system is very thick, they do this to make it stronger of course, but the enclosure also acts as a huge heat sink. It was just over room temperature when I touched it, but they assured me that it was engineered to handle much more heat than could be generated at a well attended trade show.
Re: Software Defined *.*
Ok, that cracked me up, made me spit Sprite Zero on my keyboard. Are you happy now?
Re: And it comes with a free unicorn.
Damn it! I missed out on writing about the free unicorn? They didn't have a unicorn at the show, or maybe it was in one of the customer-only whisper suites.
The way I understood this is that we're essentially talking about using the entire set of drives in the array (48? 64? or more) to rebuild the failed drive. This would be a lot faster, of course, then having five or ten drives rebuilding a single drive.
As to the internal bandwidth and the limitations there, you make a good point. However, I'm pretty sure we're talking about SATA 3 rather than SATA 2, and PCIe 3 as well. I'm not positive on these points, but it makes sense to me that they'd use the latest/greatest on this new storage box.
Re: Computer Scientists
You guys both make some good points, but these Student Cluster Competitions aren't just about putting together computers and racing them. These kids have to run real scienfitic apps, running the gamut from weather forecasting to molecular dynamics to financial modeling. In order to do this, they have to investigate and understand the apps and the science behind them. Many of these students are majoring in scientific fields, which, as you point out, require some fairly deep knowledge of computer science.
The laptop was not connected to the same speakers as the desktop. The audio on the laptop was playing through the laptop speakers and headphone jack. Hmm....I didn't check to see if both computers were playing the SAME audio at the same time - that was an oversight on my part, probably caused by panic..lol
Re: Poorly Solved
Yeah, you're right. I may not have a solution here at all. But so far, so good. The rest of your post scared me to the point where I almost peed my pants. But it's not anything that I haven't been wondering about.
Very interesting situation with your twisted wires picking up boradcasts. In my case, I could hear enough audio snippets to realize that they were coming from locations very far away from me - not local TV or radio stations.
You should charge admission to your demonstration of your audio problem. If it confounds and amazes co-workers, they should pay at least a little something for the experience, eh?
Re: Rootkit playing tunes?
You make some great points above. And I was scared to death about what the rootkit might have been doing while it was distracting me with the audio. I did look at my network traffic on my switch (also 16 ports to support desktop computers and home infrastructure). I didn't see much, if any traffic coming out of the infected system, which at least sort of put my heart at ease.
I did boot the system from a recovery DVD and repair the O/S as part of my own troubleshooting. It didn't seem to work, or else the virus reinstalled itself upon boot.
I wish I had the skills to use a debugger and those kind of deep diagnostics you cite above. That would have come in very handy back then!
Re: Tired of self professed "security experts"
Damn it! Where the hell were you when I was going through all of this??!! I could have really used your help and would have paid handsomely for it.
Just to keep things straight, I wasn't going through some random security auditors or 'experts'. I contracted with a major security firm to have them fix this box. From what I could tell from watching what they were doing on my screen, you're absolutely right, they were primarily using the same tools that I was using before I contacted them.
I didn't see any evidence of them using a debugger, or checking processes, or doing any sort of deep dive into figuring out how this virus functioned. To me, I figured it would be a matter of finding out what process is compromised and then eliminating it. But, according to them, it's not that easy these days. They said the virus code could have been buried inside other processes - which would make finding it more problematic.
On the other hand, there was SOME process that was pushing audio to my sound card, would it be impossible to trace that chain back and see what was issuing those commands?
Ok, here we go, this is more like it...lol. The charge was for $100 per month and it was with one of the biggest and most reputable security firms in the industry. So i wouldn't put them on par with monkeys - unless you know some really smart and experience monkeys.
I ended up reflashing back way more than a few days and that seems to have solved the problem, while losing valuable work at the same time, of course.
You're right that my NAS wouldn't have helped me if someone had come in and encrypted all the files on the server and NAS box. That sort of 'data kidnapping' scheme is pretty tough to protect against. How would you do it?
Re: Your own ethics?
Hold on, wait a minute. I didn't receive ANY benefit from their services. I thought I made this clear in the article, but maybe not.
The security company mainly just cost me time. They didn't tell me anything I didn't know. They weren't able to identify the virus, how it got into my system, or fix it. The did tell me that my current security status was top notch and exactly what they would recommend.
In their pitch to me, they said that they would certainly be able to fix my system without any intervention on my part. Was it an ironclad guarantee? Nope, but no one gives those types of guarantees on these types of services. The problem here is that they simply didn't have the experience or tools to handle my particular problem. Since the reason I purchased the subscription was to solve this problem, I feel that canceling it was fine ethically.
Re: This may be a really obvious question
Great question. Here's what I observed:
1) I couldn't seem to find a specific process that fired up when the audio was active. The process list (hundreds of processes and services) looked identical when the virus was 'sleeping' vs. when it was actively playing sound. I also didn't see any particular process taking more memory or CPU than normal.
2) The audio played even when there wasn't any other application open and without me doing anything that would prompt a system sound. There also weren't any system issues that would cause an alert - at least nothing I could find with deep hardware and software scans.
3) The audio was unfamiliar to me. It wasn't playing anything that I had ever heard before (nothing from my media library, for example). It sounded like it was current or cached versions of web broadcasts, but the snippets weren't long enough for me to figure out exactly what they were.
Re: Sorry if this is a stupid question....
NO! Not a stupid question at all! I spent hours wondering the same thing. I also closely checked network activity when the virus was active to see if it was busily sucking files out of my system or feverishly sending commands to a bot army. I didn't see much of any outbound network activity at all, and very little inbound traffic too.There also hasn't been any problems that are identity theft related either.
I think whoever penetrated me (whether I was targeted or it was random) probably used some code that was both sophisticated and amateurish at the same time. It was able to penetrate multiple layers of protection to get into my boxes and hide itself, but the payload didn't do much of anything other than annoy me.
On the back-up issue. I would heartily suggest that you keep more than three weeks of back-up images. One thing I didn't mention in the article (I didn't want to make it even longer and more tedious) was that one of the first things I tried was to use Windows System Restore to get back to an earlier version of the system. No luck on that, even with going back weeks in time.
So if I didn't have as many back-images as I did, I would have been even more screwed than I was. Storage is cheap these days, you can get a 3TB drive for less than $100. Get one and fill it up with back-ups, it's cheap insurance.
Re: In concerns me that this is the case
I see your point about remote vs. keyboard access. But from looking at what they were doing on my system, it looked like they did do a boot from an external source into a proprietary shell o/s.
It didn't work, of course. Which sucked. But is reformatting the only solution here? That's going to take a lot of time - longer than it took me to reimage a spare drive. I get your point that there's every possibility for reinfection, but I would hope our current set of tools can do at least a decent job of ensuring that your existing files aren't compromised.
I took a quick scan through the lastest Top500 list and, not surprisingly, didn't see any Itanium based systems on the list. It's not surprising, since these boxes were mainly sold to customers who need large SMP instances. There also aren't very many HPC apps available for HP-UX, which is a huge hurdle as well.
Yep, that's another great point and something that I've been considering. Most of the industry system components are manufacturered. Systems vendors today are much more assemblers and integrators than anything else. So how much difference is there really between a Lenovo branded system or a rebadged IBM set of boxes? Technically? Probably none. But according to procurement rules, probably it's the difference between a completed deal and a quick rejection.
Re: IBM will slide further down
You're making a great point in your comments above. Lenovo will absolutely face problems with trying to replace govt. and lab systems. How much of a problem? I'm not sure yet but am asking around. I'll post a follow up story as I find out more. Thanks for the great comment!
Crappier? Eh, I don't agree
You think their PCs have gotten crappier over time? Hmmm...that hasn't been my experience. I had an original ThinkPad when I was with IBM for a few years in the late 90's until 2001. I was impressed with the design and durability - very efficient box. Just for fun, I priced out my business laptop as if I were buying it retail and was shocked to see how much the thing would have cost me.
After leaving Big Blue and starting my own company, I had HP, Acer, Fujitsu, and Sony laptops before giving Lenovo a try again. While the other laptops worked fine and were functional, I found that the Lenovo had that same durability and functional design that I remembered from my first ThinkPads. Moreover, when I started buying Lenovos, the prices had reached parity with the rest of the industry.
So, from my perspective and for my needs, their laptops are a cut above the rest. I haven't tried every laptop out there - or any other brand for the last three years, so maybe things have changed.
Re: "Successes? Oh, we've had a few...........
Ok, that one made me spit Sprite Zero all over my keyboard....great job....lol
Re: Forgot one thing
Hmm...I'm not so sure it was as big a swap-out as you're saying. There are still a boatload of Lenovo folks in Raleigh, NC. How many of those are legacy IBM, I'm not sure. I also know that they've hired a number of IBM'ers to staff their earlier server efforts.
I seriously doubt that the carnage for the people moving from IBM to Lenovo will be anywhere close to what you're saying. They're picking up an ENTIRE division of completely new products. I'm assumnig that they truly do want to be a power in the x86 server market. That said, they can't afford to blow out the people they're going to need to market, sell, service, and design these boxes. It just doesn't make sense to me.
Unpaid internships suck
I hate the idea of an unpaid internship. If the intern is providing any sort of value, then pay them at least something to recognize that fact.
What's great about these student cluster competitions is that these kids are finding good opportunities out there - whether we're talking internships, advanced degrees, or full time jobs. I have another webcast coming up that talks with Cray's John Lee about this....
It's a short video, but longer than zero. It's working fine now - at least it did for me.
Not sure what you mean....
...but the name of the university is truly "National University of Defense Technology". These are the same folks who built/own Top500 chart topping boxes like Tianhe-1 and Tianhe-2
I actually like CC
I'm a guy who needs to use a fair amount of the big Adobe tools like PS, Premiere, After Effects, Audition, and Illustrator. Before CC, I had debated buying either the package or the individual components I needed, but had always balked at the $3,000+ purchase price and then the extra money to stay current (assuming I wanted to stay current).
For my $49 per month, I get everything I need. The crossover point between me being ahead of the game financially is 61 months (3,000/49), meaning that I'm ahead of the game for five years. For me, this is a better deal than having to put out three grand to get what I need.
My analysis would be different if I had already owned any of these admittedly expensive tools. But in my case, Adobe's CC brought them a new customer that they probably wouldn't have had before.
Re: Why stop there..
Good idea, but that would have reduced the hyperbole in my sentence by perhaps as much as 37%. My goal is to provide all the information about the competition, but with as much hyperbole as I can possibly pack in. All of this at NO additional charge to you, the reader.
Re: 26 amps?
Ahhh...damn it, I should have mentioned the voltage! We're talking 26 amps at standard US power of 110 volts. They have two 13 amp circuits they can draw from. Sorry for omitting that key fact and thanks for pointing it out to me.
Re: And still
I see what you mean, but I remember the days when we had SMP systems using buses and crossbars with flat flat access times (ie. non-NUMA). The system I'm most familiar with, the Sun E10000, had a 16x16 crossbar with 1.3 GB/sec bandwidth at about 550 ns latency. The E10k scaled like a scorched weasel (which is pretty good) when compared with the systems of that era.
Today, you can get 56Gb FDR Infiniband that offers 6.8 GB bandwidth at around 700 ns latency. That's not too bad at all, and should be enough to ensure reasonably linear scalability on a good portion of workloads.
In any case, a SMP system should be able to scale better than a cluster running message passing. While sitting here writing out this reply, I can't think of any application that would scale better in an MPP system vs. SMP, but I could be wrong.
Re: And still
Yeah, very good point on that. And this solution should be a hell of a lot faster on those types of workloads vs. message passing on clusters. But we'll need to see some number to really know for sure.
I expect to see this kind of technology filter down into enterprise fairly soon. ScaleMP is part of the secret sauce in SAP's Hana in-memory analytics product set.
End of an era
I've learned more from reading and talking with TPM than I ever learned in school. (Hmm...is this a comment on TPM's guru-like knowledge or the poor quality of my schooling? Maybe both?)
I'm glad that he's staying in the business and hasn't pulled up stakes to pursue his other interests. Some of you might know that Tim is a mean home brewer. (He's actually a mean everything, but what I mean in this case is that he has some serious brewing skills.) As he mentioned in the story above, he is a master of the holiday fruit cake. His 2012 edition won the prestigious Olds household "Best Mailed Food" award for that year.
I know that Reg readers will miss his heaping helpings of tech news and views. I'll reading him in The Reg as well.....
Re: Background or explanation
Here's the problem: there are now three major international student cluster competitions, each with 8+ teams, 5 or more apps, and three or more winners. I probably end up writing somewhere around 20-30 articles for each competition. I can't face a life where each and every one of these stories has to contain verbiage summarizing the entire competition. It would be way tedious to read and more tedious to write.
The solution? I wrote up a comprehensive guide to Student Cluster Competitions earlier this year. That article (here: http://www.theregister.co.uk/2013/02/14/guide_to_student_cluster_compos/) explains pretty much everything you need to know about these events.
While that article isn't linked in each and every story, the stories that I have linked to do have a link to that original document. (Now that's a great sentence, very clear.) . So, like the other posters have said, if you're interested, check out the links - they have the details.
Re: Background or explanation
Hey Steve....I did write an article about the configurations, you can find it here: http://www.theregister.co.uk/2013/06/24/isc_13_cluster_comp_configurations/
We didn't see any exotic cooling this year, although at least one team had signaled they might try for a liquid cooled box.
I'm going to do little deeper dive into the hardware, looking at node count, core counts, memory to core ratios, and whatever else looks interesting. Look for that soon.
Re: Colorado is also home to the NIST
I just finished an interview with a couple of WRF experts where we discussed the app and how to optimize it. I'll have a link to that interview in a near future blog post here.
If you're looking for past Student Cluster Competition results, you can find my articles by searching my name on The Reg, of course. This stuff goes back to 2010. If you're looking for all of it, you can also visit www.studentclustercomp.com - that's a new portal site we built to be a one-stop-shop for all things related to the Student Cluster Competitions.
That's a story I've been pitching to Reg editors for a few years now, plus additional coverage on other models who are trying to make their way in the rough and tumble world of HPC. In order to do this story justice, I'd need to travel to various locations and have the resources to blend into the world of the super model. So far, I haven't had any luck convincing them that: 1) this is a good story for The Reg and...2) that I'm the ONLY PERSON who can pull this off.
I'll keep trying, though.....
As a long-time resident of the Portland tech suburbs, I've watched the city spiral down to the point where it can generate it's own field of hipness and pretentiousness without needing any inputs from outside the city limits. For the unitiated, take a look at any episode of the TV series Portlandia....while it's put forward as a comedy, it's much closer to a documentary.
Guess I'm the exception....
..but Adobe gained me as a customer with this package. I run a small business and found that I needed Premiere, Photoshop, Audition, and Illustrator to do our videos and other projects. I had a definite need, but couldn't quite justify spending $1,500 for the Premium Product suite or just buying a few of the products ($650 for Premiere, $600 for PS, $500 for Illustrator, and $350 for Audition).).
For $50 per month, I get access to all of these products plus everything else I would ever need. I also get support and won't have to shell out more $$ for upgrades.Over the time I've been with the program (about a year now), they've added more products to my subscription, some useful to me, some not.
Before Creative Cloud, I used point products (Adobe PS/Prem Elements, Audacity, others) and was constantly frustrated at what I couldn't do with them and the amount of time to get over several learning curves.. The Adobe products are extremely high quality and are well integrated. This has saved me a lot of time and I've been much happier with the results.
On a straight financial basis, my $1,500 outlay would pay for 30 months of Creative Cloud subscriptions, so I would be in the hole on the 31st month of my subscription. But to me, paying $50 per month for everything I need, no charge upgrades, and support is well worth it. And if I find a better alternative, I can convert anything I produced with Adobe to a different format and use it in whatever software I end up using.
Without the Creative Cloud, I wouldn't be an Adobe customer....
...maybe next time? And that highlights a problem with the internet, it's really difficult to find adult-oriented content on it.
- Review Apple iPhone 6: Looking good, slim. How about... oh, your battery died
- Review + Vid Apple iPhone 6 Plus: What a waste of gorgeous pixel density
- +Comment EMC, HP blockbuster 'merger' shocker comes a cropper
- Moon landing was real and WE CAN PROVE IT, says Nvidia
- Apple's iPhone 6 first-day sales are MEANINGLESS, mutters analyst