Feeds

back to article AMD: Why we had to evacuate 276TB from Oracle DB to Hadoop

AMD has migrated terabytes of information from an Oracle Database installation to an Apache Hadoop stack, claiming Oracle's pricey software was suffering from scaling issues. The chip maker's chief information officer, Jake Dominguez, revealed further details of the transfer in a chat with The Reg. "Within the common Oracle …

COMMENTS

This topic is closed for new posts.
Bronze badge

sounds like

they didn't know what they were doing to begin with. Or perhaps weren't able to do the right thing to begin with and had an organically growing system that nobody seriously managed and they just let it grow until it hit a major breaking point.

Not that I'd use Hadoop to replace Oracle, rather go with something like Vertica instead especially for a small data set like 276TB.

2
12

Re: sounds like

"especially for a small data set like 276TB."

lol

20
0
Bronze badge

Re: sounds like

...they're doing data mining, which is dumb to put in a relational database. It's not that the Oracle DB failed, it's that Oracle failed to come up with practical distributed computing systems for non-relational data. I know Oracle created some distributed systems in the past but I don't think anyone made it to the end of the installation instructions.

9
0
Anonymous Coward

Re: sounds like

Oracle usually shits a brick well before that mark. Depends in your query rate and how much horribly written PL/SQL you're throwing at your doddering old mainframes. I've lost count of the number of places that have come to us wanting to replace their Oracle boxes, thinking they'd pick up something like Teradata or Vertica, only to end up settling for Hadoop.

Between the completely hassle free configuration and administration, the resilience and fault tolerance, the as-good-as-free hardware and licensing and the massive toolkit you get, it's pretty much a no brainer. Especially now the major distributions come with their MPP SQL engines (Impala, Stinger etc.). Throw in Parquet and Snappy and you literally have a carbon copy of your garden variety Teradata architecture, but open source and really, really sodding cheap.

Those in traditional data warehousing should be worried.

10
0
Silver badge

@intrigid

276 TB is small in the database world.

1
4
Bronze badge

Re: @intrigid

"276 TB is small in the database world."

I'd wager that more than 99.9% of databases in existence are smaller than 276 TB

13
0

Re: @intrigid

99.9999% even.

2
0
Silver badge
Linux

Re: sounds like

I also found the remark about the 100K result set limit quite odd.

2
0
Silver badge
Linux

Re: @intrigid

> "276 TB is small in the database world."

>

> I'd wager that more than 99.9% of databases in existence are smaller than 276 TB

That just means there's been a proliferation of a lot of small databases as the technology has become cheaper to use. The big jobs continue to get bigger all the time and people aren't terribly impressed when you start throwing around non-trivial numbers.

0
0
Silver badge

Re: @intrigid

Perhaps I was exaggerating (or should have said "276TB isn't overly large") but to me how many databases are smaller is less relevant than how unusual a database larger than that size is. Having consulted on a project a couple years ago where one database utilized several fully loaded VMAX arrays, perhaps I'm jaded.

Probably 99.9% of all programs are shorter than 10K lines, but that doesn't make a 10K line program large.

0
0

Re: sounds like

When most rack configs are sized at 1PB, 276TB is relatively small.

0
0
Anonymous Coward

Re: sounds like

When most rack configurations for big data are sized for around 500TB+, 276TB is relatively small.

0
0

"According to AMD, the Hadoop software was able to reduce query times by 300 per cent"

So let's say you have a report that takes 6 hours to run... When your boss says "and I need it yesterday!", you can say "You've got it, literally!"

15
1
(Written by Reg staff) Silver badge

Re: intrigid

Yes, OK. AMD claimed it and we weren't awake enough at the time to think it through, but now we've tweaked that sentence.

Please, don't forget to email corrections@thereg is you spot something odd - we see those emails, but we can't read every comment.

C.

5
4

Re: intrigid

What would be the fun in that? If your buddy's fly is hanging open during a funeral, you don't quietly mention it to him, you point and laugh! Ok, maybe you have a point.

5
0
Pint

Re: intrigid

Actually, that depends entirely on whether your buddy has pledged to buy the pints at the after-funeral trip to the pub.

3
0
Silver badge

>>So let's say you have a report that takes 6 hours to run... When your boss says "and I need it yesterday!", you can say "You've got it, literally!"

Only if they get you the request before 6:00pm that day.

And typically that's when the sadists let you know there's something urgently needed right away, ime. :( >:|

1
0
Anonymous Coward

Re: intrigid

> Please, don't forget to email corrections@thereg is you spot something odd

Couldn't you set up an online form instead of email? I would occasionally send the odd correction but I'd rather not divulge my "useful" email addresses and I really cannot be arsed to log into a throwaway one for this.

0
0

Humm "We made the pivot to Hadoop [and] it not only increased our reliability but our response time," I thought you normally tried to reduce your response times?

12
0
Silver badge
WTF?

"row limit for query results [of] 100,000 rows on the Oracle system" - !?

Can someone who works with Oracle's software confirm, deny, or qualify this?

I cannot believe a modern database system of any kind would have such a tiny row limit...

4
0

Re: "row limit for query results [of] 100,000 rows on the Oracle system" - !?

never seen this limit, (going back as far as version 7.3.4) so not sure what they are talking about, there is a limit of 1000 columns in a table although that would indicate your doing something really stupid.

Possibly they have performance problems with data > 100,000 rows being returned but there is no hard limit.

2
0
Anonymous Coward

Re: "row limit for query results [of] 100,000 rows on the Oracle system" - !?

I work for Oracle as a developer. Every day I launch lots of queries that return tens of millions of rows. Never been a problem.

1
0
Silver badge

Bah!

I can believe it.

I can believe almost anything after watching Oracle "specialists" fail to get Oracle RAC running cleanly on a Sun/Solaris stack in a reasonable time, or to be able to say "yes, we know exactly why <very popular document manager application> hosted by our Weblogic Application Server running in Solaris 10 on a dedicated 4450 is acting as though it is running in treacle on wet tar, and here is how to fix it".

I was approached by an Oracle afterservice drone a couple of years back who asked "How do you like Weblogic on that 4450?" and I had the great pleasure of telling him we liked it so much we were now running the application in webSHPERE on an LPAR in a crowded P7 - and had seen an order of magnitude pick up in response time as a result".

The fun when doing that is to keep a straight face.

7
0
Silver badge
Mushroom

wow

Its a serious question if Oracle is going to outlive Ellison. Even if it does Oracle may end up being Wang a few decades later.

4
0
Roo
Bronze badge
Pint

Re: wow

"Even if it does Oracle may end up being Wang a few decades later."

It looks as though Oracle have been firmly wanged in the short and curlies by AMD + Open Source of all people. If a sufficiently detailed white paper is published I suspect this will develop into a Tsunami of woe for Larry.

5
1
Silver badge

Not surprised

It's a normal development cycle that incumbents get bloated, expensive and lose touch with what customers want.

7
0
Anonymous Coward

Re: Not surprised

But what is surprising is just how long it's taking Oracle to spiral down the plughole. I expected it to happen a decade ago.

1
0
Anonymous Coward

Re: Not surprised

I concur. I've always thought of Oracle as Apple without the cool people factor.

0
0
Bronze badge

Good software, stupid name

From my experience as a consultant I think there would be a lot more installs of Hadoop if the name didn't sound so goofy. I can't count the number of times I've gone in front of management and had them laugh off the software because of its name.

8
0
Anonymous Coward

Re: Good software, stupid name

If that's the reason they won't use it, they deserve to go bankrupt. I've heard the same over Beastie. Apparently so did FreeBSD. The effing suits... Yawn...

9
1
Bronze badge
Joke

Re: Good software, stupid name

Heard of Apache Pig ?

2
0
Anonymous Coward

Re: Good software, stupid name

Hadoop is in at least 95 of the FTSE100. It doesn't have a branding problem.

5
0
Silver badge

Re: Good software, stupid name

Yep, I eventually got FreeBSD used in a previous place of employment - ultimately, it's ability spoke for itself - however, in the initial stages, 'suits' weren't impressed with the word 'free' in the title.

And I''m sure we've all heard of religious American nutjobs who are against it because it 'glorifies satan' etc.

0
0
Anonymous Coward

Re: Good software, stupid name

That would be Management from the 1980s. There are a lot of silly names out there that dominate the industry now. Example: Google. Puppet. Red Hat. Git. Linux. Any versions of software with "X" in them (makes it "edgy"!). Android. I mean, comeon, the computer industry is rife with quirky names. In fact, it's pretty much a tradition.

2
0
Silver badge

Re: Good software, stupid name

True, I am going back to the early 90's - when open source and free software weren't generally known, and there was still a big 'Nobody gets fired for buying Microsoft' ethos..

(Yes, I know the phrase originally mentioned IBM, but it was used for Microsoft in the 90's)

1
0
Anonymous Coward

Didn't

Oracle stop using AMD processors?

1
0
Anonymous Coward

Re: Didn't

Yes, we have some of their AMD-based heads for the 7410 storage system and it is no longer fully supported less than 5 years after it was introduced.

We are not going back to Oracle in the future, but migrating away as fast as we can.

0
0
Anonymous Coward

Re: Didn't

LMAO - It was Sun Microsystems who stopped using AMD, not AMD not providing anymore..

0
0

Poor Larry -- looks like he'll be building his next America's Cup techno-boat from fiberglass instead of nano-tube-glass!

0
0
This topic is closed for new posts.