Sphinx - text search The Pirate Bay way

STOP THE POSTGRE MADNESS. WE KNOW. WE UNDERSTAND.

"I'm still amused by how many people use the nightmare that is MySQL when PostgreSQL is more scalable, quicker, powerful and featured. "

Well that would be amusing indeed. Except, of course, PostgreSQL is not actually more scalable, nor is it quicker, nor does it have nearly the same amount of features and flexibility (aka power) that MySQL does.

What Postgresql fans seem to like most about Postgresql is that it's simply not MySQL.

This is a point that has become VERY clear every single time that MySQL comes up in any article anywere on the entire fucking internet. I could write a entire 400 page article about the history of databases and computing and put a single line mentioning MySQL anywere and, with absolute reliability, you'll have at least one PostgreSQL fanboy stand up and say how much he loves Postgresql becuase it is not MySQL and he does not understand why MySQL is so popular despite the fact that it's MySQL and MySQL sucks.

The reality of hte situation is that there are lot of things that are not MySQL (Dirt, humans, Oracle, IBM, video cards, desks, etc etc) and while Postgresql is one of them it's not a really good reason to actually use Postgresql.

While there are actually very good reasons to use MySQL over Postgresql and people actually do know databases very well actually do end up choosing MySQL for really valid reasons. Despite the fact that a average postgre fanboi doesn't see this or understand why...

Index servers

Great article. Sounds like you're pulling a lot from your Pressflip experience.

I can emphasize with all the developers trying to figure out enterprise search. For our site, I went through nearly a year, and many index engines to finally get it right. There are some clever things programmers have to do do beyond the normal Lucene and engine tweaks. By thinking out of the box for their particular application (wrappers, pre-processing, etc), results can be much more useful than the normal search engine scoring scheme.

@STOP THE POSTGRE MADNESS. WE KNOW. WE UNDERSTAND.

You just managed to sum up nearly all debates about why certain people love certain pieces of software. Especially true in the Windows vs. Linux vs Mac vs. evryone lese wars that have gone on for decades now....

@ David Wiernicki

for making me laugh. Other wise all this is way over my head, I am sad to say.

@ David Wiernicki

Hah! Thanks :)

Can I be the first to say it also works really well with Rails?

Just install the Thinking Sphinx gem and off y'go.

We've used it to implement combined searches across many heterogenous tables (as in type in a reference to a company, client or claim and find all of the places it's referenced for example)

It Just Works

The MySql/Postgres debate - err ... guys ... relational databases are just a commodity now. Just somewhere to stuff things you want to be stored permanently. Soon as you start using the extensions you've locked yourself in, which is why using another engine for free text makes sense. You can swap it out if it starts not doing what you want without having a ton of embedded code.

This is why I'd never use the Oracle Enterprise search thing, but that's a discussion for another day...

@Big Chris

Way to miss the point here.

The point is that I know you fanboys love your database and every time you see the term 'MySQL' printed anywere... in comments or in a article, even in a article (like this one) that talks about how you DO NOT WANT TO USE MYSQL... you people show up and talk about how much postgre is. It's like a compulsion, a disease, and it has to stop.

Nobody cares anymore.

Beyond that... I HATE ALL SQL DATABASES.

Why? Because they are the most abused thing imaginable.

First off... The vast majority of websites that use SQL database for storage.. well.. they don't need it. In fact they would be much faster and be better designed if they didn't use SQL at all and used flat files and file-system backed storage rather then shoving everything into some binary blob. They would be simplier, easier to back up, easier to setup, permissions handled easier, less prone to corruption, and quite a bit faster.

That covers about 90% of the users that go on and on about Postgre vs MySQL. They are wrong and stupid for using a SQL database in the first place. It's just stupid.

There are SQL databases that store my URL history... I have video games that store scores in databases... there are music playing applications that store music databases in SQL format. If I try to install a stupid web application for doing some simple-stupid automation on a remote machine it tries to pull in MySQL or Postgresql or some other shitastic thing.

Why? Because web developers don't know how to do jack shit other then using SQLfor storage. They think it scales, they think having websites running from multiuser relational databases make sense. They are under some delusion that SQL databases are fast and secure.. they won't think out of the box and never learned how to do anything else. Then they stop being web designers and try to be application designers and well... when all you know how to use a hammer then every problem is a nail waiting to get hammered into submission. Round hole, Square Peg? Goddammit.. just smash it hard enough with SQLite and it'll fit perfectly!!! The repair manual says to tighten to 15 foot pounds? Well.. dammit Postgre is ACID COMPLIANT! Smash that bolt down with that and your garrenteed success. If you hit it hard enough then the leaks are going to have to stop, it's physics!

Sure, sure for a lot of things SQL makes sense, but usually it doesn't.

DBMS fanboy war

I for one just hate all Databases, they are 'orrible, finicky things .

I can't imagine why anyone would like ANY DBMS to the point of enjoining an INTERNET FLAMEWAR over them!

It's like a bunch inmates at the Guantanamo Hilton having a fist fight regarding the relative merits of waterboarding vs extreme sleep deprivation.

Keep up the good work chaps <reaches for beer and crisps>

@Francis Fish

Relational Databases are not a commodity. At least to those that know about relational databases.

Its like saying that cars are a commodity. Just try and say that a Porche 911 and a Ford Mustang are equivalent vehicles.

The real problem is that a lot of the tips and tricks that are in the commercial products are not done well in the 'open source' community. Sorry, but query optimization developers isn't a commodity skill.

If the author did his homework, he would have found that IBM's IDS has a VTI (Virtual Table Interface) that also has a way of creating advanced indexing. So you can write you own indexing scheme.

The point is that what you praise in the OpenSource world has been around in the commerical world. Too bad certain web 2.0 companies would rather be a billionaire on paper than actually develop the rock solid infrastructure needed to efficiently support high volumes of users.