Tuesday, September 25, 2007

So I have these two little databases...

... in these databases are some tables, database A contains table A..a and database B contains table B..b, the schemas for A..a and B..b are identical. The data stored in them is sequential, that is, table A..a contains the first half of the data and table B..b contains the second half of the data. Sound familiar? Probably does, this is a fairly common archive arrangement, you put all that data from each month / year / decade / whatever into a new db to keep the size manageable... yeah yeah, not ideal, but hey, these things are everywhere.

So now I want to run queries across the full depth of A..a and B..b in one go - don't ask why, I just want to ok? (hint: no clean logical way to decided which db will contain the data you want) Create a view you say, a union all kind of arrangement? Take advantage of the fact that both tables are nicely indexed? Nice idea, but no. Might work on small tables with a few gb in each one... but what if they have a few hundred gb? You better have a monstrous tempdb even to do something simple like:
set rowcount 100;
select * from AB_view;

So, what do you do? Surely there has to be some better way than freeing up a mega amount of space and bcp'ing the data out of one table, putting it into the other one, then re-indexing, that will take a considerable amount of time. I want to be able to just grab hold of the table objects in the db and tell them to join together "vertically". I'm sure there's many reasons why doing that isn't practical, but I don't care... at all.

Databases suck sometimes... And don't even bother asking me about DBArtisan... Surely there must be a good DB query tool out there somewhere that is light weight and, well, works? Someone could make a crap load of money if they built one...

No comments: