MySQL preps business upgrade

TalkBack 15 of 19:

Next »

« Previous

• Thread View
• Flat View

On the other hand

"DB designers brought up on normalization often have a problem moving to the data warehousing world. Many will obstinately continue to develop 3rd normal form solutions, and then complain about the hardware when performance dogs."

But isn't data warehousing a symptom of problems in data management rather than a cure? Having spent many unhappy and unproductive hours explaining discrepancies between accounting systems and data warehouses I think that in practice your claims that duplicating data doesn't lead to inconsistences don't stand up to close examination in the real world.

"Data consistency means something very different in large historical data stores. Duplicating data in data warehousing is not a sin, but instead a primary means of bringing data related by business processes together in a data mart. Where there are no updates (or, no business-initiated updates), data consistency is no longer sacrificed by data duplication."

No historical data has to be consistent too and has to updated from somewhere.

"There are a number of different solutions buzzing around right now that try to ressurect virtual data warehousing under the unbrella of EII (Enterprise Information Integration) solutions, but even most of the EII zealots will admit that this only supplements but does not replace a traditional physical, integrated data store for the core of data warehouse."

The proper goal should be not duplicating the data. We should accept that anything else is a compromise until the proper technical solutions come along that make this unnecessary. This is certainly not unrealisable if you are able to step away from the assumption of current implementations.

"Now, transaction process in a whole different beast. And there is nothing, absoultely nothings, about current SQL RDBMS's that prevents you from maintaining you data at 100% consistency, if that's what you're aiming for. If you checks and foreign key constraints won't enforce your consistency, then triggers can."

No you are mistaken. Triggers are event handlers not constraints and there are table and database level constraints that cannot be handled by triggers.

"My experience has been that most production transaction systems, even reliable and robust ones, share the responsibility of data consistency between the application and the db, often for performance reasons. But, if you wanted to do it all in the DB, you certainly could."

The database is the right place for enforcing data consistency and integrity, in this case your data is always protected, even against ad-hoc updates.

Again I would reiterate that getting up to date with some of the theoretical fundamentals of data management is well worth while and has some serious practical implications both in terms of the quality and performance of the work we do. If you think a DBMS is just somewhere to store data then exploring the true nature of the relational model should open your eyes a bit. After all for years the relational model was just a theory dismissed by many as theoretical and impractical. The success of even such a compromised implementation as SQL is an indication of the strength of the ideas. There obviously isn't room to explain it all here so I strongly recommend you to take the time to do some background reading.

Posted by: jorwell   Posted on: 10/20/05 You are currently: a Guest | Members login | Terms of Use

Alert moderator to an offensive message

Subscribe to this discussion via Email or RSS

SponsoredWhite Papers, Webcasts, and Downloads

• Open Standards Technologies Provide the Ingredients for Delivering Security Across the Papa Gino's Enterprise Dell Papa Gino's Holdings Corporation founded by the entrepreneur operates one ... Download Now
• VMware Infrastructure: A Guide to Bottom-Line Benefits VMware Frustrated by the costs of maintain ever larger data centers?or building ... Download Now
• Building the Virtualized Enterprise with VMware Iinfrastructure VMware VMware virtualization software has been adopted by over 120,000 enterprise ... Download Now