Monday, January 21, 2013

There's a place for everything and everything in its place - except when there isn't

Charles Goodrich (1790 - 1862) popularized the epigram "A place for everything and everything in its place". However I suspect he wasn't thinking about what to do with data. We are accustomed to thinking in hierarchies, that our data are neatly arranged - like an English formal garden. But in reality, what we have is a wild  and untamed data wilderness. We never know what nuggets we will pick up where. What "gifts" flocks of birds excreting seeds will deliver allowing new things to sprout in unusual places.
Organization and arrangement are fantastic and efficient for transactional activities - those things where we, by force of will have designed things. But where we have non transactional, undesigned things we have to adapt to what we find.
This distinction between adaptation to what we find and purposeful design is crucial. If we attempt to impose purposeful design over things we find, we commit a variety of sins:
  • We abstract too early - ending up with meaningless abstractions. After all the ultimate data model has one box and one line. Thing is the box. Relates to is the line. Then we end up with a series of triples. But we can't see any reality in there.
  • We name things that don't really have names. But categorization demands it.
  • We box ourselves in - we put "everything in its place", without considering that actually there are multiple places
  • We throw stuff away because it doesn't fit our notion of predined categories.
We are now in a "big data" world, where we have the opportunity to keep more stuff, where we have multiple classification systems, where we can draw inference from meta-data observation. Yet we still try to impose "design ahead" schemata on things outside of our control. This is often where master data management, large data modeling projects and other attempts to impose order often falter. We argue over nits, create beautiful (but sometimes irrelevant) abstractions.
Let's not forget that there is a place for order. But also remember that sometimes things just are and it is our job to interpret rather than to order.

No comments:

Post a Comment