I’m sure some people have bigger data, but Facebook has some serious daily growth. The highlight? 500TB a day.
That’s serious data. I’ve never had a database that was a TB, must less growth by over a TB a day.
To be fair, lots of the Facebook data is unstructured, stored in Hadoop or NoSQL structures for images/video, but there’s a ton of database-like status updates, links, comments, etc. that have to be managed.
Rather impressive, but not something I want to administer.
This editorial was originally published on August 22, 2006.
My son asked me about physics the other day and what it meant. He’s been on a bit of a science kick, looking to understand and explain his world. I can understand that because most of us are curious about how things work in our world, not necessarily in a scientific way, but just in general.
I somewhat butchered the definition because I was caught up slightly in the actual concepts, but one of the things we talked about was inertia. Maybe because I’d seen this article on overcoming institutional inertia the same day.
We all have things at work that we do a certain way. Maybe we learned it that way, maybe we’re more comfortable, or maybe someone yelled at us daily until we did it that way. Regardless, fighting the way people work is a difficult thing to undertake. The article gives a few examples of process changes and how they were effectively moved forward and how the issues were resolved at a high level. It’s common sense if you’re wondering, use diplomacy and take your time.
I’ve seen changes occur from both sides of the fence, from someone trying to implement change and from someone who was told changes were occurring. I was never completely comfortable in either situation mostly because with change there’s usually conflict. Someone doesn’t want to change.
As DBAs we often try to control our world and ensure a high degree of stability, often at the cost of slowing or severely impeding changes in our systems. While I think it’s good to aim for a stable, reliable environment, I don’t think this should necessarily impede change.
Understanding the business needs and goals is critical both for DBAs in protecting their systems and developers in building new applications. In either case, it also means that the needs of your organization need to dictate how quickly things change.
Not an employee’s personal feelings.
A few years ago my accountant caught a mistake in one of our tax returns. This was during the next year’s filing and so we had to amend to previous return, filing against almost two years later to correct an issue. Fortunately the error was in our favor and we ended up receiving a refund.
In most businesses you typically do not change data that was completed in the past unless there is a severe error. For some types of data, such as audit data, you never want to change it. And in many cases, even if there were some change, because of the way that you have operated the business based on those past values, you might decide not to change things.
However as more and more data is accumulated, and used in business decisions, legal proceedings, etc, it is likely that more data professionals will be faced with the questions about when it is appropriate to alter something. With that in mind, this Friday’s poll is:
When would you change archived data?
Are there circumstances that you are aware of where it is valid to change data that might be archived? Is it only to make corrections that were recorded incorrectly in the past? Is it to normalize data, so events like companies merging are able to run reports with pre-merge data?
This is a thorny subject, but it seems like one that we should be thinking about as data professionals, looking to give guidance to our clients.
(Originally published at http://www.sqlservercentral.com/articles/Editorial/72232/)