An Impressive SQL Server

This editorial was originally published on May 6, 2011. It is being re-run for the Christmas holiday.

Merry Christmas and Happy Holidays!

Recently I heard about one SQL Server with 1TB of RAM and another one that was handling over 10,000 transactions a second. I read a note about a 1.1PB database, built on SQL Server to hold data from telescopes. That particular data warehouse is producing 1.4TB a day. I know it’s mostly binary data, but that’s a lot of 1s and 0s being stored in SQL Server.

Over the years I’ve heard regular complaints and concerns from executives about SQL Server scalability. Those complaints have dwindled, but I think there are people in the relational database world that still question SQL Server’s ability to handle high volumes or data sets.

This Friday, I wanted to see what might change their mind, or maybe what might make you view SQL Server differently. While I suspect most of you are SQL Server fans, that doesn’t mean that you don’t have concerns at times about how far you can push SQL Server. The question this week is:

What company would impress you if SQL Server backed their main systems?

This isn’t scientific, and I’m not defining which systems impress you. It could be their web site, their supply chain management, their largest data warehouse, or anything else. What company talking about using SQL Server in one of their systems would impress you? And which system would you like to see backed by SQL Server?

When I think about a lot of data, high volumes and rates, I think of a few things. The stock market, sports, and Amazon. If Amazon switched their website and ordering systems to SQL Server, I’d be impressed. If Major League Baseball, or the National Football League were to run all their statistical systems on SQL Server, I’d be impressed. With all the fantasy sports fans out there, the gathering and processing live data for real time calculations of performance, would be impressive.

I’ve left out other companies. Wal-Mart using SQL Server for supply chain management would be impressive. UPS managing packages on SQL Server would be something. I’m sure have your own ideas, and let us know this week.

Steve Jones

Are We Suckers?

I written about learning to say no. I’ve noted that we must find a balance in life with work. I try to encourage you to work hard, get better at your job, and bring value to your employer. I certainly know there are plenty of employees that coast along in their jobs, doing often less than what I would consider the minimum. I’m hoping that more and more of you are trying to do better.

However, when I read pieces like this one where a CEO wonders why employees don’t work harder, I get frustrated by managers that don’t seem to understand that work is merely work for many of us. We enjoy it, we want to do well, but it’s just a part of our lives. We have other things in life that we love and are more important than work.

However I also know there are plenty of you that go the extra mile. You routinely work outside the normal business hours to solve problems, or just deploy changes to system. I’ve been there, and I’ve wondered why I’d bother working from 11pm Sat – 4am Sun for “routine maintenance” if I wasn’t being paid an hourly wage. This week, what do you think about this?

Are we suckers for working extra hours?

I know plenty of people enjoy the work, and they rise to the challenge of responding in a crisis. One of the hardest things I had to manage was getting people to go home and rest when faced with an extended crisis. It seems I had no shortage of employees that wanted to work 30, 40 or more hours in a row, and I’d argue to send them home to rest because I’d need them to work tomorrow night.

I think that we should make an honest effort to give our employers value. We should do the work assigned, perhaps do it better today than yesterday, and be flexible when more work is needed. I also think employers should do the same thing. Extra hours at work should be extra time off. Compensation should relate to to performance, but not hours worked. Sacrifice by employees should be met with sacrifice by employers.

In other words, just like my balance between work and life, there should be a balance between employers and employees with each side respecting the other.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.3MB) podcast or subscribe to the feed at iTunes and LibSyn.

The Lost Laptop Crisis

I wrote about the dangers of travel awhile back, where I had concerns about information when crossing the borders of a country. However that’s a case that most of us encounter rarely. However many of us do move information around on a regular basis with our devices, carrying them wherever we go. Each time we change locations, there’s a chance we’ll forget a device, or that one might be stolen from us.

There is a limited amount of information in our phones and tablets, but in many laptops, we can carry more information, and often do. No shortage of us keep database backups or imports/exports of data for development purposes. Our clients certainly have reports or data in Excel, but often that’s a limited amount of data compared to what some of us might keep on our machines. In thinking about this, I wanted to ask this question:

What would you do if you lost your laptop?

I’m talking about the main laptop that you use for work. What’s on it? What’s important? How would you react? Have you even thought about it?

I have, and I’ve had laptops die, though never stolen. However I have taken some precautions. When I get a new laptop, the first thing I do is enable the whole disk encryption in Windows or OSX. I keep backups of the recovery codes at home and I make sure I’ve taken advantage of strong passwords and other encryption for information I carry. I don’t have passwords saved for any VPN or other secure connections to banks or work.

It’s not a perfect solution, but it’s a start. Let me know today if you take other precautions.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.1MB) podcast or subscribe to the feed at iTunes and LibSyn.

Positive or Negative

While watching a presentation recently on a SQL Server topic, I was surprised by a design technique that the speaker used. I hadn’t ever thought of the issue, but it caught my eye and made me rethink the way I look at design, and even data.

In this case, the speaker was building a table and looking to make a row active or inactive. This person chose to use a column name of “inactive,” which surprised me. I’ve always used an “active” column if I am trying to mark a row in a binary fashion.

That got me thinking. I wonder if there is some impact to designing entities either way, but I wanted to ask this question:

Are you positive or negative. Would you use an active flag or an inactive flag?

This could be extended to any other type of choice, but do you look to flag an action as happening, or not happening. As readable or not readable? As usable or ignored? I’m not sure it matters, but I’d certainly be curious if anyone has any reason why you might choose one or the other. Is there some meaning conveyed? Let us know this week.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 1.9MB) podcast or subscribe to the feed at iTunes and LibSyn.

ER Modeling

I saw someone recently note that an Entity-Relationship model is important for your database development project. That UML is important, and the model should be actively maintained. I agree.

However, I’ve usually not seen this done. Typically when I’ve seen an ER diagram in a shop, it resides on a wall, printed out on multiple sheets of paper. Often there are as many handwritten additions on the pages as there are marks from a printer. The diagram is also usually out of date, even allowing for handwritten notes.

Maintaining a diagram is hard. Deciding to actually spend time working on one instead of just talking about a few entities and building them, may be harder. This week, I wanted to ask you if you’re up to the challenge in your organization.

How many of you maintain an ER model for your database?

Do you have an ER diagram you’ve built? Whether it’s in ErWin, Visio, or Paint. Is there a diagram that’s actively maintained and that everyone can use for designing changes to the database(s) in use? Do people actually consult this and use it in design meetings? Let us know this week.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 1.5MB) podcast or subscribe to the feed at iTunes and LibSyn.

How many environments?

One of the things I’ve learned over time is that in order to develop software, you need multiple environments. I think that outside of school projects, I’ve always used at least two environments: development and production. At school there were times I worked on software that I also used, and would make changes on my live (and only) system. I had no shortage of regrets, and whenever I question using a Version Control System (VCS) or separate environments, I think back to those days.

People I meet these days that develop software almost always have more than one environment, and most have at least 3. The most common model might be development, test, and production, but plenty of people work with even more. Maybe not as many as the seven Grant once had to deal with, but I have worked in environments where we had six database environments for one of our software pipelines.

How many different environments do you work with?

The poll this week asks for the count in your organization. I realize that you probably have multiple applications and test environments, but choose one or more and give us some answers.

The most I’ve worked with was in a financial services company. We had a development database and a separate QA system for testing. There was a staging system to mimic production, a Beta database for users to actually test against and production. That worked well, and we were able to easily refresh environments as needed without impacting our development efforts. Mostly because we have a strong DevOps type process.

Let us know this week how you’ve got your systems set up and what each one is used for in your software pipeline.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.8MB) podcast or subscribe to the feed at iTunes and LibSyn.

Is Skynet Coming?

That’s the question this week: Is Skynet coming?

I read a piece recently that noted Stephen Hawking, Elon Musk, and Bill Gates have warned us about artificial intelligence. They have been quoted as this is potentially something that the human race needs to be cognizant of. However, I wonder.

I know what IBM’s Watson does is amazing, but is it intelligence? Is it anywhere close to sentience? Or is it really just pattern recognition and matching with facts? I think more of the latter, and I’m not sure we’re moving closer to a computer intelligence.

I guess there are narrowly defined domains where computers seem to be improving their capabilities, but it seems to me that these areas are defined by the programmers and the systems are tailored to a specific ability.

I do agree with the article that combinations of massive computing power and humans will make fundamental changes in the world. I think many, many jobs are potentially going to be lost and workers dislocated because of the ability of computers to do many jobs that humans perform today. I don’t have any solutions here, but I am glad I work in technology.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 1.6MB) podcast or subscribe to the feed at iTunes and LibSyn. feed

Does It Count?

This editorial was originally published on Jan 28, 2011. It is being re-run as Steve is out of town.

I wrote a blog recently about downtime and SLAs, and the need to have not only downtime SLAs, but also a data loss SLA. Thanks to Paul Randal (blog | @PaulRandal) for clueing me in to the need for having both. In the comments someone mentioned that scheduled downtime ought not be counted against your uptime measurements. That’s a debate I’ve had in the past with employers, and I thought it would be a good Friday poll:

Should scheduled maintenance be counted against your downtime SLA?

There’s an argument that says since the outage is scheduled, and people are informed, that this shouldn’t really count against your work in keeping servers running. The flip side is that when the database instance is down, it’s not useable, and of limited use to the business.

I know that companies that calculate the usage of their vehicles, and maintenance counts against usage time. While it’s necessary, the idea is that mechanics should be looking to minimize it, and perhaps ensure that extra checks are done when the vehicle is being worked on to help prevent future issues. A similar argument could be made for database servers.

Do you feel that scheduled maintenance is downtime? Is it calculated that way at your employer? Let us know this Friday.

Steve Jones

Statistical Analysis

I ran across a book recently that is written to address some of the statistical issues the author has found in many scientific papers. It’s been released online for free, but if you like it and would like a more portable format, you can buy it for a reasonable price.

I haven’t dug into the book much, and I certainly don’t consider myself to be a statistician. That wasn’t one of my stronger subjects in school, and it’s a weakness I’d like to correct at some point. However I have had to use some of the statistical functions in past applications, and I wonder if I was using them wrong.

This week I decided to see how many of you are using more complex math in your systems. I’m hoping you understand how the functions work, but I wanted to ask what you’re using:

What statistical functions have you implemented in a production system?

I am thinking of functions beyond basic aggregates like SUM and AVG. Are you using standard deviations, linear regressions or some other complex functions? Have you made use of built-in functions in T-SQL, R, or some other language? Are you implementing custom functions in code or CLR Aggregates?

I think this is one of the areas where our profession will grow more and more across the next decade. As we deal with lots of data of varying types, and our organizations look to gain some strategic advantage through deeper insight into their information, we will have lots of chances to experiment and learn more about complex data analysis.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.1MB) podcast or subscribe to the feed at iTunes and LibSyn. feed

How Virtualized?

I went to a talk recently where I saw this statistic: “50% of all workloads were virtualized in 2009. That number is 72% today.”

That’s a really big number, at least in my mind. That implies the vast majority of all servers, file, print, database, email, etc. are virtualized. Inside of companies that have their own data centers and machines, they must be heavily virtualized. I’m sure that all those instances in the “cloud” also count, but still, 72%? That’s big.

However I’m sure that’s skewed towards those machines that don’t require a lot of resources, like file and print servers, DNS hosts, etc. This week, I thought I’d see what the percentage is inside of your organization.

What percentage of your SQL Servers are virtualized?

Give us numbers of physical v virtual if you can. I’d combine all instances, from development to test to production, not worrying about size or workload. If you have a single guest on a host, using almost all the resources, that’s a virtual server.

My suspicion is that the percentage of SQL Servers is much lower than that of other workloads, but I’m curious. With the low overhead of modern hypervisors, and the free (or low) cost, it makes sense to virtualize servers. If for no other reason than to remove any weird hardware dependencies for DR purposes. However I’m sure that there are large workloads that require more resources than the current hypervisors can expose, at least for some database instances, and those need to remain on physical machines, but my guess is more often than not, it’s the human concerns or lack of confidence that prevents virtualization.

Let us know this week how your organization is doing in the trend towards virtual servers.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.4MB) podcast or subscribe to the feed at iTunes and LibSyn.