Skip to content

Consolidating Again and Again and Again

This editorial was originally published on November 12, 2009. It is being re-run as Steve is on holiday. 

In reading about virtualization, one of the main things that drives its acceptance is cost. It’s the reduction in actual physical resources needed to run multiple servers that makes it popular with many IT departments. It doesn’t help with administrative costs, at least not the costs of the system administrators because you still have the same number of servers to manage. There are just less physical servers to buy, less power needed, less cooling, and probably other savings if you virtualize.

Virtualization is really a one-time cost savings. Once you’ve virtualized a server, you can’t do I again, so there are no repeat savings from this event. Is consolidation the same? After all, the main driving force for that process is also cost, and once you’ve consolidated a server you can’t do it again.

Or can you?

There’s a great white paper by Allen Hirt that talks about the process of consolidation, and I’d urge you to read it before beginning a consolidation project. However I don’t know that consolidation is, or should be, a large, one-time project.

If I consolidate 3 or 5 SQL Server instances onto one machine, I have a choice of how I can do this. It can be by moving databases to a new instance, having multiple instances, or even using virtual machines. But is that the end? It’s not because I can add more databases later, or I could even move a database that is receiving a heavy load back to its own server and find other instances to consolidate onto this one.

Consolidation, unlike  virtualization ought to be something you periodically examine in your environment. On a regular basis review your baseline for all instances and see if it makes sense to perform some consolidation. Server sprawl can creep up on you, but with a little management and the multi-instance nature of SQL Server, you can keep it under control.

And if you don’t have a baseline to examine, consider setting one up.

Steve Jones

Placeholders for Emergencies

A long time ago I worked in a large corporation where we managed lots of servers, each of which performed a variety of different functions. Some were Exchange, some SQL, some file servers, etc. and they would come and go at a frightening pace.

We had monitoring in place, but we’d also have plenty of situations where a runaway process would use up a lot of disk space at a rate that exceeded the ability of the monitoring software to alert us before the user encountered a problem. To help us more effectively monitor things, we started adding placeholders do our server build process.

Placeholders

What’s a placeholder? It’s a large file that just takes up space on a disk. For example, I’ve created a few on my system:

placeholders2

These can be text files, movies, images, whatever you want. The idea is that you just save a particular amount of space. In this case, I have 3 1GB files that are taking up space.

That’s it.

If I run out of disk space, I just delete one of these, and voila, instant 1GB space available.

These have become really, really handy for me. I actually have a few on my laptop. When I run low on space, which will often happen at THE WORST possible time, like the morning of a talk, I can remove a file and free up space.

Of course, I’ve just deferred the issue, but at least then I can work at that moment and then clean up old files as I have a moment.

Placeholders are a great way to save yourself in emergencies.

How Often Does Production Drift?

One of the new products that Red Gate Software is working on is SQL Lighthouse, which is designed to track and alert you on “database drift“, which are the changes in your schema. I hadn’t really heard the concept in the past, though I’ve found there are a few people using the term to describe the changes being made to production that aren’t inline with the development project that created (and updates) the database.

Vendors that sell products backed by a SQL Server database, like Microsoft Dynamics, often consider database drift to be a violation of their EULA. When a customer changes the database, even by just adding indexes, it causes them support costs, though arguably those costs are lower than if no indexes were ever added by enterprising DBAs.

However this also happens in databases built by companies. It seems there’s no shortage of “quick fixes” in production, in addition to the “improvements” that DBAs and others seem to make to their live databases in order to solve some short term issue. I know there’s a need for this, and even seen recommendations to ensure production differs from development schemas. However I haven’t had many issues with this in the past and I’ve tried to prevent any drift.

Maybe because I’m a controlling DBA, or maybe because I’ve been lucky. In either case, while I’ve had to implement hot fixes and changes in production, it’s been rare. Most of the time I’ve been able to backfill these changes in development (and test) environments, or delay the fix until it could be tested.

However I’m sure there’s a need for a way to manage drift, and detect it as the number of databases you manage grows, so I’m excited by SQL Lighthouse and hope it helps DBAs avoid those late night phone calls by being aware of what’s changing in their systems.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.2MB) podcast or subscribe to the feed at iTunes and LibSyn.

Off

I’m off for a few days. Actually, today and most of next week. I completed my achievement, getting all my vacation scheduled, and this one of those days. I also have most of next week off, so I’ll be taking time with family over the Thanksgiving holiday to relax a bit, and likely, work on my latest woodworking project.

The blog will be a bit hit and miss as I work around time off. Apart from next week, and the week of Christmas, I’ve mostly got some random days each week, 1-2, where I hope to have views like this:

Photo Nov 18, 12 45 28 PM

Real World T-SQL Tricks

I’ve been playing more with T-SQL this year, doing some testing and development in various places. I’ve been trying to improve my skills, and keeping up with the advancements the language has made in the last few versions. Like many of you, if I haven’t had the need for a particular feature, or the chance to implement it, I haven’t done much with it.

I think that’s the case with many people, who may not catch all the changes in T-SQL with new versions, especially if you upgrade after a year or two and don’t often do more than refactor the existing code you have in production. In the spirit of improving skills, and also learning more about how others work in this profession, I want to ask you:

Have you used any of these T-SQL items in production code?

  • APPLY
  • THROW
  • LAG/LEAD
  • a tally table
  • symmetric or asymmetric keys
  • TRY..CONVERT
  • CONCAT/FORMAT
  • IIF
  • SEQUENCE

Some of these have been around for a long time, some are new in SQL Server 2012, but they are all improvements beyond the core T-SQL language that I see so many people posting and asking questions about.

Let us know this week if you’ve used any of these constructs, and if you can, how they solve a problem that you struggled with in the past, or maybe how these new features have improved performance or sped up code development.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.0MB) podcast or subscribe to the feed at iTunes and LibSyn.

Encryption in Colorado Springs – Encrypting in the Application?

Last night was my annual presentation at the Colorado Springs SQL Server User Group. I try to make sure I get down there at least once a year, and it’s been only once a year for the last few years. Far too busy, and I’m sorry for that, but I am glad I get invited to go down.

I presented The Encryption Primer, and there were a few interesting questions asked. Always good to see people debating and asking questions.

One interesting one from a developer – If I can perform encryption in the application, why would I do use something like TDE or column level encryption?

To me, I prefer to do encryption as close to the source as possible. If I can do the encryption in the application front end, I’d do it there. It reduces the chances of having the data accidentally disclosed. I don’t have to worry about having data read across the wire, or in a backup tape, or anywhere else.

However that takes time and effort. Developers are expensive, and they have to write good, solid, secure code in the application. They also have to write this encryption code in every application that accesses the database (reports, ETL, etc.).

Something like TDE is much easier to setup and use. Column level encryption, while still coding, is centralized.

It’s a balance, and one you need to consider carefully and thoroughly. It also helps to debate and discuss the decisions about what you protect, why, and what it costs.

Achievement Unlocked: Balance 0

Earlier this week I got a notice that I hadn’t completed a task this year. It’s one that I’ve rarely finished in the past, and probably not in any year in the last 10. However I decided this year that I should buckle down and get this done. I’m proud to say that I spent about an hour working on the task and got it done.

What was the task? Taking all my vacation.

I’ve got some amazing benefits from Red Gate Software, and my holiday allowance is one of them. However I’ve often struggled to allocate out my time, and with travel and my concerns over keeping this site running, I usually carry over a few days. I often sell a few back as well, and I’m actually glad that there are limits on both of those actions. If left to my own devices, I might become much more unbalanced than I have been in the past.

I had a couple of weeks left, but to keep SQLServerCentral running in top form and ensure that you are getting information to help you improve your SQL Server skills, I can’t disappear on short notice. As a result, I’ve spread out my time off across the rest of this year and will be absent a number of days in December. Some is real vacation, some are days off that where I’ll have to decide what I want to do with the time.

I know far too many people in technology skip vacations or don’t plan enough time off. I’ve been trying to get better at this each year, and I’m happy to say that I’ve finally managed to use up all my holiday for 2014. Don’t be like me. Schedule your time off and take it. Recharge, refresh, and enjoy the rest of your life. Remember that you work to live, you don’t live to work.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.2MB) podcast or subscribe to the feed at iTunes and LibSyn.

Practical Problems with CI

I agree. Continuous Integration has issues in the real world, at least it probably does in many companies. The more tests you have, the more likely you refactoring or changes will break something. The more things break, the more people will look to leave them alone, deferring the "fix" into the future.

This defeats the purpose of CI. The whole idea is to keep your software in releasable condition. That way you can push it out if you need to.

How do you deal with the constant build breaks? The first thing I’d do is see if I mad a mistake and can fix my code. That should be the first reaction.

The next thing is stop for a moment and see what tests are failing. It’s entirely possible that the tests aren’t valid or aren’t needed. Maintenance of tests is an issue. If this is a test that regularly breaks, I might call a quick meeting and see if we should remove it.

The last thing is that I’d rollback my code and then go back to work on it. Perhaps there’s another way to solve the issue and not break the build.

This isn’t magic. It’s also not easy. However if you are keeping up with your CI process and it’s healthy, I believe it will pay off over time with better software.

SQL Konference 2015

My first speaking date of the new year is scheduled. I’ll be at the SQL Server Konference 2015, near Frankfurt, Germany. Red Gate arranged things, and I’m looking forward to heading over and renting a car to drive in Germany.

And speaking, of course.

I don’t know the agenda or details, or which session I’m presenting, but it’s exciting.

Slightly bittersweet as I won’t make SQL Saturday #358 in Albuquerque. I’ve been to the last two, taking a day to ski at Taos, but I can’t make it this year as I’ll be speaking that week in Europe.

T-SQL Tricks – Custom Templates

I wrote about the Template Explorer, which comes with the SQL Server tools and is visible in Management Studio (SSMS). It’s handy, but there are limited code items in there. What if I want more?

That’s easy. Suppose I decide that I often need to create procedures with the EXECUTE AS clause. I usually do this:

CREATE PROCEDURE MyProc
   @id INT
WITH EXECUTE AS OWNER
AS
BEGIN

-- do work
BEGIN TRY

  COMMIT
END TRY
BEGIN CATCH
  ROLLBACK
  EXEC uspErrorHandler;
END CATCH

END

It’s a basic template of stuff I do. Let’s stick this in our Template Explorer.

The first thing I do is go to the Stored Procedure folder. I can right click it and I’ll see this:

templates12

I choose template and a new one is created. I enter a name and I have a template. The first 6 templates here are defaults. The last one, highlighted below, is the one I created.

templates13

Now I right click it again and select Edit. At this point, it will open in a query window. This is just a file in my file system (under ), and like any other query, I can edit it. I paste in my script from above, and change a few items to parameters.

templates14

Now I can save this, and the next time I need this, just drag it into the main window and customize it.

templates15

Follow

Get every new post delivered to your Inbox.

Join 4,918 other followers