The Relentless Cloud

The march to the cloud is inevitable. No, I don’t think all your services will be run from a cloud service, but I do think that some of them will be. Maybe 10% for your company, maybe 90%, but likely somewhere in between. In the efforts to reduce costs, and maybe more likely, the idea of shifting costs to expenses, I would bet that most organiztions will take serious looks at the cloud in the next decade for a significant portion of all new services. They might not move most applications, but they’ll think about it.

I saw a note recently on Geekwire that Azure is estimated to exceed  Amazon Web Services (AWS) by 2019. Most of us might not care about the race between these two tech giants to build large cloud infrastructures, but their capital expenditure is based on the idea that your organizations will want to use some of those services. They have good reasons to think this with surveys and conversations with many of their customers.

In the absence of changing tax policy in many nations, it also makes more sense to make move costs to expense, rather than capital expenditures. And that’s the way things are going, at least according to what CIOs are saying. More and more applications and services will be moving to the cloud in the next few years. Up to 30% from the current 14% of applications today. As I’ve seen more and more entreprises, public and private, moving mail and web services to outsourcing providers like O365, I also think we’ll see other applications moving.

I don’t think every application is suited to an outsourced area, but I do think there’s a case to be made for many of them, including databases. For those with privacy or security issues, cloud providers are working to mitigate the issues. Amazon has built separate data centers for government services, and I believe Azure is (or has) done so as well. The challenges that exist can be solved, and many of the risks or concerns will be mitigated, though you’ll still need to evaluate your situation on a case by case basis, which will change over time.

My recommendation: learn about the cloud.  Really learn, don’t just assume you know. It will be good for your career.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 3.9MB) podcast or subscribe to the feed at iTunes and Mevio .

Going On-Prem?

When I started in this business, every company owned their own servers, and had their own data center. Now, some of those data centers were closets (including that for one state government Senate). Some were offices-converted-to-computer rooms, rarely with separate air conditioning until I requested it. Some were actual data center rooms, though usually for larger companies. The idea of putting your computing resources off-site was seen as ludicrous.

As time passed, the idea of using a co-location facility grew. In fact, SQLServerCentral went from a couple machines in a residential basement to a locked rack at a facility in Denver. Many companies started to use third party data centers, and in the late 90s and early 2000s, this was common, with facilities springing up. Full time employees will often racked equipment and visited data centers when they needed physical access.

This changed over time as virtualization grew, and more and more companies started to rent a VM, rather than purchase a physical box. Some still used physical machines, but they rented the machine, allowing a third party to setup and mount the equipment, sometimes even installing the OS and then providing access to the lessee. We’ve come to the place now where we have cloud services that are often just platforms or services, where we have no idea of the underlying equipment. In the case of things like Salesforce, Azure SQL Databases, or Amazon’s RDS, we don’t even care, just asking for a certain level of performance.

Is this the future? Will we now be asking ourselves if we “want to go on-prem” as the default question? I ran across the blog linked, which is from a SaaS perspective, that is trying to always get customers to use the service (or platform) and never have a local install. I know Microsoft would like to do this, especially for Azure databases as they make more profit on compute services.

Some of you think this is crazy. In fact, I know plenty of industries that struggle with this from a regulatory standpoint. However, I’d point out that the idea of letting someone else run your email system was seen as crazy 20 years ago. Today many people (myself included) would never think to install Exchange or any other email software. We would always purchase this service from someone else.

Tim Mitchell had a podcast interview recently talking about whether the on-premise data warehouse is dead. While I think the Azure Data Warehouse has some great advantages, and is worth considering, I’m not sure the local DW is dead. In fact, for lots of tasks I perform, including software Continuous Integration builds (.NET or database), I prefer to have some local resource doing the work. Especially when I can’t predict the number of times I’ll build and that task is easy to scale on local machines.

I don’t think that we are at the point where we’d put more databases in a cloud service than on premise, but I do think that for new applications, it’s a valid question to ask whether the database and application could be hosted in the cloud.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 4.7MB) podcast or subscribe to the feed at iTunes and Mevio .

Cloud First

SQL Server 2016 is the first “cloud first” release of SQL Server, as told to us by Remus Rusanu, former Microsoft SQL Server development team member. The features and enhancements in SQL Server 2016 have been running in the Azure SQL Database cloud for some time before they will be packaged up and released as the on-premise SQL Server product that most of us expect to work with.

There are a number of items about this worth discussing, but one stands out to me. The idea of using large numbers of feature flags and deploying code to the “cloud” for use by customers and internal Microsoft people is interesting. On one hand, it’s really just a faster way of having beta software examined by users other than developers, with no install/uninstall/upgrade for users. Speed is good, but being on the bleeding edge and having systems break isn’t necessarily what any of us want. However the use of these flags to turn features off quickly means that disruptions can be minimized for individual customers.

Some of the engineering process changes made to be cloud first were important for Microsoft to have one main branch of code for SQL Server. Having a separate branch for the cloud and on-premise versions had to be inefficient and problematic. However, that’s not something we, as customers, care about. We just want the database platform to work, wherever we run may run it.

I do think that having features come out in Azure, whether private testing, public preview, or general availability is good. The people that can test these features give feedback quickly, and the rest of us aren’t affected by the problem code. More importantly, the developers at Microsoft get the chance to learn more about how the features will be used and tune them before a very wide set of on-premise customers get code. Personally I was glad to see things like Row Level Security (RLS) appear in Azure first (Mar 2015), and come later to the on-premise version (SQL Server 2016).

I really expect in the future that almost every feature that is added to the SQL Server platform will be run in some form in Azure before it’s released for on-premise use. In fact, I’d be surprised if features are added to the on-premise without 6-12 months of testing in the cloud. That’s good, and I hope it leads to an increase in code quality for those of us that will run SQL Server on our own hardware.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 3.0MB) podcast or subscribe to the feed at iTunes and LibSyn.

Two Days Off

I almost couldn’t believe this when I saw the article. The Verizon Cloud is shutting down for 48 hours. Apparently they have maintenance scheduled for this weekend and notified their customers that their virtual machines will be shut down early Saturday morning. There are some legacy Verizon cloud-type services that will be available, but the platform they’ve been pushing to customers will be down.

This isn’t good news for Verizon or their customers, but it also doesn’t help the cloud overall as a service. This outage reinforces the idea that reliability isn’t necessarily better for vendors than individuals. If costs for the cloud are anywhere near that of on-premise hosting, this event would certainly make me think twice about moving anything really important in my organization to a single cloud vendor.

I suspect that most cloud vendors have outages like this, but they don’t shut down their entire clouds. When a large amount of maintenance is needed at a data center, most vendors would migrate customers to a separate data center or another part of their cloud while they perform their work. Either this mainenance is a major change to Verizon’s entire infrastructure that can’t be staged on just a part of their system, or they’ve poorly planned their architecture and maintenance.

Either way, this weekend will certainly be a good DR test for enterprises that might have important applications hosted with Verizon. It will also be a chance to test how these clients notify their own customers of potential issues or how they respond to problems. I wouldn’t want to have an application hosted with Verizon as it would be a lot of work for me and likely a weekend away from family, but I know I’d learn a lot about how well I’ve prepared my own systems for fault tolerance.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.2MB) podcast or subscribe to the feed at iTunes and LibSyn.

Load Balanced IIS Machines

I noticed a contest this week while working on the Database Weekly newsletter. It’s the Cloud Hero contest, with the chance to win a Surface Pro 3. I could always use another device, or at least a device I could give away, so I decided to enter.

There are a few things you can do, all of which are interesting to me in terms of a direction that I, and Red Gate, want to move. I don’t know if Azure works everywhere, but we are considering moving SQLServerCentral, or perhaps parts of it, to Azure, so this was a good chance for me to try out some new Azure stuff.

I’ve messed with a few things in Azure, but mostly on the PaaS side. That interests me more, and I’ve done little with IaaS. I certainly haven’t really worked with IIS much in Azure. I decided to go through the VM setup, to create two IIS machines, load balanced on the same URL. I used this blog post with a cartoon and demo to run through the process.

It was a bit more than 10 minutes, mostly because some of the allocation stuff in Azure took time, and the responsiveness from the VM in Azure was slow. From the time I connected to the time Server Manager popped up was over two minutes for each machine. Since I was going through some of the steps sequentially, that meant it was slow to get going.

The video and the portal bring to light some of the issues of Azure. It’s a great tutorial and I was able to get the two machines load balancing IIS in 20 minutes (or less). It was surprised how quickly it went, but I also had to stop and think. The load balancing and cloud services are different now than they were when the post was written.

I’m sure that’s the case with lots of Azure content. In some sense, this means that we will have lots of issues with people trying to learn how to use Azure as they’ll find content and information that is woefully out of date, sometimes quickly. I wonder if we need to think about having some code on blogs for Azure that marks the content as potentially out of date after it’s been out for 6 months.

It’s a challenge to keep the content up to date, and luckily the changes weren’t too different in the portal.

I am glad that I was able to get to IIS machines up and load balanced, delete them, and bring them back. That makes me think I may find some use for this Azure stuff, yet. I have a few projects in mind, including rebooting my personal site. Perhaps Azure will be the place I give it a go.

The Next Five Years

On one hand, the next five years don’t seem too far away. Five years, that’s a year more than most of us spent in high school or college. On the the other hand, 5 years ago was a long time in the computer world. Azure first gave us the ability to create a database, as opposed to just a key value store. SQL Server 2008 was the current version of the platform. SQL Saturday #26 was held the first weekend in October in Redmond. 8GB was the large size for flash drives and the iPad hadn’t started the current tablet revolution.

The world of data has grown dramatically in 5 years. Our cell phones and other devices are drastically changed the amount of data that is collected and consumed. Advances in bandwidth have almost removed the need for us to move all but the largest sizes of data without any physical media. If we look to the next five years, does anyone think they can accurately predict how much storage we’ll maintain in our pockets or how fast we’ll expect to move data on a daily basis?

I ran across a prediction of surprising things in the next five years, and while I’m not sure I think they will all come true, I do think that similar things will come to pass because of two things: amazing increases in computational power available to any individual and tremendous amounts of data. That’s no great surprise, but I am excited. I can’t help but think that there will be so many opportunities for data professionals that most of us will be gainfully employed for decades.

I do think, however, that the mundane, easy jobs of administering individual instances, checking logs and backups, and setting security will be few and far between. Unless you know how to do those things for thousands of instances at a time.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.5MB) podcast or subscribe to the feed at iTunes and LibSyn. 

The Subtle Push to the Cloud

With SQL Server 2014 released, there’s the temptation to upgrade for many DBAs. However the licensing costs and debatable improvements in the product will temper the DBA’s enthusiasm with the reality of the ROI seen by management. While reading about the licensing changes, I also saw this note from Tom LaRock, where he wrote about the features most of us aren’t using. It made me think about upgrades, and perhaps the strategy Microsoft is employing.

As Tom mentioned, the features not being used are Enterprise Edition features. This prevents many of us from upgrading to use them because Enterprise Edition is so expensive. Actually, even Standard Edition is expansive these days, given the per-core licensing, and I suspect lots of companies with SQL Server 2008, 2005, even 2000 are debating whether or not the upgrade is really worth the cost.

The piece also mentions that Azure is much less expensive in the short term, and you should consider it. Of course at some point the money you pay will be more than buying a license and a server, but that assumes you run them for the same length of time. You’ll likely upgrade at some point, and if you upgrade both hardware and software, the cloud starts to look more attractive.

It’s not simple to move into the cloud, nor is it appropriate for some applications, but it does seem that Microsoft would prefer to have most companies running in Azure, on a single platform they run, they patch, and they support. Many companies might prefer the same thing, though I do worry that we might find the on-premises version of many products becoming second class citizens. Vendors will spend less on development and support if too many customers move to the cloud.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.0MB) podcast or subscribe to the feed at iTunes and LibSyn. feed

The Voice of the DBA podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at

The Data Platform Vision

Recently the Microsoft Data Platform blog published a piece on what drives Microsoft’s vision. It’s an interesting look at the ways in which data is being used in some businesses as well as the variety of types of situations that must be dealt with. There is this idea that the value derived from having lots of different data was once only used by specialized companies or applications. However today, it’s being more widely analyzed and accepted by more businesses. Of course, the vision is that Microsoft can help those businesses.

In the cloud.

I know this piece is more marketing than technical, but I have concerns that Microsoft is not driven by what customers want and need, but is instead seeking to drive customers to their own tooling and offerings in the Azure cloud. While I do like many things about the cloud and think it has a place in the future of many companies, I also think that there are still lots of opportunities, as well as demand from customers, for on-premises solutions.

I worry that the focus of Microsoft, which can be tightly bound in one idea as they eye more sales revenue, is driven with the idea that everyone, and all applications, will some day be in the cloud. I can’t see that in the future for many businesses, and certainly hope that Microsoft recognizes this. Otherwise many of our hybrid solutions that push some data into the cloud might be backed by PostgreSQL or some other platforms as the on-premises database.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.2MB) podcast or subscribe to the feed at iTunes and Mevio . feed

The Voice of the DBA podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at

The Cloud

I like the cloud. It’s very handy for a lot of the things I do. I like having backups restored for my iPhone upgrade. I appreciate having my documents synced through Evernote, Dropbox, and OneDrive (formerly SkyDrive) automatically moving to new machines. These services are incredibly convenient and handy. I expect that my mail will be accessible from multiple machines and my blog posts can be created, edited, and published from anywhere. Many of us work in a similar fashion, using the cloud as a service construct in much of our lives. In fact, many of us expect our own knowledge will be supplemented by the cloud. We use Google to search, getting us syntax,  solutions, or ideas from sources that we expect to just be there.

Someone was talking to me about their concerns with the cloud recently. This person thought the cloud would be a fad that passes quickly and businesses will get back to owning, hosting, and managing their servers from the bare metal up to the network connection. I disagreed, and not because I use the cloud personally.

I remember setting up a web server in 1996 for a company. I remember installing an email server in 1999. I would never do either of those things again for my own company, and I wouldn’t expect it from most companies. These days we’ve learned that email can easily be handled by a third party and more and more companies are using email services instead of managing their own systems. Web servers are mostly a shared service that we rent from any number of other companies, sometimes even stitching together a presence across multiple providers. Even many companies that want to manage their own host operating system are turning to third party companies to manage the hardware and networking. More and more we accept renting the things we need from others.

We’ll get there with databases as well. Not all databases, but more and more of them will be hosted at third party companies. Whether this is in some type of VM environment or a service that hosts data, it will still be hosted at a third party. We will have concerns and we will have reasons to not do this, but I think for more and more of our data, we won’t care.

Companies have lots of systems in place. They have legacy investments in infrastructure and facilities. In those cases, it’s hard to justify the cost of cloud computing. However as we grow into new areas, or with new companies, we’ll have to re-evaluate whether it is a good idea to continue to make those investments. As the management of software grows to allow the quick scale up and down of assets, I think we’ll be much more likely to consider moving applications into the cloud than ever before.

We’ll never be 100% in the cloud, or out of it, in the future, but increasingly, we’ll be partners with the cloud.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 3.4MB) podcast or subscribe to the feed at iTunes and Mevio . feed

The Voice of the DBA podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at

T-SQL Tuesday #48–Cloud Atlas

tsqltuesdayI never saw the movie, though it’s on my list to watch sometime. I love the Wachowski’s as directors.

This month Jorge Segarra is hosting. He is better known as the SQLChicken, and his theme this month is Cloud Atlas. He’s asking us to write about the cloud and what we think of it. What’s our take on the cloud. He wants to know if we’ve used it, what we think, probably how we even define it. It’s a great question with all the media and hype about the cloud, especially for data professionals.

This is a monthly blog party, where bloggers write on a specific topic every month, on the second Tuesday. I’ve got a complete list of topics on this blog, and to participate, do this:

  • write a blog
  • publish it on the Tuesday, November 12, 2013 (for this month)
  • be sure a trackback or comment is on Jorge’s blog invitation
  • include the T-SQL Tuesday logo, linked to the invitation.

That’s it. Write, and enjoy.

The Cloud Atlas

What is the cloud? It’s been hyped in commercials and numerous articles, and to some extent I think we don’t have a good definition. We have PaaS, IaaS, and SaaS, all of which are legitimately the “cloud” for different people.

As far as I’m concerned, the cloud really is any computing service that is provided by a vendor. This means you don’t own the hardware and rent out either a machine (physical or virtual), a service, a set of APIs, etc. This exclude co-location, but anything else from renting a VM to using IFTTT, is cloud computing.

With that in mind, what do I think? There are certainly problems with the cloud in terms of security and recovery of data. However I think the promise of renting out computing resources (machines/platform/software/services) in a scalable fashion is amazing to me.

Do I use the cloud? Sure. I have Evernote, Skydrive and Dropbox to keep my data in sync. My family uses OurGroceries to manage a shopping list. We daughter is hooked on Spotify, which alleviates the need to keep much music on her device. I even have backups of data on Glacier.

On the career side, we (SQLServerCentral and Red Gate) offer AdventureWorks on Azure for users to play with. I’ve done minor work in Azure, and spun up a SQL instance on AWS. I haven’t actually done anything more than test, personally, but certainly the cloud is useful for SQLServerCentral. We host our database server and web server on virtual machines we rent from Rackspace. We use a Content Delivery Network for some videos and YouTube for others. We’ve dramatically lowered costs and increased scale with the cloud.

The cloud works.


For some applications.