NoSQL Complaints

I think that NoSQL databases are a really interesting set of technologies, however they aren’t necessarily a panacea for many problems, including scalability. There are likely some problem domains where a particular NoSQL technology might work better than an RDBMS, and perhaps some situations where many types of databases would work well.

I ran across a piece that looked at some of the complaints that companies have with NoSQL technologies. It’s a generic piece, perhaps too generic, since a graph database is not like a document database. However it does talk about the issues that companies see, such as performing analytic queries. While a row-based RDBMS isn’t ideal for these queries, often the platforms work very well for a wide variety of reporting requirements.

I do think some of the problems people have noted have to do with the maturity of the systems. There are a lack of references for how to run a high uptime NoSQL database, not many data models to review that may or may not work well, and relatively few administrative techniques to maintain these databases. The problem isn’t that NoSQL databases can’t meet your needs, or run quickly, or provide many 9s of uptime, but rather many enterprise professionals might not know how to find the information to accomplish these goals. That’s something easily solved across time.

Ultimately I think the NoSQL world has a lot of maturing to do, and we, as professionals and media, need to start to differentiate among the various flavors of databases. I am glad to start seeing articles and blogs refer to graph databases when they cover a product like Neo4j rather than writing NoSQL. Slowly I am seeing MongoDB referred to more often as a document database, not a NoSQL database. It’s good to start to separate technologies and not confuse one with another.

One thing in the piece I did notice is that it’s likely that we will see more platforms and products incorporate both RDBMS and non-RDBMS technologies together. We see that in SQL Server already with the addition of columnstore technologies in the last few versions and Polybase to SQL Server 2016. I expect we’ll also start to see some of the capabilities of DocumentDB added to SQL Server, along with, perhaps, graph based technologies in the future.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.7MB) podcast or subscribe to the feed at iTunes and LibSyn.

Python Command Line Calls

There was a time I worked in a company any we lived in the command line. This was in the early 90s, prior to Windows v3.1 being released and we primarily used DOS on a Novell network.

We also had paper phone books on every desk for the 1,000+ people on the property. However, as you might guess, the phone books went out of date, and were only updated a couple times a year. We might get a new sheet, but mostly people learned to cross out a name and write a new extension in for the people they dealt with regularly.

However updates to the master list happened regularly, every few days. These were done in a small Lotus 1-2-3 file that an administrative assistant maintained. As a network person, I knew where this lived, and arranged a text export of this every night with a scheduled task.

Why text? Well, I’d come from a university where we had our phone details in a text file and would grep the file to find information. In DOS, I knew we could do the same thing with FIND. However, rather than write the find command with parameters, I put a batch file on our global share that called FIND with the parameters needed and the path to the phone book. I called this 411.bat. When I needed a phone number, I could type

411 “Andy Warren”

I’d get Andy’s name, location, and phone number back. It was a trivial piece of programming for me, but the rest of the network team, all non-programmers, were thrilled. I even added a /? check to return help information to the user.

With my playing with Python last week, I decided to do this for myself as well. I took my Python program to send tweets and changed it to send a tweet when the program was called, and to send the parameter as the tweet. The code looked like this::

import sys
import tweepy
def send_a_tweet(tweettext):
consumer_key = "X1GWqgKpPP4OuqWJZ7hw6"
consumer_secret = "QW3EkMHlyzF69AHn8DjyWyRG5CAQ0wjK9RqUZ2"
access_token = '146009-MqpeZFslj0VGgTSik1fq5klvJpqc1x6HAsiu'
access_token_secret = '8wIZlPbIqXtczc2LOsJgjMP3dCRRw5ajMvkjEspF'

auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)

api = tweepy.API(auth)


if __name__ == '__main__':

I placed this in a “\tools” folder that I have in my path. I also added a “tweet.cmd” file in this folder with this code:

python c:\tools\ %1

Since Python.exe is in my path as well, I can do this:

2015-11-06 15_25_19-Command Prompt

And I’ll see this on my timeline. I guess you’ll all be able to see this as well.

2015-11-06 15_25_26-Steve Jones (@way0utwest) _ Twitter


Why bother? Well, it was partially playing around. As I have been learning Python, I have mostly been playing in an IDE, solving small problems, but not really doing things useful. I also like the idea of command line tools, since I find them quick. Tweetdeck is big and bloated, and if I wanted to send a tweet from my desk, this is a quick way to do it. I could do a “readtweets” as well, and may.

However I also learned how to call Python programs with a command line, which is a good step to starting to build more useful programs that I can customize. This is also the start of me being able to schedule a program, and perhaps build more automation into my life with Python.

Mostly, however, it was just fun.

Test if a File Exists with Powershell

Another post for me that is simple and hopefully serves as an example for people trying to get blogging as#SQLNewBloggers.

Awhile back I wrote a PowerShell script (PoSh) to download the SQL Saturday XML files used for the Guidebook application. These contain a bunch of the information that I wanted to collect, transform, and keep around.

However the script wasn’t great. It basically looped a set number of times and re-downloaded the files. Not the more efficient solution, especially if I want this to run regularly.

One of the enhancements I wanted to make was check if the file exists, and if not, then download it. However, if it does exist, then I’ll skip the file. I know this means I don’t get updated files if schedules change, which is possible, but in that case, I can just delete the file from my desktop and go from there.

I made a quick search, and found a few links to the Test-Path cmdlet. Essentially you give this a file path and it returns true or false. Almost exactly what I need. This is what I added to my code:

if (Test-Path $DestinationFile) {

#do something


However I want to take action if the file doesn’t exist. In most languages, I’d add a ! in front to signify "not". However that doesn’t work in PoSh, just like > doesn’t mean greater than.

Another search showed me the -NOT operator. That’s what I need, but I can’t do this:

if (-NOT Test-Path $DestinationFile) {

#do something


Instead, I need to have a single expression for -NOT, which means more parenthesis. Not a big deal. I used this code:

if (-NOT (Test-Path $DestinationFile)) {

#do something


That worked great and now I only download the files I need. If I want to re-download (for newer events), I just delete those files and re-run the script.


This post came about when I started working on the script. It actually took longer to write this than find the solution, add it to code, and test it. That process was about 5 minutes.

This post took about 10 minutes to write. I also had a second post based on similar modifications to the script, so I’m did that in another 5 minutes.


A few of the links I used:

Just Say No

Just say no was a tagline from my youth. As I recall, it didn’t work well then, but I hope it will work better for a different audience and in a different context today. I was reading Brent Ozar’s note about one of the toughest things for consultants to do: saying no. While I certainly think this isn’t a problem for some people that can’t find work, I do think this is an issue, and one that isn’t just a problem for consultants.

I have worked with many people across my career, and one of the things that most of them have struggled with is saying no. Maybe it’s fear over being scolded (or worse) by a manager. Maybe it’s the simple desire to please others. Perhaps it’s because we simple are eternal optimists, and we think we can handle everything thrown at us. No matter what your reason, many people find the inability to say no plagues them throughout their careers.

I’ve learned to say no across the last decade. I probably say no to more things than I agree to accomplish. There are times that I worry about how much I decline to tackle things at work. However I have learned to deliver what I agree to, and deliver it well. I’ve learned that to do that, I can’t tackle every project, idea, or assignment. There are times I have to push back and refuse (politely) to do something.

It’s a luxury for sure. I have been successful in my career, and I’m not in a position where the loss of my job would wreck my life. However I also realize time is the most valuable resource I have at work. Time is also the most valuable resource I have in life, and I have to learn to balance my use of that resource to continue to be successful. Hopefully you learn that as well.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.1MB) podcast or subscribe to the feed at iTunes and LibSyn. feed

The Biggest Data Breech (For Now)

I keep reading the words “the largest data breach in history” in a variety of stories. In fact, given the tremendous growth of data acquisition, I’m guessing that this headline will continue to repeat itself over and over. I think I’m getting to the point where I’d rather just see a story say that xxx million customers were affected. At least then I’d be able to easily put some scale to the loss of data.

What’s interesting in this case involving JP Morgan is there are indictments being handed down, to at least two men that somehow participated in hacks that copied over 100million people’s data. JPMorgan admits 76 million households and 7 million small businesses were compromised, which isn’t 100, but perhaps there’s something I’m missing. However the data wasn’t just sold, but rather hackers used the information to market stocks to the individuals compromised. That’s an interesting level of sophistication, and a scary one.

Can you start to imagine criminals using the information intelligently to not directly sell the data but to make a secondary use of the information. Perhaps they will enagage social engineering by bundling the information with other data to perform some other attack on individuals? It’s entirely possible that we will see more sophisticated uses in the future as criminals work to evade or avoid the fraud detection systems that have been put in place.

I have no doubt that bigger data breaches are coming. Perhaps we could reduce the impact and frequency with better security frameworks and development practices, but I’m not sure that any company out there will place a high priority on security over ease of access and speed of development. I do continue to hope that market forces will drive companies to build better detection and protection mechanisms, and our vendors will build better security mechanisms into all platforms.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.4MB) podcast or subscribe to the feed at iTunes and LibSyn.

Visual Studio Subscriptions

Many of us that work with SQL Server do so exclusively through SQL Server Management Studio (SSMS). I find so many people really do the majority of their jobs with SSMS, Outlook, and a web browser. Even back in 2003 when I was a full time DBA, I probably spent the majority of my time in those three applications.
However I also see more and more people using Visual Studio and other tools to accomplish their jobs. The growth of new tools, like Powershell, the expansion of our work into BI areas, and more mean that more and more people are using tools besides SSMS to work with SQL Server data.
This past week there was an announcement that MSDN subscriptions were changing. At most of my jobs, I’ve had an MSDN subscription available to me. In fact, some of you might remember the large binders of CDs (and later DVDs) that arrived on a regular basis and contained copies of all Microsoft software. However many of you out there haven’t had MSDN available to you, or you’ve struggled to justify the yearly $1000+ cost, but you do want to work on your careers and practice with Microsoft software.
At first I saw the yearly cost of MSDN at $799, which is a pretty large investment. However as I looked to the side, I saw a monthly subscription, no large commitment, available for $45. That’s not an extremely low cost for much of the world, but it’s very reasonable in the US. It’s also a great way to build a setup that allows you to work with a variety of Microsoft technologies at an affordable cost. What’s more, you can stop paying at any time. Or start again at any time.
I know that it can be a struggle to invest in your own career, probably more difficult to find time than money. However this is a good way to get access to the various development and server tools for a period of time if you want to tackle a project or force yourself to learn a new skill.
I’m glad that Microsoft has moved to a subscription model for MSDN. I expect to see this subscription growing as small companies use a small investment that scales linearly with new hires to provide their employees with tools. I can only hope that many other vendors adopt this same model and allow us to rent our tools, and upgrade, for a very reasonable cost. I just hope they all let us backup and save our settings in case we interrupt our subscription for a period of time.
Steve Jones

Technical Debt

I was speaking with one of the development teams at Redgate earlier this year. They were working on a product, and had planned out a few sprints worth of work. Each sprint, called a train, was a couple weeks long, with specific goals and ideas to be implemented. That was all good, but I noticed that there was a sprint in the middle of the list that was devoted to technical debt.

Technical debt is a strange term. It’s one that many managers don’t understand well, often because the code may work fine. I ran across an interesting piece that looks at what the concept means, with what I think is a good explanation. We get technical debt when we sacrifice maintainability to meet another requirement. The piece also looks at the accumulation of the debt and why it becomes problematic later. Certainly the more debt that piles up, the mode difficult it can be to change code. Since we are almost always going to go back and maintain code, this becomes a problem.

I think the ideas given to keep technical debt under control are good ones. We should make an effort to clean code as we can, though not make it such a priority that we end up causing more work with constant refactoring. We do need to get work done. However the suggestions given require a good amount of discipline and buy in from management, and I’m glad Redgate tries to keep debt under control. I think our developers like the debt trains as well.

I thought the idea was pretty cool until I was looking for a feature to be completed and the technical debt train was running that week. I thought about complaining, but I decided to have patience and wait a week. After all, if the debt isn’t kept under control, I might be waiting much longer than a week for some fix or feature next year.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.4MB) podcast or subscribe to the feed at iTunes and LibSyn.

Branding Yourself for a Dream Job–Powerpoint and Questions

I had a few people ask about the deck, and I’ve uploaded it here: Branding Yourself for a Dream Job.

I also had a couple good questions at the end after people had left, so I thought I’d put them here.

What about a picture on my LinkedIn/resume?

I have had a few people ask this in the past, and I forgot to cover this in the talk. My only advice is that if you want to put one up, use a professional picture. Have someone take one.

Other than that, it’s a hit and miss prospect. We all have prejudices, and hiring managers/HR people are no different. If you’re a women, or old, or a minority, you might have some people exclude you based on your picture.

Personally I don’t see much benefit to adding a picture. Keep that information back until you need to disclose it.

Does it matter how many blogs I have?

No. However you will likely want to maintain them, so don’t tackle too much. I would suggest no more than 2, one for your career and one for something else.

Python and Tweepy

One of the projects that’s been on my list lately is to programmatically access Twitter for a few ideas I want to play with. Since I’ve been trying to learn some Python, I thought I would take a look using Python to update status and read status.

A quick Google search showed me lots of Python clients, but Tweepy caught my eye for some reason. I’m not sure why, but I ended up popping a command line open and downloading the library.

From there, I saw a short tutorial over at Python Central. I started by creating an app at Twitter for myself, which was very simple. Once that was done, I had a set of consumer tokens (key and secret), that I could use. Another click got me to the access key and secret. Note, the easy way to do this is over at

My first attempt was using this sample code.

import tweepy

consumer_key = “ABCDE”
consumer_secret = “12345”
access_token = ‘asdfasdf’
access_token_secret = ‘98765’

auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)

api = tweepy.API(auth)

public_tweets = api.user_timeline()
for tweet in public_tweets:

This gives me a list of my tweets. At least, the last 20.

2015-11-06 13_30_24-SDTIG - [C__Users_Steve_OneDrive_Documents_Python_SDTIG] - ..._Last_10_twitter_l

That’s progress.

I then went to make an update by using this:

api.update_status('Hello, Pyton')

However that returned an error:

2015-11-06 13_50_53-Cortana

Hmmm, I double checked the API, and this should work, but I’m guessing there’s another issue. I searched, and found there’s a bug. However, I should be using named parameters, so no big deal. Change the code.

api.update_status(status='Hello, Pyton')

Now it works.

2015-11-06 13_53_06-Steve Jones (@way0utwest) _ Twitter

This is the first step for me to look at building an app that might send some tweets on my behalf, perhaps with the data stored somewhere, like, I don’t know, maybe a database?

Shrinking the Budget

Most of us rarely have to build or manage budgets in our organizations, but almost all of us are affected by the budget process. It’s tempting to ignore budgets and just do your job, but sooner or later you might find yourself arguing for more funding for a project, training, or even hardware.

I ran across a short piece on 10 ways to shrink your IT budget and found it a bit scary. I would bet that many of you have had conversations, or been affected by decisions, that follow some of the advice in the piece. The push for open source, virtualization, hosted (or cloud) migrations, or more can cause stress and anxiety when your job is making sure the servers run without complaints from end users.

I don’t expect that all of these would be followed in a specific organization, but some of these might be proposed to you. What I would recommend is you understand the reasons why, or why not, you might adopt any of these ideas. Certainly the time lost from retraining people or rewriting code can overwhelm the savings for many years, but even seemingly smaller changes, like changing priorities can affect the way your clients and customers view IT. It seems that sometimes budgets get changed to provide a short term view that technology spending is being managed efficiently, only to find out later we will remove all our savings by undoing a poor decision.

I’d suggest that you approach budget issues with transparency and honesty. Take a hard look at your costs and determine what items are really needed, and what items aren’t. However, I’d also urge you to carefully consider whether it’s really valuable to save money by not taking care of your staff. While labor is an expensive part of your IT cost, good staff are worth much more than they cost, often by an order of magnitude.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.0MB) podcast or subscribe to the feed at iTunes and LibSyn.