The Data Dewey Decimal System

As a small child, I learned about the Dewey Decimal System in school. Those were the days when you wanted to look up some information or check a fact, you needed to peruse the dead tree books in a library, with information organized by the Dewey Decimal System in the US. At least, all libraries I used in school adhered to this.

These days we usually use a computer of some sort for learning, research, or really most any work with data. Often I start with Google to find my way to the source of information, but that’s not something that necessarily works well with finding sets of data. It certainly doesn’t work well within a an organization.

I saw recently that Microsoft announced the general availability of the Azure Data Catalog, which is designed to provide a catalog of data sets. In essence the Data Catalog is an index of the data sets that might be produced by your organization, with the information about the data filled in by the producer of data. Users that are looking for data can query the catalog instead of asking coworkers, wandering through the enterprise databases, or even relying on their own memory of where data might be located.

At first this seems silly, after all, don’t people inside of an organization know where data is kept? Don’t they learn the servers, databases, and connection methods? Certainly many do, but as with the pace of change these days, as well as the rapidly growing number of ways to publish data these days, it’s entirely possible that many people aren’t aware of all the data sources available inside of an organization. Even at Redgate Software, with a few hundred employees, it is fairly difficult to keep track of what data exists in which location.

The functionality of the Data Catalog seems a bit basic, and really almost like an extension of adding extended properties to various tables. Certainly things are centralized here, which is good. There are also ways to add other sources, such as SSRS reports, files, and even other relational sources. I’ll have to experiment a bit and see what’s available, and I might encourage you to do the same. The product relies on crowdsourcing, which can go really well, or really poorly, depending on how cooperative your crowd is.

In any case, I do like the idea of having a central catalog that individuals can update as they produce data sources for others to consume and change what’s available. If it works well, with good searching and tagging, it might eliminate some of the redundant work often performed to surface data inside of any organization and let employees know how to find the answers to their questions.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.9MB) podcast or subscribe to the feed at iTunes and LibSyn. feed

The Machines are Learning

One of the technologies that Microsoft is promoting heavily is machine learning. This has rapidly gone from a technology that I heard little about to being in multiple keynotes that I’ve seen at large conferences this year. It almost seems that I can’t go a week without seeing some sort of machine learning article, announcement, or reference.

How applicable is machine learning for most of us? I’m not sure, but banks have certainly taken advantage of machine learning to reduce their risks associated with fraud and their systems work well. In fact, given the ways in which I travel, I’m amazed that I don’t get more calls about fraud related to my card use. With me in a new state almost every month, on a very irregular schedule, sometimes buying computer supplies far from home, I get called by a bank once or twice a year.

Across the ten or so calls I’ve gotten in the last few years, only one questioned legitimate purchases. The rest caught fraud on the same day that someone tried to use my card without my knowledge. That’s a very successful rate of both true positives, and a very, very low rate of false positives. I’m impressed.

If you haven’t played with machine learning, Microsoft has made it easy to give a try in Azure. You don’t even need to put in a credit card to get up and working with machine learning. Whether it’s applicable to your industry or not, you’ll have to decide, but I do think that lots of tedious analysis that humans do now could be better done by a machine.

At least after one of us humans has configured the algorithms and trained the machine to recognize patterns. And, of course, with one of us technologists periodically tuning the system to work better  and monitoring the analysis as data changes.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.6MB) podcast or subscribe to the feed at iTunes and LibSyn.

Crossing Azure Borders

The speed at which Azure is evolving actually amazes me at times. I know that the Azure platform is a focus for most, if not all, groups at Microsoft. The breadth of changes occurring every quarter continue to enhance capabilities to the point where Azure is a viable platform for more and more applications all the time. It’s not for everyone, but it does work well for more and more organizations all the time.

I’ve been watching the Azure SQL database evolve, and been doing a little work with it for a few years. It has grown from a fairly limited RDBMS to a full-featured database, albeit with limitations. Some of those issues have meant that tools or techniques I’ve been familiar with haven’t worked well, if at all.

One of those items was the ability to query across databases. Previously I’ve had to open two connections and pull data to a client to combine it. However just recently that with the elastic database query in Azure now supports queries across databases. This is cool, as we can now build applications that easily query across shards, or even just across different databases without requiring additional client work.

The list of features in an on-premise SQL Server instance that aren’t in the Azure SQL Database continues to shrink. While there’s more work to be done, I think we can start to experiment with systems and see where the Azure platform might support our needs. Certainly our business people like the ongoing rather than capital expense nature of cloud platforms.

We should be sure we understand where, and where not, we could use an Azure SQL Database. I’m curious if any of you have had success with Azure and what types of applications you use the platform to host.

Steve Jones

The Voice of the DBA Podcast

Listen to the MP3 Audio ( 2.4MB) podcast or subscribe to the feed at iTunes and LibSyn.

Upgrading a SQL Azure Database to v12

Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.

I was looking to test some of the Azure v12 capabilities, but I needed to upgrade a database. Here’s a quick look at how.

In the old portal, you’ll see your databases.

2015-08-14 13_49_10-SQL Databases - Microsoft Azure

I love that there’s no information on database version. I’m sure the Azure people love this, but I don’t. Very, very annoying.

If I pick one of these, and go to the “Quick Glance” items on the lower right, I’ll see the “update to v12” message.

2015-08-14 13_49_34-SQL Databases - Microsoft Azure

However in the new portal, it’s different. At first glance it looks the same, but zoom in.

2015-08-14 13_51_00-SQL databases - Microsoft Azure

Now you can see the icons are different. I couldn’t read the 12 at first, but I did notice the color difference.

2015-08-14 13_52_12-SQL databases - Microsoft Azure

Now I can go into the properties of the database and see those. Mine said v2 here, which was strange. Anyway, pick that.

2015-08-14 13_54_02-Movies & TV

You’ll get a new blade, which will show you the upgrade benefits. Select “Upgrade this server”.

2015-08-14 13_54_07-Latest SQL database update - Microsoft Azure

Now you get a blade to upgrade the server.

You must type the server name, not the database name, to upgrade. Note that all databases on this server will be upgraded. Be aware of that. However, it’s not the full server name.

2015-08-14 13_54_48-New notification

It’s just the host, but a tool tip will prompt you if you mess that up.

2015-08-14 13_55_00-

Once that’s done, the upgrade will start on the blade. You can close this.

2015-08-14 13_55_07-Movies & TV

The upgrade will also appear in the main blade for the database. These are scheduled, not real time.

2015-08-14 13_55_14-Movies & TV

However, within about 3-4 minutes, mine changed to “upgrading” from the schedule notice.

2015-08-14 14_00_05-Movies & TV

This was a small database, with just one table in it. However the complete upgrade took awhile. The database is available and usable while this is happening, but it does take time to complete. I think mine took about 15 minutes before it was done, but I wasn’t watching the entire time, so I could be wrong.

Once it’s done, it has the new version

2015-08-14 14_04_30-Mytester - Microsoft Azure

Easy enough, and if you need to upgrade your systems, here’s a simple way. If you have lots of them, I’d use PoSh.

SQLNewBlogger

This post didn’t take that long to write. I had the database ready, and I took a few screen shots then started the upgrade. I wrote most of this while waiting for the process to complete and then went on to other work and came back for the final screenshot.

Easy enough, and you should practice this and document it. You’ll have some idea of how to use Azure if someone asks you to later along with some notes for yourself.

References

None

tSQLt in Azure SQL Database

I was excited to hear about the v12 Azure databases supporting CLR assemblies. Mainly because I’ve been doing testing work with tSQLt and wanted to run tests in an Azure database.

I upgraded a SQL Azure database to v12 and connected in SSMS. My first step was to open the tSQLt file.

2015-08-14 14_27_22-Start

I had the file open, connected to Azure. This was the easy part. Now I need to actually send the batch across and compile the code:

2015-08-14 14_27_33-Start

My next step was to execute it. However that didn’t work as well as I expected.

2015-08-14 14_27_58-New notification

There are multiple errors here, but it’s possible that one error causes others. I pinged Sebastian Meine, the creator of tSQLt about the External_Access and he noted there was only one method that needs it.

So I decided to make a change. First, a search.

2015-08-14 14_28_18-tSQLt.class.sql - dkranchapps.database.windows.net,1433.Predictions (sjones (54)

That got me the assembly installation.

2015-08-14 14_28_28-Start

I decided to try and change this to something that’s contained inside the database. Since Azure is a bit of a black box, I thought safe was the way to go.

2015-08-14 14_28_42-Start

With that change made, I compiled the entire file again. This was the only change I made.

2015-08-14 14_30_18-Movies & TV

That’s cool, but does it work? I connected in Object Explorer and then opened SQL Test. I saw my database, and I could create a test.

2015-08-14 14_31_36-Cortana

However I got an error on the connection and creation of a procedure.

2015-08-14 14_42_31-Movies & TV

Even from Object Explorer, I refreshed the procedures, but got this:

2015-08-14 14_42_44-Microsoft SQL Server Management Studio

This appears to be a SQL Server 2014 RTM error. Actually I was on a CU, but not SP1. It is supposed to be corrected in SP1.

However the procedure was created, and I could alter it.

2015-08-14 14_43_50-Movies & TV

More importantly, I can execute it.

2015-08-14 14_44_04-SQLQuery4.sql - dkranchapps.database.windows.net,1433.Predictions (sjones (52))_

If I change the code.

2015-08-14 14_44_23-Movies & TV

Now it fails.

2015-08-14 14_44_33-SQLQuery4.sql - dkranchapps.database.windows.net,1433.Predictions (sjones (52))_

It appears that tSQLt can now work in v12 databases in Azure, so start adding those unit tests to your database projects.

Pick the Right Database in Azure

Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.

Another quick post, because this is something that’s annoyed me regularly. Hopefully writing this will help me make fewer mistakes.

Let’s suppose you have an Azure SQL Database. You go to connect from SSMS, as you do every day.

2015-08-14 12_30_11-SQLQuery3.sql - ARISTOTLE.SQLServerCentral_Trunk (ARISTOTLE_Steve (57)) - Micros

you log in, and you’re connected. But your default, by default, is master. This probably happens to you when connecting to many on-premise SQL instances. What do you do?

2015-08-14 13_21_01-Cortana

Most of us either click the drop down, which you can do, and even in Azure, you’ll see all your databases.

2015-08-14 13_21_10-Cortana

You can also enter a USE statement (which I do as I like the keyboard). However when you run that, you have an issue.

2015-08-14 13_21_34-SQLQuery4.sql - o8c1w0tm5u.database.windows.net,1433.master (sjones (59))_ - Mic

These databases, while they may be grouped on the same server for communications, do not necessarily reside on the same instance. Therefore you cannot ever assume you can change from one to the other with a USE. In fact, you need to actually change your connection.

The way to do that is to click “Options” on the connection dialog and then choose the second tab, Connection Properties.

2015-08-14 13_21_46-Connect to Database Engine

Here you can specify the database to connect to. Once you do that, everything is great.

SQLNewBlogger

This took about 5 minutes to write. I changed my connection back to master, reshot the screens as I reconnected, and things worked.

You can do this. Show your next manager that you’re constantly working on your SQL Server skills.

References

No references here. This is something I should know well enough.

Create a SQL Azure Database–Old Portal

Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers.

I worry about beta software, so when I wanted to create a new Azure database, I stuck with the old portal. Call me nervous, but the new portal worries me at times. It works, but since the default is the old one, I often use that one.

You can use either, but here I’ll show you how to create a database with the old (which will be gone someday) portal.  It’s simple and easy, and this only takes a few minutes.

Creating a Database

Here’s the “old” portal. I wish it had a major version number, because that’s what it needs.

2015-08-14 13_36_27-Movies & TV

I’ve got a few things running here. In the lower left corner is a “New” button, which I can pick. When I do, I choose Data Services and get this.

2015-08-14 13_36_42-SQL Databases - Microsoft Azure

I select SQL Database and then click “Custom.” You could click quick create, but I wanted to show more options. There aren’t a lot, but you can put in a few.

You get the settings page. The first thing you need is a database name. You can pick your subscription as well as the size of the system. Be aware of charges when you pick sizes. S0 is not cheap.

2015-08-14 12_21_27-SQL Databases - Microsoft Azure

I had an existing server, so I used that, but the system will create one for you if you need it. This is really the name you connect to, so if you care, create one first and then use it here.

The next screen is really for the main admin account. Enter a decent password and save it. You can reset this form the portal once it’s up, so don’t be too concerned, but build good habits. Don’t use the same password everywhere and use a password manager so you pick strong ones.

2015-08-14 12_22_28-Store

Once that’s done, create the database. It will take a little time, but you’ll see a moving “Creating” item in the status. There’s no percentage, but I think it was about 3-4 minutes for me one afternoon.

2015-08-14 12_22_39-SQL Databases - Microsoft Azure

Once it’s done, the database list will show you the database as online.

2015-08-14 12_28_16-Store

If you go to the dashboard, you’ll see  a number of items listed.

2015-08-14 12_28_47-Greenshot capture form

Scrolling down, you might see the important item that you care about. How to connect. The server is listed, and if you’re using an app with ADO, PHP, OLEDB, etc, you can click the link and get the strings you will use.

2015-08-14 12_29_01-SQL Databases - Microsoft Azure

That’s it. Simple, easy, and ready for you to use.

SQLNewBlogger

I was actually working on something in the new portal and wanted to go back to the old portal to see how things worked. As I waited for the main screen to refresh, I though, hey, write a post.

So I captured screen shots. It took maybe 10-15 minutes to type this up.

References

None used.

Azure SQL Database – Link User to Login

I’m writing this post as a way to help motivate the #SQLNewBloggers out there. Read the bottom for a few notes on structuring a post.

I created a login for an Azure SQL Database, but couldn’t log in. I would get this message, which makes perfect sense.

2015-06-23 14_52_41-Connect to Database Engine

Just like an on-premises instance, I need to link a user to the login for access. However, unlike an on-premises SQL Server, I can’t willy-nilly change databases. If I do this:

Use SQLServerCentral GO

I’ll get this message.

2015-06-23 15_00_31-SQLQuery3.sql - mhknbn2kdz.database.windows.net,1433.SQLServerCentral (jt (58))_

I can’t switch, I need to reconnect. In this case, I use the same login (an admin level) and reconnect to a different database. This certainly can make scripts more cumbersome, and imply that your work, whether through T-SQL or PoSh, needs to include the connection strings for the correct database. In fact, I might recommend PoSh over SSMS for this reason.

Once I’ve connected, I can use standard commands. In this case, I’ll use this code:

add user to SQLServerCentral database connect to database CREATE USER sscdev FROM LOGIN sscdev; add a role for ddl admin EXEC sp_addrolemember db_ddladmin, sscdev;

Now when I click connect with my user, I see this:

2015-06-23 15_02_22-SQLQuery4.sql - mhknbn2kdz.database.windows.net,1433.sqlservercentral (sscdev (5

Just what I need.

SQLNewBlogger

This was a second post as part of the previous one. I was adding a user and login as part of a bit of work and when it didn’t work, I captured screen shots and split this part from the last one.

Five minutes writing this one, probably no more than five minutes slower in running the code to capture the images.

You can do this.

References

Azure SQL Database – Adding a Login

I’m writing this post as a way to help motivate the #SQLNewBloggers out there. Read the bottom for a few notes on structuring a post.

I am trying to slowly do a little work in the Azure SQL Database world, building some skills that I can use, and I can teach you a bit more about. One of the things I needed to do lately was add a new login to my system, which isn’t intuitive in the portal.

I did go to the Managing Databases and Logins in Azure SQL Database to get some help, but it wasn’t completely clear how to do this.

I setup a database recently for a test project, but I wanted a different user for this database than others I have on this particular Azure SQL Server. I am using the new (2015) portal, and there didn’t seem to be a good way to add a new login, so I connected to do this in SQL. The reference below showed me the standard “CREATE LOGIN” statement I’d use with an on-premise instance, so I tried to run that.

CREATE LOGIN sscqa WITH PASSWORD = mypwd; GO

I got this:

2015-06-23 14_45_19-SQLQuery3.sql - mhknbn2kdz.database.windows.net,1433.AdventureWorks2012 (sjones

OK, no issues. I changed the connection to the use the master database. I used the options after clicking the “Change Connection” button and put in master.

2015-06-23 14_47_06-SQLQuery3.sql - mhknbn2kdz.database.windows.net,1433.AdventureWorks2012 (sjones

I connected and ran the code to get this:

2015-06-23 14_47_30-SQLQuery3.sql - mhknbn2kdz.database.windows.net,1433.master (sjones (63))_ - Mic

I needed to not only change my database user connection, but also use the admin user I have. In this case, it’s the jt user I inherited from Jamie Thomson as part of the Hosted AdventureWorks project he started.

Once I did that, the login was created without an issue. Now I need to link it to a user.

SQLNewBlogger

This took about 5 minutes to write, and 5 minute to reshoot some screen shots. The rest was really the 20 minutes I spent mucking around, researching, and reading how to do this. Hopefully this is quicker for me, and you, if you have this need.

Anyone could write this, and I’d encourage you to write your own stories about working with an Azure database if you want to learn about the topic.

Reference

Load Balanced IIS Machines

I noticed a contest this week while working on the Database Weekly newsletter. It’s the Cloud Hero contest, with the chance to win a Surface Pro 3. I could always use another device, or at least a device I could give away, so I decided to enter.

There are a few things you can do, all of which are interesting to me in terms of a direction that I, and Red Gate, want to move. I don’t know if Azure works everywhere, but we are considering moving SQLServerCentral, or perhaps parts of it, to Azure, so this was a good chance for me to try out some new Azure stuff.

I’ve messed with a few things in Azure, but mostly on the PaaS side. That interests me more, and I’ve done little with IaaS. I certainly haven’t really worked with IIS much in Azure. I decided to go through the VM setup, to create two IIS machines, load balanced on the same URL. I used this blog post with a cartoon and demo to run through the process.

It was a bit more than 10 minutes, mostly because some of the allocation stuff in Azure took time, and the responsiveness from the VM in Azure was slow. From the time I connected to the time Server Manager popped up was over two minutes for each machine. Since I was going through some of the steps sequentially, that meant it was slow to get going.

The video and the portal bring to light some of the issues of Azure. It’s a great tutorial and I was able to get the two machines load balancing IIS in 20 minutes (or less). It was surprised how quickly it went, but I also had to stop and think. The load balancing and cloud services are different now than they were when the post was written.

I’m sure that’s the case with lots of Azure content. In some sense, this means that we will have lots of issues with people trying to learn how to use Azure as they’ll find content and information that is woefully out of date, sometimes quickly. I wonder if we need to think about having some code on blogs for Azure that marks the content as potentially out of date after it’s been out for 6 months.

It’s a challenge to keep the content up to date, and luckily the changes weren’t too different in the portal.

I am glad that I was able to get to IIS machines up and load balanced, delete them, and bring them back. That makes me think I may find some use for this Azure stuff, yet. I have a few projects in mind, including rebooting my personal site. Perhaps Azure will be the place I give it a go.