Creating Storage fast with Azure Blob Storage

Picture this: it's Friday night. The wife has gone back to East Texas to visit some friends and I am all alone in the house with a couple of beers and a lot of free time. This opportunity doesn't present itself very often. So I did any other red blooded developer would do, I wrote an app and deployed it to Azure.

My goal was to write the basic app as fast as possible. I was planning on using this app for part of a talk I was scheduled for at a user group. I'll detail the app in other blog posts but all that it does is convert a picture into ascii art. It was a fun little thing to do.

The part I want to talk about is the storage behind the app. When you go to the homepage, you can upload an image to be converted into ascii characters. When it's done processing, you can go to the gallery to see all of the converted images. I told you, this was a very simple app.

Choosing the right storage

Given the functionality, I didn't need some giant or complicated data store. I had a couple of different options. I could write the converted image, which is now just a string of text, to some SQL store. That meant setting up a VM with a SQL Server Image. It's not complicated but its quite a bit overkill for this little app. Also, I didn't want to have manage the VM by spinning it up and down as needed when I was doing the presentation or and future development/testing.

Another option is simply writing the text to files on disk. This would be fine except I wanted be able to show how easy it would be to scale the app in Azure. If I had two instances, I'd have to do some work for either instance to know of the already converted files. Not ideal.

Azure Blob Storage

I chose to use the Azure Blob Storage.  If you aren't familiar with the Azure blob storage, it's just a key/value type of storage. A (not very good) analogy is a series of post office boxes. When you want to use a P.O. box, you must have a key to open it. The box can hold just about anything you need to store. As long as you have the key, you can access that specific P.O. box. If you want another box, you have to get another key.

This is pretty much how the blob storage works. You create a name for the blob (key) and you give it some arbitrary data to store (value). The data can be in whatever format you want; the blob doesn't care. It just knows to hold on to it for you until you want it back. These blobs (Block blobs) have a limit of 200 GB. If you happen to need more than that, you can use a Page blob which supports 1TB. Looking for more information on it? Check out the documentation.

Setting up the blob storage took a couple of minutes - wicked fast.

Creating a Storage Account

Azure provides three storage options to choose from: Blob, Queues, and Tables. In order to use any of them, you first must create a storage account.

Go to the Azure Management Portal and clicked the 'NEW' button at the bottom.

[caption id="attachment_1101" align="alignleft" width="500"]Creating a new item in the Azure management Portal. Creating a new item in the Azure management Portal.[/caption]






Then click on Data Services > Storage > Quick Create.

[caption id="attachment_1181" align="alignleft" width="300"]Creating a storage account. Creating a storage account.[/caption]






Here you pick a name, decide where (geographically) the storage account should be located and a redundancy option.

The name must be unique. You can access blobs over a HTTP so the storage account name is going to be apart of the url. It must be all lowercase letters, numbers and dashes.

For lower latency when access the storage from your apps, be mindful of which region they will be located in. If you are hosting in your apps in the East US region, that's where you want to create your storage accounts.

Choosing a Redundancy Strategy

The last option is where you chose the type of replication you want/want to pay for. Azure is nothing if not resilient.

As of this post, Azure offers three redundancy options: Locally Redundant, Geo-Redundant and Read Access Geo-Redundant.

Locally redundancy provides redundancy within the same data center. Your data will be replicated across three different nodes within the same data center. It is also the cheapest option.

Geo-Redundancy replicates your data in a secondary physical location. So if your account is in East US, then it would replicate to West US. The fail over process is pretty slick. If East US went down for whatever reason, the DNS entry for your account would simply be changed to point to the West US region. You wouldn't have to change anything in your app. There is, of course a time penalty associated with this. It takes time to configure hundreds of thousands of accounts so your app needs to be prepared to not have access to data for a little bit. Here's more detailed information from the Azure Storage blog about Geo-Redundancy and the failover process.

If that's an issue, then you can choose Read Access Geo-Redundancy. It's all the features of Geo-Redundancy with the added benefit of being able to read from the secondary location whenever you want. There are issues with eventual consistency because data replication takes time, but hey it's better than nothing.

I choose the local option because its the cheapest and I don't need to support some SLA for this stupid little app.

Click 'Create Storage Account' and, after a minute or so, you'll see your new account in the list.

[caption id="attachment_1131" align="alignleft" width="300"]The Storage accounts list. The Storage accounts list.[/caption]






Azure Storage Accounts will allow you to store up to 200TB of data. That's a lot of ascii art.

Blobs reside inside of containers and containers are under an account. You've created an account now you just need to create a container. A container is just a logical grouping of your blobs. Creating one is a cinch. Just click on your storage account, click on 'Containers' from the menu, then click on the add button at the bottom.

[caption id="attachment_1111" align="alignleft" width="300"]Creating a new storage container. Creating a new storage container.[/caption]










The same naming rules ally here, too. Only lowercase letters, numbers and dashes. You also must choose the type of access you want for the container. If you don't want anyone to see it except those who have the secret key, keep it Private. If you want to expose the blob itself, but none of the meta data about the container, then choose Public Blob. If you want all the data exposed, choose Public Container.

Windows Azure Tools For Visual Studio - It Rocks

That's it! Now to use it in your app there are two things we need to do. First, use nuget to install the WindowsAzure.Storage package.

The second thing is to install the Azure Tools. This will add a Windows Azure emulator so you don't have to use your Azure account when testing something locally. Since what I'm doing costs almost nothing to do in Azure, I'm not going to use the emulator.

Installing the tools also gives you a server explorer for Azure.

[caption id="attachment_1221" align="alignleft" width="243"]The Server Explorer add-in for windows Azure in Visual Studio The Server Explorer add-in for windows Azure in Visual Studio[/caption]











This allows you to get details on your account without having to go to the Azure portal or do it through code. Once you expand the 'Windows Azure' node, you will be prompted to sign into Azure. If you are using Visual Studio 2013 and have signed in with the same account that your Azure account is tied to, you won't need to log in again.

A slick little feature that the Explorer gives is you is it tells you the connections strings for your accounts, which is needed for your app to use. So to get the connection string to the storage account you create, click on the name of your storage account. It will be a node underneath the Storage node. Then right click and select 'Properties'.

[caption id="attachment_1231" align="alignleft" width="300"]The properties of your storage account from the Server Explorer in Visual Studio. The properties of your storage account from the Server Explorer in Visual Studio.[/caption]










See that connection string? Let's copy that guy and put it into your app config under app settings.

[caption id="attachment_1241" align="alignleft" width="300"]Put your connection string for the storage account in your app settings. Put your connection string for the storage account in your app settings.[/caption]








Put your connection string where I highlighted in the picture. On a side note, please don't judge my inability to highlight in a straight line. It is a lot harder than you would think to do it on a track pad.

Now we need a little bit of code.

var blobConnectionString = CloudConfigurationManager.GetSetting("BlobStorage.ConnectionString");

var storageAccount = CloudStorageAccount.Parse(blobConnectionString);
var blobClient = storageAccount.CreateCloudBlobClient();
var container = blobClient.GetContainerReference("conerted-images");

var blobName = "TestBlob";
var blob = container.GetBlockBlobReference(blobName); 

if (blob.Exists())
    //If the blob already exists, then you can delete it, overwrite it, view it, 
    //notify the user or whatever your app needs to do. 
blob.UploadText("This text will appear in blob storage!");

The first line grabs the connection string from the config file. Then it parse the connection string to and create's a CloudStorageAccount object that manages the Azure Storage Account.

The blobClient is the client object you use to interact with your blob containers in your storage account. Then the code grabs a reference to the actual container that was created earlier. Remember that blobs exist inside of containers so we get a reference to the blob by name from the container.

You can check to see if the blob current exists and do whatever your app would require.

The last line uploads some text to the blob... That's it!

Now you can use the Server Explorer and drill down to the blob and see the contents.

So to recap here's what you need to do to start using the Azure Blob Storage fast:

  1. Login to the Azure Portal and create a new storage account and blob container
  2. Install WindowsAzure.Storage package via Nuget
    1. (Optionally but highly recommened) Install the Windows Azure Tools for Visual Studio
  3. Add connection string to the storage account in your config file
  4. Write Code
For me, I was able to skip steps 2 and 2.1 since I already had those installed. I was adding ascii art to my container in about 3 minutes.

Lastly, I want to link to some resources I find very helpful with learning about Azure and dealing with the Storage in particular.