Azure storage: services, tiers, redundancy, account types, and moving data in

Why this matters

Almost every Azure workload stores something: website images, log files, a shared drive for a finance team, the disks under a virtual machine, or backups that must be kept for years. Azure offers several storage services, and each one is built for a different shape of data. Choosing the wrong one costs money or breaks an application. Put a legacy application's mapped network drive into object storage and it will not mount. Keep seven years of untouched backups in the most expensive tier and the bill grows every month for no benefit.

A manager needs this lesson to understand why two storage accounts holding the same data can cost very different amounts. A salesperson needs it to answer "what happens to our data if a datacenter fails?" honestly. A junior administrator needs it to pick a redundancy option, a tool for copying files, and a sensible way to move a whole server estate into Azure. Get redundancy wrong and a regional outage takes the data with it. Get the migration tool wrong and a transfer that should take a week takes months.

The lesson

Comparing the Azure Storage services

Most Azure storage lives inside a storage account, a named container that groups data services under one set of endpoints, one bill and one set of settings. A standard account can hold four data services. Managed disks are a fifth option that sits alongside them.

Blob Storage is object storage. Each blob has its own address, so a retailer can store millions of product photos and let its website hand them straight to shoppers' browsers. It is also the landing zone for large volumes of raw data such as logs and sensor readings, and Azure Data Lake Storage is built on top of it for analytics.

Azure Files gives the company a file share without a file server to build or patch. Clients mount it like a network drive over Server Message Block (SMB), and premium shares can also be mounted over Network File System (NFS). The contrast with Blob Storage matters: Azure Files presents shares that an operating system mounts, while Blob Storage holds objects that applications reach through REST calls over HTTPS. Both work with Windows, Linux and macOS.

Queue Storage carries work items. A web front end adds a message for each new order and a background worker picks the messages up later. If the worker is offline for a while, the orders wait in the queue rather than being lost.

Table Storage keeps structured data as entities identified by keys. Each entity can carry a different set of properties because there is no fixed schema. It is not a relational database and it is the wrong home for large media files.

Azure managed disks are block storage for virtual machines (lesson 7). The operating system sees them as ordinary physical disks, and Azure manages and replicates them. A file share can be mounted by a VM over the network, but it cannot be the VM's own operating system disk.

Access tiers for blob data

Blob data can sit in one of four access tiers. The trade-off is simple: as you move from hot to archive, the cost of storing data falls and the cost of reading it rises.

• Hot: the highest storage cost and the lowest cost per read. Use it for data read often, such as images for articles published this month.
• Cool: cheaper to store, dearer to read. For data kept for a while and seldom read. The data stays online and writable.
• Cold: cheaper again, and still online. It is the lowest-cost tier that can return data immediately, which suits compliance records an auditor may ask for without warning.
• Archive: the lowest storage cost and the highest cost to read. The data is offline. Before an archived blob can be read it must be rehydrated to the hot, cool or cold tier, which can take hours.

Three further rules are worth knowing.

• The archive tier is set only on individual blobs. An account's default access tier must be an online tier, usually hot. No role assignment, not even Owner, makes an archived blob readable while it is offline.
• Cool, cold and archive each have a minimum storage period (30, 90 and 180 days respectively). Delete or move a blob before its period ends and an early deletion charge appears on the bill.
• A lifecycle management policy on the storage account moves blobs between tiers, or deletes them, based on their age or when they were last used. It runs without anyone doing it by hand. Do not confuse it with Azure Policy (lesson 13), which governs how resources are configured, not where individual blobs sit.

Redundancy options

Azure Storage always keeps several copies of your data. The redundancy option decides where those copies live, and so which failures the data survives. Lesson 5 explains availability zones and region pairs.

• Locally redundant storage (LRS) keeps three copies in a single datacenter in the primary region. It is the cheapest and least durable option. If that datacenter fails, the data can become unavailable. It suits data that is easy to recreate.
• Zone-redundant storage (ZRS) copies data synchronously across three availability zones in the primary region. Losing one zone leaves the data available, and nothing leaves the region.
• Geo-redundant storage (GRS) keeps three copies in one location in the primary region, exactly as LRS does, and then copies the data asynchronously to a secondary region. Where the primary region has a pair, the secondary is the paired region; the customer does not pick it.
• Geo-zone-redundant storage (GZRS) combines the two: three zones in the primary region, plus an asynchronous copy in the secondary region. It survives a zone loss without failover and keeps a copy if the whole region is lost.

By default the secondary copy in GRS and GZRS cannot be read. The read-access versions, RA-GRS and RA-GZRS, let an application read the secondary copy at any time, even when no failover has happened.

If the primary region has a long outage, an account failover makes the secondary region the new primary, and the company can read and write there. Changing the redundancy setting to LRS would throw the secondary copy away rather than use it, and access tiers have nothing to do with which region serves the data.

A quick test: which options keep data available when an entire zone in the primary region fails? Only those with "zone" in the name, ZRS and GZRS (and RA-GZRS). GRS and RA-GRS store their primary copies in one location, just like LRS.

Storage account types and settings

Microsoft offers a small number of account types.

• Standard general-purpose v2 is the recommended default for most scenarios. It supports Blob Storage, Azure Files, Queue Storage and Table Storage in one account, and every redundancy option.
• Premium block blobs keep block blobs on solid-state drives for workloads that write huge numbers of small objects and need high transaction rates or consistently low latency.
• Premium file shares keep Azure file shares on solid-state drives. This is the account type that supports NFS shares.
• Premium page blobs hold page blobs only, a specialist format for random read and write access.

The general rule is that premium accounts use solid-state drives for low latency and cost more, while standard accounts trade some performance for a lower price. Premium accounts also offer a narrower choice of redundancy.

Some settings apply to every account.

• The name must be unique across all of Azure, not just within your subscription, and may use only lowercase letters and numbers. The name becomes part of each service's public endpoint: an account called contosomedia has its blobs at https://contosomedia.blob.core.windows.net, its file shares at contosomedia.file.core.windows.net, and similar addresses for queues and tables. So if a stranger already took the name, you cannot use it.
• Data at rest is encrypted automatically in every account, and that encryption cannot be turned off. By default Microsoft manages the keys. A customer can choose to manage its own keys instead, but encryption does not depend on that choice.

Tools for moving files: AzCopy, Storage Explorer and File Sync

Three tools move files, and each suits a different person and job.

AzCopy is a command-line utility that copies blobs or files to and from a storage account. Because it is a command, it fits naturally into a scheduled script, such as a nightly copy of an on-premises folder into a blob container. Its sync command works in one direction only: it makes the destination match the source, and changes made at the destination are not copied back.

Azure Storage Explorer is a graphical desktop application for Windows, macOS and Linux. An analyst who avoids the command line can browse containers and file shares and upload or download files with a mouse. Behind the scenes, Storage Explorer uses AzCopy to perform its transfers. AzCopy has not been retired.

Azure File Sync is different in kind. It is not a one-off copy but a continuous service. An agent installed on a Windows Server turns that server into a fast local cache of an Azure file share, and one share can be synchronised with many servers in different branch offices. Its cloud tiering feature keeps frequently used files on the local server and replaces rarely used ones with pointers; when someone opens a pointer, the file is recalled from Azure. Note the trap: cloud tiering is not the same as blob access tiers. File Sync also assumes the company still runs Windows Servers. A company that wants to stop running file servers altogether wants Azure Files alone.

Migration options: Azure Migrate and Azure Data Box

Copying files is one task. Moving whole workloads, or hundreds of terabytes, is another.

Azure Migrate is a central hub for moving servers, databases and web applications from on premises into Azure. It works in stages:

1. Discover the on-premises servers, for example a few hundred VMware machines.
2. Assess them: check readiness for Azure, and estimate the sizing and monthly cost of each one there.
3. Migrate: replicate running virtual machines to Azure and then cut over with minimal downtime, which works well when the network link to Azure is fast and reliable.

Azure Advisor (lesson 16) is not a substitute, because it only makes recommendations about resources already running in Azure.

Azure Data Box solves a bandwidth problem, not a planning one. Microsoft ships a rugged physical storage device to the customer, who loads the data locally and sends the device back. Microsoft then uploads the data into a storage account from inside the Azure datacenter. Smaller (Data Box Disk) and larger (Data Box Heavy) members of the family exist for different volumes. Key facts:

• It suits a one-time move of a very large data set when the internet connection would take weeks or months. AzCopy would still push every byte over that slow link.
• It works in both directions: import into Azure, and export out of Azure, for example for disaster recovery copies or to leave the cloud.
• After the upload, Microsoft securely wipes the device's disks following NIST guidelines for media sanitisation, so the next customer never sees the data.
• It does not assess servers, stream over ExpressRoute, or keep file servers in sync.

What the exam is really asking

Most storage questions describe data and ask for the service, tier or option that fits it. Decide in this order.

• Shape of the data. Objects served over HTTPS: Blob Storage. A drive that mounts over SMB or NFS: Azure Files. Work items between components: Queue Storage. Key-value entities: Table Storage. A VM's own disk: managed disks.
• How often it is read. Often: hot. Rarely but must open at once: cold (or cool for data read a little more often). Almost never, and hours of waiting are fine: archive. Archive is offline, blob-level only and must be rehydrated.
• Which failure it must survive. One datacenter: ZRS at least, since LRS lives in one datacenter. A zone and the region: GZRS. Read the secondary without failover: an RA- option. Data must not leave the region: LRS or ZRS, never a geo option.
• Tool by person and job. Script: AzCopy. Mouse: Storage Explorer. Branch cache on Windows Server: Azure File Sync. Plan and move whole servers: Azure Migrate. Too much data for the network: Azure Data Box.

The classic traps are Azure Files versus Azure File Sync (a managed share versus syncing it with servers you still run), cloud tiering versus access tiers, GRS versus GZRS on a zone failure, and lifecycle management versus Azure Policy.