Overview
Diagnosing and troubleshooting issues in cloud storage applications can be complex especially if they are not considered upfront. When an issue does occur, it can involve parsing and analyzing multiple log files (Azure Storage Analytics, client-side logs from Azure Storage Client libraries, and network traces) to fully understand and mitigate the issue. To assist with this, we have released a set of Azure Storage specific assets in Microsoft Message Analyzer, including parsers, color rules, charts, filters, and view layouts. This blog provides a brief overview, but for a complete hands-on tutorial see
End-to-End Troubleshooting using Azure Storage Metrics and Logging, AzCopy, and Message Analyzer.
Sample Scenario – Searching Storage Logs for Storage Service Versions
For this blog post, we’ll examine a scenario where the customer wants to determine what REST versions their client applications are currently using, so they can prepare (if needed) for the planned
Azure Storage Service Removal of several of the earlier service versions.
Assumption is you have Message Analyzer with the Azure Storage Assets installed and Azure storage logging and metrics enabled – for info on that see the complete hands-on tutorial.
Step 1: Download the Azure Storage server-side logs using AzCopy
AzCopy is available for download on the
Azure Downloads page. For details about using AzCopy, see
How to use AzCopy with Microsoft Azure Storage. As an example, the following command will download the log files for blob operations that took place on January 2, 2015 to the folder C:\Temp\Logs\Server; replace
<storageaccountname> with the name of your storage account, and
<storageaccountkey> with your account access key:
AzCopy.exe /Source:https://<storageaccountname>.blob.core.windows.net/$logs /Dest:C:\Temp\Logs\Server /Pattern:"blob/2015/01/02" /SourceKey:<storageaccountkey> /S /V
NOTE: It can take up to an hour for log data to become available because of the frequency at which the storage service flushes the log writers.
Step 2: Import your server-side logs into Message Analyzer
1. On the
File menu in Microsoft Message Analyzer, click
New Session >
Files >
Add Files to browse to the location where you downloaded your server-side logs. Select your server-side logs and click on the
Open button.
2. In the
Session Details panel, set the
Text Log Configuration drop-down for each server-side log file to
AzureStorageLog (if not already set) to ensure that Microsoft Message Analyzer can parse the log file correctly, and click on the
Start button.
Step 3: Add the RequestVersionHeader column to the Analysis Grid
In Microsoft Message Analyzer, under
Column Chooser > Azure Storage Log > Azure Storage Log Entry, right click on
RequestVersionHeader and select “Add as Column” to make it visible in the Analysis Grid.
Step 4: Search for earlier REST versions
Add the following session filter to determine if there are any requests using an Azure Storage service version that is scheduled to be removed:
AzureStorageLog.RequestVersionHeader < "2012-02-12"
You can double click on each row (if any) in the Analysis Grid to get more information on the individual request. You can also read the “What Should I do?” section on our
blog.
Performance Tip: Note that Message Analyzer loads log files into memory. If you have a large set of log data, you will want to apply a session filter before you load the data, in order to get the best performance from Message Analyzer.
Summary
In the above scenario we demonstrated how Message Analyzer along with AzCopy can be used to identify clients that are still using an old REST version. You can use the same combination of tools for your own debugging and analysis when working with Azure Storage.
Next Steps
Follow the complete hands-on tutorial
here for more advance scenarios including correlating storage, network, and client side logs to troubleshoot performance issues.
For more information visit the following resources:
•
E2E Troubleshooting using Azure Storage Metrics and Logging, AzCopy, and Message Analyzer
•
Monitor, diagnose, and troubleshoot Microsoft Azure Storage
•
Microsoft Azure Storage Service Version Removal
