Fork me on GitHub

Programming, Internet & more

Analyze and Visualize your Azure Logfiles with Alfa

I’ve been using Microsoft’s Cloud Platform Azure for several years now. Azure has built in logging support which can be used by applications deployed to Azure.

Logging is as simple as calling one of the methods from the System.Diagnostics.Trace class.


Trace.TraceInformation();  
Trace.TraceWarning();  
Trace.TraceError()

All these logging information will NOT be stored in a file, but in a table called WADLogsTable which is located in the Azure Storage.

Using the Azure Storage to display log data

To display these log data you have to query the Azure Storage. The problem with that, is that querying the storage is a real pain as Gaurav Mantri has already pointed out back in 2012.

The Azure Storage has a few disadvantages when it comes to displaying log data:

  • Querying for logs in a specific timeframe using the Timestamp attribute is incredibly slow
  • You can’t filter the logs by attributes like Level, Role, DeploymentId etc. because the query gets so slow that it becomes unuseable
  • You cannot filter for strings in the Message attribute

I searched a long time for tools that will display the log data in the Azure Storage in a nice way but couldn’t find one.

Alfa (Azure Logfile Analyzer)

Back in 2013, I decided to create my own tool called Alfa (Azure Logfile Analyzer) which should solve all these flaws with the Azure Logs.

Alfa is an application which is designed as a background service. The idea behind it is that Alfa periodically fetches the log data from Azure and stores it into its own datastore called ElasticSearch. From there you can go forward and use all visualization tools which can work with data stored in ElasticSearch. One great example is Kibana which is what I’m using to visualize my log data.

I’m using Alfa in production for almost 1 year now and I’m happy to announce that it has pushed my log analysis experience a big step forward compared to using the default Azure Storage.

The main features of Alfa are:

  • Filter logs by date/time
  • Filter logs by any attribute
  • Search in log messages
  • Aggregate all logfiles in one place (even from different Azure applications)

So, if you feel the same pain like I did, while you’re trying to view your Azure log data, make sure to give Alfa a try.

Alfa runs under Windows and Linux. Under Windows you can run it as a service.

As always, the source code is available at GitHub where you can also find all release binaries for download.

Category: open-source, programming
-->

Post a Comment

Your email is kept private. Required fields are marked *

*
*