Running big complex web applications can be overwhelming at times. Especially, when it comes to troubleshooting. Developers often need to access application log data from the servers that run the web applications. Collecting that information from all the servers, let alone searching through them, sounds like an impossible task especially if you have thousands of servers running.

Introducing: Log Management Tools

Log management tools help engineering teams collect, parse, and search through log data from all of your servers, across different clusters.

Two of my favorite log management tools are: SumoLogic and Loggly.

Things to consider when picking a log management tool:

1. Does the tool index your log upon receipt?
2. Does the tool support your log data format?
3. What is the daily upload limit?
4. What is the burst limit? (i.e. what if you exceed the daily limit on certain busy days?)
5. What is the data retention policy?
6. Do they come with API access?
7. Do queries return results quickly?
8. Do they have alerting feature?
9. Do they have adequate charting/graphing feature?
10. What is the delay between event trigger moment to events appearing in the dashboard?

Sumologic vs Loggly

Based on my experience, having used both services, here are my opinions on them:

Sumologic

1. Great charting feature
2. Amazing query language/syntax
3. Super slow query time

Loggly

1. Super fast query time
2. Very limited querying options
3. Limited charting feature

How do you currently manage your logs?