CompleteFusion

Splunk is a great tool that helps organize, manage, and search IT data. Although Splunk's primary marketing focus seems to be on making logs searchable, it is also readily usable to analyze and report on usage since tracking usage is basically the process of summarizing log data.

Configuring Splunk for website visitor tracking is certainly more work than using AWStats or Google Analytics, but if you have non-standard sources of log data, Splunk provides unparalleled flexibility. For example, I recently wanted to aggregate usage statistics from custom log4j log data from an application running on a large number of Tomcat servers. Since I was already using log4j to handle writing the logs to local storage on each server, I simply needed to add a SyslogAppender to my Tomcat log4j.properties, as follows:

log4j.appender.statssyslog=org.apache.log4j.net.SyslogAppender
log4j.appender.statssyslog.SyslogHost=YourSplunkHostIP
log4j.appender.statssyslog.layout=org.apache.log4j.PatternLayout
log4j.appender.statssyslog.layout.ConversionPattern="%d{ISO8601}",%m
log4j.appender.statssyslog.Facility=USER

After configuring Splunk to receive syslog data on UDP port 514, I started to see the data appear in Splunk. Stay tuned for Part 2, which will provide details about configuring Splunk's Summary Indexing and reporting on our gathered statistics.