Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Java logstash logger #87

Open
wants to merge 2 commits into
base: gh-pages
Choose a base branch
from
Open

Java logstash logger #87

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
615a430
Added index.md for java-logstash-logger
mpucholblasco Jan 19, 2014
cfee76a
Modified index.md to include java-logstash-logger
mpucholblasco Jan 19, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,11 @@ ready to spit out logstash compatible log files

Have a NodeJS app? Get your logs into logstash efficiently with this cookbook.

## [ Logging from Java ](recipes/java-logstash-logger/)

Have a Java or Java Application Server? Get your logs using this
cookbook.

## [ Cisco ASA ](recipes/cisco-asa/)

Sample configuration for parsing syslog messages from a Cisco ASA firewall
Expand Down
55 changes: 55 additions & 0 deletions recipes/java-logstash-logger/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
---
layout: article
title: Java logstash logger
tags: java
---

* Goal: Ship Java logs to Logstash using Java logging libraries.
* Audience: Folks who use Java or Java Application Servers to direct log output in Logstash format.

# Preface: capture Java logs is difficult

The most common way to capture Java logs is based on using Java logging libraries
(such as log4j, Java Logging API, Apache Commons Logging, JBoss logmanager, etc.),
write those logs in a specified format, and then, capture it using multiline and grok.
But there is no a perfect regular expression. In addition, multiline and does
not capture logs correctly when log files are not flushed.

Another way to obtain Java logs is based on using log4j, and adding a SocketAppender
to send logs directly to Logstash. The main problem here is: what happen if
Logstash is down or busy during a long time? The answer is simple: some
logs can be lost.

# Solution: write logs directly on Logstash format

Do not transform logs, write them directly in your desired format!

[logstash-loggers](https://github.com/mpucholblasco/logstash-loggers) can help you
with this. It is a library to write Logstash formatted logs, plus interfaces
to those most common Java logging libraries.

It is a project in progress, so
not all Java logging libraries are supported. Supported logging libraries are
present on subsequent subsections.

## JBoss logmanager (JBoss 7)

JBoss logmanager is supported. You only should install and configure [ logstash-jboss-logmanager ]
(https://github.com/mpucholblasco/logstash-loggers/tree/master/logstash-jboss-logmanager)
to write JBoss 7 logs directly to Logstash format without any regular expression,
nor headaches on researching why your logs are splitted.

The following configuration sample will configure your JBoss to write your
server.log file in Logstash format:

<custom-handler name="HANDLER_NAME" class="net.logstash.loggers.logstash_jboss_logmanager.LogstashSizeRotatingFileHandler" module="net.logstash.loggers.logstash_jboss_logmanager">
<properties>
<property name="fileName" value="${jboss.server.log.dir}/server.log"/>
<property name="timeZone" value="UTC"/>
<property name="autoFlush" value="true"/>
<property name="maxBackupIndex" value="10"/>
<property name="rotateSize" value="209715200"/>
</properties>
</custom-handler>

More features can be seen on [logstash-jboss-logmanager readme](https://github.com/mpucholblasco/logstash-loggers/tree/master/logstash-jboss-logmanager).