Linux环境下将log4j的日志存储到mongoDB

转自：http://my.oschina.net/chinaearl/blog/111552

 

单位有很多java项目，一般上生产的项目日志都是通过log4j打印在本地，平常开发同学要查看日志还要给他们开个专门的FTP，不利于集中维护管理，近日因工作需要，在网上看到相关技术资料，可把log4j日志直接存放到mongoDB，在mongoDB的可以更好的查询日志。非常利于集中管理和维护。以下是自己和开发同学配置部署的过程，可谓是一波三折：

Log4mongo for Java官方介绍：

• MongoDbAppender - stores the LogEvent in a BSONified form
• ExtendedMongoDbAppender - Extends MongoDbAppender by allowing you to add top level elements
• MongoDbPatternLayoutAppender - supports logging data in a custom format

 

 

1.首先下载并安装mongoDB，并运行 ，下载安装请参考我的上篇博文
版本要求（MongoDB Server v2.0+ (tested with 2.2.2)）

2.下载Mongo Java driver jar包

3.下载 Log4J jar包 注意版本，一定要1.2.16及以上。

4.下载 log4mongo-java jar包

5.请将这三个jar文件放置在你的项目classpath中。

6.在你的classpath中创建一个名为log4j.properties的文件

7.在log4j.properties文件尾添加以下几行

?

1 2 3 4

log4j.rootLogger=INFO, MongoDB                               log4j.appender.MongoDB=org.log4mongo.MongoDbAppender         log4j.appender.MongoDB.databaseName=appname                  log4j.appender.MongoDB.collectionName=log

如果你的mongoDB不是在本地，那添加mongoDB数据库的IP地址和端口

 

?

1 2	log4j.appender.MongoDB.hostname=172.17.0.9  log4j.appender.MongoDB.port=27017

创建一个Java类，如：

 

?

1 2 3 4 5 6 7 8

import org.apache.log4j.Logger;   public class LogTest {     public static void main(String[] args) {         Logger logger = Logger.getLogger(LogTest.class);         logger.error("Don't panic");     } }

 

编译和运行这个类

碰到问题，mongoDB里的日志数据始终不见出来。后来发现问题，原因是项目要部署到tomcat上才可能生效。

以下是开发同学整理的部署情况

 

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71

1、建立一个web项目，项目名为logmongo，在项目当中引入jar包，见步骤2~4（指你写的）描述，共3个jar包。 2、log4j.properties内容： log4j.rootLogger=debug, stdout, R, MongoDB   log4j.appender.stdout=org.apache.log4j.ConsoleAppender log4j.appender.stdout.layout=org.apache.log4j.PatternLayout log4j.appender.stdout.layout.ConversionPattern=%-d{MM-dd HH:mm:ss.SSS} %-5p - [%c{1}.%t] %m%n   log4j.appender.R=org.apache.log4j.RollingFileAppender log4j.appender.R.File=logs/client.log log4j.appender.R.MaxFileSize=1000KB log4j.appender.R.MaxBackupIndex=1 log4j.appender.R.layout=org.apache.log4j.PatternLayout log4j.appender.R.layout.ConversionPattern=%-d{MM-dd HH:mm:ss.SSS} %-5p - [%c{1}.%t] %m%n   log4j.appender.MongoDB=org.log4mongo.MongoDbAppender log4j.appender.MongoDB.databaseName=appname log4j.appender.MongoDB.collectionName=log log4j.appender.MongoDB.hostname=172.17.0.9 log4j.appender.MongoDB.port=27017   3、web.xml内容 <?xml version="1.0" encoding="UTF-8"?> <web-app version="3.0" xmlns="http://java.sun.com/xml/ns/javaee"  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  xsi:schemaLocation="http://java.sun.com/xml/ns/javaee  http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd">  <display-name></display-name>  <welcome-file-list>  <welcome-file>index.jsp</welcome-file>  </welcome-file-list>    <context-param>  <param-name>log4jConfigLocation</param-name>  <param-value>classpath:/log4j.properties</param-value>  </context-param>     <servlet>  <servlet-name>log4jServlet</servlet-name>  <servlet-class>log4jServlet</servlet-class>  </servlet>    <servlet-mapping>  <servlet-name>log4jServlet</servlet-name>  <url-pattern>/log4jServlet</url-pattern>  </servlet-mapping> </web-app>   4、log4jServlet.java import java.io.IOException; import javax.servlet.http.HttpServlet; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import org.apache.log4j.Logger;   public class log4jServlet extends HttpServlet {     private static Logger logger = Logger.getLogger(log4jServlet.class);       public void doPost(HttpServletRequest request, HttpServletResponse response)             throws IOException {         this.doGet(request, response);     }       public void doGet(HttpServletRequest request, HttpServletResponse response)             throws IOException {         logger.error("test for tomcat");     }   }   5、部署到tomcat上运行，浏览器中输入http://localhost:8080/logmongo/log4jServlet即可。

 

然后查看mongoDB，看是否有记录

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

[root@zabbix bin]# ./mongo MongoDB shell version: 2.2.2 connecting to: test > show dbs admin   (empty) appname 0.203125GB local   (empty) > use appname switched to db appname > db.log.findOne() {     "_id" : ObjectId("513427c2abef7a8b93610a70"),     "timestamp" : ISODate("2013-03-04T04:49:06.060Z"),     "level" : "ERROR",     "thread" : "http-8080-1",     "message" : "Don't panic",     "loggerName" : {         "fullyQualifiedClassName" : "log4jServlet",         "package" : [             "log4jServlet"         ],         "className" : "log4jServlet"     },     "fileName" : "log4jServlet.java",     "method" : "doGet",     "lineNumber" : "24",     "class" : {         "fullyQualifiedClassName" : "log4jServlet",         "package" : [             "log4jServlet"         ],         "className" : "log4jServlet"     },     "host" : {         "process" : "6752@yangyang",         "name" : "yangyang",         "ip" : "10.1.19.2"     } }

接下去，可以用PHP写个查看mongoDB数据的程序，友好的把日志通过网页形式显示出来。

MongoDB的PHP扩展安装

1.下载php扩展包：http://pecl.php.net/package/mongo

2.安装php的mongodb扩展

?

1 2 3 4 5 6

tar zxvf mongo-1.3.4.tgz cd mongo-1.3.4/ #/usr/local/webserver/php/ 您的php实际安装目录  /usr/local/webserver/php/bin/phpize ./configure --with-php-config=/usr/local/webserver/php/bin/php-config make && make install

php读取mongodb日志数据程序如下

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92

<?php     error_reporting(7);     $conn = new Mongo();     $db = $conn->appname;     $collection = $db->log; ?> <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html>  <head>   <title> log4j for mongodb </title>   <meta name="Generator" content="EditPlus">   <meta name="Author" content="">   <meta name="Keywords" content="">   <meta name="Description" content=""> <style type="text/css">          table {              border: 1px solid #999999;              padding:0;               margin:0 auto;              border-collapse: collapse;          }                      td,th {              border: 1px solid #999999;              background: #fff;              font-size:13px;              padding: 3px 3px 3px 8px;              color: #000;          }  </style>  </head>    <body>     <table border="1" width="100%">     <tr>     <th rowspan="2">ID</th>     <th rowspan="2">TIMESTAMP</th>     <th rowspan="2">LEVEL</th>     <th rowspan="2">THREAD</th>     <th rowspan="2">MESSAGE</th>     <th colspan="3">LOGGERNAME</th>     <th rowspan="2">FILENAME</th>     <th rowspan="2">METHOD</th>         <th rowspan="2">lineNumber</th>     <th colspan="3">CLASS</th>     <th colspan="3">HOST</th>   </tr>   <tr>     <td>fullyQualifiedClassName</td>     <td>package</td>     <td>className</td>     <td>fullyQualifiedClassName</td>     <td>package</td>     <td>className</td>     <td>process</td>     <td>name</td>     <td>ip</td>   </tr>   <?php     $cursor = $collection->find();     while($cursor->hasNext()){         $r = $cursor->getNext(); ?>   <tr>         <td><?php echo $r[_id]?></td>         <td><?php echo $r[timestamp]?></td>         <td><?php echo $r[level]?></td>         <td><?php echo $r[thread]?></td>         <td><?php echo $r[message]?></td>         <td><?php echo $r[loggerName][fullyQualifiedClassName]?></td>         <td><?php echo implode('.', $r[loggerName][package]);?></td>         <td><?php echo $r[loggerName][className]?></td>         <td><?php echo $r[fileName]?></td>         <td><?php echo $r[method]?></td>         <td><?php echo $r[lineNumber]?></td>         <td><?php echo $r['class'][fullyQualifiedClassName];?></td>         <td><?php echo implode('.', $r['class'][package]);?></td>         <td><?php echo $r['class'][className];?></td>         <td><?php echo $r[host][process]?></td>         <td><?php echo $r[host][name]?></td>         <td><?php echo $r[host][ip]?></td>   </tr> <?php } ?>   </table>    </body> </html>

运行效果如图:

另：log4j还可以通过配置写到MySQL数据库里去。不过论现在形式而言，存到nosql里去会更优雅.