Hi,

My first topic i hope i will not make a duplicate :frowning:

I got a problem

I custom my logstash filter to ingest catalina.out logs, the pattern included doesn't fit me so i build this:

Exemple of log:

> 2017-09-13 12:01:20 [f05e3812] info    [native] Command line is /usr/java/default/bin/java -Djava.util.logging.config.file=/opt/tomcat-servers/TOMCAT/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Xmx256m -Xms256m -server -Dorg.apache.catalina.security.SecurityListener.UMASK=0007 -Djava.awt.headless=true -Dsun.net.inetaddr.ttl=600 -XX:+UseG1GC -XX:+UseStringDeduplication -DLOG_ROOT_PATH=/opt/applogs/TOMCAT -Dcom.decathlon.environment=PRE_PRODUCTION -Djava.security.egd=file:///dev/urandom -Dinstance.tomcat=TOMCAT -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=rtdkz1rfi01.dktetrix.net --Dmqjazzconfig=/opt/mqjazz/mqmapp/config/flux.properties -Dflows=true -Djava.security.egd=file:///dev/urandom -XX:MaxMetaspaceSize=128M -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/applogs/RFID_API01 -XX:OnOutOfMemoryError=/opt/tomcat-servers/bin/manageHPROF.sh_%p -Djdk.tls.ephemeralDHKeySize=2048 -Djava.endorsed.dirs=/opt/apache-tomcat/endorsed -classpath /opt/apache-tomcat/bin/bootstrap.jar:/opt/apache-tomcat/bin/tomcat-juli.jar

my filter

> `20%{YEAR:Year}-%{MONTHNUM:Month}-%{MONTHDAY:Day} %{TIME:Time} \[%{GREEDYDATA:sequence}\] %{LOGLEVEL:Loglevel} %{GREEDYDATA:error_message}`

The pattern work successful

![40|690x221](upload://sZE4l5HCG3MVLOF1SdQrg4Sr5f4.png)

BUT !!!

When the logstash doesnt apply the filter

> {
>     "@timestamp" => 2017-09-14T09:36:43.281Z,
>         "offset" => 2047,
>       "@version" => "1",
>     "input_type" => "log",
>           "beat" => {
>         "hostname" => "rtdkz1rfi01.dktetrix.net",
>             "name" => "rtdkz1rfi01.dktetrix.net",
>          "version" => "5.5.2"
>     },
>           "host" => "rtdkz1rfi01.dktetrix.net",
>         "source" => "/opt/tomcat-servers/RFID_API01/logs/catalina.out",
>        "message" => "2017-09-13 12:08:35 [554fc844] info    [native] Hot Sensor Placement ........ available",
>           "type" => "tomcat_catalina_out",
>           "tags" => [
>         [0] "beats_input_codec_plain_applied"
>     ]
> }

Here is my beat configuration (i got multine for information)

> - input_type: log
>   paths:
>     - /opt/tomcat-servers/RFID_API01/logs/catalina.out
>   document_type: tomcat_catalina_out
>   exclude_lines: ["^---------","DEBUG"]
>   include_lines: ["^20"]

And my logstash filter:

>       if [type] == "tomcat_catalina_out" {
>     grok {
>       match => { "message" => "20%{YEAR:Year}-%{MONTHNUM:Month}-%{MONTHDAY:Day} %{TIME:Time} %{GREEDYDATA:error_message}" }
>          }
>    }

Do you have any clue ??
Thanks a lot

---
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB