Logstash 그로크 필터가 메시지를 구문 분석하지 않습니다.

Question

그 로크에서는 디버거가 logstash를 시작할 때 괜찮지만 구문 분석하지 않습니다.

로그 라인 :

# Time: 170904 10:16:01\n# User@Host: mmcite[mmcite] @ localhost []\n# Thread_id: 18712 Schema: mmcite QC_hit: No\n# Query_time: 0.502068 Lock_time: 0.000030 Rows_sent: 0 Rows_examined: 1\n# Rows_affected: 1\nuse mmcite;\nSET timestamp=1504512961;\nUPDATE `ajedem456456` SET `id`='cotjo4mim2j7fp3ui2kit7gns6' WHERE id='pvueh0rm6l2meiguootdfqsan7'; 

필터 :

grok { 

    match => { "message" => "#%{SPACE}Time:%{SPACE}%{NUMBER}%{SPACE}%{TIME}(.|\n)*%{HOSTNAME}\[%{HOSTNAME:mysql_host}\]%{SPACE}@%{SPACE}localhost \[\](.|\n)*#%{SPACE}Thread_id:%{SPACE}%{NUMBER}%{SPACE}Schema:%{SPACE}%{WORD}%{SPACE}%{WORD}:%{SPACE}%{WORD}(.|\n)*#%{SPACE}Query_time:%{SPACE}%{BASE16FLOAT:mysql_query_time}%{SPACE}Lock_time:%{SPACE}%{BASE16FLOAT:mysql_lock_time}%{SPACE}Rows_sent:%{SPACE}%{NUMBER:mysql_rows_sent}%{SPACE}Rows_examined:%{SPACE}%{NUMBER:mysql_rows_examined}(.|\n)*%{SPACE}Rows_affected:%{SPACE}%{NUMBER:mysql_rows_affected}(.|\n)*%{WORD}%{SPACE}%{WORD};(.|\n)*SET%{SPACE}timestamp=%{NUMBER:timestamp};\\n%{GREEDYDATA:mysql_query}" } 

} 

출력 :

{ 
    "@timestamp" => 2017-09-04T13:08:06.260Z, 
     "offset" => 3441, 
     "@version" => "1", 
    "input_type" => "log", 
      "beat" => { 
     "hostname" => "server.jerewan.cz", 
      "name" => "server.jerewan.cz", 
     "version" => "5.1.1" 
    }, 
      "host" => "server.jerewan.cz", 
     "source" => "/usr/home/admin/filebeat/mysql.slow.log", 
     "message" => "# Time: 170904 10:16:01\n# User@Host: mmcite[mmcite] @ localhost []\n# Thread_id: 18712 Schema: mmcite QC_hit: No\n# Query_time: 0.502068 Lock_time: 0.000030 Rows_sent: 0 Rows_examined: 1\n# Rows_affected: 1\nuse mmcite;\nSET timestamp=1504512961;\nUPDATE `PAJKA` SET `id`='cotjo4mim2j7fp3ui2kit7gns6' WHERE id='pvueh0rm6l2meiguootdfqsan7';", 
      "type" => "mysql_slow_log", 
      "tags" => [ 
     [0] "beats_input_codec_plain_applied", 
     [1] "_grokparsefailure" 
    ] 
} 

당신의 도움을 대단히 감사합니다.

Answer 1

나는 어떻게 작동하는지 전혀 모르지만 작동합니다.

grok {  
    match => { "message" => "#%{SPACE}Time:%{SPACE}%{NUMBER}%{SPACE}%{TIME}(.|\n)*%{HOSTNAME}\[%{HOSTNAME:mysql_host}\]%{SPACE}@%{SPACE}localhost \[\](.|\n)*#%{SPACE}Thread_id:%{SPACE}%{NUMBER}%{SPACE}Schema:%{SPACE}%{WORD}%{SPACE}%{WORD}:%{SPACE}%{WORD}(.|\n)*#%{SPACE}Query_time:%{SPACE}%{BASE16FLOAT:mysql_query_time}%{SPACE}Lock_time:%{SPACE}%{BASE16FLOAT:mysql_lock_time}%{SPACE}Rows_sent:%{SPACE}%{NUMBER:mysql_rows_sent}%{SPACE}Rows_examined:%{SPACE}%{NUMBER:mysql_rows_examined}(.|\n)*%{SPACE}Rows_affected:%{SPACE}%{NUMBER:mysql_rows_affected}(.|\n)*%{WORD}%{SPACE}%{WORD};(.|\n)*SET%{SPACE}timestamp=%{NUMBER:timestamp};(.|\n)%{GREEDYDATA:mysql_query}" }  
} 

나는 (.|\n)에 의해 %{GREEDYDATA:mysql_query} 전에 \\n를 교체했습니다.