003 SpringCloud整合-LogStash安装及ELK日志收集

SpringCloud整合-LogStash安装及ELK日志收集

1.安装ElasticSearch和kibana

ElasticSearch7.x 、IK分词器、Kibana 环境搭建、安装

2.Docker安装logstash

1.拉取docker镜像

docker pull logstash:6.6.0

2.创建外部挂载目录

mkdir  /dockerData/logstash/config

3.拷贝容器内部文件到宿主机

也可直接创建第四步文件

#拷贝容器内部文件到宿主机
docker container cp logstash:/usr/share/logstash/pipeline/logstash.conf  /dockerData/logstash/config
docker container cp logstash:/usr/share/logstash/config/logstash.yml   /dockerData/logstash/config

4.修改外部挂载文件

logstash.yml

http.host: "0.0.0.0"
xpack.monitoring.elasticsearch.url: http://127.0.0.1:9200
#es认证配置，没有就不用配
xpack.monitoring.elasticsearch.username: logstash_system
xpack.monitoring.elasticsearch.password: logstash_system

logstash.conf

input {tcp {mode => "server"host => "0.0.0.0"port => 4560#日志文件json输出codec => json_lines}
}
output {elasticsearch {action => "index"# ES地址hosts  => ["127.1.0.1:9200"]#es认证配置，没有就不用配user => "logstash_system"password => "logstash_system"# 输出到的es索引index  => "index-logstash"}	stdout {codec => rubydebug}
}

5.运行docker容器

docker run  -p 9600:9600 -p 4560:4560 \
--name logstash \
-v /dockerData/logstash/config/logstash.conf:/usr/share/logstash/pipeline/logstash.conf \
-v /dockerData/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml \
-d --restart=always logstash:6.6.0 

3.整合kibana

1.进入kibana

2.点击Create index pattern创建es索引的匹配模式

3.进入discover查看创建的Index pattern

4.SpringCloud整合ELK日志采集

1.引入logstash依赖

        <!--集成logstash--><dependency><groupId>net.logstash.logback</groupId><artifactId>logstash-logback-encoder</artifactId><version>6.6</version></dependency>

2.配置logback日志文件

logback-spring.xml

<?xml version="1.0" encoding="UTF-8"?>
<configuration debug="false" scan="true" scanPeriod="1 seconds"><include resource="org/springframework/boot/logging/logback/base.xml" /><contextName>logback</contextName><appender name="stash" class="net.logstash.logback.appender.LogstashTcpSocketAppender"><destination>127.0.0.1:4560</destination><!-- encoder必须配置,有多种可选 --><encoder charset="UTF-8" class="net.logstash.logback.encoder.LogstashEncoder" /><!-- 日志输出编码 --><!--<encoder charset="UTF-8" class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder"><providers><timestamp><timeZone>UTC</timeZone></timestamp><pattern><pattern>{"logLevel": "%level","serviceName": "${springAppName:-}","pid": "${PID:-}","thread": "%thread","class": "%logger{40}","rest": "%message"}</pattern></pattern></providers></encoder>--></appender><root level="info"><appender-ref ref="stash" /></root>
</configuration>

yml引用

#日志收集
logging:config: classpath:logback-spring.xml

3.启动测试，查看kibana