Kibana

Export index-pattern curl http://xxx.xxx.xxx.xxx:5601/api/saved_objects/index-pattern/f1836c20-e880-11e8-8d66-7d7b4c3a5906 > index-pattern-export.json visualization curl http://xxx.xxx.xxx.xxx:5601/api/saved_objects/visualization/1eb85311-f901-11e8-864c-bd4880954537 > visual-export.json Import index-pattern curl -v -XPOST kibana:5601/api/kibana/dashboards/import?force=true -H “kbn-xsrf:true” -H “Content-type:application/json” -d @/usr/share/config/config/index-pattern-export.json visualization curl -v -XPOST kibana:5601/api/kibana/dashboards/import?force=true -H “kbn-xsrf:true” -H “Content-type:application/json” -d @/usr/share/config/config/visual-export.json PS： visualization can be multi { "objects": [ {"id":"0c298010-f901-11e8-864c-bd4880954537",...}, {"id":"1eb85300-f901-11e8-864c-bd4880954537",...} ]}

1. Change logstash add fields or grok some fields. Kibana DISCOVER Table can see new fields & ! 2. Kibana Management -> Index Patterns can “Fefresh field list” ! will be disappear. 3. Logstash set some fields Type “geo_point”. see Kibana DISCOVER Table field Type still “TEXT”. Try to delete index. GET _cat/indices?v GET _cat/indices?v&s=index GET filebeat-6.5.1-2018.12.06 DELETE filebeat-6.5.1-2018.12.06 After DELETE index (real index), index rebuiled. Geo_point usually can see. 4. Every things sure is ok, export index-pattern.json https://sueboy.blogspot.com/2018/11/kibana-default-index-pattern.html ...

Now check again…. 1、template_filebeat.json Can only { "index_patterns": ["filebeat*"], "settings": { "number_of_shards": 1 }, "mappings": { "doc": { "properties": { "geoip.location": { "type": "geo_point" }, "geoip.coordinates": { "type": "geo_point" } } } } } Here Import： “location” is Error, Must “geoip.location” But sometime why no use, because my way insert index-pattern, so geoip.location no field, always is geoip.location.lat and geoip.location.lon overwrtie. see 2. 2、index-pattern index-pattern-export.json one way just try to put ...

Filebeat + Elasticsearch + Kibana 轻量日志收集与展示系统 https://wzyboy.im/post/1111.html?utm_source=tuicool&utm_medium=referral 提到 beat -> logstash -> elk 可以 beat -> elk ingest plugs ( Elasticsearch Ingest Node ) Elasticsearch Ingest Node 是 Elasticsearch 5.0 起新增的功能。在 Ingest Node 出现之前，人们通常会在 ES 前置一个 Logstash Indexer，用于对数据进行预处理。有了 Ingest Node 之后，Logstash Indexer 的大部分功能就可以被它替代了，grok, geoip 等 Logstash 用户所熟悉的处理器，在 Ingest Node 里也有。对于数据量较小的 ES 用户来说，省掉一台 Logstash 的开销自然是令人开心的，对于数据量较大的 ES 用户来说，Ingest Node 和 Master Node, Data Node 一样也是可以分配独立节点并横向扩展的，也不用担心性能瓶颈。 目前 Ingest Node 已支持数十种处理器，其中的 script 处理器具有最大的灵活性。 与 /_template 类似，Ingest API 位于 /_ingest 下面。用户将 pipeline 定义提交之后，在 Beats 中即可指定某 pipeline 为数据预处理器。 ...

1、filebeat /var/log/secure 2、 filter { grok { #type => "syslog" match => ["message", "%{SYSLOGBASE} Failed password for (invalid user |)%{USERNAME:username} from %{IP:src_ip} port %{BASE10NUM:port} ssh2"] add_tag => "ssh_brute_force_attack" } grok { #type => "syslog" match => ["message", "%{SYSLOGBASE} Accepted password for %{USERNAME:username} from %{IP:src_ip} port %{BASE10NUM:port} ssh2"] add_tag => "ssh_sucessful_login" } geoip { source => "src_ip" target => "geoip" add_tag => [ "ssh-geoip" ] add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ] add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ] add_field => [ "geoipflag", "true" ] } }