ELK

Wesley13
• 阅读 604

一、基本概念

1 Node 与 Cluster

Elastic 本质上是一个分布式数据库,允许多台服务器协同工作,每台服务器可以运行多个 Elastic 实例。单个 Elastic 实例称为一个节点(node)。一组节点构成一个集群(cluster)。

2 Index

Elastic 会索引所有字段,经过处理后写入一个反向索引(Inverted Index)。查找数据的时候,直接查找该索引。

所以,Elastic 数据管理的顶层单位就叫做 Index(索引)。它是单个数据库的同义词。每个 Index (即数据库)的名字必须是小写

3 Document

Index 里面单条的记录称为 Document(文档)。许多条 Document 构成了一个 Index。

Document 使用 JSON 格式表示。同一个 Index 里面的 Document,不要求有相同的结构(scheme),但是最好保持相同,这样有利于提高搜索效率。

4 Type

Document 可以分组,比如weather这个 Index 里面,可以按城市分组(北京和上海),也可以按气候分组(晴天和雨天)。这种分组就叫做 Type,它是虚拟的逻辑分组,用来过滤 Document。

不同的 Type 应该有相似的结构(schema),举例来说,id字段不能在这个组是字符串,在另一个组是数值。这是与关系型数据库的表的一个区别。性质完全不同的数据(比如products和logs)应该存成两个 Index,而不是一个 Index 里面的两个 Type(虽然可以做到)。

根据规划,Elastic 6.x 版只允许每个 Index 包含一个 Type,7.x 版将会彻底移除 Type。

二、操作

1:查看ES版本信息

[elk@es logs]$ curl -GET 'http://localhost:9250'
{
  "name" : "elk01",
  "cluster_name" : "elk-cluster",
  "cluster_uuid" : "KW6Nr_pTSVuwT0gR0agtOA",
  "version" : {
    "number" : "5.3.1",
    "build_hash" : "5f9cf58",
    "build_date" : "2017-04-17T15:52:53.846Z",
    "build_snapshot" : false,
    "lucene_version" : "6.4.2"
  },
  "tagline" : "You Know, for Search"
}
[elk@es logs]$ 

ES返回一个json文本信息,包括版本、当前节点、集群等等信息

默认情况下,Elastic 只允许本机访问,如果需要远程访问,可以修改 Elastic 安装目录的config/elasticsearch.yml文件,去掉network.host的注释,将它的值改

0.0.0.0,然后重新启动 Elastic。

上面代码中,设成0.0.0.0让任何人都可以访问。线上服务不要这样设置,要设成具体的 IP。

2:查看索引列表

[elk@es logs]$ curl -X GET 'http://localhost:9250/_cat/indices?v'
health status index     uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   textindex r8Mj0h23TBO6uP6tBLGylQ   5   1          0            0       650b           650b
[elk@es logs]$ 

此处返回我刚创建的一个索引(创建步骤下面叙述),具体各个含义值,可以使用help查看

[elk@es logs]$ curl -X GET 'http://localhost:9250/_cat/indices?help'
health                           | h                              | current health status                                                                                            
status                           | s                              | open/close status                                                                                                
index                            | i,idx                          | index name                                                                                                       
uuid                             | id,uuid                        | index uuid                                                                                                       
pri                              | p,shards.primary,shardsPrimary | number of primary shards                                                                                         
rep                              | r,shards.replica,shardsReplica | number of replica shards                                                                                         
docs.count                       | dc,docsCount                   | available docs                                                                                                   
docs.deleted                     | dd,docsDeleted                 | deleted docs                                                                                                     
creation.date                    | cd                             | index creation date (millisecond value)                                                                          
creation.date.string             | cds                            | index creation date (as string)                                                                                  
store.size                       | ss,storeSize                   | store size of primaries & replicas                                                                               
pri.store.size                   |                                | store size of primaries                                                                                          
completion.size                  | cs,completionSize              | size of completion                                                                                               
pri.completion.size              |                                | size of completion                                                                                               
fielddata.memory_size            | fm,fielddataMemory             | used fielddata cache                                                                                             
pri.fielddata.memory_size        |                                | used fielddata cache                                                                                             
fielddata.evictions              | fe,fielddataEvictions          | fielddata evictions                                                                                              
pri.fielddata.evictions          |                                | fielddata evictions                                                                                              
query_cache.memory_size          | qcm,queryCacheMemory           | used query cache                                                                                                 
pri.query_cache.memory_size      |                                | used query cache                                                                                                 
query_cache.evictions            | qce,queryCacheEvictions        | query cache evictions                                                                                            
pri.query_cache.evictions        |                                | query cache evictions                                                                                            
request_cache.memory_size        | rcm,requestCacheMemory         | used request cache                                                                                               
pri.request_cache.memory_size    |                                | used request cache                                                                                               
request_cache.evictions          | rce,requestCacheEvictions      | request cache evictions                                                                                          
pri.request_cache.evictions      |                                | request cache evictions                                                                                          
request_cache.hit_count          | rchc,requestCacheHitCount      | request cache hit count                                                                                          
pri.request_cache.hit_count      |                                | request cache hit count                                                                                          
request_cache.miss_count         | rcmc,requestCacheMissCount     | request cache miss count                                                                                         
pri.request_cache.miss_count     |                                | request cache miss count                                                                                         
flush.total                      | ft,flushTotal                  | number of flushes                                                                                                
pri.flush.total                  |                                | number of flushes                                                                                                
flush.total_time                 | ftt,flushTotalTime             | time spent in flush                                                                                              
pri.flush.total_time             |                                | time spent in flush                                                                                              
get.current                      | gc,getCurrent                  | number of current get ops                                                                                        
pri.get.current                  |                                | number of current get ops                                                                                        
get.time                         | gti,getTime                    | time spent in get                                                                                                
pri.get.time                     |                                | time spent in get                                                                                                
get.total                        | gto,getTotal                   | number of get ops                                                                                                
pri.get.total                    |                                | number of get ops                                                                                                
get.exists_time                  | geti,getExistsTime             | time spent in successful gets                                                                                    
pri.get.exists_time              |                                | time spent in successful gets                                                                                    
get.exists_total                 | geto,getExistsTotal            | number of successful gets                                                                                        
pri.get.exists_total             |                                | number of successful gets                                                                                        
get.missing_time                 | gmti,getMissingTime            | time spent in failed gets                                                                                        
pri.get.missing_time             |                                | time spent in failed gets                                                                                        
get.missing_total                | gmto,getMissingTotal           | number of failed gets                                                                                            
pri.get.missing_total            |                                | number of failed gets                                                                                            
indexing.delete_current          | idc,indexingDeleteCurrent      | number of current deletions                                                                                      
pri.indexing.delete_current      |                                | number of current deletions                                                                                      
indexing.delete_time             | idti,indexingDeleteTime        | time spent in deletions                                                                                          
pri.indexing.delete_time         |                                | time spent in deletions                                                                                          
indexing.delete_total            | idto,indexingDeleteTotal       | number of delete ops                                                                                             
pri.indexing.delete_total        |                                | number of delete ops                                                                                             
indexing.index_current           | iic,indexingIndexCurrent       | number of current indexing ops                                                                                   
pri.indexing.index_current       |                                | number of current indexing ops                                                                                   
indexing.index_time              | iiti,indexingIndexTime         | time spent in indexing                                                                                           
pri.indexing.index_time          |                                | time spent in indexing                                                                                           
indexing.index_total             | iito,indexingIndexTotal        | number of indexing ops                                                                                           
pri.indexing.index_total         |                                | number of indexing ops                                                                                           
indexing.index_failed            | iif,indexingIndexFailed        | number of failed indexing ops                                                                                    
pri.indexing.index_failed        |                                | number of failed indexing ops                                                                                    
merges.current                   | mc,mergesCurrent               | number of current merges                                                                                         
pri.merges.current               |                                | number of current merges                                                                                         
merges.current_docs              | mcd,mergesCurrentDocs          | number of current merging docs                                                                                   
pri.merges.current_docs          |                                | number of current merging docs                                                                                   
merges.current_size              | mcs,mergesCurrentSize          | size of current merges                                                                                           
pri.merges.current_size          |                                | size of current merges                                                                                           
merges.total                     | mt,mergesTotal                 | number of completed merge ops                                                                                    
pri.merges.total                 |                                | number of completed merge ops                                                                                    
merges.total_docs                | mtd,mergesTotalDocs            | docs merged                                                                                                      
pri.merges.total_docs            |                                | docs merged                                                                                                      
merges.total_size                | mts,mergesTotalSize            | size merged                                                                                                      
pri.merges.total_size            |                                | size merged                                                                                                      
merges.total_time                | mtt,mergesTotalTime            | time spent in merges                                                                                             
pri.merges.total_time            |                                | time spent in merges                                                                                             
refresh.total                    | rto,refreshTotal               | total refreshes                                                                                                  
pri.refresh.total                |                                | total refreshes                                                                                                  
refresh.time                     | rti,refreshTime                | time spent in refreshes                                                                                          
pri.refresh.time                 |                                | time spent in refreshes                                                                                          
refresh.listeners                | rli,refreshListeners           | number of pending refresh listeners                                                                              
pri.refresh.listeners            |                                | number of pending refresh listeners                                                                              
search.fetch_current             | sfc,searchFetchCurrent         | current fetch phase ops                                                                                          
pri.search.fetch_current         |                                | current fetch phase ops                                                                                          
search.fetch_time                | sfti,searchFetchTime           | time spent in fetch phase                                                                                        
pri.search.fetch_time            |                                | time spent in fetch phase                                                                                        
search.fetch_total               | sfto,searchFetchTotal          | total fetch ops                                                                                                  
pri.search.fetch_total           |                                | total fetch ops                                                                                                  
search.open_contexts             | so,searchOpenContexts          | open search contexts                                                                                             
pri.search.open_contexts         |                                | open search contexts                                                                                             
search.query_current             | sqc,searchQueryCurrent         | current query phase ops                                                                                          
pri.search.query_current         |                                | current query phase ops                                                                                          
search.query_time                | sqti,searchQueryTime           | time spent in query phase                                                                                        
pri.search.query_time            |                                | time spent in query phase                                                                                        
search.query_total               | sqto,searchQueryTotal          | total query phase ops                                                                                            
pri.search.query_total           |                                | total query phase ops                                                                                            
search.scroll_current            | scc,searchScrollCurrent        | open scroll contexts                                                                                             
pri.search.scroll_current        |                                | open scroll contexts                                                                                             
search.scroll_time               | scti,searchScrollTime          | time scroll contexts held open                                                                                   
pri.search.scroll_time           |                                | time scroll contexts held open                                                                                   
search.scroll_total              | scto,searchScrollTotal         | completed scroll contexts                                                                                        
pri.search.scroll_total          |                                | completed scroll contexts                                                                                        
segments.count                   | sc,segmentsCount               | number of segments                                                                                               
pri.segments.count               |                                | number of segments                                                                                               
segments.memory                  | sm,segmentsMemory              | memory used by segments                                                                                          
pri.segments.memory              |                                | memory used by segments                                                                                          
segments.index_writer_memory     | siwm,segmentsIndexWriterMemory | memory used by index writer                                                                                      
pri.segments.index_writer_memory |                                | memory used by index writer                                                                                      
segments.version_map_memory      | svmm,segmentsVersionMapMemory  | memory used by version map                                                                                       
pri.segments.version_map_memory  |                                | memory used by version map                                                                                       
segments.fixed_bitset_memory     | sfbm,fixedBitsetMemory         | memory used by fixed bit sets for nested object field types and type filters for types referred in _parent fields
pri.segments.fixed_bitset_memory |                                | memory used by fixed bit sets for nested object field types and type filters for types referred in _parent fields
warmer.current                   | wc,warmerCurrent               | current warmer ops                                                                                               
pri.warmer.current               |                                | current warmer ops                                                                                               
warmer.total                     | wto,warmerTotal                | total warmer ops                                                                                                 
pri.warmer.total                 |                                | total warmer ops                                                                                                 
warmer.total_time                | wtt,warmerTotalTime            | time spent in warmers                                                                                            
pri.warmer.total_time            |                                | time spent in warmers                                                                                            
suggest.current                  | suc,suggestCurrent             | number of current suggest ops                                                                                    
pri.suggest.current              |                                | number of current suggest ops                                                                                    
suggest.time                     | suti,suggestTime               | time spend in suggest                                                                                            
pri.suggest.time                 |                                | time spend in suggest                                                                                            
suggest.total                    | suto,suggestTotal              | number of suggest ops                                                                                            
pri.suggest.total                |                                | number of suggest ops                                                                                            
memory.total                     | tm,memoryTotal                 | total used memory                                                                                                
pri.memory.total                 |                                | total user memory                                                                                                
[elk@es logs]$ 

3:创建索引

[elk@es logs]$ curl -X PUT 'localhost:9250/abctest'
{"acknowledged":true,"shards_acknowledged":true}
[elk@es logs]$ 

创建索引名称,必须是小写,并且索引名称一旦创建,则不可以修改,acknowledged表示执行结果,true或false

4:删除索引

删除索引,只需要将PUT替换为DELETE即可

[elk@es logs]$ curl -X DELETE 'localhost:9250/abctest'
{"acknowledged":true}
[elk@es logs]$
点赞
收藏
评论区
推荐文章
Wesley13 Wesley13
3年前
mongo复制集脑裂问题如何处理
mongo replication 脑裂问题如何处理:一、问题描述:一套mongo replication有4个节点。1个仲裁节点。在停止实例(或实例毁坏)的时候,导致所有节点都变为SECONDARY节点。不管如何重启节点。或者停止到最后一个节点。此时mongo节点都是SECONDARY。二、出现的现象:1、如果此时所有节点都重新
Stella981 Stella981
3年前
Centos7.6部署redis6.0.8集群(redis cluster模式)
集群规划:三台服务器各部署一个主节点、一个从节点,同一台服务器上不是直接主从关系。服务器角色ip:端口node1master1/slave2192.168.146.199:6379/192.168.146.199:6380node2master2/slave3192.168.146.200:6379/192.168.1
Stella981 Stella981
3年前
PostgreSQL与MySQL比较
特性MySQLPostgreSQL实例通过执行MySQL命令(mysqld)启动实例。一个实例可以管理一个或多个数据库。一台服务器可以运行多个mysqld实例。一个实例管理器可以监视mysqld的各个实例。通过执行Postmaster进程(pg\_ctl)启动实例。一个实例可以管理一个或多个数据库,这些数据库组成一个
Easter79 Easter79
3年前
Swarm 中最重要的概念
第94篇Swarm 中最重要的概念从主机的层面来看,DockerSwarm管理的是DockerHost集群。所以先来讨论一个重要的概念集群化(Clustering)。服务器集群由一组网络上相互连接的服务器组成,它们一起协同工作。一个集群和一堆服务器最显著的区别在于:集群能够像 单个 系统那样工作,同时提
Wesley13 Wesley13
3年前
001.SQLServer高可用简介
一SQLServer高可用集群相关概念1.1Windows故障转移群集Windows故障转移群集是由多个服务器组成的共同提供某高可用服务,该服务用于防止单台服务器故障导致服务失效。故障转移群集是一种高可用性的基础结构层,由多台计算机组成,每台计算机相当于一个冗余节点,整个群集系统允许某部分节点掉线
Stella981 Stella981
3年前
Linux集群架构
一、集群介绍集群,简单地说是指一组(若干个)相互独立的计算机,利用高速通信网络组成一个较大的计算机服务系统,每个集群节点(即集群中的每台计算机)都是运行各自服务的独立服务器。集群的特点:高性能(Performance)、价格有效(Costeffectiveness)、可伸缩性(Scalability)、高可用性(Ava
Stella981 Stella981
3年前
Mongodb集群节点故障恢复场景分析(转)
一个适当配置的Mongodb分片集群是没有单点故障。本文描述了分片集群中存在的几种不同的潜在的节点故障场景,以及Mongodb对这些节点故障是怎么处理的。1、Mongos节点宕机一个Mongos进程应该运行在每一个应用程序服务器上,这个服务器应该独占这个Mongos进程,并且通过它与分片集群来通讯。Mongos进程不是持久化的,相反,
Stella981 Stella981
3年前
Linux实战教学笔记30:Nginx反向代理与负载均衡应用实践
1.1集群简介简单地说,集群就是指一组(若干个)相互独立的计算机,利用高速通信网络组成的一个较大的计算机服务系统,每个集群节点(即集群中的每台计算机)都是运行各自服务的独立服务器。这些服务器之间可以彼此通信,协同向用户提供应用程序,系统资源和数据,并以单一系统的模式加以管理。当用户客户机请求集群系统时,集群给用户的感觉就是
Wesley13 Wesley13
3年前
ES 源代码阅读(二)
1基本概念集群:一个集群有一个或多个节点组织在一起,并将数据组织在一起,提供索引和搜索服务.节点:一个节点是一个集群中的服务器,提供存储数据,提供搜索服务.索引:文档的逻辑的集合分片:一个逻辑索引有若干分片,其中一个分片被设置为主分片.分片为索引的存储位置.会涉及到分布式问题.类型:文档的类型文档:与lucene中的d
京东云开发者 京东云开发者
11个月前
ElasticSearch集群灾难:别放弃,也许能再抢救一下 | 京东云技术团队
1前言Elasticsearch作为一个分布式搜索引擎,自身是高可用的;但也架不住一些特殊情况的发生,如:集群超过半数的master节点丢失,ES的节点无法形成一个集群,进而导致集群不可用;索引shard的文件损坏,分片无法被正常恢复,进而导致索引无法正常