ElasticSearch Size Management – Shards, logstash

Elasticsearch’s default capacity management concept is automatic capacity growth with the addition of nodes. If the capacity is insufficient, a new node is automatically added to the entire capacity of the added node.

Check out the URL below for more details.

https://www.elastic.co/videos/big-data-search-and-analytics

동영상 · 느긋이 누워 즐기세요 | Elastic Videos

If you want to manage capacity, you need to manage Shards size by index. Shards is the default storage unit managed by ElasticSearch, which allows you to set the maximum Index size.

https://www.elastic.co/blog/found-sizing-elasticsearch#over-allocation-and-routing

Sizing Elasticsearch

If you read the above article, you can manage it through the total size of shards when managing with single index. If it is generated based on time like logstash, you should manage it by deleting the index manually.

However, this is very easy and intuitive. If you want to delete the yearly data from logstash, you can easily delete the 2018 index via DELETE / logstash-2018*.

However, please keep in mind that big data systems such as Elasticsearch are used as simple commands, so care should be taken to ensure that clients accessing the system are able to use their privileges.