Elasticsearch Upgrade to 8.0

Breaking changes of elasticsearch have been places in below page,

https://www.elastic.co/guide/en/elasticsearch/reference/master/breaking-changes-8.0.html

https://www.elastic.co/guide/en/elasticsearch/reference/master/migrate-to-java-time.html

Below are my notes on my learnings on elastic search with the upgrade.

1. In Elasticsearch 7.x, allowing multiple types in Indices were removed. Upgrade to 8.0 possible only from 7.x, no upgrade offered from 6.x

2. There is a REST API _xpack change, not sure why this has been changed, not sure of its usage except for login & security options provided by Elasticsearch as a proprietary soln.

3. The percentiles aggregation’s percents parameter no longer supports duplicate values.

I could learn what is a percentile, percentile range and how rankings are calculated. One of my reference is here. It is an interesting topic in terms of algorithms.

Elasticsearch reference

https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-percentile-aggregation.html#_missing_value_9

4. Block Allocation Changes - An optional flag has been removed.

The block allocation are done on each node, it is important the block allocation does not exceed the capacity of the disk. This is a very rare situation but not impossible situation.

https://opster.com/elasticsearch-glossary/elasticsearch-flood-stage-disk-watermark/

5. The nGram and edgeNGram token filter names have been removed. Actually they are renamed as ngram or edge_ngram

https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-edgengram-tokenfilter.html

6. Circuit breaker API name change

use the inflight_requests stat in place of in_flight_requests.

Circuit breaker count no. of failures before declaring a node/component/request/ram dead.

https://www.elastic.co/guide/en/elasticsearch/reference/current/circuit-breaker.html

7. Clustering Changes - The voting configuration exclusions API endpoint has changed.

 The cluster.join.timeout setting has been removed.

https://www.elastic.co/guide/en/elasticsearch/reference/current/add-elasticsearch-nodes.html

https://www.elastic.co/guide/en/elasticsearch/reference/current/restart-cluster.html

8. Discovery - discovery.zen settings have been removed.

https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html

9. HTTP Changes & Network Changes - There are few changes, looks  some hacks have been removed.

10. Index lifecycle management changes

https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html

lifecycle policies can trigger actions such as:

• Rollover: Creates a new index for a rollover target when the existing index reaches a certain size, number of docs, or age. A rollover target can be either an index alias or a data stream.
• Shrink: Reduce the number of primary shards in an index.
• Force merge: Manually trigger a merge to reduce the number of segments in each shard of an index and free up the space used by deleted documents.
• Freeze: Make an index read-only and minimize its memory footprint.
• Delete: Permanently remove an index, including all of its data and metadata.

11. Indices Changes

The force merge API’s max_num_segments and only_expunge_deletes parameters cannot both be specified in the same request.

https://www.elastic.co/guide/en/elasticsearch/reference/7.x/indices-update-settings.html

12. Java API Changes

Fuzziness: To create Fuzziness instances, use the fromString and fromEdits method instead of the build method

https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-fuzzy-query.html#query-dsl-fuzzy-query

Repository has no dependency on IndexShard anymore. Improves testability

https://www.elastic.co/guide/en/elasticsearch/reference/current/snapshots-register-repository.html

13. Node settings

The node.max_local_storage_nodes setting was deprecated in 7.x and has been removed in 8.0

https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-node.html

14. Java 11 is required.

Elasticsearch 8.0 only works in JAVA 11.

15. Mapping API Changes,

The maximum number of completion contexts per field is now 10.

https://www.elastic.co/guide/en/elasticsearch/reference/current/search-suggesters.html

Mapping API endpoints containing mapping types have been removed.

Single typed since 7.x, so now there is no type in mapping defn.

Multi-fields within multi-fields is no longer supported.

https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-fields.html

The _field_names metadata field’s enabled parameter has been removed - Simple change

The boost parameter on field mappings has been removed

https://www.elastic.co/guide/en/elasticsearch/reference/6.8/search-request-index-boost.html

Java-time date formats replace joda-time formats

16. Reindex changes

"Type" in path removed. size parameter in delete by query and update by query removed.

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html

17. REST API changes - Local node query support has been removed.

18. Rollup error response change - 

An experimental feature - daily, weekly, monthly rollups

https://www.elastic.co/guide/en/elasticsearch/reference/current/rollup-overview.html

19. Search API changes, security, kibana users and role changes, snapshot & restore changes, thread pool and transport changes are few more changes that has been done.