Unsecured Elasticsearch and Hadoop clusters leak petabytes of data
Thousands of unsecured Elasticsearch and Hadoop clusters can be found on the Internet that leak petabytes of data, according to an inventory by search engine Shodan. In recent years, numerous data breaches have been in the news due to Elasticsearch and Hadoop installations that were accessible to everyone on the Internet. Despite these reports, the situation has only gotten worse.
Elasticsearch is search engine software for indexing all kinds of information. It is used, among other things, for searching websites, documents and applications, but can also be used for analytics, monitoring and data analysis. Last year, Elastic, developer of the software, decided to make a number of security measures available for the free version of the platform.
Nevertheless, figures from search engine Shodan show that the amount of data that can be found via unsecured Elasticsearch clusters has only increased. In 2018 it was still about 900 terabytes, which has now risen to 3.2 petabytes. In total, Shodan found 28,000 Elasticsearch clusters that are accessible via the internet.
The number of unsecured Hadoop clusters has actually decreased since 2018, but the data that these clusters contain actually increased. The Hadoop Distributed File System (HDFS) is a distributed file system for storing large amounts of data. In 2018, Shodan had around 4,000 unsecured HDFS clusters. That has now fallen to some eight hundred clusters. However, the amount of data leaked by these clusters increased from 5.1 petabytes in 2018 to 13.1 petabytes this year. Most of the unsecured Elasticsearch and Hadoop clusters are located in China.
Sources: https://blog.shodan.io/elastic-data-exposure-grows-to-3-2-pb/
