Elasticsearch has rapidly—and deservedly—established itself as a popular choice for enterprise application developers. The one negative associated with the highly capable distributed open-source search and log analytics engine is that it has garnered headlines for security hiccups. This reputation for security is arguably undeserved. I think it says more about the fact that organizations too often fail to treat Elasticsearch security with the respect required for any data storage solution. With the proper attention and an accurate understanding of Elasticsearch’s specific needs, the technology can be made sufficiently secure for enterprise needs.
The basic default Elasticsearch configuration lacks enterprise-grade security features. This combination—a simple deployment that’s just as easy to then ignore when it comes to subsequent security hardening—can easily lead to lax access restrictions and data protection. And, it has, as those aforementioned headlines have shown over the past couple of years. But, by implementing enterprise-grade security and adhering to best practices, enterprises can eliminate the errors that put Elasticsearch data in peril.
Understand Your Two Options for Enterprise-Grade Elasticsearch Security
For most of Elasticsearch’s existence, the Elastic Stack extension X-Pack has been the single reliable choice for effective Elasticsearch security. But, it’s also a matter of getting what you pay for: X-Pack’s enterprise-grade security requires buying Elastic’s expensive enterprise subscription.
A second option emerged in 2019 with the debut of the Amazon-backed Open Distro for Elasticsearch. The open-source project provides numerous enterprise-grade security features including encryption of data-in-transit utilizing OpenSSL or TLS 1.2. With this encryption, Open Distro for Elasticsearch equips enterprises to meet regulatory compliance rules around data protection, easily integrate with public key infrastructures, and secure both internal cluster node and external client traffic.
The Open Distro also integrates with authentication infrastructures, enabling enterprises to leverage LDAP/Active Directory, Kerberos, SAML, and other widely-used authentication protocols. The Open Distro offers granular role-based access controls (RBACs) as well in order to restrict individual user access to only their required cluster operations, documents, indices, and fields. The open-source option further provides informed responses to security incidents, as well as audit logs and monitoring that tracks and records all user actions, as required by governmental and industry regulatory compliance frameworks.
X-Pack offers a largely equivalent collection of features, including SSL or TLS encryption, user authentication, RBACs, IP filtering, compliant audit trails, and security monitoring, and more. X-Pack also now features an open code base with many useful features, such as monitoring, available to use for free. However, because licensing fees are still required in order to utilize the full features of X-Pack, it remains the considerably more expensive choice for achieving Elasticsearch security.
Enterprise Elasticsearch Security Best Practices
An enterprise’s Elasticsearch security strategy, whichever technology supports it, should incorporate these best practices:
- Encrypt all internal and external data traffic: Leverage TLS and be sure to encrypt both traffic inside the Elasticsearch cluster and all data source traffic that connects to the cluster.
- Tightly control access: Leverage RBACs and secure methods of authentication to closely control access to all indices, documents, and other sensitive elements of Elasticsearch.
- Put audit logging in place: Implement audit logs so that all users’ actions within the Elasticsearch cluster are tracked. Monitor those logs for suspicious activities and leverage this information to respond to security incidents more effectively.
- Only expose the Elasticsearch cluster to the internet if required and with protections: Studying Elasticsearch security breach incidents, a common error occurs when teams put convenience ahead of security, and expose Elasticsearch to the open internet during development and testing phases. The teams that make headlines neglect to shift Elasticsearch to a secure configuration when deploying to production, leaving data exposed to external access. When Elasticsearch exposure to the internet is necessary, safeguard internet-facing servers with secure configurations, firewalls, access controls based on least-privilege policies, proxies, and further available security measures.
- Enlist provider support when required: In many cases, enterprises may find it expedient to add external support and expertise from a managed Elasticsearch provider experienced in executing effective security strategies. Providers can apply their knowledge to meet an enterprise’s specific regulatory compliance requirements around data security, and implement key features out-of-the-box, including encryption, access controls, monitoring, and alerts.
Elasticsearch is a valuable solution that can be leveraged safely and securely with the right data security strategy and protections in place. Enterprises should approach Elasticsearch with the same care that any data-layer solution calls for and align their deployments with best practices to optimize both their data security and the benefits Elasticsearch delivers.
By Mussa Shirazi, the Senior Consultant at Instaclustr