CodeSolutionsHub

Introduction

OpenSearch, a powerful open-source search and analytics suite, is designed to handle the ever-growing demand for scalable and reliable data search and visualization. Whether used for real-time log analytics, application monitoring, or complex data queries, OpenSearch provides robust solutions for data-intensive applications. Deploying OpenSearch locally on Minikube allows developers to experiment, configure, and troubleshoot in a controlled, cost-effective environment.

In this guide, we’ll walk through deploying OpenSearch and OpenSearch Dashboards on Minikube using Kubernetes manifests. Along the way, we’ll discuss strategies for making your local deployment efficient and reliable, so you’re better prepared to transition to a production environment when the time comes.

Why Use Minikube for OpenSearch?

Using Minikube for OpenSearch offers several key benefits:

• Experiment Safely: Make adjustments to your OpenSearch configurations without risking production data or infrastructure.
• Rapid Iteration: Test new plugins, data sources, or configurations locally, minimizing the cycle time for testing and deployment.
• Resource Savings: Minikube is open-source and doesn’t require cloud resources, making it a great solution for early-stage development or testing on a budget.
• Skill Building: Working with Kubernetes and OpenSearch locally provides hands-on learning that translates well to larger-scale production setups.

By deploying on Minikube, you gain the flexibility to build and refine configurations, experiment with cluster setup, and even troubleshoot security configurations without leaving your development environment.

Prerequisites

To get started, ensure you have the following tools installed:

• Minikube: Install it from here.
• Kubectl: Download it from here.

Deployment Steps

Step 1: Set Up Your Local Environment

Start by cloning the Git repository containing all necessary manifests and configuration files:

git clone https://github.com/kaushal540/opensearch-minikube
cd opensearch-minikube

This repository includes everything needed to spin up OpenSearch, including security configurations and storage settings.

Step 2: Start Minikube

Next, initialize your Minikube cluster. We recommend configuring Minikube with enough memory and CPU resources to handle OpenSearch and OpenSearch Dashboards efficiently.

minikube start --cpus=4 --memory=8192mb

Step 3: Set Up Secure Connections

To ensure secure communication, generate SSL/TLS certificates with the provided script, generate_certs.sh. This script creates certificates for OpenSearch, which you’ll add as secrets in your Kubernetes environment.

bash generate_certs.sh

Encode each certificate and key in Base64 format to use them in Kubernetes secrets. For example:

base64 -i certificates/opensearch/opensearch.key -o opensearch.key.base64

Add these encoded values to manifests/secret.yaml for secure storage in Kubernetes.

Step 4: Deploy OpenSearch and OpenSearch Dashboards

Once certificates are configured, apply each manifest in the appropriate order:

kubectl apply -f manifests/namespace.yaml
kubectl apply -f manifests/configmap.yaml
kubectl apply -f manifests/opensearch-pv-pvc.yaml
kubectl apply -f manifests/secret.yaml
kubectl apply -f manifests/opensearch-deployment.yaml
kubectl apply -f manifests/opensearch-dashboards.yaml

With this setup, you’ll have a separate namespace for OpenSearch, configuration data, persistent storage for data, and deployments for both OpenSearch and OpenSearch Dashboards.

Step 5: Initialize Security Settings

To activate OpenSearch’s security features, apply the opensearch-security-init.yaml manifest once the main OpenSearch pod is running. This initializes the security plugin, providing built-in protections against unauthorized access.

kubectl apply -f manifests/opensearch-security-init.yaml

Step 6: Accessing OpenSearch and OpenSearch Dashboards

To access your OpenSearch setup, you have two options:

Using Minikube Service Command: This opens the services in your default browser:

minikube service opensearch-service -n opensearch
minikube service opensearch-dashboards-service -n opensearch

OR

Using Port Forwarding: An alternative that forwards ports to your localhost:

kubectl port-forward service/opensearch-service 9200:9200 -n opensearch
kubectl port-forward service/opensearch-dashboards-service 5601:5601 -n opensearch

Optimizing Your Local OpenSearch Deployment

Resource Management

By default, Kubernetes manifests may set low resource limits. For better performance, especially on larger data sets or complex queries, allocate more CPU and memory in the opensearch-deployment.yaml and opensearch-dashboards.yaml files.

Custom Configuration

Use the configmap.yaml to tailor OpenSearch’s settings to your specific requirements. For example, you can modify parameters such as shard allocation, thread pool sizes, or logging levels to improve response times or debugging output.

Data Persistence

Persistent storage is essential for keeping your data intact between sessions. The opensearch-pv-pvc.yaml defines a persistent volume claim, which allocates space on Minikube’s VM. You can adjust the storage size here to accommodate larger data sets as needed.

Cleanup Process

To remove all resources and clean up your environment, run:

kubectl delete -f manifests/

This will delete the namespace, deployments, ConfigMaps, and other resources created by the setup, allowing you to start fresh if needed.

Troubleshooting Common Issues

High Memory Usage

If you notice high memory usage, consider reducing the number of OpenSearch nodes or adjusting the JVM heap size in the deployment manifest.

Port Conflicts

If you encounter conflicts on ports 9200 or 5601, ensure no other processes are using these ports. Alternatively, you can modify the port-forwarding configuration to use different ports.

Certificate Errors

Certificate issues can arise if the Base64 encoding step is skipped or incorrectly configured. Double-check the base64 commands and verify that the secret.yaml file contains the correct encoded values.

Conclusion

Setting up OpenSearch on Minikube is a valuable practice for developers looking to build and test search and analytics applications in a local Kubernetes environment. This setup enables you to experiment with configurations, test security settings, and become comfortable with OpenSearch’s capabilities. Whether you’re preparing for a production deployment or just exploring OpenSearch’s features, Minikube provides an excellent starting point for developing search and analytics solutions.

Reference: GitHub Repository