25 minutes
Prerequisites:
Use Hazelcast Caching in Open Liberty and Spring Boot based Microservices and deploy to Kubernetes
You will learn how to use Hazelcast distributed caching with Spring Boot, bundle with openliberty and deploy to a local Kubernetes cluster. You will then create a Kubernetes Service which load balance between containers and verify that you can share data between Microservices.
The microservice you will deploy is called hazelcast-caching. The hazelcast-caching microservice simply
helps you put a data and read it back. As Kubernetes Service will send the request to different pod each time
you initiate the request, the data will be served by shared hazelcast cluster between hazelcast-caching pods.
You will use a local single-node Kubernetes cluster. However, you can deploy this application on any kubernetes distributions such as IBM Cloud Private.
Hazelcast is an open source In-Memory Data Grid (IMDG). It provides elastically scalable distributed In-Memory computing, widely recognized as the fastest and most scalable approach to application performance.
Hazelcast is designed to scale up to hundreds and thousands of members. Simply add new members and they will automatically discover the cluster and will linearly increase both memory and processing capacity
Spring Boot makes it easy to create stand-alone, production-grade Spring based Applications that you can "just run". To learn more about Spring Boot. http://spring.io/projects/spring-boot
Before you begin, you need a containerization software for building containers. Kubernetes supports various container runtimes. You will use Docker in this guide. For Docker installation instructions, refer to the official Docker documentation.
WINDOWS
MAC
LINUX
Use Docker Desktop, where a local Kubernetes environment is pre-installed and enabled. If you do not see the Kubernetes tab, then upgrade to the latest version of Docker Desktop.
Complete the setup for your operating system:
Set up Docker for Windows.
After you complete the Docker setup instructions for your operating system, ensure that Kubernetes (not Swarm) is selected as the orchestrator in Docker Preferences.
Use Docker Desktop, where a local Kubernetes environment is pre-installed and enabled. If you do not see the Kubernetes tab, then upgrade to the latest version of Docker Desktop.
Complete the setup for your operating system:
Set up Docker for Mac.
After you complete the Docker setup instructions for your operating system, ensure that Kubernetes (not Swarm) is selected as the orchestrator in Docker Preferences.
You will use Minikube as a single-node Kubernetes cluster that runs locally in a virtual machine.
Make sure you have kubectl installed. If you need to install kubectl, see the kubectl installation instructions.
For Minikube installation instructions, see the Minikube documentation.
The fastest way to work through this guide is to clone the Git repository and use the projects that are provided inside:
git clone https://github.com/openliberty/guide-caching-microservices-hazelcast.git
cd guide-caching-microservices-hazelcastThe start directory contains the starting project that you will build upon.
The finish directory contains the finished project that you will build.
Before you begin, make sure you have all the necessary prerequisites.
Start your Kubernetes cluster.
WINDOWS
MAC
LINUX
Start your Docker Desktop environment.
Ensure that Kubernetes is running on Docker Desktop and that the context is set to docker-desktop.
Run the following command from a command-line session:
minikube startNext, validate that you have a healthy Kubernetes environment by running the following command from the active command-line session.
kubectl get nodesThis command should return a Ready status for the master node.
WINDOWS
MAC
LINUX
You do not need to do any other step.
Run the following command to configure the Docker CLI to use Minikube’s Docker daemon. After you run this command, you will be able to interact with Minikube’s Docker daemon and build new images directly to it from your host machine:
eval $(minikube docker-env)The microservice in the start directory has in-process cache which is not accessible between microservices.
Firstly, you need to configure Hazelcast to be used in Spring Boot Application.
replace Application.java with the following java class.
package io.openliberty.guides.hazelcast;
import com.hazelcast.config.Config;
import com.hazelcast.config.JoinConfig;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
@SpringBootApplication
public class Application {
private static Config config;
@Bean
public Config hazelcastConfig() {
Config config = new Config();
config.setProperty( "hazelcast.logging.type", "slf4j" );
JoinConfig joinConfig = config.getNetworkConfig().getJoin();
joinConfig.getMulticastConfig().setEnabled(false);
joinConfig.getKubernetesConfig().setEnabled(true);
return config;
}
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}This configuration will tell Spring Boot what kind of configuration is needed when each hazelcast member is started.
Now it is time to use Hazelcast in the hazelcast-caching microservice.
if you check the source code of CommandController.java, you would notice that ConcurrentHashMap is used
for backing key value store. In order to make this cache distributed, replace CommandController.java
with Hazelcast version.
package io.openliberty.guides.hazelcast;
import com.hazelcast.core.HazelcastInstance;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import java.util.concurrent.ConcurrentMap;
@RestController
public class CommandController {
@Value("#{environment.MY_POD_NAME}")
private String podName;
@Autowired
HazelcastInstance hazelcastInstance;
private ConcurrentMap<String,String> retrieveMap() {
return hazelcastInstance.getMap("map");
}
@RequestMapping("/put")
public CommandResponse put(@RequestParam(value = "key") String key, @RequestParam(value = "value") String value) {
retrieveMap().put(key, value);
return new CommandResponse(value,podName);
}
@RequestMapping("/get")
public CommandResponse get(@RequestParam(value = "key") String key) {
String value = retrieveMap().get(key);
return new CommandResponse(value,podName);
}
}You would notice that getting hazelcast member is Autowired by Spring Boot and made available to the CommandController
and there is no change in /put and /get endpoints.
The first step of deploying to Kubernetes is to build your microservices and containerize them with Docker.
The starting Java project, which you can find in the start directory, is a spring boot based microservice.
It is called hazelcast-caching which has two simple endpoints, /put and /get. This guide uses boost-maven-plugin
to bundle spring boot app and openliberty together and build docker image for the deployment.
Boost plugin does not require you to create your own Dockerfile but instead opionated docker image.
If you wanna learn more about boost maven plugin, you can visit the project website.
https://github.com/OpenLiberty/boost/tree/master/boost-maven
If you want to create your own Dockerfile then you can always use dockerfile-maven plugin. please refer to https://openliberty.io/guides/kubernetes-intro.html#starting-and-preparing-your-cluster-for-deployment
To build hazelcast-caching microservice, run the following command in the start folder.
This will build the docker image and push it to docker registry provided by minikube
mvn packageExecute following command to see available docker images
docker imagesVerify that the hazelcast-caching:latest is listed among them, for example:
If you don’t see the hazelcast-caching:latest image, then check the Maven build log for any potential errors.
In addition, if you are using Minikube, make sure your Docker CLI is configured to use Minikube’s Docker daemon and not your host’s as described in the previous section.
Now that your Docker images are built, deploy them using a Kubernetes resource definition.
To deploy the hazelcast-caching microservice, first create the kubernetes.yaml file in the start directory:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: hazelcast-caching-statefulset
labels:
app: hazelcast-caching
spec:
replicas: 2
serviceName: hazelcast-caching-service
selector:
matchLabels:
app: hazelcast-caching
template:
metadata:
labels:
app: hazelcast-caching
spec:
containers:
- name: hazelcast-caching-container
image: hazelcast-caching:latest
imagePullPolicy: IfNotPresent
ports:
- name: openliberty
containerPort: 9080
env:
- name: MY_POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
---
apiVersion: v1
kind: Service
metadata:
name: hazelcast-caching-service
spec:
type: NodePort
selector:
app: hazelcast-caching
ports:
- protocol: TCP
port: 9080
targetPort: 9080
nodePort: 31000This file defines two Kubernetes resources: one statefulset and one service. StatefulSet is preferred solution for Hazelcast because it enables controlled scale out/in of your microservices for easy data distribution. To learn more about StatefulSet, you can visit Kubernetes documentation https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/
By default, we create 2 replicas of hazelcast-caching microservice behind the hazelcast-caching-service which forwards
requests to one of the pods available in the kubernetes cluster.
MY_POD_NAME is an environment variable made available to the pods so that each microservice knows which pod they are in.
This is going to be used in this guide in order to show which pod is responding to the http request.
As a second step, we will create rbac.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: default-cluster
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: view
subjects:
- kind: ServiceAccount
name: default
namespace: defaultRole Based Access Controller(RBAC) configuration is used to give access to Kubernetes Master API from pods which runs microservices. Hazelcast requires a read access to autodiscover other hazelcast members and form hazelcast cluster.
Run the following commands to deploy the resources as defined in kubernetes.yaml and rbac.yaml in the specified order:
kubectl apply -f rbac.yamlkubectl apply -f kubernetes.yamlrun the following command to check the status of your pods:
kubectl get podsYou’ll see an output similar to the following if all the pods are healthy and running:
NAME READY STATUS RESTARTS AGE
hazelcast-caching-statefulset-0 1/1 Running 0 20m
hazelcast-caching-statefulset-1 1/1 Running 0 20mYou should also check if hazelcast cluster is formed by checking one of the pod’s log file:
kubectl logs hazelcast-caching-statefulset-1Members {size:2, ver:2} [
Member [172.17.0.4]:5701 - 71009ef7-ee18-45f0-8a8f-e9321931e9ce this
Member [172.17.0.5]:5701 - 99222e16-93e5-4453-ac9e-cdf3e80069c6
]Next you will make requests to your services.
Open your favorite terminal and send http get requests in a loop via curl command.
This request asks for the value of key=1 and prints value and which pod has replied to the request.
You might need to wait up to 30 sec before microservices accepts traffic.
while true; do curl 192.168.99.100:31000/get?key=1;echo; sleep 2; doneYou should see an output like below. The value is null because we have not put any data yet.
podname shows that which kubernetes pod replied to the request.
{"value":null,"podName":"hazelcast-caching-statefulset-1"}
{"value":null,"podName":"hazelcast-caching-statefulset-1"}
{"value":null,"podName":"hazelcast-caching-statefulset-0"}
{"value":null,"podName":"hazelcast-caching-statefulset-1"}
{"value":null,"podName":"hazelcast-caching-statefulset-0"}Break the current loop and put some data into hazelcast-caching microservice.
curl "http://192.168.99.100:31000/put?key=1&value=hazelcast_springboot_openliberty"Although request has been executed by one specific kubernetes pod, by querying the data in a loop will show data the data is actually shared by multiple pods. To see this, execute following command again.
while true; do curl 192.168.99.100:31000/get?key=1;echo; sleep 2; doneAs you can see both pods are returning the same data.
{"value":"hazelcast_springboot_openliberty","podName":"hazelcast-caching-statefulset-1"}
{"value":"hazelcast_springboot_openliberty","podName":"hazelcast-caching-statefulset-0"}
{"value":"hazelcast_springboot_openliberty","podName":"hazelcast-caching-statefulset-1"}
{"value":"hazelcast_springboot_openliberty","podName":"hazelcast-caching-statefulset-0"}
{"value":"hazelcast_springboot_openliberty","podName":"hazelcast-caching-statefulset-1"}Scale the cluster with one more pod and see that you still retrieve the shared data.
kubectl scale statefulset hazelcast-caching-statefulset --replicas=3Run following command to see the latest status of the pods
kubectl get podsAs you can see, a new pod hazelcast-caching-statefulset-2 has joined to the cluster.
NAME READY STATUS RESTARTS AGE
hazelcast-caching-statefulset-0 1/1 Running 0 8m
hazelcast-caching-statefulset-1 1/1 Running 0 8m
hazelcast-caching-statefulset-2 1/1 Running 0 31sRun the following command again to see the output
while true; do curl 192.168.99.100:31000/get?key=1;echo; sleep 2; doneAs you can see, hazelcast-caching-statefulset-2 is returning correct data.
{"value":"hazelcast_springboot_openliberty","podName":"hazelcast-caching-statefulset-1"}
{"value":"hazelcast_springboot_openliberty","podName":"hazelcast-caching-statefulset-2"}
{"value":"hazelcast_springboot_openliberty","podName":"hazelcast-caching-statefulset-0"}Create aHazelcastCachingITclass.src/test/java/it/io/openliberty/guides/hazelcast/HazelcastCachingIT.java
HazelcastCachingIT.java
1/*
2 * Copyright 2014-2016 the original author or authors.
3 *
4 * Licensed under the Apache License, Version 2.0 (the "License");
5 * you may not use this file except in compliance with the License.
6 * You may obtain a copy of the License at
7 *
8 * http://www.apache.org/licenses/LICENSE-2.0
9 *
10 * Unless required by applicable law or agreed to in writing, software
11 * distributed under the License is distributed on an "AS IS" BASIS,
12 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13 * See the License for the specific language governing permissions and
14 * limitations under the License.
15 */
16package it.io.openliberty.guides.hazelcast;
17
18import io.openliberty.guides.hazelcast.CommandResponse;
19import org.junit.Assert;
20import org.junit.BeforeClass;
21import org.junit.Test;
22import org.springframework.http.HttpStatus;
23import org.springframework.http.ResponseEntity;
24import org.springframework.web.client.RestTemplate;
25
26
27public class HazelcastCachingIT {
28
29 private static String clusterUrl;
30
31
32 @BeforeClass
33 public static void oneTimeSetup() {
34 String clusterIp = System.getProperty("cluster.ip");
35 String nodePort = System.getProperty("cluster.port");
36 clusterUrl = "http://" + clusterIp + ":" + nodePort ;
37 }
38
39
40 @Test(timeout = 60000)
41 public void testHazelcastCache()
42 throws Exception {
43
44
45 String key = "1";
46 String value = "hazelcast-springboot-openliberty";
47
48 String put_url = String.format(clusterUrl +"/put?key=%s&value=%s", key, value);
49 RestTemplate rest = new RestTemplate();
50
51 ResponseEntity<CommandResponse> putResponse = rest.getForEntity(put_url, CommandResponse.class);
52 Assert.assertTrue(putResponse.getStatusCode() == HttpStatus.OK);
53
54 String firstPod = putResponse.getBody().getPodName();
55
56 //GET call to see data is coming from another pod
57 String get_url = String.format(clusterUrl+"/get?key=%s", key);
58
59 while (true) { // it will try every second until it timeouts in 60 seconds
60 ResponseEntity<CommandResponse> getResponse = rest.getForEntity(get_url, CommandResponse.class);
61 String secondPod = getResponse.getBody().getPodName();
62 if (!secondPod.equals(firstPod)) break; // we get the response from different pod so SUCCESS!!
63 Thread.sleep(1000);
64 }
65 }
66
67}The testHazelcastCache test makes sure that the /put endpoint is handled by one pod
and /get methods returns the same data from the other kubernetes pod.
It first puts a key/value pair to hazelcast-caching microservice and keeps podname in the firstpod
variable. In the second part, tests submits multiple /get requests until to see that podname is different then the pod
which initially handled /put request.
In order to run integration tests, you must have a running hazelcast-caching microservices
in minikube environment. As you have gone through all previous steps, you already have it.
The default properties defined in the pom.xml are:
| Property | Description |
|---|---|
|
IP or hostname for your cluster, |
|
Port of the Kubernetes Service wrapping the |
Navigate back to the start directory.
Thank you for your feedback!
Raise an issue to share feedback
Create a pull request to contribute to this guide
Ask a question on Stack Overflow
