Distributed Caching at Scale: Hazelcast Integration in Java

Introduction

Hazelcast is a leading open-source in-memory computing platform that provides distributed caching, computing, and data storage capabilities. Unlike simple key-value stores like Memcached, Hazelcast offers advanced features including distributed data structures, event processing, and seamless clustering, making it ideal for high-performance, scalable Java applications.

Hazelcast vs Other Caching Solutions

Key Advantages

True distributed caching with automatic data partitioning
Built-in clustering with automatic node discovery
Rich data structures beyond simple key-value pairs
Persistence and off-heap memory options
Event listeners and continuous querying
ACID transactions support
Native Java integration with minimal configuration

Setup and Dependencies

Maven Dependencies

<properties> <hazelcast.version>5.3.6</hazelcast.version> </properties> <dependencies> <!-- Hazelcast Core --> <dependency> <groupId>com.hazelcast</groupId> <artifactId>hazelcast</artifactId> <version>${hazelcast.version}</version> </dependency> <!-- Spring Boot Starter --> <dependency> <groupId>com.hazelcast</groupId> <artifactId>hazelcast-spring</artifactId> <version>${hazelcast.version}</version> </dependency> <!-- For Management Center --> <dependency> <groupId>com.hazelcast</groupId> <artifactId>hazelcast-kubernetes</artifactId> <version>${hazelcast.version}</version> </dependency> </dependencies>

Configuration

1. Programmatic Configuration

package com.example.cache.config; import com.hazelcast.config.*; import com.hazelcast.core.Hazelcast; import com.hazelcast.core.HazelcastInstance; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; @Configuration public class HazelcastConfig { @Bean public Config hazelcastConfiguration() { Config config = new Config(); // Cluster name config.setClusterName("production-cluster"); // Network configuration NetworkConfig networkConfig = config.getNetworkConfig(); networkConfig.setPort(5701) .setPortAutoIncrement(true); JoinConfig joinConfig = networkConfig.getJoin(); joinConfig.getMulticastConfig().setEnabled(false); joinConfig.getTcpIpConfig().setEnabled(true) .addMember("192.168.1.10") .addMember("192.168.1.11"); // Alternatively, use Kubernetes discovery // joinConfig.getKubernetesConfig().setEnabled(true); // Map configuration for user cache MapConfig userMapConfig = new MapConfig(); userMapConfig.setName("users") .setBackupCount(2) .setTimeToLiveSeconds(3600) .setMaxIdleSeconds(1800); // Eviction policy userMapConfig.getEvictionConfig() .setEvictionPolicy(EvictionPolicy.LRU) .setMaxSizePolicy(MaxSizePolicy.PER_NODE) .setSize(10000); // Near cache for frequently accessed data userMapConfig.setNearCacheConfig(new NearCacheConfig() .setInMemoryFormat(InMemoryFormat.OBJECT) .setCacheLocalEntries(true)); config.addMapConfig(userMapConfig); // Product cache configuration MapConfig productMapConfig = new MapConfig(); productMapConfig.setName("products") .setTimeToLiveSeconds(86400) .setBackupCount(1); config.addMapConfig(productMapConfig); // Management Center config.setManagementCenterConfig( new ManagementCenterConfig() .setEnabled(true) .setUrl("http://localhost:8080/hazelcast-mancenter")); return config; } @Bean public HazelcastInstance hazelcastInstance(Config config) { return Hazelcast.newHazelcastInstance(config); } }

2. XML Configuration (hazelcast.xml)

<?xml version="1.0" encoding="UTF-8"?> <hazelcast xmlns="http://www.hazelcast.com/schema/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.hazelcast.com/schema/config http://www.hazelcast.com/schema/config/hazelcast-config-5.0.xsd"> <cluster-name>production-cluster</cluster-name> <network> <port auto-increment="true" port-count="100">5701</port> <join> <multicast enabled="false"/> <tcp-ip enabled="true"> <member>192.168.1.10</member> <member>192.168.1.11</member> </tcp-ip> </join> </network> <map name="users"> <backup-count>2</backup-count> <time-to-live-seconds>3600</time-to-live-seconds> <max-idle-seconds>1800</max-idle-seconds> <eviction eviction-policy="LRU" max-size-policy="PER_NODE" size="10000"/> <near-cache> <in-memory-format>OBJECT</in-memory-format> <cache-local-entries>true</cache-local-entries> </near-cache> </map> <map name="products"> <time-to-live-seconds>86400</time-to-live-seconds> <backup-count>1</backup-count> </map> <management-center enabled="true"> <url>http://localhost:8080/hazelcast-mancenter</url> </management-center> </hazelcast>

Core Caching Implementation

1. Generic Cache Service

package com.example.cache.service; import com.hazelcast.core.HazelcastInstance; import com.hazelcast.map.IMap; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.stereotype.Service; import java.util.Collection; import java.util.Map; import java.util.Set; import java.util.concurrent.TimeUnit; @Service public class HazelcastCacheService { private static final Logger logger = LoggerFactory.getLogger(HazelcastCacheService.class); private final HazelcastInstance hazelcastInstance; public HazelcastCacheService(HazelcastInstance hazelcastInstance) { this.hazelcastInstance = hazelcastInstance; } // Basic cache operations public <K, V> void put(String cacheName, K key, V value) { IMap<K, V> map = hazelcastInstance.getMap(cacheName); map.put(key, value); } public <K, V> void putWithTTL(String cacheName, K key, V value, long ttl, TimeUnit timeUnit) { IMap<K, V> map = hazelcastInstance.getMap(cacheName); map.put(key, value, ttl, timeUnit); } public <K, V> V get(String cacheName, K key) { IMap<K, V> map = hazelcastInstance.getMap(cacheName); return map.get(key); } public <K, V> boolean containsKey(String cacheName, K key) { IMap<K, V> map = hazelcastInstance.getMap(cacheName); return map.containsKey(key); } public <K, V> void remove(String cacheName, K key) { IMap<K, V> map = hazelcastInstance.getMap(cacheName); map.delete(key); } // Bulk operations public <K, V> void putAll(String cacheName, Map<K, V> entries) { IMap<K, V> map = hazelcastInstance.getMap(cacheName); map.putAll(entries); } public <K, V> Map<K, V> getAll(String cacheName, Set<K> keys) { IMap<K, V> map = hazelcastInstance.getMap(cacheName); return map.getAll(keys); } // Cache statistics public <K, V> long getSize(String cacheName) { IMap<K, V> map = hazelcastInstance.getMap(cacheName); return map.size(); } public <K, V> void clear(String cacheName) { IMap<K, V> map = hazelcastInstance.getMap(cacheName); map.clear(); } // Advanced operations public <K, V> V getOrDefault(String cacheName, K key, V defaultValue) { IMap<K, V> map = hazelcastInstance.getMap(cacheName); return map.getOrDefault(key, defaultValue); } public <K, V> boolean putIfAbsent(String cacheName, K key, V value) { IMap<K, V> map = hazelcastInstance.getMap(cacheName); return map.putIfAbsent(key, value) == null; } public <K, V> V computeIfAbsent(String cacheName, K key, java.util.function.Function<K, V> mappingFunction) { IMap<K, V> map = hazelcastInstance.getMap(cacheName); return map.computeIfAbsent(key, mappingFunction); } }

2. Typed Cache Manager

package com.example.cache.manager; import com.example.cache.service.HazelcastCacheService; import com.fasterxml.jackson.core.JsonProcessingException; import com.fasterxml.jackson.databind.ObjectMapper; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.stereotype.Component; import java.util.concurrent.TimeUnit; @Component public class TypedHazelcastCacheManager { private static final Logger logger = LoggerFactory.getLogger(TypedHazelcastCacheManager.class); private final HazelcastCacheService cacheService; private final ObjectMapper objectMapper; // Cache names public static final String USER_CACHE = "users"; public static final String PRODUCT_CACHE = "products"; public static final String SESSION_CACHE = "sessions"; public static final String CONFIG_CACHE = "configurations"; public TypedHazelcastCacheManager(HazelcastCacheService cacheService, ObjectMapper objectMapper) { this.cacheService = cacheService; this.objectMapper = objectMapper; } // User cache operations public void cacheUser(String userId, Object userData) { cacheService.put(USER_CACHE, userId, userData); } public void cacheUserWithTTL(String userId, Object userData, long ttlMinutes) { cacheService.putWithTTL(USER_CACHE, userId, userData, ttlMinutes, TimeUnit.MINUTES); } public <T> T getUser(String userId, Class<T> clazz) { return cacheService.get(USER_CACHE, userId); } // Product cache operations public void cacheProduct(String productId, Object productData) { cacheService.put(PRODUCT_CACHE, productId, productData); } public <T> T getProduct(String productId, Class<T> clazz) { return cacheService.get(PRODUCT_CACHE, productId); } // Session management public void cacheSession(String sessionId, Object sessionData, long ttlMinutes) { cacheService.putWithTTL(SESSION_CACHE, sessionId, sessionData, ttlMinutes, TimeUnit.MINUTES); } public <T> T getSession(String sessionId, Class<T> clazz) { return cacheService.get(SESSION_CACHE, sessionId); } public void invalidateSession(String sessionId) { cacheService.remove(SESSION_CACHE, sessionId); } // JSON serialization for complex objects public <T> void cacheJson(String cacheName, String key, T value, long ttlMinutes) { try { String json = objectMapper.writeValueAsString(value); cacheService.putWithTTL(cacheName, key, json, ttlMinutes, TimeUnit.MINUTES); } catch (JsonProcessingException e) { logger.error("Failed to serialize object for caching", e); throw new RuntimeException("Serialization error", e); } } public <T> T getJson(String cacheName, String key, Class<T> clazz) { try { String json = cacheService.get(cacheName, key); return json != null ? objectMapper.readValue(json, clazz) : null; } catch (Exception e) { logger.error("Failed to deserialize cached object", e); return null; } } // Cache-aside pattern with loader public <T> T getOrLoad(String cacheName, String key, Class<T> clazz, CacheLoader<T> loader, long ttlMinutes) { T value = cacheService.get(cacheName, key); if (value == null) { value = loader.load(); if (value != null) { cacheService.putWithTTL(cacheName, key, value, ttlMinutes, TimeUnit.MINUTES); } } return value; } @FunctionalInterface public interface CacheLoader<T> { T load(); } }

Advanced Hazelcast Features

1. Distributed Data Structures

package com.example.cache.distributed; import com.hazelcast.core.HazelcastInstance; import com.hazelcast.cp.IAtomicLong; import com.hazelcast.multimap.MultiMap; import com.hazelcast.collection.IQueue; import com.hazelcast.collection.IList; import com.hazelcast.topic.ITopic; import org.springframework.stereotype.Component; @Component public class DistributedDataStructures { private final HazelcastInstance hazelcastInstance; public DistributedDataStructures(HazelcastInstance hazelcastInstance) { this.hazelcastInstance = hazelcastInstance; } // Distributed Atomic Long for counters public long incrementAndGet(String counterName) { IAtomicLong atomicLong = hazelcastInstance.getCPSubsystem().getAtomicLong(counterName); return atomicLong.incrementAndGet(); } public long getCounter(String counterName) { IAtomicLong atomicLong = hazelcastInstance.getCPSubsystem().getAtomicLong(counterName); return atomicLong.get(); } // MultiMap for one-to-many relationships public <K, V> void addToMultiMap(String mapName, K key, V value) { MultiMap<K, V> multiMap = hazelcastInstance.getMultiMap(mapName); multiMap.put(key, value); } public <K, V> Collection<V> getFromMultiMap(String mapName, K key) { MultiMap<K, V> multiMap = hazelcastInstance.getMultiMap(mapName); return multiMap.get(key); } // Distributed Queue for task processing public <T> void addToQueue(String queueName, T item) { IQueue<T> queue = hazelcastInstance.getQueue(queueName); queue.add(item); } public <T> T takeFromQueue(String queueName) throws InterruptedException { IQueue<T> queue = hazelcastInstance.getQueue(queueName); return queue.take(); } // Distributed Topic for pub/sub public <T> void publish(String topicName, T message) { ITopic<T> topic = hazelcastInstance.getTopic(topicName); topic.publish(message); } public <T> String subscribe(String topicName, com.hazelcast.core.MessageListener<T> listener) { ITopic<T> topic = hazelcastInstance.getTopic(topicName); return topic.addMessageListener(listener); } }

2. Event Listeners and Continuous Query

package com.example.cache.listeners; import com.hazelcast.core.EntryEvent; import com.hazelcast.core.EntryListener; import com.hazelcast.map.IMap; import com.hazelcast.map.MapEvent; import com.hazelcast.query.Predicate; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.stereotype.Component; import java.util.concurrent.ConcurrentHashMap; @Component public class CacheEventListeners { private static final Logger logger = LoggerFactory.getLogger(CacheEventListeners.class); private final Map<String, String> listenerRegistrations = new ConcurrentHashMap<>(); public <K, V> void addUserCacheListener(IMap<K, V> userMap) { String registrationId = userMap.addEntryListener(new EntryListener<K, V>() { @Override public void entryAdded(EntryEvent<K, V> event) { logger.info("User added to cache: {} -> {}", event.getKey(), event.getValue()); // Trigger business logic } @Override public void entryUpdated(EntryEvent<K, V> event) { logger.info("User updated in cache: {} -> {}", event.getKey(), event.getValue()); } @Override public void entryRemoved(EntryEvent<K, V> event) { logger.info("User removed from cache: {}", event.getKey()); } @Override public void entryEvicted(EntryEvent<K, V> event) { logger.info("User evicted from cache: {}", event.getKey()); } @Override public void mapEvicted(MapEvent event) { logger.info("User cache evicted: {}", event.getNumberOfEntriesAffected()); } @Override public void mapCleared(MapEvent event) { logger.info("User cache cleared"); } }, true); listenerRegistrations.put("userCache", registrationId); } // Continuous query for real-time updates public <K, V> void addContinuousQuery(IMap<K, V> map, Predicate<K, V> predicate) { map.addEntryListener(new EntryListener<K, V>() { @Override public void entryAdded(EntryEvent<K, V> event) { if (predicate.apply(event)) { handleMatchingEntry(event); } } @Override public void entryUpdated(EntryEvent<K, V> event) { if (predicate.apply(event)) { handleMatchingEntry(event); } } // ... other methods private void handleMatchingEntry(EntryEvent<K, V> event) { logger.info("Matching entry detected: {}", event.getKey()); // Process matching entries } }, predicate, true); } }

Spring Cache Integration

1. Hazelcast Spring Cache Manager

package com.example.cache.spring; import com.hazelcast.core.HazelcastInstance; import com.hazelcast.spring.cache.HazelcastCacheManager; import org.springframework.cache.CacheManager; import org.springframework.cache.annotation.EnableCaching; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.context.annotation.Primary; @Configuration @EnableCaching public class HazelcastCacheManagerConfig { @Bean @Primary public CacheManager cacheManager(HazelcastInstance hazelcastInstance) { return new HazelcastCacheManager(hazelcastInstance); } }

2. Using Spring Cache Annotations

package com.example.service; import org.springframework.cache.annotation.Cacheable; import org.springframework.cache.annotation.CacheEvict; import org.springframework.cache.annotation.CachePut; import org.springframework.cache.annotation.Caching; import org.springframework.stereotype.Service; import java.util.List; @Service public class UserService { @Cacheable(value = "users", key = "#userId") public User getUserById(String userId) { // This will only execute if user is not in cache return userRepository.findById(userId) .orElseThrow(() -> new UserNotFoundException(userId)); } @Cacheable(value = "users", key = "#username") public User getUserByUsername(String username) { return userRepository.findByUsername(username); } @CachePut(value = "users", key = "#user.id") public User updateUser(User user) { User updatedUser = userRepository.save(user); return updatedUser; } @Caching(evict = { @CacheEvict(value = "users", key = "#userId"), @CacheEvict(value = "user-sessions", allEntries = true) }) public void deleteUser(String userId) { userRepository.deleteById(userId); } @CacheEvict(value = "users", allEntries = true) public void clearAllUserCache() { // This will clear all entries in 'users' cache } // Conditional caching @Cacheable(value = "users", key = "#userId", unless = "#result == null") public User findUserConditional(String userId) { return userRepository.findById(userId).orElse(null); } }

Production Configuration

1. Kubernetes Configuration

@Configuration public class KubernetesHazelcastConfig { @Bean public Config kubernetesConfig() { Config config = new Config(); config.getNetworkConfig().getJoin().getMulticastConfig().setEnabled(false); config.getNetworkConfig().getJoin().getTcpIpConfig().setEnabled(false); // Kubernetes discovery config.getNetworkConfig().getJoin().getKubernetesConfig() .setEnabled(true) .setProperty("namespace", "hazelcast-namespace") .setProperty("service-name", "hazelcast-service"); // Resource management config.setInstanceName("hazelcast-cluster"); // Memory configuration NativeMemoryConfig memoryConfig = new NativeMemoryConfig(); memoryConfig.setEnabled(true) .setSize(new MemorySize(1024, MemoryUnit.MEGABYTES)) .setAllocatorType(NativeMemoryConfig.MemoryAllocatorType.POOLED); config.setNativeMemoryConfig(memoryConfig); return config; } }

2. Security Configuration

@Configuration public class SecurityConfig { @Bean public Config secureConfig() { Config config = new Config(); // Security configuration SecurityConfig securityConfig = config.getSecurityConfig(); securityConfig.setEnabled(true); // SSL configuration SSLConfig sslConfig = new SSLConfig(); sslConfig.setEnabled(true) .setFactoryClassName("com.hazelcast.nio.ssl.BasicSSLContextFactory") .setProperty("keyStore", "/path/to/keystore") .setProperty("keyStorePassword", "password") .setProperty("trustStore", "/path/to/truststore") .setProperty("trustStorePassword", "password"); config.getNetworkConfig().setSSLConfig(sslConfig); return config; } }

Monitoring and Management

1. Health Check

package com.example.cache.health; import com.hazelcast.core.HazelcastInstance; import com.hazelcast.cluster.Member; import org.springframework.boot.actuate.health.Health; import org.springframework.boot.actuate.health.HealthIndicator; import org.springframework.stereotype.Component; import java.util.Set; @Component public class HazelcastHealthIndicator implements HealthIndicator { private final HazelcastInstance hazelcastInstance; public HazelcastHealthIndicator(HazelcastInstance hazelcastInstance) { this.hazelcastInstance = hazelcastInstance; } @Override public Health health() { try { Set<Member> members = hazelcastInstance.getCluster().getMembers(); boolean running = hazelcastInstance.getLifecycleService().isRunning(); if (running && !members.isEmpty()) { return Health.up() .withDetail("clusterSize", members.size()) .withDetail("clusterName", hazelcastInstance.getConfig().getClusterName()) .withDetail("localMember", hazelcastInstance.getCluster().getLocalMember().getAddress()) .build(); } else { return Health.down() .withDetail("reason", "Cluster not properly formed") .build(); } } catch (Exception e) { return Health.down(e).build(); } } }

2. Metrics Collection

package com.example.cache.metrics; import com.hazelcast.core.HazelcastInstance; import com.hazelcast.map.IMap; import io.micrometer.core.instrument.Gauge; import io.micrometer.core.instrument.MeterRegistry; import org.springframework.scheduling.annotation.Scheduled; import org.springframework.stereotype.Component; import java.util.concurrent.atomic.AtomicLong; @Component public class HazelcastMetrics { private final HazelcastInstance hazelcastInstance; private final MeterRegistry meterRegistry; public HazelcastMetrics(HazelcastInstance hazelcastInstance, MeterRegistry meterRegistry) { this.hazelcastInstance = hazelcastInstance; this.meterRegistry = meterRegistry; initializeMetrics(); } private void initializeMetrics() { // Cluster metrics Gauge.builder("hazelcast.cluster.size", hazelcastInstance.getCluster(), cluster -> cluster.getMembers().size()) .register(meterRegistry); // Map-specific metrics registerMapMetrics("users"); registerMapMetrics("products"); } private void registerMapMetrics(String mapName) { IMap<Object, Object> map = hazelcastInstance.getMap(mapName); Gauge.builder("hazelcast.map.size", map, IMap::size) .tag("map", mapName) .register(meterRegistry); Gauge.builder("hazelcast.map.owned.entry.count", map, m -> m.getLocalMapStats().getOwnedEntryCount()) .tag("map", mapName) .register(meterRegistry); } @Scheduled(fixedRate = 60000) public void collectStats() { // Collect and log additional statistics hazelcastInstance.getDistributedObjects().forEach(obj -> { if (obj instanceof IMap) { IMap<?, ?> map = (IMap<?, ?>) obj; meterRegistry.gauge("hazelcast.map.backup.entry.count", map.getLocalMapStats().getBackupEntryCount()); } }); } }

Testing

1. Integration Test

@SpringBootTest class HazelcastIntegrationTest { @Autowired private HazelcastInstance hazelcastInstance; @Autowired private HazelcastCacheService cacheService; @Test void testDistributedCache() { String key = "test-key"; String value = "test-value"; // Test basic put/get cacheService.put("test-map", key, value); String retrieved = cacheService.get("test-map", key); assertEquals(value, retrieved); // Verify distribution IMap<String, String> map = hazelcastInstance.getMap("test-map"); assertEquals(1, map.size()); } @Test void testClusterFormation() { Set<Member> members = hazelcastInstance.getCluster().getMembers(); assertFalse(members.isEmpty()); assertTrue(hazelcastInstance.getLifecycleService().isRunning()); } }

2. Embedded Test Configuration

@TestConfiguration class TestHazelcastConfig { @Bean public Config testConfig() { Config config = new Config(); config.getNetworkConfig().getJoin().getMulticastConfig().setEnabled(false); config.getNetworkConfig().getJoin().getTcpIpConfig().setEnabled(true).addMember("127.0.0.1"); config.setProperty("hazelcast.logging.type", "slf4j"); return config; } }

Best Practices

1. Configuration Best Practices

# application.yml hazelcast: cluster: name: ${APP_NAME:my-app} network: port: 5701 port-auto-increment: true map: users: backup-count: 2 time-to-live-seconds: 3600 max-idle-seconds: 1800 eviction: eviction-policy: LRU max-size-policy: PER_NODE size: 10000 products: time-to-live-seconds: 86400 backup-count: 1

2. Memory Management

@Configuration public class MemoryConfig { @Bean public Config memoryOptimizedConfig() { Config config = new Config(); // Off-heap memory configuration NativeMemoryConfig memoryConfig = new NativeMemoryConfig(); memoryConfig.setEnabled(true) .setAllocatorType(NativeMemoryConfig.MemoryAllocatorType.POOLED) .setSize(new MemorySize(2, MemoryUnit.GIGABYTES)) .setMinBlockSize(16) .setPageSize(1 << 20); config.setNativeMemoryConfig(memoryConfig); // Near cache configuration EvictionConfig evictionConfig = new EvictionConfig() .setEvictionPolicy(EvictionPolicy.LRU) .setMaxSizePolicy(MaxSizePolicy.USED_NATIVE_MEMORY_PERCENTAGE) .setSize(90); return config; } }

Conclusion

Hazelcast provides a comprehensive distributed caching solution for Java applications with several key advantages:

True distributed architecture with automatic data partitioning
Rich data structures beyond simple key-value storage
Seamless Spring integration with minimal configuration
Enterprise-grade features including persistence, security, and monitoring
Cloud-native ready with Kubernetes support

When implementing Hazelcast:

Design cache keys strategically for optimal distribution
Configure appropriate TTL and eviction policies
Use near cache for frequently accessed data
Implement proper monitoring and health checks
Consider security in production environments
Test cluster behavior under different network conditions

Hazelcast is particularly well-suited for applications requiring:

High-performance distributed caching
Real-time data processing
Session clustering
Distributed computing
Event-driven architectures

With proper implementation, Hazelcast can significantly improve application performance, scalability, and reliability in distributed environments.