Client-Side Load Balancing with Ribbon in Java

Ribbon is a client-side load balancer that gives you control over the behavior of HTTP and TCP clients in distributed systems. It integrates seamlessly with service discovery tools and provides intelligent routing and resilience features.

What is Ribbon?

Ribbon is a client-side load balancer that provides:

Load balancing - Distribute traffic across multiple service instances
Fault tolerance - Handle failures gracefully with circuit breakers
Service discovery integration - Work with Eureka, Consul, etc.
Custom routing rules - Implement complex routing logic

Setup and Dependencies

Maven Dependencies

<properties> <spring-cloud.version>2022.0.0</spring-cloud.version> </properties> <dependencies> <!-- Spring Cloud Starter Netflix Ribbon --> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-ribbon</artifactId> </dependency> <!-- For service discovery (optional) --> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId> </dependency> <!-- For REST communication --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> </dependency> <!-- For reactive support --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-webflux</artifactId> </dependency> </dependencies> <dependencyManagement> <dependencies> <dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-dependencies</artifactId> <version>${spring-cloud.version}</version> <type>pom</type> <scope>import</scope> </dependency> </dependencies> </dependencyManagement>

Configuration

# application.yml spring: application: name: user-service-client # Ribbon Configuration user-service: ribbon: listOfServers: localhost:8081,localhost:8082,localhost:8083 NIWSServerListClassName: com.netflix.loadbalancer.ConfigurationBasedServerList NFLoadBalancerRuleClassName: com.netflix.loadbalancer.RoundRobinRule ConnectTimeout: 1000 ReadTimeout: 3000 MaxAutoRetries: 1 MaxAutoRetriesNextServer: 2 OkToRetryOnAllOperations: true # Eureka Configuration (if using service discovery) eureka: client: serviceUrl: defaultZone: http://localhost:8761/eureka/ fetchRegistry: true registerWithEureka: false # This service doesn't register, just discovers instance: preferIpAddress: true # Logging for debugging logging: level: com.netflix.loadbalancer: DEBUG com.example.ribbon: DEBUG

Basic Ribbon Implementation

Example 1: Manual Ribbon Client with RestTemplate

import org.springframework.cloud.client.loadbalancer.LoadBalanced; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.web.client.RestTemplate; @Configuration public class RibbonConfig { @Bean @LoadBalanced // This enables Ribbon load balancing public RestTemplate restTemplate() { return new RestTemplate(); } } @Service public class UserServiceClient { private final RestTemplate restTemplate; // Service ID (as registered in Eureka) or configured in application.yml private static final String USER_SERVICE_URL = "http://user-service"; public UserServiceClient(RestTemplate restTemplate) { this.restTemplate = restTemplate; } public User getUserById(Long userId) { String url = USER_SERVICE_URL + "/api/users/{userId}"; try { ResponseEntity<User> response = restTemplate.getForEntity(url, User.class, userId); return response.getBody(); } catch (RestClientException e) { throw new ServiceUnavailableException("User service is unavailable", e); } } public List<User> getAllUsers() { String url = USER_SERVICE_URL + "/api/users"; try { ResponseEntity<User[]> response = restTemplate.getForEntity(url, User[].class); return Arrays.asList(response.getBody()); } catch (RestClientException e) { throw new ServiceUnavailableException("User service is unavailable", e); } } public User createUser(User user) { String url = USER_SERVICE_URL + "/api/users"; try { ResponseEntity<User> response = restTemplate.postForEntity(url, user, User.class); return response.getBody(); } catch (RestClientException e) { throw new ServiceUnavailableException("User service is unavailable", e); } } public void updateUser(Long userId, User user) { String url = USER_SERVICE_URL + "/api/users/{userId}"; try { restTemplate.put(url, user, userId); } catch (RestClientException e) { throw new ServiceUnavailableException("User service is unavailable", e); } } public void deleteUser(Long userId) { String url = USER_SERVICE_URL + "/api/users/{userId}"; try { restTemplate.delete(url, userId); } catch (RestClientException e) { throw new ServiceUnavailableException("User service is unavailable", e); } } } // Domain object public class User { private Long id; private String name; private String email; private LocalDateTime createdAt; // Constructors, getters, and setters public User() {} public User(Long id, String name, String email) { this.id = id; this.name = name; this.email = email; this.createdAt = LocalDateTime.now(); } // Getters and setters public Long getId() { return id; } public void setId(Long id) { this.id = id; } public String getName() { return name; } public void setName(String name) { this.name = name; } public String getEmail() { return email; } public void setEmail(String email) { this.email = email; } public LocalDateTime getCreatedAt() { return createdAt; } public void setCreatedAt(LocalDateTime createdAt) { this.createdAt = createdAt; } }

Example 2: Advanced Ribbon Configuration

import com.netflix.client.config.IClientConfig; import com.netflix.loadbalancer.*; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; @Configuration public class AdvancedRibbonConfig { // Custom load balancing rule @Bean public IRule ribbonRule() { // You can choose from various built-in rules or create custom ones return new WeightedResponseTimeRule(); // Uses response time for weighting // return new RoundRobinRule(); // Simple round-robin // return new RandomRule(); // Random selection // return new AvailabilityFilteringRule(); // Skip servers with high failure rates // return new ZoneAvoidanceRule(); // Consider zone and server availability } // Custom ping mechanism (health check) @Bean public IPing ribbonPing() { return new PingUrl(); // HTTP ping to check server health // return new NoOpPing(); // No health checking // return new DummyPing(); // Always return true } // Server list filter @Bean public ServerListFilter<Server> serverListFilter() { return new ZonePreferenceServerListFilter(); // Prefer servers in the same zone } // Custom configuration @Bean public IClientConfig ribbonClientConfig() { DefaultClientConfigImpl config = new DefaultClientConfigImpl(); config.setClientName("user-service"); config.set(CommonClientConfigKey.ConnectTimeout, 1000); config.set(CommonClientConfigKey.ReadTimeout, 3000); config.set(CommonClientConfigKey.MaxAutoRetries, 1); config.set(CommonClientConfigKey.MaxAutoRetriesNextServer, 2); config.set(CommonClientConfigKey.OkToRetryOnAllOperations, true); return config; } } // Custom load balancing rule @Component public class CustomLoadBalancingRule extends RoundRobinRule { private static final Logger logger = LoggerFactory.getLogger(CustomLoadBalancingRule.class); @Override public Server choose(ILoadBalancer lb, Object key) { if (lb == null) { logger.warn("No load balancer available"); return null; } Server server = null; int count = 0; while (server == null && count++ < 10) { List<Server> reachableServers = lb.getReachableServers(); List<Server> allServers = lb.getAllServers(); int upCount = reachableServers.size(); int serverCount = allServers.size(); if ((upCount == 0) || (serverCount == 0)) { logger.warn("No available servers to choose from"); return null; } // Custom logic: Skip servers that are under maintenance List<Server> availableServers = reachableServers.stream() .filter(this::isServerAvailable) .collect(Collectors.toList()); if (!availableServers.isEmpty()) { int nextServerIndex = incrementAndGetModulo(availableServers.size()); server = availableServers.get(nextServerIndex); } else { // Fallback to original logic if no filtered servers available int nextServerIndex = incrementAndGetModulo(upCount); server = reachableServers.get(nextServerIndex); } if (server == null) { // Transient exception - give the thread a chance to do other work Thread.yield(); continue; } if (server.isAlive() && server.isReadyToServe()) { return server; } // Next server in line server = null; } if (count >= 10) { logger.warn("No available alive servers after 10 tries"); } return server; } private boolean isServerAvailable(Server server) { // Custom logic to check if server is available // You could check maintenance mode, load, etc. return true; // Implement your custom logic } private int incrementAndGetModulo(int modulo) { for (;;) { int current = nextServerCyclicCounter.get(); int next = (current + 1) % modulo; if (nextServerCyclicCounter.compareAndSet(current, next)) return next; } } }

Reactive Ribbon with WebClient

Example 3: Reactive Ribbon Client

import org.springframework.cloud.client.loadbalancer.reactive.LoadBalancedExchangeFilterFunction; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.web.reactive.function.client.WebClient; @Configuration public class ReactiveRibbonConfig { @Bean @LoadBalanced public WebClient.Builder loadBalancedWebClientBuilder() { return WebClient.builder(); } } @Service public class ReactiveUserServiceClient { private final WebClient webClient; private static final String USER_SERVICE_URL = "http://user-service"; public ReactiveUserServiceClient(WebClient.Builder webClientBuilder) { this.webClient = webClientBuilder.baseUrl(USER_SERVICE_URL).build(); } public Mono<User> getUserById(Long userId) { return webClient.get() .uri("/api/users/{userId}", userId) .retrieve() .bodyToMono(User.class) .timeout(Duration.ofSeconds(3)) .onErrorResume(WebClientResponseException.class, ex -> { if (ex.getStatusCode().is5xxServerError()) { return Mono.error(new ServiceUnavailableException("User service unavailable")); } return Mono.error(ex); }) .retryWhen(Retry.backoff(3, Duration.ofMillis(100)) .doOnSuccess(user -> log.info("Successfully retrieved user: {}", user.getId())) .doOnError(error -> log.error("Failed to retrieve user: {}", error.getMessage())); } public Flux<User> getAllUsers() { return webClient.get() .uri("/api/users") .retrieve() .bodyToFlux(User.class) .timeout(Duration.ofSeconds(5)) .onErrorResume(throwable -> { log.error("Error fetching users: {}", throwable.getMessage()); return Flux.empty(); }); } public Mono<User> createUser(User user) { return webClient.post() .uri("/api/users") .bodyValue(user) .retrieve() .bodyToMono(User.class) .timeout(Duration.ofSeconds(3)); } public Mono<Void> updateUser(Long userId, User user) { return webClient.put() .uri("/api/users/{userId}", userId) .bodyValue(user) .retrieve() .bodyToMono(Void.class) .timeout(Duration.ofSeconds(3)); } public Mono<Void> deleteUser(Long userId) { return webClient.delete() .uri("/api/users/{userId}", userId) .retrieve() .bodyToMono(Void.class) .timeout(Duration.ofSeconds(3)); } // Batch operations with load balancing public Flux<User> getUsersByIds(List<Long> userIds) { return Flux.fromIterable(userIds) .flatMap(this::getUserById, 5) // Concurrency of 5 .onErrorContinue((error, userId) -> log.warn("Failed to fetch user {}: {}", userId, error.getMessage())); } }

Advanced Features

Example 4: Circuit Breaker with Ribbon

import io.github.resilience4j.circuitbreaker.CircuitBreaker; import io.github.resilience4j.circuitbreaker.CircuitBreakerConfig; import io.github.resilience4j.circuitbreaker.CircuitBreakerRegistry; import org.springframework.stereotype.Component; import java.time.Duration; import java.util.function.Supplier; @Component public class CircuitBreakerUserServiceClient { private final UserServiceClient userServiceClient; private final CircuitBreaker circuitBreaker; public CircuitBreakerUserServiceClient(UserServiceClient userServiceClient) { this.userServiceClient = userServiceClient; // Configure circuit breaker CircuitBreakerConfig circuitBreakerConfig = CircuitBreakerConfig.custom() .failureRateThreshold(50) // Open circuit if 50% of calls fail .waitDurationInOpenState(Duration.ofSeconds(30)) // Wait 30s in open state .slidingWindowSize(10) // Last 10 calls are considered .permittedNumberOfCallsInHalfOpenState(5) // Allow 5 calls in half-open state .recordExceptions(ServiceUnavailableException.class, RuntimeException.class) .build(); CircuitBreakerRegistry circuitBreakerRegistry = CircuitBreakerRegistry.of(circuitBreakerConfig); this.circuitBreaker = circuitBreakerRegistry.circuitBreaker("userService"); } public User getUserByIdWithCircuitBreaker(Long userId) { Supplier<User> decoratedSupplier = CircuitBreaker .decorateSupplier(circuitBreaker, () -> userServiceClient.getUserById(userId)); try { return decoratedSupplier.get(); } catch (Exception e) { // Fallback logic return getFallbackUser(userId); } } public List<User> getAllUsersWithCircuitBreaker() { Supplier<List<User>> decoratedSupplier = CircuitBreaker .decorateSupplier(circuitBreaker, () -> userServiceClient.getAllUsers()); try { return decoratedSupplier.get(); } catch (Exception e) { // Fallback logic log.warn("Using fallback for getAllUsers due to: {}", e.getMessage()); return Collections.emptyList(); } } private User getFallbackUser(Long userId) { // Return a fallback user or cached data log.warn("Using fallback user for ID: {}", userId); return new User(userId, "Fallback User", "[email protected]"); } // Get circuit breaker metrics public CircuitBreaker.Metrics getCircuitBreakerMetrics() { return circuitBreaker.getMetrics(); } public CircuitBreaker.State getCircuitBreakerState() { return circuitBreaker.getState(); } }

Example 5: Request-Level Load Balancing

import com.netflix.loadbalancer.Server; import org.springframework.cloud.client.ServiceInstance; import org.springframework.cloud.client.loadbalancer.LoadBalancerClient; import org.springframework.stereotype.Service; @Service public class ManualLoadBalancingService { private final LoadBalancerClient loadBalancerClient; private final RestTemplate restTemplate; public ManualLoadBalancingService(LoadBalancerClient loadBalancerClient, RestTemplate restTemplate) { this.loadBalancerClient = loadBalancerClient; this.restTemplate = restTemplate; } public User getUserWithManualLoadBalancing(Long userId) { // Choose a server manually ServiceInstance instance = loadBalancerClient.choose("user-service"); if (instance == null) { throw new ServiceUnavailableException("No available instances of user-service"); } String url = String.format("http://%s:%s/api/users/%d", instance.getHost(), instance.getPort(), userId); log.info("Calling user service instance: {}:{}", instance.getHost(), instance.getPort()); try { ResponseEntity<User> response = restTemplate.getForEntity(url, User.class); return response.getBody(); } catch (Exception e) { // Mark this instance as problematic and retry with another log.warn("Instance {}:{} failed, trying another", instance.getHost(), instance.getPort()); // Ribbon will automatically avoid this instance for a while return getUserWithManualLoadBalancing(userId); // Recursive retry } } public void executeOnAllInstances(Runnable task) { // Get all available instances List<ServiceInstance> instances = loadBalancerClient.getInstances("user-service"); instances.parallelStream().forEach(instance -> { try { log.info("Executing task on instance: {}:{}", instance.getHost(), instance.getPort()); // Execute your task for each instance task.run(); } catch (Exception e) { log.error("Failed to execute task on instance {}:{} - {}", instance.getHost(), instance.getPort(), e.getMessage()); } }); } }

Monitoring and Metrics

Example 6: Ribbon Metrics and Monitoring

import com.netflix.loadbalancer.Server; import com.netflix.loadbalancer.ServerStats; import io.micrometer.core.instrument.MeterRegistry; import org.springframework.scheduling.annotation.Scheduled; import org.springframework.stereotype.Component; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.atomic.AtomicLong; @Component public class RibbonMetricsCollector { private final MeterRegistry meterRegistry; private final ILoadBalancer loadBalancer; private final Map<String, AtomicLong> requestCounters; public RibbonMetricsCollector(MeterRegistry meterRegistry, @LoadBalanced RestTemplate restTemplate) { this.meterRegistry = meterRegistry; this.loadBalancer = getLoadBalancerFromRestTemplate(restTemplate); this.requestCounters = new ConcurrentHashMap<>(); initializeMetrics(); } @Scheduled(fixedRate = 30000) // Every 30 seconds public void collectRibbonMetrics() { if (loadBalancer == null) return; List<Server> allServers = loadBalancer.getAllServers(); List<Server> reachableServers = loadBalancer.getReachableServers(); // Basic server metrics meterRegistry.gauge("ribbon.servers.total", allServers.size()); meterRegistry.gauge("ribbon.servers.available", reachableServers.size()); // Server-specific metrics for (Server server : allServers) { String serverKey = server.getHost() + ":" + server.getPort(); ServerStats stats = ((AbstractLoadBalancer) loadBalancer).getLoadBalancerStats().getSingleServerStat(server); meterRegistry.gauge("ribbon.server.requests.active", Tags.of("server", serverKey), stats.getActiveRequestsCount()); meterRegistry.gauge("ribbon.server.response.time.avg", Tags.of("server", serverKey), stats.getResponseTimeAvg()); meterRegistry.gauge("ribbon.server.connection.failure.count", Tags.of("server", serverKey), stats.getSuccessiveConnectionFailureCount()); } } public void recordRequest(String serviceId, String endpoint, boolean success) { String counterName = "ribbon.requests." + serviceId; AtomicLong counter = requestCounters.computeIfAbsent(counterName, key -> meterRegistry.gauge(key, new AtomicLong(0))); counter.incrementAndGet(); // Record success/failure meterRegistry.counter("ribbon.requests", Tags.of("service", serviceId, "endpoint", endpoint, "status", success ? "success" : "failure")) .increment(); } private ILoadBalancer getLoadBalancerFromRestTemplate(RestTemplate restTemplate) { // This is a simplified example - in real implementation, you'd need to access Ribbon's internals return null; } private void initializeMetrics() { // Initialize custom metrics meterRegistry.gauge("ribbon.loadbalancer.rule", Tags.of("rule", "RoundRobinRule"), 1); } } // Custom Ribbon event listener @Component public class RibbonEventListener { private static final Logger logger = LoggerFactory.getLogger(RibbonEventListener.class); @EventListener public void handleRibbonEvent(ClientRequestEvent event) { logger.debug("Ribbon client request: {} to {}", event.getRequest().getMethod(), event.getRequest().getURI()); } @EventListener public void handleServerStatusChange(ServerStatusChangeEvent event) { Server server = event.getServer(); logger.info("Server status changed: {}:{} - {}", server.getHost(), server.getPort(), event.getStatus()); } }

Testing Ribbon Configuration

Example 7: Testing Ribbon Load Balancing

import org.junit.jupiter.api.Test; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.mock.mockito.MockBean; import org.springframework.cloud.client.ServiceInstance; import org.springframework.cloud.client.loadbalancer.LoadBalancerClient; import org.springframework.http.ResponseEntity; import org.springframework.web.client.RestTemplate; import java.util.Arrays; import java.util.List; import static org.mockito.ArgumentMatchers.*; import static org.mockito.Mockito.*; import static org.junit.jupiter.api.Assertions.*; @SpringBootTest public class RibbonLoadBalancingTest { @MockBean private LoadBalancerClient loadBalancerClient; @MockBean private RestTemplate restTemplate; @Test public void testLoadBalancingAcrossMultipleServers() { // Mock service instances ServiceInstance instance1 = mock(ServiceInstance.class); when(instance1.getHost()).thenReturn("server1"); when(instance1.getPort()).thenReturn(8080); ServiceInstance instance2 = mock(ServiceInstance.class); when(instance2.getHost()).thenReturn("server2"); when(instance2.getPort()).thenReturn(8080); // Mock load balancer to return different instances when(loadBalancerClient.choose("user-service")) .thenReturn(instance1) .thenReturn(instance2); // Mock REST responses when(restTemplate.getForEntity(anyString(), eq(User.class))) .thenReturn(ResponseEntity.ok(new User(1L, "User1", "[email protected]"))) .thenReturn(ResponseEntity.ok(new User(2L, "User2", "[email protected]"))); ManualLoadBalancingService service = new ManualLoadBalancingService(loadBalancerClient, restTemplate); // First call should go to server1 User user1 = service.getUserWithManualLoadBalancing(1L); assertNotNull(user1); assertEquals("User1", user1.getName()); // Second call should go to server2 User user2 = service.getUserWithManualLoadBalancing(2L); assertNotNull(user2); assertEquals("User2", user2.getName()); // Verify load balancer was called twice verify(loadBalancerClient, times(2)).choose("user-service"); } @Test public void testCircuitBreakerFunctionality() { CircuitBreakerUserServiceClient circuitClient = new CircuitBreakerUserServiceClient(mock(UserServiceClient.class)); // Test circuit breaker metrics CircuitBreaker.Metrics metrics = circuitClient.getCircuitBreakerMetrics(); assertNotNull(metrics); CircuitBreaker.State state = circuitClient.getCircuitBreakerState(); assertEquals(CircuitBreaker.State.CLOSED, state); } } // Integration test with multiple server instances @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) @TestPropertySource(properties = { "user-service.ribbon.listOfServers=localhost:8091,localhost:8092,localhost:8093", "user-service.ribbon.NFLoadBalancerRuleClassName=com.netflix.loadbalancer.RoundRobinRule" }) public class RibbonIntegrationTest { @LocalServerPort private int port; @Test public void testRoundRobinLoadBalancing() { // This test would require actual server instances running // or mocked servers to demonstrate round-robin behavior RestTemplate restTemplate = new RestTemplate(); // Make multiple requests and verify they're distributed for (int i = 0; i < 6; i++) { try { ResponseEntity<String> response = restTemplate.getForEntity( "http://localhost:" + port + "/api/users/" + i, String.class); assertEquals(200, response.getStatusCodeValue()); } catch (Exception e) { // Expected if no actual servers are running System.out.println("Request failed (expected in test): " + e.getMessage()); } } } }

Best Practices

Configuration: Always configure timeouts and retry policies
Monitoring: Implement comprehensive metrics and logging
Circuit Breaking: Use circuit breakers for fault tolerance
Health Checks: Ensure proper health checking mechanisms
Service Discovery: Integrate with service discovery for dynamic instance management
Testing: Test load balancing behavior with multiple instances
Fallbacks: Implement graceful fallback mechanisms

Common Ribbon Configuration Properties

user-service: ribbon: # Server list configuration listOfServers: host1:port1,host2:port2 NIWSServerListClassName: com.netflix.loadbalancer.ConfigurationBasedServerList # Load balancing rules NFLoadBalancerRuleClassName: com.netflix.loadbalancer.RoundRobinRule # Timeout configuration ConnectTimeout: 1000 ReadTimeout: 5000 SocketTimeout: 5000 # Retry configuration MaxAutoRetries: 1 MaxAutoRetriesNextServer: 2 OkToRetryOnAllOperations: false # Connection management MaxTotalConnections: 200 MaxConnectionsPerHost: 50 # Server stats ServerListRefreshInterval: 30000

Conclusion

Ribbon provides powerful client-side load balancing capabilities:

Key Features:

Multiple load balancing algorithms (Round Robin, Random, Weighted, etc.)
Integration with service discovery
Fault tolerance with retry mechanisms
Customizable routing rules
Comprehensive metrics and monitoring

Use Cases:

Microservices communication
Load distribution across multiple instances
Graceful degradation and fault tolerance
Dynamic service discovery and routing

Migration Note: While Ribbon is in maintenance mode, many concepts translate to Spring Cloud LoadBalancer, which is the recommended solution for new projects. However, understanding Ribbon is valuable for maintaining existing systems and understanding load balancing concepts.