Latency budget calculation is a critical practice for designing and monitoring distributed systems. It involves allocating time constraints across service calls to ensure end-to-end performance requirements are met.
Core Concepts
What is a Latency Budget?
- A time allocation for each service in a call chain to meet overall SLA
- Distributed time budget across multiple service calls
- Ensures system-wide performance guarantees
- Helps identify performance bottlenecks and set timeouts
Key Components:
- Total Budget: Maximum acceptable end-to-end latency
- Service Allocation: Time budget for each service
- Overhead: Network latency, serialization, queuing time
- SLO/SLA Compliance: Meeting service level objectives/agreements
Dependencies and Setup
Maven Dependencies
<properties>
<spring-boot.version>3.1.0</spring-boot.version>
<micrometer.version>1.11.0</micrometer.version>
</properties>
<dependencies>
<!-- Spring Boot -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
<version>${spring-boot.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
<version>${spring-boot.version}</version>
</dependency>
<!-- Metrics -->
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-core</artifactId>
<version>${micrometer.version}</version>
</dependency>
<!-- Utilities -->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-math3</artifactId>
<version>3.6.1</version>
</dependency>
</dependencies>
Core Implementation
1. Latency Budget Models
@Data
@Builder
public class LatencyBudget {
private final String requestId;
private final String operationName;
// Total budget for entire operation
private final Duration totalBudget;
// Already consumed time
private final Duration consumedTime;
// Start time of the operation
private final Instant startTime;
// Budget allocations per service
private final Map<String, Duration> serviceBudgets;
// Calculated remaining budget
public Duration getRemainingBudget() {
Duration elapsed = Duration.between(startTime, Instant.now());
return totalBudget.minus(elapsed);
}
// Percentage of budget consumed
public double getConsumedPercentage() {
if (totalBudget.isZero()) return 100.0;
Duration elapsed = Duration.between(startTime, Instant.now());
return (elapsed.toMillis() * 100.0) / totalBudget.toMillis();
}
// Check if budget is exhausted
public boolean isBudgetExhausted() {
return getRemainingBudget().isNegative();
}
// Get budget for a specific service
public Duration getServiceBudget(String serviceName) {
return serviceBudgets.getOrDefault(serviceName, Duration.ZERO);
}
}
@Data
@Builder
public class ServiceLatencyProfile {
private String serviceName;
private Duration p50Latency;
private Duration p90Latency;
private Duration p95Latency;
private Duration p99Latency;
private Duration timeout;
private double weight; // Importance factor for budget allocation
public boolean isWithinSLA(Duration actualLatency) {
return actualLatency.compareTo(p95Latency) <= 0;
}
public double getSlaComplianceScore(Duration actualLatency) {
if (actualLatency.compareTo(p50Latency) <= 0) return 1.0;
if (actualLatency.compareTo(p95Latency) > 0) return 0.0;
// Linear interpolation between p50 and p95
long p50Millis = p50Latency.toMillis();
long p95Millis = p95Latency.toMillis();
long actualMillis = actualLatency.toMillis();
return 1.0 - ((double) (actualMillis - p50Millis) / (p95Millis - p50Millis));
}
}
2. Budget Calculator
@Component
@Slf4j
public class LatencyBudgetCalculator {
private final LatencyMetricsService metricsService;
public LatencyBudgetCalculator(LatencyMetricsService metricsService) {
this.metricsService = metricsService;
}
/**
* Calculate latency budget distribution for a call chain
*/
public LatencyBudget calculateBudget(String operationName,
List<String> serviceChain,
Duration totalSla) {
return calculateWeightedBudget(operationName, serviceChain, totalSla,
getDefaultWeights(serviceChain));
}
/**
* Calculate budget with custom service weights
*/
public LatencyBudget calculateWeightedBudget(String operationName,
List<String> serviceChain,
Duration totalSla,
Map<String, Double> serviceWeights) {
Instant startTime = Instant.now();
String requestId = generateRequestId();
// Reserve 10% for overhead (network, serialization, etc.)
Duration availableBudget = totalSla.multipliedBy(90).dividedBy(100);
// Calculate total weight
double totalWeight = serviceWeights.values().stream()
.mapToDouble(Double::doubleValue)
.sum();
// Allocate budget proportionally to weights
Map<String, Duration> serviceBudgets = new HashMap<>();
for (String service : serviceChain) {
double weight = serviceWeights.getOrDefault(service, 1.0);
double proportion = weight / totalWeight;
Duration serviceBudget = Duration.ofMillis(
(long) (availableBudget.toMillis() * proportion)
);
serviceBudgets.put(service, serviceBudget);
}
return LatencyBudget.builder()
.requestId(requestId)
.operationName(operationName)
.totalBudget(totalSla)
.consumedTime(Duration.ZERO)
.startTime(startTime)
.serviceBudgets(serviceBudgets)
.build();
}
/**
* Calculate budget based on historical performance data
*/
public LatencyBudget calculateDataDrivenBudget(String operationName,
List<String> serviceChain,
Duration totalSla) {
Instant startTime = Instant.now();
String requestId = generateRequestId();
// Reserve 15% for overhead
Duration availableBudget = totalSla.multipliedBy(85).dividedBy(100);
// Get historical latency percentiles
Map<String, ServiceLatencyProfile> profiles = new HashMap<>();
for (String service : serviceChain) {
profiles.put(service, metricsService.getLatencyProfile(service));
}
// Calculate weights based on historical p95 latencies
Map<String, Double> weights = new HashMap<>();
long totalHistoricalLatency = profiles.values().stream()
.mapToLong(profile -> profile.getP95Latency().toMillis())
.sum();
for (String service : serviceChain) {
ServiceLatencyProfile profile = profiles.get(service);
double weight = (double) profile.getP95Latency().toMillis() / totalHistoricalLatency;
weights.put(service, weight);
}
// Allocate budget
Map<String, Duration> serviceBudgets = new HashMap<>();
for (String service : serviceChain) {
double weight = weights.get(service);
Duration serviceBudget = Duration.ofMillis(
(long) (availableBudget.toMillis() * weight)
);
serviceBudgets.put(service, serviceBudget);
}
LatencyBudget budget = LatencyBudget.builder()
.requestId(requestId)
.operationName(operationName)
.totalBudget(totalSla)
.consumedTime(Duration.ZERO)
.startTime(startTime)
.serviceBudgets(serviceBudgets)
.build();
log.debug("Calculated data-driven budget: {}", budget);
return budget;
}
/**
* Calculate adaptive budget based on current system load
*/
public LatencyBudget calculateAdaptiveBudget(String operationName,
List<String> serviceChain,
Duration totalSla) {
// Get current system load factors
Map<String, Double> loadFactors = metricsService.getCurrentLoadFactors(serviceChain);
// Calculate base budget
LatencyBudget baseBudget = calculateBudget(operationName, serviceChain, totalSla);
// Adjust budgets based on load
Map<String, Duration> adjustedBudgets = new HashMap<>();
for (String service : serviceChain) {
double loadFactor = loadFactors.getOrDefault(service, 1.0);
Duration baseServiceBudget = baseBudget.getServiceBudget(service);
// Reduce budget for heavily loaded services, increase for lightly loaded
double adjustment = 1.0 / loadFactor; // Inverse relationship
adjustment = Math.max(0.5, Math.min(2.0, adjustment)); // Clamp between 0.5x and 2x
Duration adjustedBudget = Duration.ofMillis(
(long) (baseServiceBudget.toMillis() * adjustment)
);
adjustedBudgets.put(service, adjustedBudget);
}
return baseBudget.toBuilder()
.serviceBudgets(adjustedBudgets)
.build();
}
private Map<String, Double> getDefaultWeights(List<String> services) {
Map<String, Double> weights = new HashMap<>();
double defaultWeight = 1.0 / services.size();
services.forEach(service -> weights.put(service, defaultWeight));
return weights;
}
private String generateRequestId() {
return UUID.randomUUID().toString().substring(0, 8);
}
}
3. Budget Manager
@Component
@Slf4j
public class LatencyBudgetManager {
private final ThreadLocal<LatencyBudget> currentBudget = new ThreadLocal<>();
private final LatencyMetricsService metricsService;
public LatencyBudgetManager(LatencyMetricsService metricsService) {
this.metricsService = metricsService;
}
/**
* Start tracking latency budget for a request
*/
public LatencyBudget startBudget(String operationName,
List<String> serviceChain,
Duration totalSla) {
LatencyBudgetCalculator calculator = new LatencyBudgetCalculator(metricsService);
LatencyBudget budget = calculator.calculateDataDrivenBudget(operationName, serviceChain, totalSla);
currentBudget.set(budget);
metricsService.recordBudgetAllocation(budget);
log.info("Started latency budget tracking: {} with {}ms total SLA",
operationName, totalSla.toMillis());
return budget;
}
/**
* Check remaining budget for current request
*/
public Duration getRemainingBudget() {
LatencyBudget budget = currentBudget.get();
if (budget == null) {
return Duration.ofSeconds(30); // Default fallback
}
return budget.getRemainingBudget();
}
/**
* Check if we have sufficient budget for a service call
*/
public boolean canMakeServiceCall(String serviceName, Duration estimatedDuration) {
LatencyBudget budget = currentBudget.get();
if (budget == null) return true; // No budget tracking
Duration remaining = budget.getRemainingBudget();
Duration serviceBudget = budget.getServiceBudget(serviceName);
// Check both overall remaining budget and service-specific budget
boolean withinBudget = !remaining.minus(estimatedDuration).isNegative() &&
!serviceBudget.minus(estimatedDuration).isNegative();
if (!withinBudget) {
log.warn("Budget constraint for service {}: remaining={}ms, estimated={}ms",
serviceName, remaining.toMillis(), estimatedDuration.toMillis());
metricsService.recordBudgetViolation(budget, serviceName, estimatedDuration);
}
return withinBudget;
}
/**
* Record time spent on a service call
*/
public void recordServiceCall(String serviceName, Duration actualDuration) {
LatencyBudget budget = currentBudget.get();
if (budget == null) return;
metricsService.recordServiceLatency(serviceName, actualDuration,
budget.getServiceBudget(serviceName));
log.debug("Recorded service call: {} took {}ms (budget: {}ms)",
serviceName, actualDuration.toMillis(),
budget.getServiceBudget(serviceName).toMillis());
}
/**
* Complete budget tracking and record results
*/
public void completeBudget(boolean success) {
LatencyBudget budget = currentBudget.get();
if (budget == null) return;
Duration totalTime = Duration.between(budget.getStartTime(), Instant.now());
double budgetUtilization = (totalTime.toMillis() * 100.0) / budget.getTotalBudget().toMillis();
metricsService.recordBudgetCompletion(budget, totalTime, success, budgetUtilization);
log.info("Completed budget tracking: {} took {}ms ({}% of budget, success: {})",
budget.getOperationName(), totalTime.toMillis(),
Math.round(budgetUtilization), success);
currentBudget.remove();
}
/**
* Get current budget information
*/
public Optional<LatencyBudget> getCurrentBudget() {
return Optional.ofNullable(currentBudget.get());
}
/**
* Calculate timeout for a service call based on remaining budget
*/
public Duration calculateServiceTimeout(String serviceName) {
LatencyBudget budget = currentBudget.get();
if (budget == null) {
return Duration.ofSeconds(5); // Default timeout
}
Duration remaining = budget.getRemainingBudget();
Duration serviceBudget = budget.getServiceBudget(serviceName);
// Use the more conservative of the two
Duration timeout = remaining.compareTo(serviceBudget) < 0 ? remaining : serviceBudget;
// Apply safety factor (use 80% of available budget)
timeout = timeout.multipliedBy(80).dividedBy(100);
// Ensure minimum timeout
if (timeout.toMillis() < 100) {
timeout = Duration.ofMillis(100);
}
return timeout;
}
}
4. Metrics Service
@Component
@Slf4j
public class LatencyMetricsService {
private final MeterRegistry meterRegistry;
// Counters
private final Counter budgetAllocations;
private final Counter budgetViolations;
private final Counter budgetCompletions;
// Gauges
private final Map<String, Gauge> serviceBudgetGauges;
private final Map<String, Gauge> serviceLatencyGauges;
// Distributions
private final Map<String, DistributionSummary> latencyDistributions;
public LatencyMetricsService(MeterRegistry meterRegistry) {
this.meterRegistry = meterRegistry;
this.serviceBudgetGauges = new ConcurrentHashMap<>();
this.serviceLatencyGauges = new ConcurrentHashMap<>();
this.latencyDistributions = new ConcurrentHashMap<>();
// Initialize counters
this.budgetAllocations = Counter.builder("latency.budget.allocations")
.description("Total latency budget allocations")
.register(meterRegistry);
this.budgetViolations = Counter.builder("latency.budget.violations")
.description("Total budget violations")
.register(meterRegistry);
this.budgetCompletions = Counter.builder("latency.budget.completions")
.description("Total budget tracking completions")
.register(meterRegistry);
}
public void recordBudgetAllocation(LatencyBudget budget) {
budgetAllocations.increment();
// Record gauge for each service budget
budget.getServiceBudgets().forEach((service, duration) -> {
String gaugeName = "latency.budget.allocated." + service;
serviceBudgetGauges.computeIfAbsent(gaugeName, name ->
Gauge.builder(name, () -> duration.toMillis())
.description("Allocated budget for " + service)
.register(meterRegistry)
);
});
}
public void recordBudgetViolation(LatencyBudget budget, String serviceName, Duration estimatedDuration) {
budgetViolations.increment();
Timer.builder("latency.budget.violations")
.tag("service", serviceName)
.tag("operation", budget.getOperationName())
.register(meterRegistry)
.record(estimatedDuration);
}
public void recordServiceLatency(String serviceName, Duration actualDuration, Duration allocatedBudget) {
// Record distribution
String distributionName = "latency.service." + serviceName;
latencyDistributions.computeIfAbsent(distributionName, name ->
DistributionSummary.builder(name)
.description("Latency distribution for " + serviceName)
.baseUnit("milliseconds")
.register(meterRegistry)
).record(actualDuration.toMillis());
// Record gauge
String gaugeName = "latency.service.current." + serviceName;
serviceLatencyGauges.computeIfAbsent(gaugeName, name ->
Gauge.builder(name, () -> actualDuration.toMillis())
.description("Current latency for " + serviceName)
.register(meterRegistry)
);
// Calculate and record budget utilization
double utilization = (actualDuration.toMillis() * 100.0) / allocatedBudget.toMillis();
Gauge.builder("latency.budget.utilization." + serviceName)
.description("Budget utilization for " + serviceName)
.register(meterRegistry)
.set(utilization);
}
public void recordBudgetCompletion(LatencyBudget budget, Duration totalTime,
boolean success, double utilization) {
budgetCompletions.increment();
Timer.builder("latency.budget.completion.time")
.tag("operation", budget.getOperationName())
.tag("success", String.valueOf(success))
.register(meterRegistry)
.record(totalTime);
Gauge.builder("latency.budget.final.utilization")
.tag("operation", budget.getOperationName())
.description("Final budget utilization")
.register(meterRegistry)
.set(utilization);
}
public ServiceLatencyProfile getLatencyProfile(String serviceName) {
// In real implementation, this would query historical metrics database
// For now, return mock data based on service patterns
return ServiceLatencyProfile.builder()
.serviceName(serviceName)
.p50Latency(Duration.ofMillis(getTypicalLatency(serviceName) * 1))
.p90Latency(Duration.ofMillis(getTypicalLatency(serviceName) * 2))
.p95Latency(Duration.ofMillis(getTypicalLatency(serviceName) * 3))
.p99Latency(Duration.ofMillis(getTypicalLatency(serviceName) * 5))
.timeout(Duration.ofMillis(getTypicalLatency(serviceName) * 10))
.weight(1.0)
.build();
}
public Map<String, Double> getCurrentLoadFactors(List<String> services) {
// In real implementation, query current system metrics
Map<String, Double> loadFactors = new HashMap<>();
Random random = new Random();
for (String service : services) {
// Simulate load factors between 0.5 (lightly loaded) and 2.0 (heavily loaded)
double loadFactor = 0.5 + (random.nextDouble() * 1.5);
loadFactors.put(service, loadFactor);
}
return loadFactors;
}
private long getTypicalLatency(String serviceName) {
// Typical latencies for different service types
return switch (serviceName) {
case "database" -> 50;
case "cache" -> 10;
case "auth-service" -> 100;
case "payment-service" -> 200;
case "inventory-service" -> 150;
case "shipping-service" -> 300;
default -> 100;
};
}
}
5. Spring Boot Integration
@Configuration
@EnableAspectJAutoProxy
public class LatencyBudgetConfig {
@Bean
public LatencyBudgetManager latencyBudgetManager(LatencyMetricsService metricsService) {
return new LatencyBudgetManager(metricsService);
}
@Bean
public LatencyMetricsService latencyMetricsService(MeterRegistry meterRegistry) {
return new LatencyMetricsService(meterRegistry);
}
}
@Aspect
@Component
@Slf4j
public class LatencyBudgetAspect {
private final LatencyBudgetManager budgetManager;
public LatencyBudgetAspect(LatencyBudgetManager budgetManager) {
this.budgetManager = budgetManager;
}
@Around("@annotation(BudgetedOperation)")
public Object trackOperationLatency(ProceedingJoinPoint joinPoint) throws Throwable {
BudgetedOperation annotation = getAnnotation(joinPoint);
String operationName = annotation.value();
Duration totalSla = Duration.ofMillis(annotation.timeoutMs());
List<String> serviceChain = Arrays.asList(annotation.serviceChain());
// Start budget tracking
budgetManager.startBudget(operationName, serviceChain, totalSla);
try {
Object result = joinPoint.proceed();
budgetManager.completeBudget(true);
return result;
} catch (Exception e) {
budgetManager.completeBudget(false);
throw e;
}
}
@Around("execution(* com.example.service.*.*(..)) && @target(org.springframework.stereotype.Service)")
public Object trackServiceLatency(ProceedingJoinPoint joinPoint) throws Throwable {
String serviceName = joinPoint.getTarget().getClass().getSimpleName();
Instant start = Instant.now();
try {
// Check if we have budget for this call
Duration estimatedDuration = Duration.ofMillis(100); // Base estimate
if (!budgetManager.canMakeServiceCall(serviceName, estimatedDuration)) {
log.warn("Proceeding with service call despite budget constraints: {}", serviceName);
}
Object result = joinPoint.proceed();
// Record successful call
Duration actualDuration = Duration.between(start, Instant.now());
budgetManager.recordServiceCall(serviceName, actualDuration);
return result;
} catch (Exception e) {
// Record failed call with time spent
Duration actualDuration = Duration.between(start, Instant.now());
budgetManager.recordServiceCall(serviceName, actualDuration);
throw e;
}
}
private BudgetedOperation getAnnotation(ProceedingJoinPoint joinPoint) {
Method method = ((MethodSignature) joinPoint.getSignature()).getMethod();
return method.getAnnotation(BudgetedOperation.class);
}
}
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface BudgetedOperation {
String value();
long timeoutMs() default 5000;
String[] serviceChain() default {};
}
6. REST Controller with Budget Tracking
@RestController
@RequestMapping("/api/orders")
@Slf4j
public class OrderController {
private final OrderService orderService;
private final LatencyBudgetManager budgetManager;
public OrderController(OrderService orderService, LatencyBudgetManager budgetManager) {
this.orderService = orderService;
this.budgetManager = budgetManager;
}
@PostMapping
@BudgetedOperation(
value = "createOrder",
timeoutMs = 3000,
serviceChain = {"auth-service", "inventory-service", "payment-service", "shipping-service"}
)
public ResponseEntity<OrderResponse> createOrder(@RequestBody CreateOrderRequest request) {
// Budget tracking is automatically handled by the aspect
OrderResponse response = orderService.createOrder(request);
return ResponseEntity.ok(response);
}
@GetMapping("/{orderId}")
public ResponseEntity<OrderResponse> getOrder(@PathVariable String orderId) {
// Manual budget tracking example
List<String> serviceChain = List.of("database", "cache", "auth-service");
budgetManager.startBudget("getOrder", serviceChain, Duration.ofSeconds(2));
try {
OrderResponse response = orderService.getOrder(orderId);
budgetManager.completeBudget(true);
return ResponseEntity.ok(response);
} catch (Exception e) {
budgetManager.completeBudget(false);
throw e;
}
}
@GetMapping("/{orderId}/budget")
public ResponseEntity<BudgetInfo> getCurrentBudgetInfo() {
return budgetManager.getCurrentBudget()
.map(budget -> {
BudgetInfo info = BudgetInfo.builder()
.requestId(budget.getRequestId())
.operationName(budget.getOperationName())
.totalBudgetMs(budget.getTotalBudget().toMillis())
.remainingBudgetMs(budget.getRemainingBudget().toMillis())
.consumedPercentage(budget.getConsumedPercentage())
.serviceBudgets(budget.getServiceBudgets())
.build();
return ResponseEntity.ok(info);
})
.orElse(ResponseEntity.notFound().build());
}
}
@Data
@Builder
class BudgetInfo {
private String requestId;
private String operationName;
private long totalBudgetMs;
private long remainingBudgetMs;
private double consumedPercentage;
private Map<String, Duration> serviceBudgets;
}
7. Service Implementation with Budget Awareness
@Service
@Slf4j
public class OrderService {
private final InventoryService inventoryService;
private final PaymentService paymentService;
private final ShippingService shippingService;
private final LatencyBudgetManager budgetManager;
public OrderService(InventoryService inventoryService,
PaymentService paymentService,
ShippingService shippingService,
LatencyBudgetManager budgetManager) {
this.inventoryService = inventoryService;
this.paymentService = paymentService;
this.shippingService = shippingService;
this.budgetManager = budgetManager;
}
public OrderResponse createOrder(CreateOrderRequest request) {
// Check budget before each service call
Duration inventoryTimeout = budgetManager.calculateServiceTimeout("inventory-service");
Duration paymentTimeout = budgetManager.calculateServiceTimeout("payment-service");
Duration shippingTimeout = budgetManager.calculateServiceTimeout("shipping-service");
log.debug("Service timeouts - Inventory: {}ms, Payment: {}ms, Shipping: {}ms",
inventoryTimeout.toMillis(), paymentTimeout.toMillis(), shippingTimeout.toMillis());
// 1. Check inventory
boolean inStock = inventoryService.checkInventory(request.getItems(), inventoryTimeout);
if (!inStock) {
throw new RuntimeException("Items out of stock");
}
// 2. Process payment
PaymentResult payment = paymentService.processPayment(request.getPayment(), paymentTimeout);
if (!payment.isSuccess()) {
throw new RuntimeException("Payment failed: " + payment.getError());
}
// 3. Schedule shipping
ShippingConfirmation shipping = shippingService.scheduleShipping(
request.getShippingAddress(), shippingTimeout);
return OrderResponse.builder()
.orderId(generateOrderId())
.paymentId(payment.getPaymentId())
.shippingId(shipping.getTrackingId())
.status("COMPLETED")
.build();
}
public OrderResponse getOrder(String orderId) {
// Implementation with budget awareness
return OrderResponse.builder()
.orderId(orderId)
.status("FOUND")
.build();
}
private String generateOrderId() {
return "ORD-" + UUID.randomUUID().toString().substring(0, 8);
}
}
8. Budget-Aware HTTP Client
@Component
@Slf4j
public class BudgetAwareHttpClient {
private final WebClient webClient;
private final LatencyBudgetManager budgetManager;
public BudgetAwareHttpClient(WebClient.Builder webClientBuilder,
LatencyBudgetManager budgetManager) {
this.budgetManager = budgetManager;
this.webClient = webClientBuilder.build();
}
public <T> Mono<T> getWithBudget(String serviceName, String url, Class<T> responseType) {
Duration timeout = budgetManager.calculateServiceTimeout(serviceName);
if (!budgetManager.canMakeServiceCall(serviceName, timeout)) {
return Mono.error(new BudgetExceededException(
"Insufficient budget for service: " + serviceName));
}
Instant start = Instant.now();
return webClient.get()
.uri(url)
.header("X-Timeout", String.valueOf(timeout.toMillis()))
.retrieve()
.bodyToMono(responseType)
.timeout(timeout)
.doOnSuccess(response -> {
Duration duration = Duration.between(start, Instant.now());
budgetManager.recordServiceCall(serviceName, duration);
})
.doOnError(error -> {
Duration duration = Duration.between(start, Instant.now());
budgetManager.recordServiceCall(serviceName, duration);
});
}
}
public class BudgetExceededException extends RuntimeException {
public BudgetExceededException(String message) {
super(message);
}
}
Monitoring and Visualization
1. Budget Health Dashboard
@Component
@Slf4j
public class BudgetHealthMonitor {
private final LatencyMetricsService metricsService;
private final MeterRegistry meterRegistry;
public BudgetHealthMonitor(LatencyMetricsService metricsService, MeterRegistry meterRegistry) {
this.metricsService = metricsService;
this.meterRegistry = meterRegistry;
}
@Scheduled(fixedRate = 30000) // Every 30 seconds
public void monitorBudgetHealth() {
// Check various budget health indicators
checkBudgetUtilization();
checkViolationRates();
checkSlaCompliance();
}
private void checkBudgetUtilization() {
// Implementation to check if budgets are properly utilized
log.debug("Checking budget utilization...");
}
private void checkViolationRates() {
// Implementation to check budget violation rates
log.debug("Checking budget violation rates...");
}
private void checkSlaCompliance() {
// Implementation to check SLA compliance
log.debug("Checking SLA compliance...");
}
public BudgetHealthReport generateHealthReport() {
return BudgetHealthReport.builder()
.timestamp(Instant.now())
.totalBudgetAllocations(getCounterValue("latency.budget.allocations"))
.totalBudgetViolations(getCounterValue("latency.budget.violations"))
.violationRate(calculateViolationRate())
.build();
}
private double getCounterValue(String counterName) {
return meterRegistry.find(counterName).counter().count();
}
private double calculateViolationRate() {
double allocations = getCounterValue("latency.budget.allocations");
double violations = getCounterValue("latency.budget.violations");
if (allocations == 0) return 0.0;
return (violations / allocations) * 100.0;
}
}
@Data
@Builder
class BudgetHealthReport {
private Instant timestamp;
private double totalBudgetAllocations;
private double totalBudgetViolations;
private double violationRate;
private Map<String, Double> serviceUtilization;
}
Best Practices
- Start Simple: Begin with fixed allocations before implementing adaptive budgets
- Monitor Continuously: Regularly review budget utilization and violation patterns
- Set Realistic SLAs: Base SLAs on historical performance data
- Implement Graceful Degradation: Have fallbacks for when budgets are exhausted
- Correlate with Business Metrics: Link latency budgets to business impact
- Review and Adjust: Regularly refine budget allocations based on performance data
// Example of graceful degradation
@Service
public class OrderServiceWithFallback {
public OrderResponse createOrderWithFallback(CreateOrderRequest request) {
try {
return createOrder(request);
} catch (BudgetExceededException e) {
log.warn("Budget exceeded, using fallback strategy");
return createOrderAsync(request); // Fallback to async processing
}
}
private OrderResponse createOrderAsync(CreateOrderRequest request) {
// Implement async processing with different SLA
return OrderResponse.builder()
.orderId(generateOrderId())
.status("PROCESSING")
.message("Order is being processed asynchronously")
.build();
}
}
Conclusion
Latency budget calculation in Java provides:
- Predictable Performance: Ensures end-to-end SLAs are met
- Resource Awareness: Makes services aware of their time constraints
- Proactive Monitoring: Identifies potential bottlenecks before they cause issues
- Adaptive Behavior: Allows systems to adjust based on current conditions
- SLA Compliance: Helps maintain service level agreements
By implementing the patterns shown above, you can build performance-aware applications that respect latency constraints and provide reliable user experiences even in complex distributed environments.