OpenTelemetry Auto-Instrumentation in Java: Complete Guide

Introduction to OpenTelemetry Auto-Instrumentation

OpenTelemetry provides automatic instrumentation for Java applications without requiring code changes. This guide covers comprehensive setup, configuration, and customization of OpenTelemetry auto-instrumentation for various Java frameworks and libraries.

OpenTelemetry Fundamentals
Auto-Instrumentation Setup
Spring Boot Integration
Database Instrumentation
HTTP Client/Server Instrumentation
Messaging System Instrumentation
Custom Instrumentation
Configuration and Exporters
Performance Best Practices
Real-World Examples

1. Project Setup and Dependencies

Maven Configuration

<properties> <opentelemetry.version>1.28.0</opentelemetry.version> <opentelemetry-instrumentation.version>1.28.0</opentelemetry-instrumentation.version> <spring-boot.version>3.1.0</spring-boot.version> </properties> <dependencies> <!-- OpenTelemetry API --> <dependency> <groupId>io.opentelemetry</groupId> <artifactId>opentelemetry-api</artifactId> <version>${opentelemetry.version}</version> </dependency> <!-- OpenTelemetry SDK --> <dependency> <groupId>io.opentelemetry</groupId> <artifactId>opentelemetry-sdk</artifactId> <version>${opentelemetry.version}</version> </dependency> <!-- OpenTelemetry Auto-Instrumentation Annotations --> <dependency> <groupId>io.opentelemetry.instrumentation</groupId> <artifactId>opentelemetry-instrumentation-annotations</artifactId> <version>${opentelemetry-instrumentation.version}</version> </dependency> <!-- OpenTelemetry Exporters --> <dependency> <groupId>io.opentelemetry</groupId> <artifactId>opentelemetry-exporter-otlp</artifactId> <version>${opentelemetry.version}</version> </dependency> <dependency> <groupId>io.opentelemetry</groupId> <artifactId>opentelemetry-exporter-logging</artifactId> <version>${opentelemetry.version}</version> </dependency> <!-- Spring Boot Starter --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> <version>${spring-boot.version}</version> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-jpa</artifactId> <version>${spring-boot.version}</version> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> <version>${spring-boot.version}</version> </dependency> <!-- Database --> <dependency> <groupId>com.h2database</groupId> <artifactId>h2</artifactId> <scope>runtime</scope> </dependency> <!-- HTTP Client --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-webflux</artifactId> <version>${spring-boot.version}</version> </dependency> <!-- Messaging --> <dependency> <groupId>org.springframework.kafka</groupId> <artifactId>spring-kafka</artifactId> <version>3.0.8</version> </dependency> <!-- Testing --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <version>${spring-boot.version}</version> <scope>test</scope> </dependency> </dependencies>

Using OpenTelemetry Java Agent (Recommended)

Download the latest Java agent JAR:

wget https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar

Run your application with the agent:

java -javaagent:path/to/opentelemetry-javaagent.jar \ -Dotel.service.name=my-service \ -Dotel.traces.exporter=otlp \ -Dotel.metrics.exporter=otlp \ -Dotel.logs.exporter=otlp \ -Dotel.exporter.otlp.endpoint=http://localhost:4317 \ -jar your-application.jar

2. Configuration Files

application.yml

# OpenTelemetry Configuration otel: service: name: order-service namespace: ecommerce version: 1.0.0 instance: id: ${HOSTNAME:localhost} # Exporters exporter: otlp: endpoint: http://localhost:4317 protocol: grpc headers: authorization: "Bearer ${OTEL_API_KEY:}" timeout: 10000 logging: enabled: true # Sampling traces: sampler: parentbased_traceidratio sampler: arg: 0.1 # 10% sampling rate # Resource attributes resource: attributes: deployment: environment: ${DEPLOYMENT_ENV:development} telemetry: sdk: language: java name: opentelemetry host: name: ${HOSTNAME:unknown} os: type: ${os.name:unknown} version: ${os.version:unknown} # Spring Boot Actuator management: endpoints: web: exposure: include: health,info,metrics,prometheus endpoint: health: show-details: always metrics: enabled: true tracing: sampling: probability: 1.0 # 100% for development # Logging correlation logging: pattern: level: "%5p [${spring.application.name:},%X{trace_id:-},%X{span_id:-}]"

Environment Variables

# Service Identification export OTEL_SERVICE_NAME=order-service export OTEL_SERVICE_NAMESPACE=ecommerce export OTEL_SERVICE_VERSION=1.0.0 # Exporters export OTEL_TRACES_EXPORTER=otlp export OTEL_METRICS_EXPORTER=otlp export OTEL_LOGS_EXPORTER=otlp export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 export OTEL_EXPORTER_OTLP_HEADERS="authorization=Bearer your-token" # Sampling export OTEL_TRACES_SAMPLER=parentbased_traceidratio export OTEL_TRACES_SAMPLER_ARG=0.1 # Resource Attributes export OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production,host.name=order-service-1 # Java Agent Specific export OTEL_JAVAAGENT_ENABLED=true export OTEL_INSTRUMENTATION_COMMON_DEFAULT_ENABLED=true export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST=content-type,content-length,user-agent export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE=content-type,content-length

3. Spring Boot Auto-Configuration

OpenTelemetry Configuration Class

package com.example.otel.config; import io.opentelemetry.api.OpenTelemetry; import io.opentelemetry.api.common.Attributes; import io.opentelemetry.api.trace.propagation.W3CTraceContextPropagator; import io.opentelemetry.context.propagation.ContextPropagators; import io.opentelemetry.exporter.otlp.trace.OtlpGrpcSpanExporter; import io.opentelemetry.sdk.OpenTelemetrySdk; import io.opentelemetry.sdk.resources.Resource; import io.opentelemetry.sdk.trace.SdkTracerProvider; import io.opentelemetry.sdk.trace.export.BatchSpanProcessor; import io.opentelemetry.sdk.trace.export.SimpleSpanProcessor; import io.opentelemetry.semconv.ResourceAttributes; import org.springframework.beans.factory.annotation.Value; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import java.util.concurrent.TimeUnit; @Configuration public class OpenTelemetryConfig { @Value("${otel.service.name:unknown-service}") private String serviceName; @Value("${otel.service.namespace:unknown-namespace}") private String serviceNamespace; @Value("${otel.exporter.otlp.endpoint:http://localhost:4317}") private String otlpEndpoint; @Bean public OpenTelemetry openTelemetry() { // Create resource Resource resource = Resource.getDefault() .merge(Resource.create(Attributes.builder() .put(ResourceAttributes.SERVICE_NAME, serviceName) .put(ResourceAttributes.SERVICE_NAMESPACE, serviceNamespace) .put(ResourceAttributes.DEPLOYMENT_ENVIRONMENT, getEnvironment()) .put(ResourceAttributes.HOST_NAME, getHostName()) .build())); // Configure span exporter OtlpGrpcSpanExporter spanExporter = OtlpGrpcSpanExporter.builder() .setEndpoint(otlpEndpoint) .setTimeout(30, TimeUnit.SECONDS) .build(); // Configure tracer provider SdkTracerProvider tracerProvider = SdkTracerProvider.builder() .setResource(resource) .addSpanProcessor(BatchSpanProcessor.builder(spanExporter) .setScheduleDelay(100, TimeUnit.MILLISECONDS) .setMaxExportBatchSize(512) .setMaxQueueSize(2048) .build()) // Add logging exporter for development .addSpanProcessor(SimpleSpanProcessor.create(LoggingSpanExporter.create())) .build(); // Build OpenTelemetry instance return OpenTelemetrySdk.builder() .setTracerProvider(tracerProvider) .setPropagators(ContextPropagators.create(W3CTraceContextPropagator.getInstance())) .buildAndRegisterGlobal(); } private String getEnvironment() { return System.getenv().getOrDefault("DEPLOYMENT_ENV", "development"); } private String getHostName() { return System.getenv().getOrDefault("HOSTNAME", "unknown"); } }

Spring Boot Actuator Configuration

package com.example.otel.config; import io.micrometer.core.instrument.MeterRegistry; import io.micrometer.core.instrument.config.MeterFilter; import org.springframework.boot.actuate.autoconfigure.metrics.MeterRegistryCustomizer; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; @Configuration public class MetricsConfig { @Bean public MeterRegistryCustomizer<MeterRegistry> metricsCommonTags() { return registry -> registry.config() .commonTags("application", "order-service", "environment", System.getenv().getOrDefault("ENV", "dev"), "instance", System.getenv().getOrDefault("HOSTNAME", "unknown")) .meterFilter(MeterFilter.ignoreTags("too.many.tags")) .meterFilter(MeterFilter.denyNameStartsWith("jvm.threads")); } }

4. Auto-Instrumented Components

HTTP Server Instrumentation (Spring MVC)

package com.example.otel.controller; import io.opentelemetry.instrumentation.annotations.WithSpan; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.http.ResponseEntity; import org.springframework.web.bind.annotation.*; import java.util.List; import java.util.Map; import java.util.concurrent.CompletableFuture; @RestController @RequestMapping("/api/orders") public class OrderController { private static final Logger logger = LoggerFactory.getLogger(OrderController.class); private final OrderService orderService; private final PaymentService paymentService; public OrderController(OrderService orderService, PaymentService paymentService) { this.orderService = orderService; this.paymentService = paymentService; } // Auto-instrumented HTTP endpoint @PostMapping public ResponseEntity<OrderResponse> createOrder(@RequestBody CreateOrderRequest request) { logger.info("Creating order for customer: {}", request.customerId()); // This will be automatically traced Order order = orderService.createOrder(request); // Payment processing with tracing PaymentResult paymentResult = paymentService.processPayment(order); return ResponseEntity.ok(new OrderResponse( order.getId(), order.getStatus(), paymentResult.transactionId() )); } // Auto-instrumented with query parameters @GetMapping public ResponseEntity<List<Order>> getOrders( @RequestParam(defaultValue = "0") int page, @RequestParam(defaultValue = "20") int size, @RequestParam(required = false) String status) { logger.debug("Fetching orders - page: {}, size: {}, status: {}", page, size, status); List<Order> orders = orderService.findOrders(page, size, status); return ResponseEntity.ok(orders); } // Path variables are automatically captured @GetMapping("/{orderId}") public ResponseEntity<Order> getOrder(@PathVariable String orderId) { logger.info("Fetching order: {}", orderId); Order order = orderService.findOrderById(orderId); if (order == null) { return ResponseEntity.notFound().build(); } return ResponseEntity.ok(order); } // Async endpoint with automatic tracing @GetMapping("/{orderId}/status") public CompletableFuture<ResponseEntity<OrderStatus>> getOrderStatus(@PathVariable String orderId) { return orderService.getOrderStatusAsync(orderId) .thenApply(ResponseEntity::ok) .exceptionally(throwable -> { logger.error("Failed to get order status: {}", orderId, throwable); return ResponseEntity.status(500).build(); }); } // Custom span with attributes @WithSpan("cancel-order") @DeleteMapping("/{orderId}") public ResponseEntity<Void> cancelOrder(@PathVariable String orderId, @RequestParam String reason) { logger.warn("Cancelling order {} with reason: {}", orderId, reason); // Add custom attributes to span io.opentelemetry.api.trace.Span.current() .setAttribute("order.id", orderId) .setAttribute("cancellation.reason", reason); boolean cancelled = orderService.cancelOrder(orderId, reason); if (cancelled) { return ResponseEntity.noContent().build(); } else { return ResponseEntity.notFound().build(); } } // Record DTOs public record CreateOrderRequest(String customerId, List<OrderItem> items, String shippingAddress) {} public record OrderResponse(String orderId, String status, String transactionId) {} public record OrderStatus(String orderId, String status, String lastUpdated) {} }

Database Instrumentation (JPA/Hibernate)

package com.example.otel.repository; import io.opentelemetry.instrumentation.annotations.WithSpan; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.data.jpa.repository.JpaRepository; import org.springframework.data.jpa.repository.Query; import org.springframework.data.repository.query.Param; import org.springframework.stereotype.Repository; import java.util.List; import java.util.Optional; @Repository public interface OrderRepository extends JpaRepository<Order, String> { // Auto-instrumented JPA methods List<Order> findByCustomerId(String customerId); List<Order> findByStatus(String status); // Custom query with automatic instrumentation @Query("SELECT o FROM Order o WHERE o.customerId = :customerId AND o.status = :status") List<Order> findOrdersByCustomerAndStatus(@Param("customerId") String customerId, @Param("status") String status); // Native query instrumentation @Query(value = "SELECT * FROM orders WHERE total_amount > :minAmount", nativeQuery = true) List<Order> findOrdersWithTotalGreaterThan(@Param("minAmount") Double minAmount); } @Service @Transactional public class OrderService { private static final Logger logger = LoggerFactory.getLogger(OrderService.class); private final OrderRepository orderRepository; private final InventoryService inventoryService; public OrderService(OrderRepository orderRepository, InventoryService inventoryService) { this.orderRepository = orderRepository; this.inventoryService = inventoryService; } // Auto-instrumented database operations @WithSpan("create-order") public Order createOrder(CreateOrderRequest request) { logger.info("Creating order for customer: {}", request.customerId()); // Start transaction (automatically traced) Order order = new Order(); order.setCustomerId(request.customerId()); order.setShippingAddress(request.shippingAddress()); order.setStatus("PENDING"); // Process order items for (OrderItem item : request.items()) { // Check inventory with automatic tracing boolean available = inventoryService.checkAvailability(item.productId(), item.quantity()); if (!available) { throw new InventoryException("Product not available: " + item.productId()); } OrderItem orderItem = new OrderItem(); orderItem.setProductId(item.productId()); orderItem.setQuantity(item.quantity()); orderItem.setUnitPrice(item.unitPrice()); order.addItem(orderItem); } // Calculate total order.calculateTotal(); // Save order (automatically traced JPA operation) Order savedOrder = orderRepository.save(order); logger.info("Order created successfully: {}", savedOrder.getId()); return savedOrder; } @WithSpan("find-order-by-id") public Order findOrderById(String orderId) { // Auto-instrumented database call return orderRepository.findById(orderId) .orElseThrow(() -> new OrderNotFoundException("Order not found: " + orderId)); } @WithSpan("find-orders") public List<Order> findOrders(int page, int size, String status) { // Auto-instrumented database call with pagination Pageable pageable = PageRequest.of(page, size, Sort.by("createdAt").descending()); if (status != null) { return orderRepository.findByStatus(status, pageable); } else { return orderRepository.findAll(pageable).getContent(); } } @WithSpan("cancel-order") public boolean cancelOrder(String orderId, String reason) { Optional<Order> orderOpt = orderRepository.findById(orderId); if (orderOpt.isPresent()) { Order order = orderOpt.get(); order.setStatus("CANCELLED"); order.setCancellationReason(reason); orderRepository.save(order); // Restore inventory for (OrderItem item : order.getItems()) { inventoryService.restoreInventory(item.getProductId(), item.getQuantity()); } return true; } return false; } // Async method with tracing public CompletableFuture<OrderStatus> getOrderStatusAsync(String orderId) { return CompletableFuture.supplyAsync(() -> { Order order = findOrderById(orderId); return new OrderStatus(order.getId(), order.getStatus(), order.getUpdatedAt().toString()); }); } }

HTTP Client Instrumentation

package com.example.otel.service; import io.opentelemetry.instrumentation.annotations.WithSpan; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.http.HttpHeaders; import org.springframework.http.MediaType; import org.springframework.stereotype.Service; import org.springframework.web.reactive.function.client.WebClient; import org.springframework.web.reactive.function.client.WebClientResponseException; import reactor.core.publisher.Mono; import java.time.Duration; @Service public class PaymentService { private static final Logger logger = LoggerFactory.getLogger(PaymentService.class); private final WebClient webClient; public PaymentService(WebClient.Builder webClientBuilder) { this.webClient = webClientBuilder .baseUrl("http://payment-service:8080") .defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE) .build(); } // Auto-instrumented HTTP client call @WithSpan("process-payment") public PaymentResult processPayment(Order order) { logger.info("Processing payment for order: {}", order.getId()); PaymentRequest request = new PaymentRequest( order.getId(), order.getCustomerId(), order.getTotalAmount(), "USD" ); try { // Auto-instrumented HTTP call with trace context propagation PaymentResponse response = webClient.post() .uri("/api/payments/process") .bodyValue(request) .retrieve() .bodyToMono(PaymentResponse.class) .timeout(Duration.ofSeconds(30)) .block(); // In production, use reactive programming properly logger.info("Payment processed successfully: {}", response.transactionId()); return new PaymentResult(true, response.transactionId(), "Payment successful"); } catch (WebClientResponseException e) { logger.error("Payment failed with status {}: {}", e.getStatusCode(), e.getResponseBodyAsString()); return new PaymentResult(false, null, "Payment failed: " + e.getMessage()); } catch (Exception e) { logger.error("Payment processing error", e); return new PaymentResult(false, null, "Payment error: " + e.getMessage()); } } // Reactive HTTP client with automatic tracing @WithSpan("validate-payment-method") public Mono<ValidationResult> validatePaymentMethod(String customerId, String paymentMethodId) { return webClient.get() .uri("/api/payments/customers/{customerId}/methods/{paymentMethodId}/validate", customerId, paymentMethodId) .retrieve() .bodyToMono(ValidationResult.class) .timeout(Duration.ofSeconds(10)) .doOnSuccess(result -> logger.debug("Payment method validated: {}", result.valid())) .doOnError(error -> logger.error("Payment method validation failed", error)); } // Record DTOs public record PaymentRequest(String orderId, String customerId, Double amount, String currency) {} public record PaymentResponse(String transactionId, String status, String processedAt) {} public record PaymentResult(boolean success, String transactionId, String message) {} public record ValidationResult(boolean valid, String reason) {} }

Messaging System Instrumentation (Kafka)

package com.example.otel.messaging; import io.opentelemetry.instrumentation.annotations.WithSpan; import org.apache.kafka.clients.consumer.ConsumerRecord; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.kafka.annotation.KafkaListener; import org.springframework.kafka.core.KafkaTemplate; import org.springframework.messaging.handler.annotation.Payload; import org.springframework.stereotype.Component; @Component public class OrderEventsConsumer { private static final Logger logger = LoggerFactory.getLogger(OrderEventsConsumer.class); private final OrderService orderService; private final KafkaTemplate<String, Object> kafkaTemplate; public OrderEventsConsumer(OrderService orderService, KafkaTemplate<String, Object> kafkaTemplate) { this.orderService = orderService; this.kafkaTemplate = kafkaTemplate; } // Auto-instrumented Kafka consumer @KafkaListener(topics = "order-created", groupId = "order-service") public void handleOrderCreated(@Payload OrderCreatedEvent event, ConsumerRecord<String, Object> record) { logger.info("Processing order created event: {}", event.orderId()); try { // Process the event (automatically traced) orderService.processOrderCreation(event); // Send to next topic sendOrderProcessedEvent(event.orderId()); } catch (Exception e) { logger.error("Failed to process order created event: {}", event.orderId(), e); // Send to DLQ sendToDlq(event, e.getMessage()); } } @KafkaListener(topics = "order-cancelled", groupId = "order-service") public void handleOrderCancelled(@Payload OrderCancelledEvent event) { logger.info("Processing order cancelled event: {}", event.orderId()); // Auto-instrumented processing orderService.processOrderCancellation(event); } @WithSpan("send-order-processed-event") private void sendOrderProcessedEvent(String orderId) { OrderProcessedEvent event = new OrderProcessedEvent(orderId, "PROCESSED", System.currentTimeMillis()); // Auto-instrumented Kafka producer kafkaTemplate.send("order-processed", orderId, event) .addCallback( result -> logger.debug("Order processed event sent: {}", orderId), failure -> logger.error("Failed to send order processed event: {}", orderId, failure) ); } @WithSpan("send-to-dlq") private void sendToDlq(Object event, String error) { DlqMessage dlqMessage = new DlqMessage("order-created", event, error, System.currentTimeMillis()); kafkaTemplate.send("order-events-dlq", dlqMessage) .addCallback( result -> logger.warn("Message sent to DLQ"), failure -> logger.error("Failed to send message to DLQ", failure) ); } // Event DTOs public record OrderCreatedEvent(String orderId, String customerId, Double totalAmount, Long timestamp) {} public record OrderCancelledEvent(String orderId, String reason, Long timestamp) {} public record OrderProcessedEvent(String orderId, String status, Long processedAt) {} public record DlqMessage(String originalTopic, Object payload, String error, Long timestamp) {} }

5. Custom Instrumentation

Custom Span Attributes and Events

package com.example.otel.instrumentation; import io.opentelemetry.api.trace.Span; import io.opentelemetry.api.trace.StatusCode; import io.opentelemetry.context.Context; import io.opentelemetry.context.Scope; import org.aspectj.lang.ProceedingJoinPoint; import org.aspectj.lang.annotation.Around; import org.aspectj.lang.annotation.Aspect; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.stereotype.Component; import java.util.HashMap; import java.util.Map; @Aspect @Component public class BusinessMetricsAspect { private static final Logger logger = LoggerFactory.getLogger(BusinessMetricsAspect.class); @Around("@annotation(TrackBusinessMetric)") public Object trackBusinessMetric(ProceedingJoinPoint joinPoint) throws Throwable { String methodName = joinPoint.getSignature().getName(); String className = joinPoint.getTarget().getClass().getSimpleName(); Span span = Span.current(); // Add custom attributes span.setAttribute("business.operation", methodName) .setAttribute("business.component", className) .setAttribute("business.invocation.time", System.currentTimeMillis()); // Add method arguments as attributes (be careful with sensitive data) Object[] args = joinPoint.getArgs(); for (int i = 0; i < args.length; i++) { if (args[i] != null && !isSensitiveArgument(methodName, i)) { span.setAttribute("business.arg." + i, args[i].toString()); } } // Record start event span.addEvent("business.operation.started", System.currentTimeMillis(), Map.of("method", methodName, "class", className)); try { Object result = joinPoint.proceed(); // Record success event span.addEvent("business.operation.completed", System.currentTimeMillis(), Map.of("status", "success")); // Add result information if applicable if (result != null) { span.setAttribute("business.result.type", result.getClass().getSimpleName()); } return result; } catch (Exception e) { // Record error event span.setStatus(StatusCode.ERROR, "Business operation failed") .recordException(e); span.addEvent("business.operation.failed", System.currentTimeMillis(), Map.of("error", e.getMessage(), "exception.type", e.getClass().getSimpleName())); throw e; } } private boolean isSensitiveArgument(String methodName, int argIndex) { // Define which arguments might contain sensitive data Map<String, int[]> sensitiveMethods = new HashMap<>(); sensitiveMethods.put("processPayment", new int[]{0}); // Payment details sensitiveMethods.put("validateUser", new int[]{1}); // Password int[] sensitiveArgs = sensitiveMethods.get(methodName); if (sensitiveArgs != null) { for (int sensitiveArg : sensitiveArgs) { if (sensitiveArg == argIndex) { return true; } } } return false; } } // Custom annotation for business metrics @Target(ElementType.METHOD) @Retention(RetentionPolicy.RUNTIME) public @interface TrackBusinessMetric { String value() default ""; boolean trackArguments() default true; boolean trackResult() default false; }

Database Query Instrumentation

package com.example.otel.instrumentation; import io.opentelemetry.api.trace.Span; import io.opentelemetry.api.trace.StatusCode; import org.hibernate.resource.jdbc.spi.StatementInspector; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.stereotype.Component; @Component public class HibernateStatementInspector implements StatementInspector { private static final Logger logger = LoggerFactory.getLogger(HibernateStatementInspector.class); @Override public String inspect(String sql) { Span span = Span.current(); if (span.getSpanContext().isValid()) { // Add SQL information to span span.setAttribute("db.statement", sql); span.setAttribute("db.operation", extractOperation(sql)); // Add query complexity metrics int queryComplexity = estimateQueryComplexity(sql); span.setAttribute("db.query.complexity", queryComplexity); logger.debug("Executing SQL: {}", sql); } return sql; // Return original SQL unchanged } private String extractOperation(String sql) { if (sql == null) return "unknown"; String upperSql = sql.trim().toUpperCase(); if (upperSql.startsWith("SELECT")) return "SELECT"; if (upperSql.startsWith("INSERT")) return "INSERT"; if (upperSql.startsWith("UPDATE")) return "UPDATE"; if (upperSql.startsWith("DELETE")) return "DELETE"; return "OTHER"; } private int estimateQueryComplexity(String sql) { if (sql == null) return 0; int complexity = 0; String upperSql = sql.toUpperCase(); // Simple heuristics for query complexity if (upperSql.contains(" JOIN ")) complexity += 2; if (upperSql.contains(" WHERE ")) complexity += 1; if (upperSql.contains(" GROUP BY ")) complexity += 1; if (upperSql.contains(" ORDER BY ")) complexity += 1; if (upperSql.contains(" UNION ")) complexity += 3; if (upperSql.contains(" SUBQUERY")) complexity += 2; return complexity; } }

6. Configuration Management

OpenTelemetry Configuration Service

package com.example.otel.config; import io.opentelemetry.api.GlobalOpenTelemetry; import io.opentelemetry.api.metrics.Meter; import io.opentelemetry.api.trace.Tracer; import org.springframework.beans.factory.annotation.Value; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; @Configuration public class OpenTelemetryBeanConfig { @Value("${otel.service.name:unknown-service}") private String serviceName; @Bean public Tracer tracer() { return GlobalOpenTelemetry.getTracer(serviceName); } @Bean public Meter meter() { return GlobalOpenTelemetry.getMeter(serviceName); } } @Service public class OpenTelemetryConfigService { private static final Logger logger = LoggerFactory.getLogger(OpenTelemetryConfigService.class); private final Tracer tracer; private final Meter meter; public OpenTelemetryConfigService(Tracer tracer, Meter meter) { this.tracer = tracer; this.meter = meter; initializeMetrics(); } private void initializeMetrics() { // Define business metrics LongCounter orderCounter = meter.counterBuilder("orders.created") .setDescription("Total number of orders created") .setUnit("1") .build(); Histogram orderAmountHistogram = meter.histogramBuilder("orders.amount") .setDescription("Order amount distribution") .setUnit("USD") .build(); LongCounter errorCounter = meter.counterBuilder("errors.total") .setDescription("Total number of errors") .setUnit("1") .build(); logger.info("OpenTelemetry metrics initialized"); } public void recordOrderCreated(double amount, String customerType) { // Get or create counter with labels LongCounter orderCounter = meter.counterBuilder("orders.created") .build(); orderCounter.add(1, Attributes.of( AttributeKey.stringKey("customer.type"), customerType, AttributeKey.stringKey("amount.range"), getAmountRange(amount) )); // Record amount distribution Histogram amountHistogram = meter.histogramBuilder("orders.amount") .build(); amountHistogram.record(amount, Attributes.of(AttributeKey.stringKey("customer.type"), customerType)); } public void recordError(String errorType, String component) { LongCounter errorCounter = meter.counterBuilder("errors.total") .build(); errorCounter.add(1, Attributes.of( AttributeKey.stringKey("error.type"), errorType, AttributeKey.stringKey("component"), component )); } private String getAmountRange(double amount) { if (amount < 50) return "0-50"; if (amount < 100) return "50-100"; if (amount < 200) return "100-200"; return "200+"; } }

7. Performance Monitoring

Performance Monitoring Aspect

package com.example.otel.monitoring; import io.opentelemetry.api.metrics.LongHistogram; import io.opentelemetry.api.trace.Span; import io.opentelemetry.api.trace.StatusCode; import org.aspectj.lang.ProceedingJoinPoint; import org.aspectj.lang.annotation.Around; import org.aspectj.lang.annotation.Aspect; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.stereotype.Component; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.TimeUnit; @Aspect @Component public class PerformanceMonitoringAspect { private static final Logger logger = LoggerFactory.getLogger(PerformanceMonitoringAspect.class); private final ConcurrentHashMap<String, LongHistogram> methodHistograms = new ConcurrentHashMap<>(); @Around("execution(* com.example.otel.service.*.*(..)) || " + "execution(* com.example.otel.repository.*.*(..)) || " + "execution(* com.example.otel.controller.*.*(..))") public Object monitorPerformance(ProceedingJoinPoint joinPoint) throws Throwable { String methodName = joinPoint.getSignature().toShortString(); String className = joinPoint.getTarget().getClass().getSimpleName(); long startTime = System.nanoTime(); boolean success = true; try { Object result = joinPoint.proceed(); return result; } catch (Exception e) { success = false; Span.current().setStatus(StatusCode.ERROR, e.getMessage()); throw e; } finally { long duration = System.nanoTime() - startTime; long durationMs = TimeUnit.NANOSECONDS.toMillis(duration); // Record method execution time recordMethodExecution(methodName, className, durationMs, success); // Log slow method executions if (durationMs > 1000) { // 1 second threshold logger.warn("Slow method execution: {} took {}ms", methodName, durationMs); } } } private void recordMethodExecution(String methodName, String className, long durationMs, boolean success) { String histogramName = "method.execution.time"; LongHistogram histogram = methodHistograms.computeIfAbsent(histogramName, name -> io.opentelemetry.api.GlobalOpenTelemetry.getMeter("performance") .histogramBuilder(name) .setDescription("Method execution time distribution") .setUnit("ms") .ofLongs() .build()); histogram.record(durationMs, io.opentelemetry.api.common.Attributes.of( io.opentelemetry.api.common.AttributeKey.stringKey("method"), methodName, io.opentelemetry.api.common.AttributeKey.stringKey("class"), className, io.opentelemetry.api.common.AttributeKey.booleanKey("success"), success )); } }

8. Log Correlation

MDC Log Correlation

package com.example.otel.logging; import org.slf4j.MDC; import org.springframework.core.annotation.Order; import org.springframework.stereotype.Component; import org.springframework.web.filter.OncePerRequestFilter; import javax.servlet.FilterChain; import javax.servlet.ServletException; import javax.servlet.http.HttpServletRequest; import javax.servlet.http.HttpServletResponse; import java.io.IOException; @Component @Order(1) public class LogCorrelationFilter extends OncePerRequestFilter { @Override protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain) throws ServletException, IOException { // Extract trace context from headers String traceId = extractTraceId(request); String spanId = extractSpanId(request); // Add to MDC for log correlation if (traceId != null) { MDC.put("trace_id", traceId); } if (spanId != null) { MDC.put("span_id", spanId); } try { filterChain.doFilter(request, response); } finally { // Clear MDC MDC.clear(); } } private String extractTraceId(HttpServletRequest request) { // Extract from W3C Trace Context header String traceParent = request.getHeader("traceparent"); if (traceParent != null && traceParent.length() > 55) { return traceParent.substring(3, 35); // Extract trace ID } // Extract from other headers if needed return request.getHeader("X-B3-TraceId"); } private String extractSpanId(HttpServletRequest request) { // Extract from W3C Trace Context header String traceParent = request.getHeader("traceparent"); if (traceParent != null && traceParent.length() > 55) { return traceParent.substring(36, 52); // Extract span ID } // Extract from other headers if needed return request.getHeader("X-B3-SpanId"); } }

9. Testing and Validation

Integration Tests

package com.example.otel.test; import io.opentelemetry.api.common.AttributeKey; import io.opentelemetry.sdk.testing.junit5.OpenTelemetryExtension; import io.opentelemetry.sdk.trace.data.SpanData; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.RegisterExtension; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.client.TestRestTemplate; import org.springframework.http.ResponseEntity; import java.util.List; import static org.assertj.core.api.Assertions.assertThat; @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) class OpenTelemetryIntegrationTest { @RegisterExtension static final OpenTelemetryExtension otelTesting = OpenTelemetryExtension.create(); private final TestRestTemplate restTemplate = new TestRestTemplate(); @Test void testHttpRequestCreatesSpan() { // Given String baseUrl = "http://localhost:" + port; // When ResponseEntity<String> response = restTemplate.getForEntity( baseUrl + "/api/orders/123", String.class); // Then assertThat(response.getStatusCode().is2xxSuccessful()).isTrue(); // Verify spans were created List<SpanData> spans = otelTesting.getSpans(); assertThat(spans).isNotEmpty(); // Verify HTTP span attributes SpanData httpSpan = spans.stream() .filter(span -> "GET /api/orders/{orderId}".equals(span.getName())) .findFirst() .orElseThrow(); assertThat(httpSpan.getAttributes().get(AttributeKey.stringKey("http.method"))) .isEqualTo("GET"); assertThat(httpSpan.getAttributes().get(AttributeKey.stringKey("http.route"))) .isEqualTo("/api/orders/{orderId}"); } @Test void testDatabaseOperationCreatesSpan() { // When restTemplate.getForEntity("/api/orders?page=0&size=10", String.class); // Then List<SpanData> spans = otelTesting.getSpans(); // Verify database span was created boolean hasDbSpan = spans.stream() .anyMatch(span -> "OrderRepository.findByStatus".equals(span.getName()) || span.getName().startsWith("SELECT")); assertThat(hasDbSpan).isTrue(); } }

10. Spring Boot Application

Main Application Class

package com.example.otel; import org.springframework.boot.SpringApplication; import org.springframework.boot.autoconfigure.SpringBootApplication; import org.springframework.scheduling.annotation.EnableAsync; @SpringBootApplication @EnableAsync public class OrderServiceApplication { public static void main(String[] args) { SpringApplication.run(OrderServiceApplication.class, args); } }

Summary

This comprehensive OpenTelemetry auto-instrumentation implementation provides:

Key Features:

Zero-Code Instrumentation: Automatic tracing for HTTP, database, messaging
Spring Boot Integration: Seamless Spring framework support
Custom Instrumentation: Business-specific metrics and spans
Log Correlation: Trace context in logs
Performance Monitoring: Method-level performance tracking
Testing Support: Integration tests for telemetry

Auto-Instrumented Components:

HTTP Servers (Spring MVC, WebFlux)
HTTP Clients (WebClient, RestTemplate)
Database Operations (JPA, Hibernate, JDBC)
Messaging Systems (Kafka, RabbitMQ)
Async Operations (CompletableFuture, @Async)
Cache Operations (Spring Cache)

Best Practices:

Use Java agent for production deployments
Configure appropriate sampling rates
Add business context to spans
Monitor performance impact
Secure sensitive data in spans
Use structured logging with trace correlation

This implementation provides production-ready observability with minimal code changes, following OpenTelemetry standards and best practices.