Introduction to OTLP
OTLP (OpenTelemetry Protocol) is the native protocol for OpenTelemetry that enables exporting telemetry data (traces, metrics, logs) to various backends. The OTLP exporter sends data to OpenTelemetry collectors or compatible backends using gRPC or HTTP.
Key Features
- Unified Protocol: Single protocol for traces, metrics, and logs
- Multiple Transports: Supports gRPC and HTTP/protobuf
- Efficient: Binary protocol with compression support
- Backend Agnostic: Works with Jaeger, Prometheus, Azure Monitor, etc.
- Production Ready: Built-in retry mechanisms and batch processing
Implementation Guide
Dependencies
Add to your pom.xml:
<properties>
<opentelemetry.version>1.34.1</opentelemetry.version>
</properties>
<dependencies>
<!-- OpenTelemetry API -->
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-api</artifactId>
<version>${opentelemetry.version}</version>
</dependency>
<!-- OpenTelemetry SDK -->
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-sdk</artifactId>
<version>${opentelemetry.version}</version>
</dependency>
<!-- OTLP Exporters -->
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-exporter-otlp</artifactId>
<version>${opentelemetry.version}</version>
</dependency>
<!-- For gRPC transport -->
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-netty-shaded</artifactId>
<version>1.59.0</version>
</dependency>
<!-- For logging -->
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-exporter-logging</artifactId>
<version>${opentelemetry.version}</version>
</dependency>
<!-- For metrics -->
<dependency>
<groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-sdk-metrics</artifactId>
<version>${opentelemetry.version}</version>
</dependency>
</dependencies>
Basic OTLP Exporter Setup
Configuration Builder
import io.opentelemetry.exporter.otlp.http.trace.OtlpHttpSpanExporter;
import io.opentelemetry.exporter.otlp.metrics.OtlpGrpcMetricExporter;
import io.opentelemetry.exporter.otlp.trace.OtlpGrpcSpanExporter;
import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.sdk.resources.Resource;
import io.opentelemetry.semconv.ResourceAttributes;
import java.time.Duration;
import java.util.concurrent.TimeUnit;
public class OTLPConfigBuilder {
public static OtlpGrpcSpanExporter createGrpcSpanExporter() {
return OtlpGrpcSpanExporter.builder()
.setEndpoint("http://localhost:4317") // OTLP gRPC endpoint
.setTimeout(30, TimeUnit.SECONDS)
.setCompression("gzip") // Optional compression
.addHeader("api-key", "your-api-key") // For authenticated backends
.build();
}
public static OtlpHttpSpanExporter createHttpSpanExporter() {
return OtlpHttpSpanExporter.builder()
.setEndpoint("http://localhost:4318/v1/traces") // OTLP HTTP endpoint
.setTimeout(30, TimeUnit.SECONDS)
.addHeader("Authorization", "Bearer your-token")
.build();
}
public static OtlpGrpcMetricExporter createGrpcMetricExporter() {
return OtlpGrpcMetricExporter.builder()
.setEndpoint("http://localhost:4317")
.setTimeout(10, TimeUnit.SECONDS)
.build();
}
public static Resource createResource() {
return Resource.getDefault()
.merge(Resource.create(Attributes.builder()
.put(ResourceAttributes.SERVICE_NAME, "my-java-service")
.put(ResourceAttributes.SERVICE_VERSION, "1.0.0")
.put(ResourceAttributes.DEPLOYMENT_ENVIRONMENT, "production")
.put("custom.attribute", "custom-value")
.build()));
}
}
Complete OpenTelemetry Setup
import io.opentelemetry.api.GlobalOpenTelemetry;
import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.api.trace.Span;
import io.opentelemetry.api.trace.Tracer;
import io.opentelemetry.api.metrics.Meter;
import io.opentelemetry.context.Scope;
import io.opentelemetry.sdk.OpenTelemetrySdk;
import io.opentelemetry.sdk.trace.SdkTracerProvider;
import io.opentelemetry.sdk.trace.export.BatchSpanProcessor;
import io.opentelemetry.sdk.metrics.SdkMeterProvider;
import io.opentelemetry.sdk.metrics.export.PeriodicMetricReader;
public class OpenTelemetrySetup {
private static final String INSTRUMENTATION_NAME = "com.example.otel";
private static OpenTelemetry openTelemetry;
private static Tracer tracer;
private static Meter meter;
public static void initialize() {
// Build resource
Resource resource = OTLPConfigBuilder.createResource();
// Configure span exporter
OtlpGrpcSpanExporter spanExporter = OTLPConfigBuilder.createGrpcSpanExporter();
// Configure metric exporter
OtlpGrpcMetricExporter metricExporter = OTLPConfigBuilder.createGrpcMetricExporter();
// Setup tracer provider
SdkTracerProvider tracerProvider = SdkTracerProvider.builder()
.addSpanProcessor(
BatchSpanProcessor.builder(spanExporter)
.setScheduleDelay(100, TimeUnit.MILLISECONDS) // Batch every 100ms
.setMaxExportBatchSize(512) // Max spans per batch
.setMaxQueueSize(2048) // Max spans in queue
.build()
)
.setResource(resource)
.build();
// Setup meter provider
SdkMeterProvider meterProvider = SdkMeterProvider.builder()
.registerMetricReader(
PeriodicMetricReader.builder(metricExporter)
.setInterval(Duration.ofSeconds(60)) // Export every 60 seconds
.build()
)
.setResource(resource)
.build();
// Build OpenTelemetry instance
openTelemetry = OpenTelemetrySdk.builder()
.setTracerProvider(tracerProvider)
.setMeterProvider(meterProvider)
.build();
// Register as global instance
GlobalOpenTelemetry.set(openTelemetry);
// Create tracer and meter
tracer = openTelemetry.getTracer(INSTRUMENTATION_NAME);
meter = openTelemetry.getMeter(INSTRUMENTATION_NAME);
// Add shutdown hook
Runtime.getRuntime().addShutdownHook(new Thread(() -> {
tracerProvider.shutdown();
meterProvider.shutdown();
}));
}
public static OpenTelemetry getOpenTelemetry() {
if (openTelemetry == null) {
initialize();
}
return openTelemetry;
}
public static Tracer getTracer() {
if (tracer == null) {
initialize();
}
return tracer;
}
public static Meter getMeter() {
if (meter == null) {
initialize();
}
return meter;
}
}
Advanced OTLP Exporter with Custom Configuration
import io.opentelemetry.exporter.internal.otlp.traces.TraceRequestMarshaler;
import io.opentelemetry.sdk.trace.export.SpanExporter;
import io.opentelemetry.sdk.common.CompletableResultCode;
import java.util.Collection;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
public class CustomOTLPExporter implements SpanExporter {
private final OTLPHttpClient httpClient;
private final ScheduledExecutorService executorService;
private final String endpoint;
private final Map<String, String> headers;
public CustomOTLPExporter(Builder builder) {
this.endpoint = builder.endpoint;
this.headers = builder.headers;
this.httpClient = new OTLPHttpClient();
this.executorService = Executors.newSingleThreadScheduledExecutor();
}
public static Builder builder() {
return new Builder();
}
@Override
public CompletableResultCode export(Collection<SpanData> spans) {
if (spans.isEmpty()) {
return CompletableResultCode.ofSuccess();
}
try {
TraceRequestMarshaler marshaler = TraceRequestMarshaler.create(spans);
byte[] data = marshaler.toByteArray();
// Send to OTLP endpoint
return httpClient.sendData(endpoint, data, headers);
} catch (Exception e) {
return CompletableResultCode.ofFailure();
}
}
@Override
public CompletableResultCode flush() {
return CompletableResultCode.ofSuccess();
}
@Override
public CompletableResultCode shutdown() {
executorService.shutdown();
try {
if (!executorService.awaitTermination(10, TimeUnit.SECONDS)) {
executorService.shutdownNow();
}
} catch (InterruptedException e) {
executorService.shutdownNow();
Thread.currentThread().interrupt();
}
return CompletableResultCode.ofSuccess();
}
public static class Builder {
private String endpoint = "http://localhost:4318/v1/traces";
private Map<String, String> headers = new HashMap<>();
private Duration timeout = Duration.ofSeconds(30);
public Builder setEndpoint(String endpoint) {
this.endpoint = endpoint;
return this;
}
public Builder addHeader(String key, String value) {
this.headers.put(key, value);
return this;
}
public Builder setTimeout(Duration timeout) {
this.timeout = timeout;
return this;
}
public CustomOTLPExporter build() {
return new CustomOTLPExporter(this);
}
}
}
OTLP HTTP Client Implementation
import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.time.Duration;
import java.util.Map;
import java.util.concurrent.CompletableFuture;
public class OTLPHttpClient {
private final HttpClient httpClient;
public OTLPHttpClient() {
this.httpClient = HttpClient.newBuilder()
.connectTimeout(Duration.ofSeconds(10))
.version(HttpClient.Version.HTTP_2)
.build();
}
public CompletableResultCode sendData(String endpoint, byte[] data,
Map<String, String> headers) {
CompletableResultCode result = new CompletableResultCode();
HttpRequest.Builder requestBuilder = HttpRequest.newBuilder()
.uri(URI.create(endpoint))
.POST(HttpRequest.BodyPublishers.ofByteArray(data))
.timeout(Duration.ofSeconds(30));
// Add headers
headers.forEach(requestBuilder::header);
requestBuilder.header("Content-Type", "application/x-protobuf");
HttpRequest request = requestBuilder.build();
httpClient.sendAsync(request, HttpResponse.BodyHandlers.ofString())
.whenComplete((response, throwable) -> {
if (throwable != null) {
// Log error and mark as failure
System.err.println("Failed to export spans: " + throwable.getMessage());
result.fail();
} else if (response.statusCode() >= 200 && response.statusCode() < 300) {
result.succeed();
} else {
System.err.println("Failed to export spans. Status: " + response.statusCode());
result.fail();
}
});
return result;
}
}
Practical Usage Examples
1. Tracing Example
import io.opentelemetry.api.trace.SpanKind;
import io.opentelemetry.api.common.AttributeKey;
public class OrderService {
private static final Tracer tracer = OpenTelemetrySetup.getTracer();
private static final AttributeKey<String> ORDER_ID_KEY = AttributeKey.stringKey("order.id");
private static final AttributeKey<Long> ORDER_AMOUNT_KEY = AttributeKey.longKey("order.amount");
public void processOrder(Order order) {
// Create a span for the order processing
Span span = tracer.spanBuilder("processOrder")
.setSpanKind(SpanKind.INTERNAL)
.setAttribute(ORDER_ID_KEY, order.getId())
.setAttribute(ORDER_AMOUNT_KEY, order.getAmount())
.startSpan();
try (Scope scope = span.makeCurrent()) {
// Add event for order received
span.addEvent("order.received");
// Process the order
validateOrder(order);
processPayment(order);
shipOrder(order);
// Add event for order completed
span.addEvent("order.completed");
} catch (Exception e) {
// Record exception
span.recordException(e);
span.setStatus(StatusCode.ERROR, "Order processing failed");
throw e;
} finally {
span.end();
}
}
private void validateOrder(Order order) {
Span span = tracer.spanBuilder("validateOrder").startSpan();
try (Scope scope = span.makeCurrent()) {
// Validation logic
Thread.sleep(100); // Simulate work
} catch (InterruptedException e) {
span.recordException(e);
Thread.currentThread().interrupt();
} finally {
span.end();
}
}
private void processPayment(Order order) {
Span span = tracer.spanBuilder("processPayment")
.setAttribute("payment.method", order.getPaymentMethod())
.startSpan();
try (Scope scope = span.makeCurrent()) {
// Payment processing logic
if (Math.random() < 0.05) { // 5% chance of failure for demo
throw new RuntimeException("Payment failed");
}
Thread.sleep(200);
} catch (Exception e) {
span.recordException(e);
span.setStatus(StatusCode.ERROR, "Payment processing failed");
throw e;
} finally {
span.end();
}
}
private void shipOrder(Order order) {
Span span = tracer.spanBuilder("shipOrder").startSpan();
try (Scope scope = span.makeCurrent()) {
// Shipping logic
Thread.sleep(150);
} catch (InterruptedException e) {
span.recordException(e);
Thread.currentThread().interrupt();
} finally {
span.end();
}
}
}
2. Metrics Example
import io.opentelemetry.api.metrics.Counter;
import io.opentelemetry.api.metrics.Histogram;
import io.opentelemetry.api.metrics.ObservableLongMeasurement;
public class OrderMetrics {
private static final Meter meter = OpenTelemetrySetup.getMeter();
private static final Counter orderCounter = meter.counterBuilder("orders.total")
.setDescription("Total number of orders processed")
.setUnit("1")
.build();
private static final Histogram orderAmountHistogram = meter.histogramBuilder("order.amount")
.setDescription("Distribution of order amounts")
.setUnit("USD")
.build();
private static final Counter failedOrdersCounter = meter.counterBuilder("orders.failed")
.setDescription("Total number of failed orders")
.setUnit("1")
.build();
public static void recordOrder(Order order) {
orderCounter.add(1, Attributes.of(
AttributeKey.stringKey("payment.method"), order.getPaymentMethod(),
AttributeKey.stringKey("currency"), order.getCurrency()
));
orderAmountHistogram.record(order.getAmount(), Attributes.of(
AttributeKey.stringKey("currency"), order.getCurrency()
));
}
public static void recordFailedOrder(Order order, String reason) {
failedOrdersCounter.add(1, Attributes.of(
AttributeKey.stringKey("failure.reason"), reason
));
}
// Gauge example for active orders
public static void setupActiveOrdersGauge(OrderService orderService) {
meter.gaugeBuilder("orders.active")
.setDescription("Number of active orders being processed")
.setUnit("1")
.buildWithCallback(measurement -> {
measurement.record(orderService.getActiveOrderCount());
});
}
}
3. Logging Correlation
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import io.opentelemetry.api.trace.Span;
import io.opentelemetry.context.Context;
public class OrderServiceWithLogging {
private static final Logger logger = LoggerFactory.getLogger(OrderServiceWithLogging.class);
public void processOrderWithLogging(Order order) {
Span span = Span.current();
String traceId = span.getSpanContext().getTraceId();
String spanId = span.getSpanContext().getSpanId();
// Log with trace context
logger.info("Processing order {} [traceId: {}, spanId: {}]",
order.getId(), traceId, spanId);
try {
// Business logic
processPayment(order);
logger.info("Successfully processed order {}", order.getId());
} catch (Exception e) {
logger.error("Failed to process order {} [traceId: {}, spanId: {}]",
order.getId(), traceId, spanId, e);
throw e;
}
}
}
Spring Boot Integration
Configuration Class
@Configuration
@EnableConfigurationProperties(OTLPProperties.class)
public class OpenTelemetryConfig {
@Bean
@ConditionalOnProperty(name = "otel.exporter.otlp.enabled", havingValue = "true")
public OpenTelemetry openTelemetry(OTLPProperties otlpProperties) {
Resource resource = Resource.getDefault()
.merge(Resource.create(Attributes.builder()
.put(ResourceAttributes.SERVICE_NAME, otlpProperties.getServiceName())
.put(ResourceAttributes.SERVICE_VERSION, otlpProperties.getServiceVersion())
.build()));
SdkTracerProvider tracerProvider = SdkTracerProvider.builder()
.addSpanProcessor(BatchSpanProcessor.builder(
OtlpGrpcSpanExporter.builder()
.setEndpoint(otlpProperties.getEndpoint())
.setTimeout(otlpProperties.getTimeout())
.build()
).build())
.setResource(resource)
.build();
return OpenTelemetrySdk.builder()
.setTracerProvider(tracerProvider)
.buildAndRegisterGlobal();
}
@Bean
public Tracer tracer(OpenTelemetry openTelemetry) {
return openTelemetry.getTracer("order-service");
}
}
@ConfigurationProperties(prefix = "otel.exporter.otlp")
@Data
public class OTLPProperties {
private boolean enabled = true;
private String endpoint = "http://localhost:4317";
private String serviceName = "unknown-service";
private String serviceVersion = "1.0.0";
private Duration timeout = Duration.ofSeconds(30);
}
Spring Boot Controller with Tracing
@RestController
@RequestMapping("/api/orders")
public class OrderController {
private final OrderService orderService;
private final Tracer tracer;
public OrderController(OrderService orderService, Tracer tracer) {
this.orderService = orderService;
this.tracer = tracer;
}
@PostMapping
public ResponseEntity<OrderResponse> createOrder(@RequestBody CreateOrderRequest request) {
Span span = tracer.spanBuilder("OrderController.createOrder")
.setSpanKind(SpanKind.SERVER)
.startSpan();
try (Scope scope = span.makeCurrent()) {
// Extract context from HTTP headers (for distributed tracing)
Span.current().updateName("POST /api/orders");
Order order = orderService.processOrder(request.toOrder());
OrderResponse response = new OrderResponse(order);
span.setAttribute("http.status_code", 201);
return ResponseEntity.status(201).body(response);
} catch (Exception e) {
span.recordException(e);
span.setStatus(StatusCode.ERROR, "Order creation failed");
span.setAttribute("http.status_code", 500);
throw e;
} finally {
span.end();
}
}
@GetMapping("/{orderId}")
public ResponseEntity<OrderResponse> getOrder(@PathVariable String orderId) {
Span span = tracer.spanBuilder("OrderController.getOrder")
.setAttribute("order.id", orderId)
.startSpan();
try (Scope scope = span.makeCurrent()) {
Order order = orderService.findOrder(orderId);
if (order == null) {
span.setAttribute("http.status_code", 404);
return ResponseEntity.notFound().build();
}
span.setAttribute("http.status_code", 200);
return ResponseEntity.ok(new OrderResponse(order));
} catch (Exception e) {
span.recordException(e);
span.setStatus(StatusCode.ERROR);
span.setAttribute("http.status_code", 500);
throw e;
} finally {
span.end();
}
}
}
Testing OTLP Exporter
Test Configuration
@TestConfiguration
public class TestOpenTelemetryConfig {
@Bean
@Primary
public OpenTelemetry testOpenTelemetry() {
// Use logging exporter for tests
SpanExporter loggingExporter = LoggingSpanExporter.create();
SdkTracerProvider tracerProvider = SdkTracerProvider.builder()
.addSpanProcessor(SimpleSpanProcessor.create(loggingExporter))
.build();
return OpenTelemetrySdk.builder()
.setTracerProvider(tracerProvider)
.build();
}
}
@SpringBootTest
@Import(TestOpenTelemetryConfig.class)
class OrderServiceTest {
@Autowired
private OrderService orderService;
@Autowired
private Tracer tracer;
@Test
void shouldProcessOrderWithTracing() {
Order order = new Order("123", 100L, "credit_card");
// Create test span
Span span = tracer.spanBuilder("testProcessOrder").startSpan();
try (Scope scope = span.makeCurrent()) {
orderService.processOrder(order);
// Verify business logic
assertThat(order.isProcessed()).isTrue();
} finally {
span.end();
}
}
}
Performance Optimization
Batch Processing Configuration
@Configuration
public class PerformanceOTLPConfig {
@Bean
public BatchSpanProcessor batchSpanProcessor() {
return BatchSpanProcessor.builder(
OtlpGrpcSpanExporter.builder()
.setEndpoint("http://collector:4317")
.setTimeout(10, TimeUnit.SECONDS)
.build()
)
.setScheduleDelay(100, TimeUnit.MILLISECONDS) // Adjust based on load
.setMaxQueueSize(5000) // Increase for high-throughput systems
.setMaxExportBatchSize(1000) // Adjust based on backend capacity
.setExporterTimeout(30, TimeUnit.SECONDS)
.build();
}
@Bean
public PeriodicMetricReader periodicMetricReader() {
return PeriodicMetricReader.builder(
OtlpGrpcMetricExporter.builder()
.setEndpoint("http://collector:4317")
.build()
)
.setInterval(Duration.ofMinutes(1)) // Less frequent for metrics
.build();
}
}
Error Handling and Retry Logic
public class RetryingOTLPExporter {
private final SpanExporter delegate;
private final int maxRetries;
private final Duration initialBackoff;
public RetryingOTLPExporter(SpanExporter delegate, int maxRetries, Duration initialBackoff) {
this.delegate = delegate;
this.maxRetries = maxRetries;
this.initialBackoff = initialBackoff;
}
@Override
public CompletableResultCode export(Collection<SpanData> spans) {
return exportWithRetry(spans, 0);
}
private CompletableResultCode exportWithRetry(Collection<SpanData> spans, int attempt) {
CompletableResultCode result = delegate.export(spans);
return result.whenComplete(() -> {
if (!result.isSuccess() && attempt < maxRetries) {
// Schedule retry with exponential backoff
long backoffMs = initialBackoff.toMillis() * (1 << attempt);
ScheduledExecutorService scheduler = Executors.newSingleThreadScheduledExecutor();
scheduler.schedule(() -> {
exportWithRetry(spans, attempt + 1);
scheduler.shutdown();
}, backoffMs, TimeUnit.MILLISECONDS);
}
});
}
// Delegate other methods...
}
Conclusion
The OTLP exporter in Java provides a robust, efficient way to send telemetry data to observability backends. This implementation covers:
- Basic setup with gRPC and HTTP exporters
- Spring Boot integration for easy adoption
- Advanced configurations for production use
- Performance optimization techniques
- Error handling and retry mechanisms
- Testing strategies for reliable operation
Key benefits include standardized protocol, backend flexibility, and production-ready features like batching and compression. Remember to monitor your exporter's performance and adjust configurations based on your specific workload and requirements.
Java Observability, Logging Intelligence & AI-Driven Monitoring (APM, Tracing, Logs & Anomaly Detection)
https://macronepal.com/blog/beyond-metrics-observing-serverless-and-traditional-java-applications-with-thundra-apm/
Explains using Thundra APM to observe both serverless and traditional Java applications by combining tracing, metrics, and logs into a unified observability platform for faster debugging and performance insights.
https://macronepal.com/blog/dynatrace-oneagent-in-java-2/
Explains Dynatrace OneAgent for Java, which automatically instruments JVM applications to capture metrics, traces, and logs, enabling full-stack monitoring and root-cause analysis with minimal configuration.
https://macronepal.com/blog/lightstep-java-sdk-distributed-tracing-and-observability-implementation/
Explains Lightstep Java SDK for distributed tracing, helping developers track requests across microservices and identify latency issues using OpenTelemetry-based observability.
https://macronepal.com/blog/honeycomb-io-beeline-for-java-complete-guide-2/
Explains Honeycomb Beeline for Java, which provides high-cardinality observability and deep query capabilities to understand complex system behavior and debug distributed systems efficiently.
https://macronepal.com/blog/lumigo-for-serverless-in-java-complete-distributed-tracing-guide-2/
Explains Lumigo for Java serverless applications, offering automatic distributed tracing, log correlation, and error tracking to simplify debugging in cloud-native environments. (Lumigo Docs)
https://macronepal.com/blog/from-noise-to-signals-implementing-log-anomaly-detection-in-java-applications/
Explains how to detect anomalies in Java logs using behavioral patterns and machine learning techniques to separate meaningful incidents from noisy log data and improve incident response.
https://macronepal.com/blog/ai-powered-log-analysis-in-java-from-reactive-debugging-to-proactive-insights/
Explains AI-driven log analysis for Java applications, shifting from manual debugging to predictive insights that identify issues early and improve system reliability using intelligent log processing.
https://macronepal.com/blog/titliel-java-logging-best-practices/
Explains best practices for Java logging, focusing on structured logs, proper log levels, performance optimization, and ensuring logs are useful for debugging and observability systems.
https://macronepal.com/blog/seeking-a-loguru-for-java-the-quest-for-elegant-and-simple-logging/
Explains the search for simpler, more elegant logging frameworks in Java, comparing modern logging approaches that aim to reduce complexity while improving readability and developer experience.