Introduction
Kibana is a powerful data visualization tool that works seamlessly with Elasticsearch to transform your Java application metrics into actionable insights. When properly configured, it provides real-time visibility into application performance, JVM health, and business metrics.
This guide covers setting up comprehensive Kibana visualizations for Java application metrics using popular monitoring tools and frameworks.
Architecture Overview
[Java Application] → [Metrics Collector] → [Elasticsearch] → [Kibana Visualizations] ↓ ↓ ↓ ↓ Micrometer/ Logstash/ Data Store Dashboards/ JMX Metrics Elastic APM & Search Engine Visualizations
Step 1: Setting Up Metrics Export from Java
Maven Dependencies
<properties>
<micrometer.version>1.12.0</micrometer.version>
<spring-boot.version>3.2.0</spring-boot.version>
</properties>
<dependencies>
<!-- Spring Boot Actuator for metrics -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<!-- Micrometer Registry for Elastic -->
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-elastic</artifactId>
<version>${micrometer.version}</version>
</dependency>
<!-- Alternatively: Prometheus registry -->
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
<version>${micrometer.version}</version>
</dependency>
</dependencies>
Application Configuration
application.yml
management: endpoints: web: exposure: include: health,metrics,prometheus endpoint: health: show-details: always metrics: enabled: true metrics: export: elastic: enabled: true host: http://localhost:9200 index: java-metrics index-date-format: yyyy-MM-dd step: 30s distribution: percentiles-histogram: http.server.requests: true percentiles: http.server.requests: 0.5, 0.95, 0.99 sla: http.server.requests: 100ms, 500ms, 1s # Custom application metrics app: metrics: prefix: myapp
Custom Metrics Configuration
MetricsConfig.java
package com.example.monitoring.config;
import io.micrometer.core.instrument.MeterRegistry;
import io.micrometer.core.instrument.Gauge;
import io.micrometer.core.instrument.Counter;
import io.micrometer.core.instrument.Timer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.util.concurrent.ConcurrentHashMap;
import java.util.Map;
@Configuration
public class MetricsConfig {
private final Map<String, Double> businessMetrics = new ConcurrentHashMap<>();
@Bean
public MeterRegistryCustomizer<MeterRegistry> metricsCommonTags() {
return registry -> registry.config()
.commonTags("application", "order-service",
"environment", "production",
"region", "us-east-1");
}
@Bean
public Gauge businessMetricsGauge(MeterRegistry registry) {
return Gauge.builder("app.business.active_orders")
.description("Number of active orders")
.register(registry, this,
config -> businessMetrics.getOrDefault("active_orders", 0.0));
}
@Bean
public Counter orderCreatedCounter(MeterRegistry registry) {
return Counter.builder("app.business.orders.created")
.description("Total orders created")
.register(registry);
}
public void setBusinessMetric(String key, Double value) {
businessMetrics.put(key, value);
}
}
Instrumented Service Example
OrderService.java
package com.example.service;
import io.micrometer.core.instrument.Counter;
import io.micrometer.core.instrument.Timer;
import io.micrometer.core.instrument.MeterRegistry;
import org.springframework.stereotype.Service;
import java.util.concurrent.TimeUnit;
@Service
public class OrderService {
private final Counter orderCounter;
private final Timer orderProcessingTimer;
private final MetricsConfig metricsConfig;
public OrderService(MeterRegistry registry, MetricsConfig metricsConfig) {
this.orderCounter = registry.counter("app.orders.total");
this.orderProcessingTimer = Timer.builder("app.orders.processing.time")
.description("Order processing time")
.register(registry);
this.metricsConfig = metricsConfig;
}
public Order processOrder(OrderRequest request) {
// Using Timer.Sample to measure method execution time
Timer.Sample sample = Timer.start();
try {
// Business logic
Order order = createOrder(request);
orderCounter.increment();
// Update business metric
metricsConfig.setBusinessMetric("active_orders",
getActiveOrdersCount());
return order;
} finally {
sample.stop(orderProcessingTimer);
}
}
private Order createOrder(OrderRequest request) {
// Simulate processing time
try {
Thread.sleep(100 + (long)(Math.random() * 100));
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
return new Order(request);
}
private double getActiveOrdersCount() {
// Implementation to get active orders from database
return Math.random() * 1000; // Example value
}
}
Step 2: Elasticsearch Index Templates
Create Index Template for Java Metrics
Using Kibana Dev Tools:
PUT _index_template/java-metrics-template
{
"index_patterns": ["java-metrics-*"],
"template": {
"mappings": {
"properties": {
"@timestamp": {
"type": "date"
},
"application": {
"type": "keyword"
},
"environment": {
"type": "keyword"
},
"metric.name": {
"type": "keyword"
},
"metric.value": {
"type": "double"
},
"jvm.memory.used": {
"type": "double"
},
"jvm.memory.max": {
"type": "double"
},
"jvm.gc.pause": {
"type": "double"
},
"http.server.requests.duration": {
"type": "double"
},
"system.cpu.usage": {
"type": "double"
}
}
},
"settings": {
"number_of_shards": 1,
"number_of_replicas": 1
}
}
}
Step 3: Kibana Visualizations
1. JVM Memory Dashboard
Kibana Lens Visualization for Memory Usage:
{
"title": "JVM Memory Usage",
"type": "lens",
"attributes": {
"visualizationType": "lnsXY",
"title": "JVM Heap Memory Usage",
"references": [],
"state": {
"datasourceStates": {
"formBased": {
"layers": {
"layer1": {
"columns": {
"x-axis": {
"columnId": "x-axis",
"field": "@timestamp",
"dataType": "date",
"isBucketed": true,
"sourceField": "@timestamp",
"params": {
"interval": "auto"
}
},
"y-axis": {
"columnId": "y-axis",
"field": "jvm.memory.used",
"dataType": "number",
"isBucketed": false,
"sourceField": "jvm.memory.used",
"params": {
"format": {
"id": "bytes"
}
}
},
"split": {
"columnId": "split",
"field": "application",
"dataType": "string",
"isBucketed": true,
"sourceField": "application"
}
}
}
}
}
}
}
}
}
2. Application Performance Overview
Create Visualization via Kibana UI:
- Go to Analytics > Dashboard > Create dashboard
- Add visualization > Lens
- Configure metrics:
HTTP Request Latency:
- X-axis:
@timestamp(Date Histogram) - Y-axis:
http.server.requests.duration(Average) - Break down by:
status(if available)
Request Rate:
- Y-axis:
http.server.requests.count(Count)
Error Rate:
- Y-axis: Count of requests with
status: 5xx
3. Custom Business Metrics Visualization
Orders Processing Dashboard:
{
"title": "Business Metrics - Orders",
"type": "lens",
"attributes": {
"visualizationType": "lnsMetric",
"title": "Active Orders",
"references": [],
"state": {
"datasourceStates": {
"formBased": {
"layers": {
"layer1": {
"columns": {
"metric-column": {
"columnId": "metric-column",
"field": "app.business.active_orders",
"dataType": "number",
"isBucketed": false,
"sourceField": "app.business.active_orders",
"params": {
"format": {
"id": "number"
}
}
}
}
}
}
}
}
}
}
}
Step 4: Kibana Query Examples
KQL (Kibana Query Language) Examples
# JVM metrics for specific application metric.name: "jvm.memory.used" and application: "order-service" # High latency requests http.server.requests.duration > 1000 # Error analysis status >= 500 and status < 600 # Memory pressure detection jvm.memory.used / jvm.memory.max > 0.8 # Business metrics with time range app.business.active_orders > 100 and @timestamp >= now-1h
Advanced Queries with Aggregations
GET java-metrics-*/_search
{
"size": 0,
"query": {
"range": {
"@timestamp": {
"gte": "now-1h"
}
}
},
"aggs": {
"applications": {
"terms": {
"field": "application.keyword"
},
"aggs": {
"avg_memory": {
"avg": {
"field": "jvm.memory.used"
}
},
"p95_latency": {
"percentiles": {
"field": "http.server.requests.duration",
"percents": [95]
}
}
}
}
}
}
Step 5: Alerting and Anomaly Detection
Kibana Alert Rules
1. High Memory Usage Alert:
{
"rule_type_id": ".index-threshold",
"name": "High JVM Memory Usage",
"params": {
"index": "java-metrics-*",
"timeField": "@timestamp",
"aggType": "avg",
"aggField": "jvm.memory.used",
"groupBy": "top",
"termSize": 5,
"termField": "application.keyword",
"timeWindowSize": 5,
"timeWindowUnit": "m",
"thresholdComparator": ">",
"threshold": [8589934592],
"actions": []
}
}
2. High Error Rate Alert:
{
"rule_type_id": "monitoring_alert",
"name": "High HTTP Error Rate",
"params": {
"criteria": [
{
"aggType": "count",
"condition": ">",
"threshold": 100,
"filter": "status:5xx"
}
],
"index": "java-metrics-*",
"timeField": "@timestamp",
"timeWindowSize": 10,
"timeWindowUnit": "m"
}
}
Anomaly Detection Jobs
Create machine learning jobs in Kibana:
- Go to Analytics > Machine Learning > Anomaly Detection
- Create job > Single metric
- Configure:
- Data source:
java-metrics-* - Field:
http.server.requests.duration - Partition field:
application
Step 6: Complete Dashboard Example
Java Application Monitoring Dashboard
Create a comprehensive dashboard with:
- JVM Health Panel:
- Heap Memory Usage (Area chart)
- GC Pause Times (Line chart)
- Thread Count (Metric)
- CPU Usage (Gauge)
- Application Performance Panel:
- Request Rate (Line chart)
- Response Time Percentiles (Line chart)
- Error Rate (Bar chart)
- Endpoint Performance (Data table)
- Business Metrics Panel:
- Active Orders (Metric)
- Orders Created (Trend)
- Processing Time (Gauge)
- System Resources Panel:
- System CPU (Line chart)
- Disk I/O (Area chart)
- Network Usage (Line chart)
Best Practices
1. Metric Naming Convention
// Good naming
Counter.builder("app.orders.created")
Counter.builder("http.requests.total")
Gauge.builder("jvm.memory.heap.used")
// Avoid
Counter.builder("ordersCreated") // inconsistent casing
Counter.builder("app_orders_created") // mixed separators
2. Tagging Strategy
registry.counter("http.requests",
"method", "GET",
"status", "200",
"uri", "/api/orders",
"application", "order-service");
3. Retention Policy
PUT _ilm/policy/java-metrics-policy
{
"policy": {
"phases": {
"hot": {
"min_age": "0ms",
"actions": {
"rollover": {
"max_size": "50gb",
"max_age": "1d"
}
}
},
"delete": {
"min_age": "30d",
"actions": {
"delete": {}
}
}
}
}
}
Conclusion
Setting up Kibana visualizations for Java metrics provides powerful insights into your application's performance and health. By combining:
- Micrometer for metric collection
- Elasticsearch for storage and search
- Kibana for visualization and alerting
You create a comprehensive monitoring solution that helps you:
- Detect performance issues proactively
- Understand application behavior under load
- Make data-driven decisions about scaling and optimization
- Quickly troubleshoot production incidents
The key to success is consistent metric naming, proper tagging, and designing visualizations that provide actionable information for your specific use case.