Cilium Hubble in Java: Deep Observability for Cloud-Native JVM Applications

In the complex world of Kubernetes networking and microservices, Java applications require more than just application-level monitoring—they need deep network observability that connects application behavior with underlying network flows. Cilium Hubble is a fully distributed networking and security observability platform built on eBPF that provides real-time visibility into network traffic, application dependency mapping, and security events for containerized Java applications.

What is Cilium Hubble?

Hubble is an observability tool built on top of Cilium and eBPF that provides deep visibility into the communication and behavior of cloud-native applications. It operates at the network and kernel level, capturing rich context about network flows, API calls, and service dependencies without requiring application changes. For Java applications, Hubble bridges the gap between JVM metrics and network behavior.

Why Hubble is Transformative for Java Microservices

  1. Application-Aware Networking: Correlates L3/L4 network flows with L7 application-layer data for HTTP, gRPC, and Kafka.
  2. Dependency Mapping: Automatically discovers and visualizes service dependencies between Java microservices.
  3. Security Observability: Identifies unexpected network connections, policy violations, and security threats.
  4. Performance Insights: Pinpoints network latency, DNS issues, and TCP problems affecting Java applications.
  5. Zero-Instrumentation Monitoring: Provides deep insights without requiring code changes or sidecar proxies.

Hubble Observability Layers for Java Applications

1. Network Flow Visibility

# Observe real-time network flows for Java services
hubble observe --pod java-app-7d8f6c9d5-abc123
# Filter by specific protocol and port
hubble observe --protocol HTTP --port 8080 -f
# Monitor inter-service communication
hubble observe --from-pod frontend-java --to-pod backend-java

2. Service Dependency Mapping

# Generate service map for Java microservices
hubble ui
# CLI-based service dependency view
hubble observe --label app=java-order-service --verdict FORWARDED

Deploying Java Applications with Hubble Observability

1. Kubernetes Deployment with Cilium Network Policies

apiVersion: apps/v1
kind: Deployment
metadata:
name: order-service
labels:
app: order-service
version: v1.2.0
component: backend
spec:
replicas: 3
selector:
matchLabels:
app: order-service
template:
metadata:
labels:
app: order-service
version: v1.2.0
component: backend
annotations:
# Enable detailed Hubble monitoring
io.cilium.hubble.metrics.enabled: "true"
io.cilium.hubble.metrics.port: "8080"
spec:
containers:
- name: order-service
image: company/java-order-service:1.2.0
ports:
- containerPort: 8080
name: http
- containerPort: 9090
name: metrics
env:
- name: JAVA_OPTS
value: "-Dspring.profiles.active=kubernetes -javaagent:/app/hubble-agent.jar"
resources:
requests:
memory: "512Mi"
cpu: "200m"
limits:
memory: "1Gi"
cpu: "500m"
livenessProbe:
httpGet:
path: /actuator/health
port: 8080
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
httpGet:
path: /actuator/health
port: 8080
initialDelaySeconds: 5
periodSeconds: 5
---
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
name: order-service-network-policy
spec:
description: "Network policy for Java order service"
endpointSelector:
matchLabels:
app: order-service
ingress:
- fromEndpoints:
- matchLabels:
app: frontend-service
toPorts:
- ports:
- port: "8080"
protocol: TCP
rules:
http:
- method: "GET"
path: "/orders/*"
- method: "POST"
path: "/orders"
egress:
- toEndpoints:
- matchLabels:
app: payment-service
toPorts:
- ports:
- port: "8080"
protocol: TCP

Real-Time Java Application Monitoring with Hubble

1. Service Dependency Discovery

@RestController
public class OrderController {
@Autowired
private PaymentService paymentService;
@Autowired 
private InventoryService inventoryService;
@PostMapping("/orders")
public Order createOrder(@RequestBody OrderRequest request) {
// Hubble will observe these service-to-service calls
PaymentResult payment = paymentService.process(request);
InventoryStatus inventory = inventoryService.reserve(request);
return assembleOrder(payment, inventory);
}
}

Hubble Observations:

  • Network flow: order-service → payment-service:8080 (HTTP POST)
  • Network flow: order-service → inventory-service:8080 (HTTP PUT)
  • Latency metrics for each service call
  • HTTP status codes and response times

2. gRPC Service Communication

@Service
public class UserServiceClient {
private final UserServiceGrpc.UserServiceBlockingStub userStub;
public UserServiceClient(Channel channel) {
this.userStub = UserServiceGrpc.newBlockingStub(channel);
}
public User getUser(String userId) {
// Hubble captures gRPC method calls and latency
GetUserRequest request = GetUserRequest.newBuilder()
.setUserId(userId)
.build();
return userStub.getUser(request);
}
}

Hubble CLI for Java Application Troubleshooting

1. Real-Time Flow Monitoring

# Monitor all flows for Java application namespace
hubble observe --namespace java-apps --follow
# Filter by specific Java service
hubble observe --label app=payment-service --follow
# View denied flows (potential security issues)
hubble observe --verdict DENIED --namespace java-apps
# Monitor DNS queries from Java applications
hubble observe --type l7 --protocol DNS

2. Performance and Latency Analysis

# Identify slow HTTP responses
hubble observe --protocol HTTP --namespace java-apps | \
jq 'select(.l7 != null) | select(.l7.latency > 1.0)'
# Monitor TCP connection issues
hubble observe --tcp-flags SYN,FIN --namespace java-apps
# Track service response codes
hubble observe --protocol HTTP --label app=order-service | \
jq '.l7.http.code' | sort | uniq -c

Advanced Hubble Integration with Java Applications

1. Custom Metrics Export

@Component
public class HubbleMetricsExporter {
private final MeterRegistry meterRegistry;
public HubbleMetricsExporter(MeterRegistry meterRegistry) {
this.meterRegistry = meterRegistry;
}
@EventListener
public void handleNetworkEvent(NetworkFlowEvent event) {
// Correlate application metrics with Hubble network data
Timer.builder("http.client.requests")
.tag("destination_service", event.getDestinationService())
.tag("protocol", event.getProtocol())
.register(meterRegistry)
.record(event.getDuration());
Counter.builder("network.bytes")
.tag("direction", "egress")
.register(meterRegistry)
.increment(event.getBytesSent());
}
}

2. Service Mesh Integration

apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
name: java-app-http-observability
spec:
description: "Enable L7 observability for Java HTTP traffic"
endpointSelector: {}
ingress:
- toPorts:
- ports:
- port: "8080"
protocol: TCP
rules:
http:
- {}
egress:
- toPorts:
- ports:
- port: "8080"
protocol: TCP
rules:
http:
- {}

Security Monitoring for Java Applications

1. Unexpected Connection Detection

# Alert on unexpected outbound connections
hubble observe --from-label app=java-app --verdict DENIED
# Monitor for port scanning attempts
hubble observe --to-label app=java-app --verdict DROPPED
# Detect DNS anomalies
hubble observe --protocol DNS --label app=java-app | \
jq 'select(.dns.rcode != 0)'

2. API Security Monitoring

# Monitor for suspicious HTTP patterns
hubble observe --protocol HTTP --label app=java-api | \
jq 'select(.l7.http.method == "POST" and .l7.http.code >= 400)'
# Track authentication failures
hubble observe --protocol HTTP --label app=auth-service | \
jq 'select(.l7.http.code == 401 or .l7.http.code == 403)'

Hubble UI for Java Application Visualization

1. Service Map Generation

# Deploy Hubble UI
kubectl apply -f https://raw.githubusercontent.com/cilium/hubble/master/kubernetes/manifests/hubble-ui.yaml
# Access service map for Java applications
# Visualizes: order-service → payment-service → user-service

2. Flow Visualization

  • Real-time network flow diagrams between Java microservices
  • HTTP/gRPC method-level visibility
  • Latency heat maps for service dependencies
  • Security policy violation highlighting

Integration with Java Monitoring Stack

1. Prometheus Metrics Integration

apiVersion: v1
kind: ConfigMap
metadata:
name: hubble-metrics-config
data:
config.yaml: |
metrics:
- source: "hubble"
provider: "prometheus"
labels:
- "source_namespace"
- "source_pod_name"
- "destination_namespace" 
- "destination_pod_name"
flows:
- query: 'rate(hubble_flows_processed_total{verdict="FORWARDED"}[1m])'
name: "forwarded_flows"

2. Grafana Dashboards for Java Applications

{
"dashboard": {
"title": "Java Microservices Network Observatory",
"panels": [
{
"title": "HTTP Request Rate",
"targets": [
{
"expr": "rate(hubble_http_requests_total{source_app=~\"java-.*\"}[1m])",
"legendFormat": "{{source_app}}"
}
]
},
{
"title": "Service-to-Service Latency",  
"targets": [
{
"expr": "histogram_quantile(0.95, rate(hubble_http_request_duration_seconds_bucket[5m]))",
"legendFormat": "P95 {{source_app}} → {{destination_app}}"
}
]
}
]
}
}

Best Practices for Java Applications with Hubble

  1. Consistent Labeling: Use standardized labels (app, version, component) for effective filtering.
  2. L7 Protocol Awareness: Ensure HTTP/gRPC traffic is properly annotated for rich observability.
  3. Network Policy Alignment: Define CiliumNetworkPolicies that reflect actual application dependencies.
  4. Correlation with App Metrics: Combine Hubble network data with JVM metrics for full-stack observability.
  5. Security Baseline: Establish normal network patterns and alert on deviations.

Troubleshooting Common Java Application Issues

1. Network Connectivity Problems

# Debug service discovery issues
hubble observe --from-pod order-service-7d8f6c9d5-abc123 --to-fqdn user-service.java-apps.svc.cluster.local
# Identify DNS resolution problems  
hubble observe --protocol DNS --from-label app=order-service

2. Performance Degradation Analysis

# Find slow service dependencies
hubble observe --protocol HTTP --label app=order-service | \
jq 'select(.l7 != null) | {source: .source.namespace, destination: .destination.namespace, latency: .l7.latency}' | \
jq -s 'sort_by(-.latency) | .[0:10]'

Conclusion

Cilium Hubble provides Java development teams with unprecedented visibility into the network behavior of their cloud-native applications. By leveraging eBPF technology, Hubble delivers rich, context-aware observability that connects application logic with network flows, service dependencies, and security events—all without requiring code changes or significant performance overhead.

For Java microservices operating in Kubernetes environments, Hubble transforms network troubleshooting from a black-box mystery into a transparent, actionable observability experience. Its ability to correlate L7 application data with L3/L4 network flows makes it an indispensable tool for maintaining performance, reliability, and security in complex distributed Java applications.


Leave a Reply

Your email address will not be published. Required fields are marked *


Macro Nepal Helper