Article
Kustomize is a template-free Kubernetes configuration management tool that allows you to customize Kubernetes manifests for different environments without duplicating configuration. For Java applications, Kustomize overlays provide a powerful way to manage environment-specific configurations, feature flags, and resource requirements across development, staging, and production environments.
What are Kustomize Overlays?
Kustomize overlays are layered configurations that build upon a base set of Kubernetes manifests. They allow you to patch, modify, and extend your base configuration for specific environments without changing the original manifests.
Key Benefits for Java Applications:
- Environment-Specific Configs: Different configurations for dev, staging, prod
- Feature Flag Management: Environment-specific feature toggles
- Resource Management: Different resource limits per environment
- Secret Management: Environment-specific secrets and configs
- Zero Duplication: DRY (Don't Repeat Yourself) configuration management
Project Structure for Java Applications
java-app-k8s/ ├── base/ │ ├── kustomization.yaml │ ├── deployment.yaml │ ├── service.yaml │ ├── configmap.yaml │ └── hpa.yaml ├── overlays/ │ ├── dev/ │ │ ├── kustomization.yaml │ │ ├── patch-deployment.yaml │ │ └── configmap-patch.yaml │ ├── staging/ │ │ ├── kustomization.yaml │ │ ├── patch-deployment.yaml │ │ └── hpa-patch.yaml │ └── production/ │ ├── kustomization.yaml │ ├── patch-deployment.yaml │ ├── configmap-patch.yaml │ └── hpa-patch.yaml └── scripts/ ├── build-and-deploy.sh └── generate-manifests.sh
Base Configuration
1. Base Kustomization
# base/kustomization.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization namespace: java-app resources: - deployment.yaml - service.yaml - configmap.yaml - hpa.yaml - serviceaccount.yaml commonLabels: app: java-app version: "1.0.0" commonAnnotations: maintainer: "[email protected]" description: "Java Spring Boot Application"
2. Base Deployment
# base/deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: java-app labels: app: java-app version: "1.0.0" spec: replicas: 2 selector: matchLabels: app: java-app template: metadata: labels: app: java-app version: "1.0.0" spec: serviceAccountName: java-app containers: - name: java-app image: my-registry/java-app:latest ports: - containerPort: 8080 name: http env: - name: SPRING_PROFILES_ACTIVE value: "default" - name: JAVA_OPTS value: "-Xmx512m -Xms256m" - name: MANAGEMENT_ENDPOINTS_WEB_EXPOSURE_INCLUDE value: "health,info,metrics" envFrom: - configMapRef: name: java-app-config livenessProbe: httpGet: path: /actuator/health port: 8080 initialDelaySeconds: 60 periodSeconds: 10 readinessProbe: httpGet: path: /actuator/health port: 8080 initialDelaySeconds: 30 periodSeconds: 5 resources: requests: memory: "512Mi" cpu: "250m" limits: memory: "1Gi" cpu: "500m"
3. Base ConfigMap
# base/configmap.yaml apiVersion: v1 kind: ConfigMap metadata: name: java-app-config data: APPLICATION_NAME: "java-app" LOGGING_LEVEL: "INFO" SERVER_PORT: "8080" SPRING_MAIN_BANNER-MODE: "off" # Database Configuration SPRING_DATASOURCE_URL: "jdbc:postgresql://localhost:5432/mydb" SPRING_DATASOURCE_USERNAME: "app_user" SPRING_JPA_HIBERNATE_DDL-AUTO: "validate" SPRING_JPA_SHOW-SQL: "false" # Redis Configuration SPRING_REDIS_HOST: "localhost" SPRING_REDIS_PORT: "6379" # Feature Flags FEATURE_NEW_SEARCH: "false" FEATURE_PREMIUM_UI: "false" FEATURE_EXPERIMENTAL_API: "false"
4. Base Service
# base/service.yaml apiVersion: v1 kind: Service metadata: name: java-app labels: app: java-app spec: type: ClusterIP ports: - port: 80 targetPort: 8080 protocol: TCP name: http selector: app: java-app
5. Base Horizontal Pod Autoscaler
# base/hpa.yaml apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: java-app spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: java-app minReplicas: 2 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70
Development Overlay
1. Development Kustomization
# overlays/dev/kustomization.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization namespace: java-app-dev resources: - ../../base nameSuffix: -dev commonLabels: environment: dev team: java-development patchesStrategicMerge: - patch-deployment.yaml - configmap-patch.yaml images: - name: my-registry/java-app newTag: "latest" configMapGenerator: - name: java-app-config behavior: merge literals: - SPRING_PROFILES_ACTIVE=dev - LOGGING_LEVEL=DEBUG - FEATURE_NEW_SEARCH=true - FEATURE_EXPERIMENTAL_API=true secretGenerator: - name: java-app-secrets behavior: create literals: - SPRING_DATASOURCE_PASSWORD=dev-password-123 - API_KEY=dev-api-key-456
2. Development Deployment Patch
# overlays/dev/patch-deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: java-app spec: replicas: 1 template: spec: containers: - name: java-app env: - name: JAVA_OPTS value: "-Xmx256m -Xms128m -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005" resources: requests: memory: "256Mi" cpu: "100m" limits: memory: "512Mi" cpu: "250m" livenessProbe: initialDelaySeconds: 120 readinessProbe: initialDelaySeconds: 60 # Development specific - debug port ports: - containerPort: 5005 name: debug
3. Development ConfigMap Patch
# overlays/dev/configmap-patch.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: java-app-config
data:
# Development-specific configurations
SPRING_DATASOURCE_URL: "jdbc:postgresql://dev-db:5432/java_app_dev"
SPRING_JPA_HIBERNATE_DDL-AUTO: "update"
SPRING_JPA_SHOW-SQL: "true"
LOGGING_PATTERN_CONSOLE: "%d{yyyy-MM-dd HH:mm:ss} - %logger{36} - %msg%n"
# Development feature flags
FEATURE_NEW_SEARCH: "true"
FEATURE_PREMIUM_UI: "true"
FEATURE_EXPERIMENTAL_API: "true"
# Development endpoints
MANAGEMENT_ENDPOINTS_WEB_EXPOSURE_INCLUDE: "health,info,metrics,env,loggers"
Staging Overlay
1. Staging Kustomization
# overlays/staging/kustomization.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization namespace: java-app-staging resources: - ../../base nameSuffix: -staging commonLabels: environment: staging team: java-development patchesStrategicMerge: - patch-deployment.yaml - hpa-patch.yaml images: - name: my-registry/java-app newTag: "v1.2.3" # Specific version for staging configMapGenerator: - name: java-app-config behavior: merge literals: - SPRING_PROFILES_ACTIVE=staging - LOGGING_LEVEL=INFO - FEATURE_NEW_SEARCH=true - FEATURE_PREMIUM_UI=false secretGenerator: - name: java-app-secrets behavior: create literals: - SPRING_DATASOURCE_PASSWORD=staging-password-789 - API_KEY=staging-api-key-012
2. Staging Deployment Patch
# overlays/staging/patch-deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: java-app spec: replicas: 2 template: spec: containers: - name: java-app env: - name: JAVA_OPTS value: "-Xmx512m -Xms256m -XX:+UseG1GC" resources: requests: memory: "512Mi" cpu: "250m" limits: memory: "1Gi" cpu: "500m" # Staging specific - more aggressive health checks livenessProbe: initialDelaySeconds: 90 periodSeconds: 15 failureThreshold: 3 readinessProbe: initialDelaySeconds: 45 periodSeconds: 10 failureThreshold: 3
3. Staging HPA Patch
# overlays/staging/hpa-patch.yaml apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: java-app spec: minReplicas: 2 maxReplicas: 5 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 60 - type: Resource resource: name: memory target: type: Utilization averageUtilization: 80
Production Overlay
1. Production Kustomization
# overlays/production/kustomization.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization namespace: java-app-production resources: - ../../base nameSuffix: -prod commonLabels: environment: production team: java-production patchesStrategicMerge: - patch-deployment.yaml - configmap-patch.yaml - hpa-patch.yaml - pdb.yaml # Production-only PodDisruptionBudget images: - name: my-registry/java-app newTag: "v1.2.3" # Specific production version configMapGenerator: - name: java-app-config behavior: merge literals: - SPRING_PROFILES_ACTIVE=production - LOGGING_LEVEL=WARN - FEATURE_NEW_SEARCH=false - FEATURE_PREMIUM_UI=true secretGenerator: - name: java-app-secrets behavior: create literals: - SPRING_DATASOURCE_PASSWORD=prod-secure-password-456 - API_KEY=prod-secure-api-key-789 replicas: - name: java-app count: 3
2. Production Deployment Patch
# overlays/production/patch-deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: java-app spec: replicas: 3 strategy: type: RollingUpdate rollingUpdate: maxSurge: 1 maxUnavailable: 0 template: metadata: annotations: prometheus.io/scrape: "true" prometheus.io/port: "8080" prometheus.io/path: "/actuator/prometheus" spec: containers: - name: java-app env: - name: JAVA_OPTS value: "-Xmx1g -Xms512m -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -Djava.security.egd=file:/dev/./urandom" - name: MANAGEMENT_ENDPOINTS_WEB_EXPOSURE_INCLUDE value: "health,info,metrics,prometheus" resources: requests: memory: "1Gi" cpu: "500m" limits: memory: "2Gi" cpu: "1000m" # Production specific - strict health checks livenessProbe: httpGet: path: /actuator/health/liveness port: 8080 initialDelaySeconds: 120 periodSeconds: 10 timeoutSeconds: 5 failureThreshold: 3 readinessProbe: httpGet: path: /actuator/health/readiness port: 8080 initialDelaySeconds: 60 periodSeconds: 5 timeoutSeconds: 3 failureThreshold: 3 # Production security context securityContext: runAsNonRoot: true runAsUser: 1000 allowPrivilegeEscalation: false capabilities: drop: - ALL
3. Production ConfigMap Patch
# overlays/production/configmap-patch.yaml apiVersion: v1 kind: ConfigMap metadata: name: java-app-config data: # Production-specific configurations SPRING_DATASOURCE_URL: "jdbc:postgresql://prod-cluster-pgbouncer:5432/java_app_prod" SPRING_DATASOURCE_HIKARI_MAXIMUM-POOL-SIZE: "20" SPRING_DATASOURCE_HIKARI_MINIMUM-IDLE: "5" SPRING_JPA_HIBERNATE_DDL-AUTO: "validate" SPRING_JPA_PROPERTIES_HIBERNATE_JDBC_BATCH_SIZE: "50" # Redis Production Config SPRING_REDIS_HOST: "redis-cluster" SPRING_REDIS_TIMEOUT: "2000ms" # Production Logging LOGGING_LEVEL_COM_EXAMPLE: "INFO" LOGGING_LEVEL_ORG_HIBERNATE: "WARN" LOGGING_LEVEL_ORG_APACHE: "WARN" # Production Feature Flags FEATURE_NEW_SEARCH: "false" # Disabled until fully tested FEATURE_PREMIUM_UI: "true" FEATURE_EXPERIMENTAL_API: "false" # Monitoring MANAGEMENT_ENDPOINT_HEALTH_SHOW-DETAILS: "always" MANAGEMENT_ENDPOINT_METRICS_ENABLED: "true"
4. Production HPA Patch
# overlays/production/hpa-patch.yaml apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: java-app spec: minReplicas: 3 maxReplicas: 20 behavior: scaleDown: stabilizationWindowSeconds: 300 policies: - type: Percent value: 50 periodSeconds: 60 scaleUp: stabilizationWindowSeconds: 60 policies: - type: Percent value: 100 periodSeconds: 30 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70 - type: Resource resource: name: memory target: type: Utilization averageUtilization: 85 - type: Pods pods: metric: name: http_requests_per_second target: type: AverageValue averageValue: "100"
5. Production PodDisruptionBudget
# overlays/production/pdb.yaml apiVersion: policy/v1 kind: PodDisruptionBudget metadata: name: java-app spec: minAvailable: 2 selector: matchLabels: app: java-app
Java-Specific Overlay Patterns
1. JVM Tuning Overlays
# overlays/performance/jvm-patch.yaml apiVersion: apps/v1 kind: Deployment metadata: name: java-app spec: template: spec: containers: - name: java-app env: - name: JAVA_OPTS value: >- -Xmx2g -Xms1g -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:G1HeapRegionSize=16m -XX:MetaspaceSize=256m -XX:MaxMetaspaceSize=512m -XX:+ParallelRefProcEnabled -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/heap-dump.hprof -Djava.security.egd=file:/dev/./urandom -Dspring.jmx.enabled=false
2. Spring Profile Overlays
# overlays/spring-profiles/cloud-patch.yaml apiVersion: apps/v1 kind: Deployment metadata: name: java-app spec: template: spec: containers: - name: java-app env: - name: SPRING_PROFILES_ACTIVE value: "cloud,kubernetes,metrics,prometheus" - name: SPRING_CLOUD_KUBERNETES_ENABLED value: "true" - name: SPRING_CLOUD_KUBERNETES_CONFIG_ENABLED value: "true" - name: SPRING_CLOUD_KUBERNETES_RELOAD_ENABLED value: "true"
3. Feature Flag Overlays
# overlays/feature-flags/new-search-patch.yaml apiVersion: v1 kind: ConfigMap metadata: name: java-app-config data: FEATURE_NEW_SEARCH: "true" SEARCH_ALGORITHM_VERSION: "v2" SEARCH_MAX_RESULTS: "100" SEARCH_CACHE_ENABLED: "true"
Build and Deployment Scripts
1. Build Manifest Script
#!/bin/bash
# scripts/generate-manifests.sh
set -e
ENVIRONMENT=${1:-dev}
VERSION=${2:-latest}
echo "Generating manifests for environment: $ENVIRONMENT"
echo "Using image version: $VERSION"
# Create output directory
mkdir -p generated/$ENVIRONMENT
# Generate manifests using kustomize
kustomize build overlays/$ENVIRONMENT \
| sed "s|my-registry/java-app:latest|my-registry/java-app:$VERSION|g" \
> generated/$ENVIRONMENT/manifests.yaml
echo "Manifests generated: generated/$ENVIRONMENT/manifests.yaml"
# Validate manifests
kubeval generated/$ENVIRONMENT/manifests.yaml
# Show differences if previous version exists
if [ -f "generated/$ENVIRONMENT/manifests.yaml.previous" ]; then
diff -u generated/$ENVIRONMENT/manifests.yaml.previous generated/$ENVIRONMENT/manifests.yaml \
|| echo "No significant changes detected"
fi
# Backup current manifests
cp generated/$ENVIRONMENT/manifests.yaml generated/$ENVIRONMENT/manifests.yaml.previous
2. Deployment Script
#!/bin/bash
# scripts/deploy.sh
set -e
ENVIRONMENT=${1:-dev}
ACTION=${2:-apply}
VERSION=${3:-latest}
if [[ ! "$ENVIRONMENT" =~ ^(dev|staging|production)$ ]]; then
echo "Error: Environment must be one of: dev, staging, production"
exit 1
fi
# Generate manifests
./scripts/generate-manifests.sh $ENVIRONMENT $VERSION
# Select kubectl context based on environment
case $ENVIRONMENT in
dev)
KUBE_CONTEXT="gke_my-project_dev"
;;
staging)
KUBE_CONTEXT="gke_my-project_staging"
;;
production)
KUBE_CONTEXT="gke_my-project_production"
;;
esac
echo "Using Kubernetes context: $KUBE_CONTEXT"
# Apply manifests
kubectl --context=$KUBE_CONTEXT $ACTION -f generated/$ENVIRONMENT/manifests.yaml
if [ "$ACTION" == "apply" ]; then
# Wait for deployment to complete
kubectl --context=$KUBE_CONTEXT rollout status deployment/java-app-$ENVIRONMENT \
--timeout=600s
# Verify services
kubectl --context=$KUBE_CONTEXT get pods -l app=java-app,environment=$ENVIRONMENT
echo "Deployment to $ENVIRONMENT completed successfully!"
fi
3. CI/CD Integration Script
#!/bin/bash
# scripts/ci-deploy.sh
set -e
BRANCH=${GITHUB_REF#refs/heads/}
COMMIT_SHA=${GITHUB_SHA:0:8}
IMAGE_TAG="${COMMIT_SHA}"
echo "Branch: $BRANCH"
echo "Commit SHA: $COMMIT_SHA"
case $BRANCH in
develop)
ENVIRONMENT="dev"
;;
release/*)
ENVIRONMENT="staging"
;;
main)
ENVIRONMENT="production"
;;
*)
echo "Skipping deployment for branch: $BRANCH"
exit 0
;;
esac
echo "Deploying to environment: $ENVIRONMENT"
# Build and push Docker image
docker build -t my-registry/java-app:$IMAGE_TAG .
docker push my-registry/java-app:$IMAGE_TAG
# Deploy using kustomize
./scripts/deploy.sh $ENVIRONMENT apply $IMAGE_TAG
# Run smoke tests
./scripts/smoke-test.sh $ENVIRONMENT
Advanced Overlay Patterns
1. Canary Deployment Overlay
# overlays/canary/kustomization.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - ../../base namePrefix: canary- commonLabels: deployment-type: canary patchesStrategicMerge: - patch-deployment.yaml replicas: - name: java-app count: 1 # Only 1 pod for canary images: - name: my-registry/java-app newTag: "v1.3.0-canary" # Canary version
2. Blue-Green Deployment Overlay
# overlays/blue-green/green/kustomization.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization resources: - ../../../base nameSuffix: -green commonLabels: deployment: green patchesStrategicMerge: - patch-deployment.yaml - patch-service.yaml
# overlays/blue-green/green/patch-service.yaml apiVersion: v1 kind: Service metadata: name: java-app spec: selector: app: java-app deployment: green # Route traffic to green deployment
Validation and Testing
1. Kustomize Validation Script
#!/bin/bash # scripts/validate-overlays.sh set -e echo "Validating all Kustomize overlays..." for overlay in overlays/*/; do if [ -f "$overlay/kustomization.yaml" ]; then echo "Validating $overlay" kustomize build "$overlay" > /dev/null kubeval <(kustomize build "$overlay") --strict fi done echo "All overlays validated successfully!"
2. Diff Between Environments
#!/bin/bash
# scripts/environment-diff.sh
ENV1=${1:-dev}
ENV2=${2:-staging}
echo "Comparing $ENV1 vs $ENV2"
kustomize build overlays/$ENV1 > /tmp/env1.yaml
kustomize build overlays/$ENV2 > /tmp/env2.yaml
diff -u /tmp/env1.yaml /tmp/env2.yaml | head -50
Best Practices
1. Structure Organization
- Keep base configurations minimal and generic
- Use overlays for environment-specific changes
- Separate concerns: config, resources, features
- Version control all overlay configurations
2. Security Practices
- Use secret generators for sensitive data
- Implement security contexts in production
- Regular security scanning of generated manifests
- RBAC configurations in appropriate overlays
3. Monitoring and Observability
- Include monitoring annotations in production
- Configure proper resource limits
- Implement comprehensive health checks
- Add logging and metrics configurations
Conclusion
Kustomize overlays provide Java teams with a powerful, maintainable approach to Kubernetes configuration management:
- Environment Consistency: Consistent base with environment-specific customizations
- Reduced Duplication: Eliminate copy-paste configuration anti-patterns
- Safe Deployments: Gradual rollout strategies with canary/blue-green
- Developer Productivity: Easy to understand and modify configurations
- GitOps Friendly: Perfect for GitOps workflows and automated deployments
By implementing Kustomize overlays, Java teams can efficiently manage complex multi-environment deployments while maintaining configuration integrity and deployment safety.
Pyroscope Profiling in Java
Explains how to use Pyroscope for continuous profiling in Java applications, helping developers analyze CPU and memory usage patterns to improve performance and identify bottlenecks.
https://macronepal.com/blog/pyroscope-profiling-in-java/
OpenTelemetry Metrics in Java: Comprehensive Guide
Provides a complete guide to collecting and exporting metrics in Java using OpenTelemetry, including counters, histograms, gauges, and integration with monitoring tools. (MACRO NEPAL)
https://macronepal.com/blog/opentelemetry-metrics-in-java-comprehensive-guide/
OTLP Exporter in Java: Complete Guide for OpenTelemetry
Explains how to configure OTLP exporters in Java to send telemetry data such as traces, metrics, and logs to monitoring systems using HTTP or gRPC protocols. (MACRO NEPAL)
https://macronepal.com/blog/otlp-exporter-in-java-complete-guide-for-opentelemetry/
Thanos Integration in Java: Global View of Metrics
Explains how to integrate Thanos with Java monitoring systems to create a scalable global metrics view across multiple Prometheus instances.
https://macronepal.com/blog/thanos-integration-in-java-global-view-of-metrics
Time Series with InfluxDB in Java: Complete Guide (Version 2)
Explains how to manage time-series data using InfluxDB in Java applications, including storing, querying, and analyzing metrics data.
https://macronepal.com/blog/time-series-with-influxdb-in-java-complete-guide-2
Time Series with InfluxDB in Java: Complete Guide
Provides an overview of integrating InfluxDB with Java for time-series data handling, including monitoring applications and managing performance metrics.
https://macronepal.com/blog/time-series-with-influxdb-in-java-complete-guide
Implementing Prometheus Remote Write in Java (Version 2)
Explains how to configure Java applications to send metrics data to Prometheus-compatible systems using the remote write feature for scalable monitoring.
https://macronepal.com/blog/implementing-prometheus-remote-write-in-java-a-complete-guide-2
Implementing Prometheus Remote Write in Java: Complete Guide
Provides instructions for sending metrics from Java services to Prometheus servers, enabling centralized monitoring and real-time analytics.
https://macronepal.com/blog/implementing-prometheus-remote-write-in-java-a-complete-guide
Building a TileServer GL in Java: Vector and Raster Tile Server
Explains how to build a TileServer GL in Java for serving vector and raster map tiles, useful for geographic visualization and mapping applications.
https://macronepal.com/blog/building-a-tileserver-gl-in-java-vector-and-raster-tile-server
Indoor Mapping in Java
Explains how to create indoor mapping systems in Java, including navigation inside buildings, spatial data handling, and visualization techniques.