Article
Dark Launching is a sophisticated deployment strategy where new features are deployed to production but remain hidden from users or enabled for only a small percentage of traffic. Unlike feature flags that typically show/hide user-facing functionality, dark launches focus on testing infrastructure, performance, and reliability without impacting the user experience.
In this guide, we'll explore how to implement dark launch strategies in Java applications to safely test new features, APIs, and infrastructure changes.
Why Implement Dark Launch Strategy?
- Performance Testing: Validate new services under real production load
- Infrastructure Validation: Test new database queries, external API calls, or microservices
- A/B Testing Backend: Compare new algorithm performance against existing implementations
- Safe Rollouts: Gradually increase traffic to new systems while monitoring metrics
- Zero User Impact: Test thoroughly without affecting customer experience
- Data Collection: Gather real-world usage data before full launch
Part 1: Core Dark Launch Architecture
1.1 Dark Launch Components
// File: src/main/java/com/darklaunch/core/DarkLaunchConfig.java
public record DarkLaunchConfig(
String featureName,
boolean enabled,
double trafficPercentage,
Map<String, Object> parameters,
Set<String> includedUsers,
Set<String> excludedUsers,
String rolloutStrategy
) {}
// File: src/main/java/com/darklaunch/core/DarkLaunchContext.java
public class DarkLaunchContext {
private final String userId;
private final String sessionId;
private final String requestPath;
private final Map<String, String> attributes;
private final Instant timestamp;
public DarkLaunchContext(Builder builder) {
this.userId = builder.userId;
this.sessionId = builder.sessionId;
this.requestPath = builder.requestPath;
this.attributes = Map.copyOf(builder.attributes);
this.timestamp = builder.timestamp;
}
public static class Builder {
private String userId;
private String sessionId;
private String requestPath;
private final Map<String, String> attributes = new HashMap<>();
private Instant timestamp = Instant.now();
public Builder userId(String userId) {
this.userId = userId;
return this;
}
public Builder sessionId(String sessionId) {
this.sessionId = sessionId;
return this;
}
public Builder requestPath(String requestPath) {
this.requestPath = requestPath;
return this;
}
public Builder attribute(String key, String value) {
this.attributes.put(key, value);
return this;
}
public Builder attributes(Map<String, String> attributes) {
this.attributes.putAll(attributes);
return this;
}
public Builder timestamp(Instant timestamp) {
this.timestamp = timestamp;
return this;
}
public DarkLaunchContext build() {
return new DarkLaunchContext(this);
}
}
// Getters...
public String getUserId() { return userId; }
public String getSessionId() { return sessionId; }
public String getRequestPath() { return requestPath; }
public Map<String, String> getAttributes() { return attributes; }
public Instant getTimestamp() { return timestamp; }
}
1.2 Dark Launch Manager
// File: src/main/java/com/darklaunch/core/DarkLaunchManager.java
@Component
public class DarkLaunchManager {
private final DarkLaunchConfigProvider configProvider;
private final DarkLaunchMetrics metrics;
private final Random random;
public DarkLaunchManager(DarkLaunchConfigProvider configProvider,
DarkLaunchMetrics metrics) {
this.configProvider = configProvider;
this.metrics = metrics;
this.random = new Random();
}
public boolean isEnabled(String featureName, DarkLaunchContext context) {
DarkLaunchConfig config = configProvider.getConfig(featureName);
if (config == null || !config.enabled()) {
return false;
}
// Check user inclusions/exclusions
if (context.getUserId() != null) {
if (config.excludedUsers().contains(context.getUserId())) {
return false;
}
if (config.includedUsers().contains(context.getUserId())) {
return true;
}
}
// Percentage-based rollout
return isInTrafficPercentage(context, config.trafficPercentage());
}
public <T> DarkLaunchResult<T> execute(String featureName,
DarkLaunchContext context,
Supplier<T> newImplementation,
Supplier<T> oldImplementation) {
boolean useNewImplementation = isEnabled(featureName, context);
Instant startTime = Instant.now();
try {
T result = useNewImplementation ?
newImplementation.get() : oldImplementation.get();
metrics.recordExecution(featureName, useNewImplementation,
Duration.between(startTime, Instant.now()), true);
return new DarkLaunchResult<>(result, useNewImplementation, true);
} catch (Exception e) {
metrics.recordExecution(featureName, useNewImplementation,
Duration.between(startTime, Instant.now()), false);
// Fallback to old implementation on failure
if (useNewImplementation) {
T fallbackResult = oldImplementation.get();
return new DarkLaunchResult<>(fallbackResult, false, true);
}
throw e;
}
}
private boolean isInTrafficPercentage(DarkLaunchContext context, double percentage) {
String trafficKey = context.getUserId() != null ?
context.getUserId() : context.getSessionId();
if (trafficKey == null) {
trafficKey = context.getRequestPath() + ":" + System.currentTimeMillis() / 1000;
}
int hash = Math.abs(trafficKey.hashCode());
double normalized = (hash % 10000) / 10000.0;
return normalized < percentage;
}
}
// File: src/main/java/com/darklaunch/core/DarkLaunchResult.java
public record DarkLaunchResult<T>(
T result,
boolean usedNewImplementation,
boolean success
) {}
Part 2: Implementation Patterns
2.1 Database Query Dark Launch
// File: src/main/java/com/darklaunch/pattern/DatabaseQueryDarkLaunch.java
@Service
public class DatabaseQueryDarkLaunch {
private final DarkLaunchManager darkLaunch;
private final UserRepository userRepository;
private final MetricsService metrics;
public DatabaseQueryDarkLaunch(DarkLaunchManager darkLaunch,
UserRepository userRepository,
MetricsService metrics) {
this.darkLaunch = darkLaunch;
this.userRepository = userRepository;
this.metrics = metrics;
}
public List<User> findActiveUsers(Date startDate, Date endDate, DarkLaunchContext context) {
return darkLaunch.execute(
"new-active-users-query",
context,
// New implementation (being tested)
() -> {
Instant start = Instant.now();
List<User> users = userRepository.findActiveUsersNewQuery(startDate, endDate);
Duration duration = Duration.between(start, Instant.now());
metrics.recordQueryPerformance("new-active-users-query", duration, users.size());
return users;
},
// Old implementation (current production)
() -> userRepository.findActiveUsersOldQuery(startDate, endDate)
).result();
}
public UserStatistics calculateUserStats(String region, DarkLaunchContext context) {
return darkLaunch.execute(
"new-user-stats-calculation",
context,
// New aggregation logic
() -> {
// New, more complex calculation being tested
UserStatsNewCalculator calculator = new UserStatsNewCalculator();
return calculator.calculateRegionalStats(region);
},
// Current production logic
() -> {
UserStatsOldCalculator calculator = new UserStatsOldCalculator();
return calculator.calculateRegionalStats(region);
}
).result();
}
}
2.2 External API Call Dark Launch
// File: src/main/java/com/darklaunch/pattern/ExternalApiDarkLaunch.java
@Service
public class ExternalApiDarkLaunch {
private final DarkLaunchManager darkLaunch;
private final PaymentService oldPaymentService;
private final PaymentService newPaymentService;
private final CircuitBreakerRegistry circuitBreakerRegistry;
public ExternalApiDarkLaunch(DarkLaunchManager darkLaunch,
@Qualifier("oldPaymentService") PaymentService oldPaymentService,
@Qualifier("newPaymentService") PaymentService newPaymentService,
CircuitBreakerRegistry circuitBreakerRegistry) {
this.darkLaunch = darkLaunch;
this.oldPaymentService = oldPaymentService;
this.newPaymentService = newPaymentService;
this.circuitBreakerRegistry = circuitBreakerRegistry;
}
public PaymentResult processPayment(PaymentRequest request, DarkLaunchContext context) {
return darkLaunch.execute(
"new-payment-service",
context,
// New payment service implementation
() -> {
CircuitBreaker circuitBreaker = circuitBreakerRegistry
.circuitBreaker("new-payment-service");
return circuitBreaker.executeSupplier(() ->
newPaymentService.processPayment(request));
},
// Current payment service
() -> oldPaymentService.processPayment(request)
).result();
}
public List<Recommendation> getRecommendations(String userId, DarkLaunchContext context) {
return darkLaunch.execute(
"new-recommendation-service",
context,
// New recommendation algorithm
() -> {
// New machine learning model
RecommendationService newService = new NewRecommendationService();
List<Recommendation> recommendations = newService.getRecommendations(userId);
// Log for comparison analysis
logRecommendationResults(userId, recommendations, "new");
return recommendations;
},
// Current recommendation algorithm
() -> {
RecommendationService oldService = new OldRecommendationService();
List<Recommendation> recommendations = oldService.getRecommendations(userId);
logRecommendationResults(userId, recommendations, "old");
return recommendations;
}
).result();
}
private void logRecommendationResults(String userId, List<Recommendation> recommendations, String version) {
// Log to analytics system for comparison
Map<String, Object> logData = Map.of(
"userId", userId,
"version", version,
"recommendationCount", recommendations.size(),
"recommendationIds", recommendations.stream()
.map(Recommendation::getId)
.collect(Collectors.toList()),
"timestamp", Instant.now()
);
// Send to analytics or log aggregation system
analyticsService.track("recommendations.generated", logData);
}
}
2.3 Cache Strategy Dark Launch
// File: src/main/java/com/darklaunch/pattern/CacheDarkLaunch.java
@Service
public class CacheDarkLaunch {
private final DarkLaunchManager darkLaunch;
private final CacheManager cacheManager;
private final MetricsService metrics;
public CacheDarkLaunch(DarkLaunchManager darkLaunch,
CacheManager cacheManager,
MetricsService metrics) {
this.darkLaunch = darkLaunch;
this.cacheManager = cacheManager;
this.metrics = metrics;
}
public <T> T getWithNewCacheStrategy(String key, Class<T> type,
Supplier<T> valueLoader, DarkLaunchContext context) {
return darkLaunch.execute(
"new-cache-strategy",
context,
// New cache implementation (Redis, Caffeine, etc.)
() -> {
Instant start = Instant.now();
T value = getFromNewCache(key, type, valueLoader);
Duration duration = Duration.between(start, Instant.now());
metrics.recordCachePerformance("new-cache", key, duration, true);
return value;
},
// Old cache implementation
() -> {
Instant start = Instant.now();
T value = getFromOldCache(key, type, valueLoader);
Duration duration = Duration.between(start, Instant.now());
metrics.recordCachePerformance("old-cache", key, duration, true);
return value;
}
).result();
}
private <T> T getFromNewCache(String key, Class<T> type, Supplier<T> valueLoader) {
// New cache implementation (e.g., Redis with different serialization)
Cache newCache = cacheManager.getCache("new-cache");
T value = newCache.get(key, type);
if (value == null) {
value = valueLoader.get();
newCache.put(key, value);
}
return value;
}
private <T> T getFromOldCache(String key, Class<T> type, Supplier<T> valueLoader) {
// Current cache implementation
Cache oldCache = cacheManager.getCache("old-cache");
return oldCache.get(key, type);
}
}
Part 3: Configuration Management
3.1 Dynamic Configuration Provider
// File: src/main/java/com/darklaunch/config/DarkLaunchConfigProvider.java
@Component
public class DynamicDarkLaunchConfigProvider implements DarkLaunchConfigProvider {
private final Map<String, DarkLaunchConfig> configs = new ConcurrentHashMap<>();
private final ConfigRefresher configRefresher;
public DynamicDarkLaunchConfigProvider(ConfigRefresher configRefresher) {
this.configRefresher = configRefresher;
loadInitialConfigs();
scheduleConfigRefresh();
}
@Override
public DarkLaunchConfig getConfig(String featureName) {
return configs.get(featureName);
}
@Override
public void updateConfig(String featureName, DarkLaunchConfig config) {
configs.put(featureName, config);
// Notify listeners
configRefresher.notifyConfigChanged(featureName);
}
private void loadInitialConfigs() {
// Load from external config (file, database, feature flag service)
configs.putAll(Map.of(
"new-payment-service", new DarkLaunchConfig(
"new-payment-service",
true, // enabled
0.1, // 10% traffic
Map.of("timeoutMs", "5000", "retryCount", "3"),
Set.of("test-user-1", "test-user-2"), // included users
Set.of(), // excluded users
"percentage"
),
"new-active-users-query", new DarkLaunchConfig(
"new-active-users-query",
true,
0.05, // 5% traffic
Map.of("queryTimeout", "30s"),
Set.of(),
Set.of(),
"percentage"
)
));
}
private void scheduleConfigRefresh() {
ScheduledExecutorService scheduler = Executors.newSingleThreadScheduledExecutor();
scheduler.scheduleAtFixedRate(this::refreshConfigs, 5, 5, TimeUnit.MINUTES);
}
private void refreshConfigs() {
// Refresh from external source
try {
// Could fetch from Unleash, LaunchDarkly, or config server
refreshFromExternalSource();
} catch (Exception e) {
log.error("Failed to refresh dark launch configs", e);
}
}
}
3.2 YAML Configuration
# src/main/resources/dark-launch-config.yaml dark-launch: features: new-payment-service: enabled: true traffic-percentage: 0.1 strategy: percentage parameters: timeout-ms: 5000 retry-count: 3 included-users: - "test-user-1" - "test-user-2" excluded-users: [] new-recommendation-service: enabled: true traffic-percentage: 0.2 strategy: percentage parameters: model-version: "v2.1" max-results: 20 included-users: [] excluded-users: [] new-cache-strategy: enabled: false # Not yet enabled traffic-percentage: 0.0 strategy: percentage parameters: cache-ttl: "3600s"
Part 4: Monitoring and Metrics
4.1 Comprehensive Metrics Collection
// File: src/main/java/com/darklaunch/metrics/DarkLaunchMetrics.java
@Component
public class DarkLaunchMetrics {
private final MeterRegistry meterRegistry;
private final Map<String, Counter> executionCounters;
private final Map<String, Timer> executionTimers;
public DarkLaunchMetrics(MeterRegistry meterRegistry) {
this.meterRegistry = meterRegistry;
this.executionCounters = new ConcurrentHashMap<>();
this.executionTimers = new ConcurrentHashMap<>();
}
public void recordExecution(String featureName, boolean usedNewImplementation,
Duration duration, boolean success) {
// Count executions
getCounter(featureName, usedNewImplementation, success).increment();
// Record timing
getTimer(featureName, usedNewImplementation).record(duration);
// Record percentage gauge
recordTrafficPercentage(featureName, usedNewImplementation);
}
public void recordComparisonResult(String featureName, boolean newImplementationBetter) {
Counter.builder("darklaunch.comparison.result")
.tag("feature", featureName)
.tag("result", newImplementationBetter ? "better" : "worse")
.register(meterRegistry)
.increment();
}
private Counter getCounter(String featureName, boolean newImpl, boolean success) {
String key = featureName + ":" + newImpl + ":" + success;
return executionCounters.computeIfAbsent(key, k ->
Counter.builder("darklaunch.execution.count")
.tag("feature", featureName)
.tag("implementation", newImpl ? "new" : "old")
.tag("success", String.valueOf(success))
.register(meterRegistry)
);
}
private Timer getTimer(String featureName, boolean newImpl) {
String key = featureName + ":" + newImpl;
return executionTimers.computeIfAbsent(key, k ->
Timer.builder("darklaunch.execution.duration")
.tag("feature", featureName)
.tag("implementation", newImpl ? "new" : "old")
.register(meterRegistry)
);
}
private void recordTrafficPercentage(String featureName, boolean newImpl) {
// This would typically be calculated from counters
Gauge.builder("darklaunch.traffic.percentage")
.tag("feature", featureName)
.register(meterRegistry, this,
metrics -> calculateCurrentPercentage(featureName));
}
private double calculateCurrentPercentage(String featureName) {
// Implementation to calculate current traffic percentage
return 0.0;
}
}
4.2 Performance Comparison Service
// File: src/main/java/com/darklaunch/analysis/PerformanceComparator.java
@Service
public class PerformanceComparator {
private final DarkLaunchMetrics metrics;
private final StatisticalAnalysisService statsService;
public PerformanceComparator(DarkLaunchMetrics metrics,
StatisticalAnalysisService statsService) {
this.metrics = metrics;
this.statsService = statsService;
}
public void analyzeFeaturePerformance(String featureName) {
Map<String, Double> oldImplMetrics = getPerformanceMetrics(featureName, false);
Map<String, Double> newImplMetrics = getPerformanceMetrics(featureName, true);
boolean isNewImplementationBetter = statsService.isStatisticallyBetter(
newImplMetrics, oldImplMetrics);
metrics.recordComparisonResult(featureName, isNewImplementationBetter);
if (isNewImplementationBetter) {
log.info("New implementation for {} shows statistically significant improvement",
featureName);
// Potentially auto-increase traffic percentage
} else {
log.warn("New implementation for {} does not show improvement", featureName);
// Consider rolling back or investigating
}
}
private Map<String, Double> getPerformanceMetrics(String featureName, boolean newImpl) {
// Collect various performance metrics
return Map.of(
"p50_response_time", getPercentileResponseTime(featureName, newImpl, 50),
"p95_response_time", getPercentileResponseTime(featureName, newImpl, 95),
"error_rate", getErrorRate(featureName, newImpl),
"throughput", getThroughput(featureName, newImpl)
);
}
}
Part 5: Testing Dark Launch Implementations
5.1 Unit Testing
// File: src/test/java/com/darklaunch/DarkLaunchManagerTest.java
@ExtendWith(MockitoExtension.class)
class DarkLaunchManagerTest {
@Mock
private DarkLaunchConfigProvider configProvider;
@Mock
private DarkLaunchMetrics metrics;
private DarkLaunchManager darkLaunchManager;
@BeforeEach
void setUp() {
darkLaunchManager = new DarkLaunchManager(configProvider, metrics);
}
@Test
void shouldUseNewImplementationWhenFeatureEnabled() {
// Given
String featureName = "test-feature";
DarkLaunchContext context = new DarkLaunchContext.Builder()
.userId("user-123")
.build();
DarkLaunchConfig config = new DarkLaunchConfig(
featureName, true, 1.0, Map.of(), Set.of(), Set.of(), "percentage");
when(configProvider.getConfig(featureName)).thenReturn(config);
Supplier<String> newImpl = () -> "new-result";
Supplier<String> oldImpl = () -> "old-result";
// When
DarkLaunchResult<String> result = darkLaunchManager.execute(
featureName, context, newImpl, oldImpl);
// Then
assertThat(result.usedNewImplementation()).isTrue();
assertThat(result.result()).isEqualTo("new-result");
}
@Test
void shouldFallbackToOldImplementationOnFailure() {
// Given
String featureName = "test-feature";
DarkLaunchContext context = new DarkLaunchContext.Builder().build();
DarkLaunchConfig config = new DarkLaunchConfig(
featureName, true, 1.0, Map.of(), Set.of(), Set.of(), "percentage");
when(configProvider.getConfig(featureName)).thenReturn(config);
Supplier<String> newImpl = () -> { throw new RuntimeException("Failure"); };
Supplier<String> oldImpl = () -> "old-result";
// When
DarkLaunchResult<String> result = darkLaunchManager.execute(
featureName, context, newImpl, oldImpl);
// Then
assertThat(result.usedNewImplementation()).isFalse();
assertThat(result.result()).isEqualTo("old-result");
assertThat(result.success()).isTrue();
}
}
5.2 Integration Testing
// File: src/test/java/com/darklaunch/DarkLaunchIntegrationTest.java
@SpringBootTest
@TestPropertySource(properties = {
"darklaunch.enabled=true"
})
class DarkLaunchIntegrationTest {
@Autowired
private DarkLaunchManager darkLaunchManager;
@Autowired
private DatabaseQueryDarkLaunch databaseDarkLaunch;
@Test
void shouldDarkLaunchDatabaseQuery() {
// Given
DarkLaunchContext context = new DarkLaunchContext.Builder()
.userId("test-user")
.requestPath("/api/users")
.attribute("userAgent", "test-runner")
.build();
Date startDate = Date.from(Instant.now().minus(7, ChronoUnit.DAYS));
Date endDate = Date.from(Instant.now());
// When
List<User> users = databaseDarkLaunch.findActiveUsers(startDate, endDate, context);
// Then
assertThat(users).isNotNull();
// Verify that metrics were recorded appropriately
}
@Test
void shouldRespectTrafficPercentage() {
// Test that traffic distribution works correctly
DarkLaunchContext context = new DarkLaunchContext.Builder()
.userId("consistent-user-123")
.build();
int newImplCount = 0;
int iterations = 1000;
for (int i = 0; i < iterations; i++) {
DarkLaunchContext iterContext = new DarkLaunchContext.Builder()
.userId("user-" + i)
.build();
boolean enabled = darkLaunchManager.isEnabled("test-feature", iterContext);
if (enabled) newImplCount++;
}
// Should be approximately 10% with some tolerance
double percentage = (double) newImplCount / iterations;
assertThat(percentage).isBetween(0.08, 0.12);
}
}
Best Practices for Dark Launch in Java
- Start Small: Begin with 1-5% traffic and gradually increase
- Monitor Aggressively: Implement comprehensive metrics and alerts
- Plan Rollback: Always have a strategy to quickly disable dark launched features
- Test Fallbacks: Ensure old implementations remain reliable
- Use Circuit Breakers: Protect systems from cascading failures
- Correlation IDs: Use tracing to follow requests through both implementations
- Data Consistency: Ensure both implementations produce semantically equivalent results
- Clean Up: Remove dark launch code once the feature is fully validated and released
Conclusion
Dark launching in Java provides a powerful mechanism for safely testing new functionality, infrastructure changes, and performance optimizations in production environments. By implementing the patterns and strategies outlined in this guide, you can:
- Reduce deployment risk by gradually exposing new code to real traffic
- Gather performance data from production under actual load conditions
- Compare implementations statistically before making permanent switches
- Maintain system stability with automatic fallbacks and circuit breakers
The key to successful dark launching is comprehensive monitoring, gradual rollout, and having clear criteria for success. When implemented properly, dark launches can significantly improve your deployment confidence and system reliability.