Synthetic Monitoring in Java: Complete Implementation Guide

Introduction

Synthetic monitoring involves simulating user interactions and transactions to proactively monitor application health, performance, and functionality. Unlike real-user monitoring, synthetic tests run on a schedule from various locations to detect issues before users are affected.

This guide covers building a comprehensive synthetic monitoring framework in Java for web applications, APIs, and business transactions.

Architecture Overview

[Synthetic Tests] → [Test Executors] → [Applications] → [Metrics Collector] → [Dashboard/Alerts] ↓ ↓ ↓ ↓ ↓ HTTP Checks Browser Microservices Metrics Storage PagerDuty API Validation Automation Databases Log Aggregator Slack/Email Business Flows Performance External APIs Time Series DB Kibana

Step 1: Project Dependencies

<properties> <spring-boot.version>3.2.0</spring-boot.version> <selenium.version>4.15.0</selenium.version> <micrometer.version>1.12.0</micrometer.version> </properties> <dependencies> <!-- Spring Boot --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> <version>${spring-boot.version}</version> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-validation</artifactId> <version>${spring-boot.version}</version> </dependency> <!-- Scheduling --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-quartz</artifactId> <version>${spring-boot.version}</version> </dependency> <!-- HTTP Client --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-webflux</artifactId> <version>${spring-boot.version}</version> </dependency> <!-- Selenium for Browser Automation --> <dependency> <groupId>org.seleniumhq.selenium</groupId> <artifactId>selenium-java</artifactId> <version>${selenium.version}</version> </dependency> <dependency> <groupId>org.seleniumhq.selenium</groupId> <artifactId>selenium-chrome-driver</artifactId> <version>${selenium.version}</version> </dependency> <!-- Metrics --> <dependency> <groupId>io.micrometer</groupId> <artifactId>micrometer-core</artifactId> <version>${micrometer.version}</version> </dependency> <dependency> <groupId>io.micrometer</groupId> <artifactId>micrometer-registry-prometheus</artifactId> <version>${micrometer.version}</version> </dependency> <!-- JSON Processing --> <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-databind</artifactId> <version>2.15.0</version> </dependency> <!-- Database --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-jpa</artifactId> <version>${spring-boot.version}</version> </dependency> <dependency> <groupId>com.h2database</groupId> <artifactId>h2</artifactId> <scope>runtime</scope> </dependency> <!-- Testing --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <version>${spring-boot.version}</version> <scope>test</scope> </dependency> </dependencies>

Step 2: Configuration

application.yml

synthetic: monitoring: enabled: true execution: thread-pool-size: 10 timeout-seconds: 300 retry-attempts: 2 retry-delay-ms: 5000 # Browser automation browser: headless: true window-width: 1920 window-height: 1080 implicit-wait-seconds: 10 page-load-timeout-seconds: 30 # Notifications alerts: slack-webhook-url: ${SLACK_WEBHOOK_URL:} pagerduty-routing-key: ${PAGERDUTY_ROUTING_KEY:} email-recipients: ${ALERT_EMAILS:} # Storage storage: retain-days: 30 cleanup-enabled: true # Metrics management: endpoints: web: exposure: include: health,metrics,prometheus endpoint: health: show-details: always metrics: enabled: true # Logging logging: level: com.example.synthetic: DEBUG pattern: level: "%5p [%X{traceId:-},%X{testId:-}]" # Scheduled tasks spring: quartz: job-store-type: jdbc jdbc: initialize-schema: always datasource: url: jdbc:h2:file:./data/synthetic-monitoring driverClassName: org.h2.Driver username: sa password: "" jpa: database-platform: org.hibernate.dialect.H2Dialect hibernate: ddl-auto: update show-sql: false

Step 3: Core Data Models

Test Definition Models

MonitorTest.java

package com.example.synthetic.model; import com.fasterxml.jackson.annotation.JsonFormat; import com.fasterxml.jackson.annotation.JsonSubTypes; import com.fasterxml.jackson.annotation.JsonTypeInfo; import jakarta.persistence.*; import jakarta.validation.constraints.NotBlank; import jakarta.validation.constraints.NotNull; import java.time.LocalDateTime; import java.util.Map; @Entity @Table(name = "monitor_tests") @Inheritance(strategy = InheritanceType.SINGLE_TABLE) @DiscriminatorColumn(name = "test_type") @JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "type") @JsonSubTypes({ @JsonSubTypes.Type(value = HttpTest.class, name = "HTTP"), @JsonSubTypes.Type(value = BrowserTest.class, name = "BROWSER"), @JsonSubTypes.Type(value = ApiValidationTest.class, name = "API_VALIDATION"), @JsonSubTypes.Type(value = BusinessFlowTest.class, name = "BUSINESS_FLOW") }) public abstract class MonitorTest { @Id @GeneratedValue(strategy = GenerationType.IDENTITY) private Long id; @NotBlank @Column(nullable = false) private String name; private String description; @NotBlank @Column(nullable = false) private String targetUrl; @NotNull @Enumerated(EnumType.STRING) @Column(nullable = false) private TestFrequency frequency; @NotNull @Enumerated(EnumType.STRING) @Column(nullable = false) private TestPriority priority; private boolean enabled = true; private int timeoutSeconds = 300; private int retryAttempts = 2; @ElementCollection @CollectionTable(name = "test_labels", joinColumns = @JoinColumn(name = "test_id")) @MapKeyColumn(name = "label_key") @Column(name = "label_value") private Map<String, String> labels; @JsonFormat(pattern = "yyyy-MM-dd HH:mm:ss") private LocalDateTime createdAt; @JsonFormat(pattern = "yyyy-MM-dd HH:mm:ss") private LocalDateTime updatedAt; // Pre-persist and pre-update methods @PrePersist protected void onCreate() { createdAt = LocalDateTime.now(); updatedAt = LocalDateTime.now(); } @PreUpdate protected void onUpdate() { updatedAt = LocalDateTime.now(); } // Getters and Setters public Long getId() { return id; } public void setId(Long id) { this.id = id; } public String getName() { return name; } public void setName(String name) { this.name = name; } public String getDescription() { return description; } public void setDescription(String description) { this.description = description; } public String getTargetUrl() { return targetUrl; } public void setTargetUrl(String targetUrl) { this.targetUrl = targetUrl; } public TestFrequency getFrequency() { return frequency; } public void setFrequency(TestFrequency frequency) { this.frequency = frequency; } public TestPriority getPriority() { return priority; } public void setPriority(TestPriority priority) { this.priority = priority; } public boolean isEnabled() { return enabled; } public void setEnabled(boolean enabled) { this.enabled = enabled; } public int getTimeoutSeconds() { return timeoutSeconds; } public void setTimeoutSeconds(int timeoutSeconds) { this.timeoutSeconds = timeoutSeconds; } public int getRetryAttempts() { return retryAttempts; } public void setRetryAttempts(int retryAttempts) { this.retryAttempts = retryAttempts; } public Map<String, String> getLabels() { return labels; } public void setLabels(Map<String, String> labels) { this.labels = labels; } public LocalDateTime getCreatedAt() { return createdAt; } public LocalDateTime getUpdatedAt() { return updatedAt; } } @Entity @DiscriminatorValue("HTTP") class HttpTest extends MonitorTest { private HttpMethod method = HttpMethod.GET; @Column(length = 2000) private String requestBody; @ElementCollection @CollectionTable(name = "http_headers", joinColumns = @JoinColumn(name = "test_id")) @MapKeyColumn(name = "header_name") @Column(name = "header_value") private Map<String, String> headers; private Integer expectedStatusCode = 200; @Column(length = 1000) private String expectedResponsePattern; private Long maxResponseTimeMs = 5000L; // Getters and Setters public HttpMethod getMethod() { return method; } public void setMethod(HttpMethod method) { this.method = method; } public String getRequestBody() { return requestBody; } public void setRequestBody(String requestBody) { this.requestBody = requestBody; } public Map<String, String> getHeaders() { return headers; } public void setHeaders(Map<String, String> headers) { this.headers = headers; } public Integer getExpectedStatusCode() { return expectedStatusCode; } public void setExpectedStatusCode(Integer expectedStatusCode) { this.expectedStatusCode = expectedStatusCode; } public String getExpectedResponsePattern() { return expectedResponsePattern; } public void setExpectedResponsePattern(String expectedResponsePattern) { this.expectedResponsePattern = expectedResponsePattern; } public Long getMaxResponseTimeMs() { return maxResponseTimeMs; } public void setMaxResponseTimeMs(Long maxResponseTimeMs) { this.maxResponseTimeMs = maxResponseTimeMs; } } @Entity @DiscriminatorValue("BROWSER") class BrowserTest extends MonitorTest { @Column(length = 2000) private String script; private String browserType = "CHROME"; private boolean takeScreenshotOnFailure = true; private Long maxLoadTimeMs = 10000L; // Getters and Setters public String getScript() { return script; } public void setScript(String script) { this.script = script; } public String getBrowserType() { return browserType; } public void setBrowserType(String browserType) { this.browserType = browserType; } public boolean isTakeScreenshotOnFailure() { return takeScreenshotOnFailure; } public void setTakeScreenshotOnFailure(boolean takeScreenshotOnFailure) { this.takeScreenshotOnFailure = takeScreenshotOnFailure; } public Long getMaxLoadTimeMs() { return maxLoadTimeMs; } public void setMaxLoadTimeMs(Long maxLoadTimeMs) { this.maxLoadTimeMs = maxLoadTimeMs; } }

Enums and Supporting Classes

TestEnums.java

package com.example.synthetic.model; public enum TestFrequency { EVERY_MINUTE(60), EVERY_5_MINUTES(300), EVERY_15_MINUTES(900), EVERY_30_MINUTES(1800), HOURLY(3600), EVERY_6_HOURS(21600), DAILY(86400); private final int seconds; TestFrequency(int seconds) { this.seconds = seconds; } public int getSeconds() { return seconds; } public String getCronExpression() { switch (this) { case EVERY_MINUTE: return "0 * * * * ?"; case EVERY_5_MINUTES: return "0 */5 * * * ?"; case EVERY_15_MINUTES: return "0 */15 * * * ?"; case EVERY_30_MINUTES: return "0 */30 * * * ?"; case HOURLY: return "0 0 * * * ?"; case EVERY_6_HOURS: return "0 0 */6 * * ?"; case DAILY: return "0 0 0 * * ?"; default: return "0 0 * * * ?"; } } } public enum TestPriority { LOW, MEDIUM, HIGH, CRITICAL } public enum HttpMethod { GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS } public enum TestStatus { SUCCESS, FAILURE, TIMEOUT, ERROR, SKIPPED }

Test Execution Results

TestExecutionResult.java

package com.example.synthetic.model; import com.fasterxml.jackson.annotation.JsonFormat; import jakarta.persistence.*; import java.time.LocalDateTime; import java.util.Map; @Entity @Table(name = "test_execution_results") public class TestExecutionResult { @Id @GeneratedValue(strategy = GenerationType.IDENTITY) private Long id; @ManyToOne(fetch = FetchType.LAZY) @JoinColumn(name = "test_id", nullable = false) private MonitorTest test; @Enumerated(EnumType.STRING) @Column(nullable = false) private TestStatus status; @JsonFormat(pattern = "yyyy-MM-dd HH:mm:ss") @Column(nullable = false) private LocalDateTime executionTime; private Long responseTimeMs; private Integer statusCode; @Column(length = 500) private String errorMessage; @Column(length = 4000) private String responseBody; @ElementCollection @CollectionTable(name = "execution_metrics", joinColumns = @JoinColumn(name = "execution_id")) @MapKeyColumn(name = "metric_name") @Column(name = "metric_value") private Map<String, Double> metrics; private String screenshotPath; private String location; // Where test was executed from @Column(length = 1000) private String additionalInfo; // Constructors public TestExecutionResult() {} public TestExecutionResult(MonitorTest test, TestStatus status) { this.test = test; this.status = status; this.executionTime = LocalDateTime.now(); } // Getters and Setters public Long getId() { return id; } public void setId(Long id) { this.id = id; } public MonitorTest getTest() { return test; } public void setTest(MonitorTest test) { this.test = test; } public TestStatus getStatus() { return status; } public void setStatus(TestStatus status) { this.status = status; } public LocalDateTime getExecutionTime() { return executionTime; } public void setExecutionTime(LocalDateTime executionTime) { this.executionTime = executionTime; } public Long getResponseTimeMs() { return responseTimeMs; } public void setResponseTimeMs(Long responseTimeMs) { this.responseTimeMs = responseTimeMs; } public Integer getStatusCode() { return statusCode; } public void setStatusCode(Integer statusCode) { this.statusCode = statusCode; } public String getErrorMessage() { return errorMessage; } public void setErrorMessage(String errorMessage) { this.errorMessage = errorMessage; } public String getResponseBody() { return responseBody; } public void setResponseBody(String responseBody) { this.responseBody = responseBody; } public Map<String, Double> getMetrics() { return metrics; } public void setMetrics(Map<String, Double> metrics) { this.metrics = metrics; } public String getScreenshotPath() { return screenshotPath; } public void setScreenshotPath(String screenshotPath) { this.screenshotPath = screenshotPath; } public String getLocation() { return location; } public void setLocation(String location) { this.location = location; } public String getAdditionalInfo() { return additionalInfo; } public void setAdditionalInfo(String additionalInfo) { this.additionalInfo = additionalInfo; } }

Step 4: Test Executors

Base Test Executor

TestExecutor.java

package com.example.synthetic.executor; import com.example.synthetic.model.MonitorTest; import com.example.synthetic.model.TestExecutionResult; import com.example.synthetic.model.TestStatus; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.stereotype.Component; import java.time.Duration; import java.time.Instant; import java.util.concurrent.Callable; public abstract class TestExecutor { protected final Logger logger = LoggerFactory.getLogger(getClass()); public abstract boolean supports(MonitorTest test); public TestExecutionResult execute(MonitorTest test) { Instant start = Instant.now(); TestExecutionResult result = new TestExecutionResult(test, TestStatus.SUCCESS); try { logger.info("Executing test: {} - {}", test.getId(), test.getName()); result = doExecute(test); } catch (Exception e) { logger.error("Test execution failed for test: {}", test.getName(), e); result.setStatus(TestStatus.ERROR); result.setErrorMessage(e.getMessage()); } finally { long duration = Duration.between(start, Instant.now()).toMillis(); result.setResponseTimeMs(duration); logger.info("Test completed: {} - Status: {} - Duration: {}ms", test.getName(), result.getStatus(), duration); } return result; } protected abstract TestExecutionResult doExecute(MonitorTest test) throws Exception; protected <T> T executeWithTimeout(Callable<T> callable, int timeoutSeconds) throws Exception { // Implementation with timeout handling // Using CompletableFuture with timeout return callable.call(); // Simplified - implement proper timeout } }

HTTP Test Executor

HttpTestExecutor.java

package com.example.synthetic.executor; import com.example.synthetic.model.HttpTest; import com.example.synthetic.model.MonitorTest; import com.example.synthetic.model.TestExecutionResult; import com.example.synthetic.model.TestStatus; import org.springframework.http.HttpHeaders; import org.springframework.http.HttpMethod; import org.springframework.http.MediaType; import org.springframework.stereotype.Component; import org.springframework.web.reactive.function.client.WebClient; import org.springframework.web.reactive.function.client.WebClientResponseException; import reactor.core.publisher.Mono; import java.time.Duration; import java.util.Map; import java.util.regex.Pattern; @Component public class HttpTestExecutor extends TestExecutor { private final WebClient webClient; public HttpTestExecutor(WebClient.Builder webClientBuilder) { this.webClient = webClientBuilder.build(); } @Override public boolean supports(MonitorTest test) { return test instanceof HttpTest; } @Override protected TestExecutionResult doExecute(MonitorTest test) throws Exception { HttpTest httpTest = (HttpTest) test; TestExecutionResult result = new TestExecutionResult(test, TestStatus.SUCCESS); try { WebClient.RequestBodySpec requestSpec = webClient .method(HttpMethod.valueOf(httpTest.getMethod().name())) .uri(httpTest.getTargetUrl()); // Set headers if (httpTest.getHeaders() != null) { httpTest.getHeaders().forEach(requestSpec::header); } // Set body for POST, PUT, PATCH if (httpTest.getRequestBody() != null && (httpTest.getMethod() == HttpMethod.POST || httpTest.getMethod() == HttpMethod.PUT || httpTest.getMethod() == HttpMethod.PATCH)) { requestSpec.contentType(MediaType.APPLICATION_JSON) .bodyValue(httpTest.getRequestBody()); } String responseBody = requestSpec .retrieve() .bodyToMono(String.class) .timeout(Duration.ofSeconds(httpTest.getTimeoutSeconds())) .block(); result.setStatusCode(200); // Assuming success since no exception result.setResponseBody(truncateResponse(responseBody, 4000)); // Validate response validateResponse(httpTest, result, responseBody); } catch (WebClientResponseException e) { result.setStatusCode(e.getStatusCode().value()); result.setResponseBody(truncateResponse(e.getResponseBodyAsString(), 4000)); if (httpTest.getExpectedStatusCode() != null && httpTest.getExpectedStatusCode() != e.getStatusCode().value()) { result.setStatus(TestStatus.FAILURE); result.setErrorMessage("Expected status " + httpTest.getExpectedStatusCode() + " but got " + e.getStatusCode().value()); } else if (httpTest.getExpectedStatusCode() == null && e.getStatusCode().isError()) { result.setStatus(TestStatus.FAILURE); result.setErrorMessage("HTTP error: " + e.getStatusCode()); } } catch (Exception e) { result.setStatus(TestStatus.ERROR); result.setErrorMessage(e.getMessage()); } return result; } private void validateResponse(HttpTest test, TestExecutionResult result, String responseBody) { // Check expected status code if (test.getExpectedStatusCode() != null && result.getStatusCode() != test.getExpectedStatusCode()) { result.setStatus(TestStatus.FAILURE); result.setErrorMessage("Expected status " + test.getExpectedStatusCode() + " but got " + result.getStatusCode()); return; } // Check response pattern if (test.getExpectedResponsePattern() != null && responseBody != null) { Pattern pattern = Pattern.compile(test.getExpectedResponsePattern()); if (!pattern.matcher(responseBody).find()) { result.setStatus(TestStatus.FAILURE); result.setErrorMessage("Response pattern not found"); return; } } // Check response time if (test.getMaxResponseTimeMs() != null && result.getResponseTimeMs() > test.getMaxResponseTimeMs()) { result.setStatus(TestStatus.FAILURE); result.setErrorMessage("Response time " + result.getResponseTimeMs() + "ms exceeds maximum " + test.getMaxResponseTimeMs() + "ms"); } } private String truncateResponse(String response, int maxLength) { if (response == null || response.length() <= maxLength) { return response; } return response.substring(0, maxLength - 3) + "..."; } }

Browser Test Executor

BrowserTestExecutor.java

package com.example.synthetic.executor; import com.example.synthetic.model.BrowserTest; import com.example.synthetic.model.MonitorTest; import com.example.synthetic.model.TestExecutionResult; import com.example.synthetic.model.TestStatus; import org.openqa.selenium.*; import org.openqa.selenium.chrome.ChromeDriver; import org.openqa.selenium.chrome.ChromeOptions; import org.openqa.selenium.support.ui.WebDriverWait; import org.springframework.beans.factory.annotation.Value; import org.springframework.stereotype.Component; import java.io.File; import java.nio.file.Files; import java.nio.file.Path; import java.nio.file.Paths; import java.time.Duration; import java.util.Base64; import java.util.HashMap; import java.util.Map; @Component public class BrowserTestExecutor extends TestExecutor { @Value("${synthetic.monitoring.browser.headless:true}") private boolean headless; @Value("${synthetic.monitoring.screenshot.directory:./screenshots}") private String screenshotDirectory; @Override public boolean supports(MonitorTest test) { return test instanceof BrowserTest; } @Override protected TestExecutionResult doExecute(MonitorTest test) throws Exception { BrowserTest browserTest = (BrowserTest) test; WebDriver driver = null; TestExecutionResult result = new TestExecutionResult(test, TestStatus.SUCCESS); try { // Setup WebDriver driver = createWebDriver(); WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(30)); // Navigate to target URL long startTime = System.currentTimeMillis(); driver.get(browserTest.getTargetUrl()); long loadTime = System.currentTimeMillis() - startTime; result.setResponseTimeMs(loadTime); // Execute custom script if provided if (browserTest.getScript() != null && !browserTest.getScript().trim().isEmpty()) { executeBrowserScript(driver, browserTest.getScript(), result); } // Check page load performance if (browserTest.getMaxLoadTimeMs() != null && loadTime > browserTest.getMaxLoadTimeMs()) { result.setStatus(TestStatus.FAILURE); result.setErrorMessage("Page load time " + loadTime + "ms exceeds maximum " + browserTest.getMaxLoadTimeMs() + "ms"); } // Capture performance metrics capturePerformanceMetrics(driver, result); } catch (TimeoutException e) { result.setStatus(TestStatus.TIMEOUT); result.setErrorMessage("Browser operation timed out: " + e.getMessage()); } catch (Exception e) { result.setStatus(TestStatus.ERROR); result.setErrorMessage("Browser test failed: " + e.getMessage()); } finally { // Take screenshot on failure if (driver != null) { if (result.getStatus() != TestStatus.SUCCESS && browserTest.isTakeScreenshotOnFailure()) { String screenshotPath = takeScreenshot(driver, test.getName()); result.setScreenshotPath(screenshotPath); } driver.quit(); } } return result; } private WebDriver createWebDriver() { ChromeOptions options = new ChromeOptions(); if (headless) { options.addArguments("--headless"); } options.addArguments("--no-sandbox"); options.addArguments("--disable-dev-shm-usage"); options.addArguments("--window-size=1920,1080"); options.addArguments("--disable-gpu"); options.addArguments("--disable-extensions"); // Performance logging options.setCapability("goog:loggingPrefs", Map.of("performance", "ALL", "browser", "ALL")); return new ChromeDriver(options); } private void executeBrowserScript(WebDriver driver, String script, TestExecutionResult result) { try { // Simple script execution - extend for more complex scenarios if (script.contains("waitForElement")) { // Example: waitForElement("#submit-button") String selector = script.split("\\(")[1].replace(")", "").replace("\"", "").trim(); WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(10)); wait.until(webDriver -> webDriver.findElement(By.cssSelector(selector)).isDisplayed()); } if (script.contains("click")) { String selector = script.split("\\(")[1].replace(")", "").replace("\"", "").trim(); driver.findElement(By.cssSelector(selector)).click(); } if (script.contains("assertText")) { // Example: assertText(".status", "Success") String[] parts = script.split(","); String selector = parts[0].split("\\(")[1].replace("\"", "").trim(); String expectedText = parts[1].replace(")", "").replace("\"", "").trim(); String actualText = driver.findElement(By.cssSelector(selector)).getText(); if (!actualText.contains(expectedText)) { result.setStatus(TestStatus.FAILURE); result.setErrorMessage("Text assertion failed. Expected: " + expectedText + ", Actual: " + actualText); } } } catch (NoSuchElementException e) { result.setStatus(TestStatus.FAILURE); result.setErrorMessage("Element not found: " + e.getMessage()); } catch (TimeoutException e) { result.setStatus(TestStatus.TIMEOUT); result.setErrorMessage("Element wait timeout: " + e.getMessage()); } } private void capturePerformanceMetrics(WebDriver driver, TestExecutionResult result) { try { JavascriptExecutor js = (JavascriptExecutor) driver; Map<String, Double> metrics = new HashMap<>(); // Navigation timing API Long loadEventEnd = (Long) js.executeScript( "return window.performance.timing.loadEventEnd"); Long navigationStart = (Long) js.executeScript( "return window.performance.timing.navigationStart"); if (loadEventEnd != null && navigationStart != null) { metrics.put("page_load_time", (double)(loadEventEnd - navigationStart)); } // Resource timing Long totalResourceSize = (Long) js.executeScript( "return window.performance.getEntriesByType('resource')" + ".reduce((acc, entry) => acc + entry.transferSize, 0)"); if (totalResourceSize != null) { metrics.put("total_resource_size_kb", totalResourceSize / 1024.0); } result.setMetrics(metrics); } catch (Exception e) { logger.warn("Failed to capture performance metrics", e); } } private String takeScreenshot(WebDriver driver, String testName) { try { Files.createDirectories(Paths.get(screenshotDirectory)); String timestamp = String.valueOf(System.currentTimeMillis()); String filename = testName.replaceAll("[^a-zA-Z0-9]", "_") + "_" + timestamp + ".png"; Path screenshotPath = Paths.get(screenshotDirectory, filename); File screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE); Files.copy(screenshot.toPath(), screenshotPath); return screenshotPath.toString(); } catch (Exception e) { logger.error("Failed to take screenshot", e); return null; } } }

Step 5: Test Orchestration Service

TestOrchestrationService.java

package com.example.synthetic.service; import com.example.synthetic.executor.TestExecutor; import com.example.synthetic.model.MonitorTest; import com.example.synthetic.model.TestExecutionResult; import com.example.synthetic.model.TestStatus; import com.example.synthetic.repository.MonitorTestRepository; import com.example.synthetic.repository.TestExecutionResultRepository; import io.micrometer.core.instrument.MeterRegistry; import io.micrometer.core.instrument.Timer; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.scheduling.annotation.Async; import org.springframework.stereotype.Service; import org.springframework.transaction.annotation.Transactional; import java.time.LocalDateTime; import java.util.List; import java.util.Map; import java.util.concurrent.CompletableFuture; import java.util.concurrent.ConcurrentHashMap; import java.util.stream.Collectors; @Service public class TestOrchestrationService { private static final Logger logger = LoggerFactory.getLogger(TestOrchestrationService.class); private final List<TestExecutor> testExecutors; private final MonitorTestRepository testRepository; private final TestExecutionResultRepository resultRepository; private final AlertService alertService; private final MeterRegistry meterRegistry; private final Map<Long, CompletableFuture<TestExecutionResult>> runningTests = new ConcurrentHashMap<>(); public TestOrchestrationService(List<TestExecutor> testExecutors, MonitorTestRepository testRepository, TestExecutionResultRepository resultRepository, AlertService alertService, MeterRegistry meterRegistry) { this.testExecutors = testExecutors; this.testRepository = testRepository; this.resultRepository = resultRepository; this.alertService = alertService; this.meterRegistry = meterRegistry; } @Async("testExecutionTaskExecutor") public CompletableFuture<TestExecutionResult> executeTest(Long testId) { return testRepository.findById(testId) .map(this::executeTest) .orElse(CompletableFuture.completedFuture( createErrorResult("Test not found with id: " + testId))); } @Async("testExecutionTaskExecutor") public CompletableFuture<TestExecutionResult> executeTest(MonitorTest test) { // Check if test is already running if (runningTests.containsKey(test.getId())) { logger.warn("Test {} is already running", test.getId()); return runningTests.get(test.getId()); } CompletableFuture<TestExecutionResult> future = CompletableFuture.supplyAsync(() -> { Timer.Sample sample = Timer.start(meterRegistry); TestExecutionResult result = null; try { TestExecutor executor = findExecutor(test); if (executor == null) { return createErrorResult("No executor found for test type: " + test.getClass().getSimpleName()); } result = executor.execute(test); } catch (Exception e) { logger.error("Test execution failed for test: {}", test.getName(), e); result = createErrorResult("Execution error: " + e.getMessage()); } finally { sample.stop(Timer.builder("synthetic.test.execution.time") .tag("testName", test.getName()) .tag("status", result != null ? result.getStatus().name() : "ERROR") .register(meterRegistry)); runningTests.remove(test.getId()); } return result != null ? result : createErrorResult("Unknown execution error"); }); runningTests.put(test.getId(), future); // Process result when complete future.thenAccept(this::processExecutionResult); return future; } @Transactional public void processExecutionResult(TestExecutionResult result) { try { // Save result to database resultRepository.save(result); // Update metrics updateMetrics(result); // Send alert if test failed if (result.getStatus() != TestStatus.SUCCESS) { alertService.sendTestFailureAlert(result); } logger.info("Processed test result for {}: {}", result.getTest().getName(), result.getStatus()); } catch (Exception e) { logger.error("Failed to process test execution result", e); } } public void executeScheduledTests() { List<MonitorTest> scheduledTests = testRepository.findEnabledTestsForExecution( LocalDateTime.now()); logger.info("Executing {} scheduled tests", scheduledTests.size()); scheduledTests.forEach(test -> { try { executeTest(test); } catch (Exception e) { logger.error("Failed to execute scheduled test: {}", test.getName(), e); } }); } private TestExecutor findExecutor(MonitorTest test) { return testExecutors.stream() .filter(executor -> executor.supports(test)) .findFirst() .orElse(null); } private TestExecutionResult createErrorResult(String errorMessage) { TestExecutionResult result = new TestExecutionResult(); result.setStatus(TestStatus.ERROR); result.setErrorMessage(errorMessage); result.setExecutionTime(LocalDateTime.now()); return result; } private void updateMetrics(TestExecutionResult result) { String testName = result.getTest().getName(); String status = result.getStatus().name().toLowerCase(); // Counter for test executions meterRegistry.counter("synthetic.test.executions", "testName", testName, "status", status ).increment(); // Response time histogram if (result.getResponseTimeMs() != null) { meterRegistry.timer("synthetic.test.response.time", "testName", testName ).record(java.time.Duration.ofMillis(result.getResponseTimeMs())); } // Gauge for success rate (you might want to calculate this differently) meterRegistry.gauge("synthetic.test.success.rate", Map.of("testName", testName), this, service -> calculateSuccessRate(testName) ); } private double calculateSuccessRate(String testName) { // Implement success rate calculation based on recent results return 95.0; // Simplified } public Map<Long, String> getRunningTests() { return runningTests.entrySet().stream() .collect(Collectors.toMap( Map.Entry::getKey, entry -> entry.getValue().isDone() ? "COMPLETED" : "RUNNING" )); } }

Step 6: Scheduling Configuration

SchedulingConfig.java

package com.example.synthetic.config; import org.springframework.boot.autoconfigure.quartz.QuartzDataSource; import org.springframework.boot.context.properties.ConfigurationProperties; import org.springframework.boot.jdbc.DataSourceBuilder; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.scheduling.annotation.EnableAsync; import org.springframework.scheduling.annotation.EnableScheduling; import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor; import javax.sql.DataSource; import java.util.concurrent.Executor; @Configuration @EnableAsync @EnableScheduling public class SchedulingConfig { @Bean("testExecutionTaskExecutor") public Executor testExecutionTaskExecutor() { ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor(); executor.setCorePoolSize(5); executor.setMaxPoolSize(10); executor.setQueueCapacity(25); executor.setThreadNamePrefix("TestExecutor-"); executor.setRejectedExecutionHandler(new ThreadPoolTaskExecutor.CallerRunsPolicy()); executor.initialize(); return executor; } @Bean @QuartzDataSource @ConfigurationProperties(prefix = "spring.datasource") public DataSource quartzDataSource() { return DataSourceBuilder.create().build(); } }

TestScheduler.java

package com.example.synthetic.scheduler; import com.example.synthetic.service.TestOrchestrationService; import org.quartz.*; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.scheduling.annotation.Scheduled; import org.springframework.stereotype.Component; @Component public class TestScheduler { private static final Logger logger = LoggerFactory.getLogger(TestScheduler.class); private final TestOrchestrationService orchestrationService; public TestScheduler(TestOrchestrationService orchestrationService) { this.orchestrationService = orchestrationService; } // Run every minute to check for scheduled tests @Scheduled(cron = "0 * * * * ?") public void executeScheduledTests() { logger.debug("Checking for scheduled tests..."); orchestrationService.executeScheduledTests(); } // Health check - run every 5 minutes @Scheduled(cron = "0 */5 * * * ?") public void healthCheck() { logger.info("Synthetic monitoring health check - Running tests: {}", orchestrationService.getRunningTests().size()); } }

Step 7: REST API for Test Management

MonitorTestController.java

package com.example.synthetic.controller; import com.example.synthetic.model.MonitorTest; import com.example.synthetic.model.TestExecutionResult; import com.example.synthetic.service.MonitorTestService; import com.example.synthetic.service.TestOrchestrationService; import org.springframework.data.domain.Page; import org.springframework.data.domain.PageRequest; import org.springframework.http.ResponseEntity; import org.springframework.web.bind.annotation.*; import java.util.List; import java.util.concurrent.CompletableFuture; @RestController @RequestMapping("/api/v1/monitor-tests") public class MonitorTestController { private final MonitorTestService testService; private final TestOrchestrationService orchestrationService; public MonitorTestController(MonitorTestService testService, TestOrchestrationService orchestrationService) { this.testService = testService; this.orchestrationService = orchestrationService; } @GetMapping public ResponseEntity<Page<MonitorTest>> getTests( @RequestParam(defaultValue = "0") int page, @RequestParam(defaultValue = "20") int size) { return ResponseEntity.ok(testService.findAll(PageRequest.of(page, size))); } @GetMapping("/{id}") public ResponseEntity<MonitorTest> getTest(@PathVariable Long id) { return testService.findById(id) .map(ResponseEntity::ok) .orElse(ResponseEntity.notFound().build()); } @PostMapping public ResponseEntity<MonitorTest> createTest(@RequestBody MonitorTest test) { return ResponseEntity.ok(testService.save(test)); } @PutMapping("/{id}") public ResponseEntity<MonitorTest> updateTest(@PathVariable Long id, @RequestBody MonitorTest test) { test.setId(id); return ResponseEntity.ok(testService.save(test)); } @DeleteMapping("/{id}") public ResponseEntity<Void> deleteTest(@PathVariable Long id) { testService.deleteById(id); return ResponseEntity.ok().build(); } @PostMapping("/{id}/execute") public CompletableFuture<ResponseEntity<TestExecutionResult>> executeTest(@PathVariable Long id) { return orchestrationService.executeTest(id) .thenApply(ResponseEntity::ok); } @GetMapping("/{id}/results") public ResponseEntity<Page<TestExecutionResult>> getTestResults( @PathVariable Long id, @RequestParam(defaultValue = "0") int page, @RequestParam(defaultValue = "50") int size) { return ResponseEntity.ok(testService.findResultsByTestId(id, PageRequest.of(page, size))); } }

Step 8: Alerting Service

AlertService.java

package com.example.synthetic.service; import com.example.synthetic.model.TestExecutionResult; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Value; import org.springframework.http.HttpEntity; import org.springframework.http.HttpHeaders; import org.springframework.http.MediaType; import org.springframework.stereotype.Service; import org.springframework.web.client.RestTemplate; import java.util.HashMap; import java.util.Map; @Service public class AlertService { private static final Logger logger = LoggerFactory.getLogger(AlertService.class); @Value("${synthetic.monitoring.alerts.slack-webhook-url:}") private String slackWebhookUrl; @Value("${synthetic.monitoring.alerts.pagerduty-routing-key:}") private String pagerdutyRoutingKey; private final RestTemplate restTemplate; public AlertService(RestTemplate restTemplate) { this.restTemplate = restTemplate; } public void sendTestFailureAlert(TestExecutionResult result) { // Send to Slack if (slackWebhookUrl != null && !slackWebhookUrl.isEmpty()) { sendSlackAlert(result); } // Send to PagerDuty for critical failures if (pagerdutyRoutingKey != null && !pagerdutyRoutingKey.isEmpty() && result.getTest().getPriority().ordinal() >= 2) { // HIGH or CRITICAL sendPagerDutyAlert(result); } logger.warn("Test failure alert sent for: {} - {}", result.getTest().getName(), result.getErrorMessage()); } private void sendSlackAlert(TestExecutionResult result) { try { Map<String, Object> slackMessage = new HashMap<>(); slackMessage.put("text", "❌ Synthetic Test Failure"); Map<String, Object> attachment = new HashMap<>(); attachment.put("color", "danger"); attachment.put("title", result.getTest().getName()); attachment.put("text", result.getErrorMessage()); Map<String, Object> fields = new HashMap<>(); fields.put("title", "Details"); fields.put("value", String.format( "Test: %s\nStatus: %s\nResponse Time: %dms\nURL: %s", result.getTest().getName(), result.getStatus(), result.getResponseTimeMs(), result.getTest().getTargetUrl() )); fields.put("short", false); attachment.put("fields", new Object[]{fields}); slackMessage.put("attachments", new Object[]{attachment}); HttpHeaders headers = new HttpHeaders(); headers.setContentType(MediaType.APPLICATION_JSON); HttpEntity<Map<String, Object>> request = new HttpEntity<>(slackMessage, headers); restTemplate.postForEntity(slackWebhookUrl, request, String.class); } catch (Exception e) { logger.error("Failed to send Slack alert", e); } } private void sendPagerDutyAlert(TestExecutionResult result) { try { Map<String, Object> pagerDutyEvent = new HashMap<>(); pagerDutyEvent.put("routing_key", pagerdutyRoutingKey); pagerDutyEvent.put("event_action", "trigger"); Map<String, Object> payload = new HashMap<>(); payload.put("summary", String.format("Synthetic test failure: %s", result.getTest().getName())); payload.put("source", "synthetic-monitoring"); payload.put("severity", "error"); payload.put("custom_details", Map.of( "test_name", result.getTest().getName(), "error_message", result.getErrorMessage(), "response_time_ms", result.getResponseTimeMs(), "target_url", result.getTest().getTargetUrl(), "test_priority", result.getTest().getPriority().name() )); pagerDutyEvent.put("payload", payload); HttpHeaders headers = new HttpHeaders(); headers.setContentType(MediaType.APPLICATION_JSON); HttpEntity<Map<String, Object>> request = new HttpEntity<>(pagerDutyEvent, headers); restTemplate.postForEntity("https://events.pagerduty.com/v2/enqueue", request, String.class); } catch (Exception e) { logger.error("Failed to send PagerDuty alert", e); } } }

Step 9: Sample Test Definitions

HTTP Health Check Test

{ "type": "HTTP", "name": "API Health Check", "description": "Check if main API is responding", "targetUrl": "https://api.example.com/health", "frequency": "EVERY_5_MINUTES", "priority": "HIGH", "method": "GET", "expectedStatusCode": 200, "expectedResponsePattern": "\"status\":\"UP\"", "maxResponseTimeMs": 1000, "headers": { "User-Agent": "Synthetic-Monitoring/1.0" } }

Browser User Journey Test

{ "type": "BROWSER", "name": "User Login Flow", "description": "Test complete user login journey", "targetUrl": "https://app.example.com/login", "frequency": "HOURLY", "priority": "HIGH", "script": "waitForElement('#username'); waitForElement('#password'); click('#login-btn'); waitForElement('.dashboard'); assertText('.welcome-message', 'Welcome');", "maxLoadTimeMs": 10000, "takeScreenshotOnFailure": true }

Benefits and Use Cases

Key Benefits:

Proactive Monitoring - Detect issues before users are affected
Performance Baselines - Establish and monitor performance benchmarks
Business Transaction Monitoring - Ensure critical flows work end-to-end
Geographic Monitoring - Test from different locations (extendable)
Trend Analysis - Track performance degradation over time

Common Use Cases:

API Health Checks - Verify microservices are responsive
User Journey Validation - Ensure critical business flows work
Third-party Integration Checks - Monitor external dependencies
Performance Regression Detection - Catch performance degradation early
SLA Monitoring - Ensure compliance with service level agreements

Conclusion

This comprehensive synthetic monitoring solution provides:

Flexible test definitions for HTTP, browser, and custom validations
Scalable execution with async processing and thread pooling
Comprehensive alerting with Slack and PagerDuty integration
Performance metrics with Micrometer and Prometheus
Persistent storage for historical analysis and trending
REST API for test management and execution

By implementing this synthetic monitoring framework, you can proactively detect issues, ensure application reliability, and maintain optimal user experience across your digital services.