Java Flight Recorder (JFR) in Native Images enables performance monitoring and diagnostics for GraalVM native executables. While traditionally a JVM feature, JFR has been adapted to work with ahead-of-time compiled native images.
JFR in Native Images Overview
Key Differences from JVM JFR:
- Limited event set compared to JVM
- Custom events work differently
- Configuration happens at build time
- Lower overhead due to AOT compilation
Setting Up JFR for Native Images
Dependencies and Configuration
<!-- Maven Dependencies --> <dependencies> <dependency> <groupId>org.graalvm.nativeimage</groupId> <artifactId>native-image-base</artifactId> <version>23.0.0</version> </dependency> <!-- JFR support --> <dependency> <groupId>org.graalvm.nativeimage</groupId> <artifactId>svm</artifactId> <version>23.0.0</version> </dependency> </dependencies>
Basic JFR Usage in Native Images
Example 1: Simple JFR Recording
import jdk.jfr.*; import java.io.IOException; import java.nio.file.*; import java.time.Duration; public class BasicJFRNativeDemo { @Startup public static void main(String[] args) throws Exception { System.out.println("=== Starting JFR Native Image Demo ==="); // Start JFR recording String recordingFile = "native-jfr-recording.jfr"; startJFRRecording(recordingFile); // Simulate application work performWorkload(); // Stop recording stopJFRRecording(); System.out.println("JFR recording saved to: " + recordingFile); System.out.println("Use JMC to analyze: jmc " + recordingFile); } private static void startJFRRecording(String filename) { try { Path path = Paths.get(filename); // Configure recording Configuration config = Configuration.getConfiguration("default"); Recording recording = new Recording(config); recording.setDestination(path); recording.setDumpOnExit(true); recording.setToDisk(true); // Enable specific events recording.enable("jdk.CPULoad").withPeriod(Duration.ofSeconds(1)); recording.enable("jdk.GarbageCollection").withPeriod(Duration.ofSeconds(1)); recording.enable("jdk.JavaMonitorEnter").withThreshold(Duration.ofMillis(10)); recording.start(); System.out.println("JFR recording started: " + filename); } catch (Exception e) { System.err.println("Failed to start JFR recording: " + e.getMessage()); } } private static void stopJFRRecording() { try { // Find and stop all recordings for (Recording recording : FlightRecorder.getFlightRecorder().getRecordings()) { if (recording.isRunning()) { recording.stop(); System.out.println("Stopped recording: " + recording.getName()); } } } catch (Exception e) { System.err.println("Error stopping recording: " + e.getMessage()); } } private static void performWorkload() { System.out.println("Performing workload simulation..."); // CPU-intensive work for (int i = 0; i < 5; i++) { cpuIntensiveTask(i); memoryAllocationTask(i); threadSleepTask(i); } System.out.println("Workload completed"); } private static void cpuIntensiveTask(int iteration) { long startTime = System.nanoTime(); // Simulate CPU work double result = 0; for (int i = 0; i < 1_000_000; i++) { result += Math.sin(i) * Math.cos(i); } long duration = System.nanoTime() - startTime; System.out.printf("CPU Task %d completed in %d ms%n", iteration, duration / 1_000_000); } private static void memoryAllocationTask(int iteration) { // Allocate some memory byte[][] arrays = new byte[100][]; for (int i = 0; i < arrays.length; i++) { arrays[i] = new byte[1024 * 1024]; // 1MB each } System.out.printf("Memory Task %d allocated %d MB%n", iteration, arrays.length); } private static void threadSleepTask(int iteration) { try { Thread.sleep(1000); System.out.printf("Sleep Task %d completed%n", iteration); } catch (InterruptedException e) { Thread.currentThread().interrupt(); } } } Custom JFR Events in Native Images
Example 2: Defining Custom Business Events
import jdk.jfr.*; import java.util.concurrent.*; @Name("com.example.BusinessTransaction") @Label("Business Transaction Event") @Category("Business") @Description("Records business transaction execution") class BusinessTransactionEvent extends Event { @Label("Transaction ID") public String transactionId; @Label("Transaction Type") public String transactionType; @Label("Amount") public double amount; @Label("Success") public boolean success; @Label("Processing Time") public long processingTimeMs; @Label("Error Message") public String errorMessage; } @Name("com.example.DatabaseQuery") @Label("Database Query Event") @Category("Database") @Description("Records database query execution") class DatabaseQueryEvent extends Event { @Label("Query Type") public String queryType; @Label("Table Name") public String tableName; @Label("Execution Time") public long executionTimeMs; @Label("Rows Affected") public int rowsAffected; @Label("Success") public boolean success; } @Name("com.example.CacheOperation") @Label("Cache Operation Event") @Category("Cache") @Description("Records cache operations") class CacheOperationEvent extends Event { @Label("Operation Type") public String operationType; // GET, PUT, REMOVE @Label("Cache Name") public String cacheName; @Label("Key") public String key; @Label("Execution Time") public long executionTimeNs; @Label("Hit") public boolean hit; } public class CustomEventsJFRDemo { private static final ConcurrentHashMap<String, String> cache = new ConcurrentHashMap<>(); public static void main(String[] args) throws Exception { System.out.println("=== Custom JFR Events Demo ==="); // Start recording String recordingFile = "custom-events.jfr"; startCustomRecording(recordingFile); // Simulate business operations simulateBusinessWorkload(); // Stop recording Thread.sleep(2000); // Allow events to be recorded stopAllRecordings(); System.out.println("Recording saved to: " + recordingFile); } private static void startCustomRecording(String filename) { try { Recording recording = new Recording(); recording.setName("Custom Events Recording"); recording.setDestination(Paths.get(filename)); recording.setDumpOnExit(true); // Enable custom events recording.enable(BusinessTransactionEvent.class); recording.enable(DatabaseQueryEvent.class); recording.enable(CacheOperationEvent.class); // Enable system events for context recording.enable("jdk.CPULoad").withPeriod(Duration.ofSeconds(2)); recording.enable("jdk.GarbageCollection"); recording.start(); System.out.println("Custom events recording started"); } catch (Exception e) { System.err.println("Failed to start recording: " + e.getMessage()); } } private static void simulateBusinessWorkload() { ExecutorService executor = Executors.newFixedThreadPool(3); for (int i = 0; i < 10; i++) { final int taskId = i; executor.submit(() -> { try { processBusinessTransaction(taskId); Thread.sleep(ThreadLocalRandom.current().nextInt(100, 500)); } catch (InterruptedException e) { Thread.currentThread().interrupt(); } }); } executor.shutdown(); try { executor.awaitTermination(10, TimeUnit.SECONDS); } catch (InterruptedException e) { Thread.currentThread().interrupt(); } } private static void processBusinessTransaction(int transactionId) { BusinessTransactionEvent transactionEvent = new BusinessTransactionEvent(); transactionEvent.begin(); try { transactionEvent.transactionId = "TXN-" + transactionId; transactionEvent.transactionType = "PAYMENT"; transactionEvent.amount = 100.0 + (transactionId * 25.0); // Simulate database operations queryDatabase("users", "SELECT * FROM users WHERE id = " + transactionId); queryDatabase("accounts", "UPDATE accounts SET balance = balance - " + transactionEvent.amount); // Simulate cache operations cacheGet("user_profile_" + transactionId); cachePut("recent_transaction_" + transactionId, "data"); // Simulate processing time Thread.sleep(ThreadLocalRandom.current().nextInt(50, 200)); transactionEvent.success = true; transactionEvent.processingTimeMs = System.currentTimeMillis() - (transactionEvent.startTime / 1_000_000); } catch (Exception e) { transactionEvent.success = false; transactionEvent.errorMessage = e.getMessage(); } finally { transactionEvent.commit(); System.out.printf("Processed transaction %s: success=%b%n", transactionEvent.transactionId, transactionEvent.success); } } private static void queryDatabase(String table, String query) { DatabaseQueryEvent dbEvent = new DatabaseQueryEvent(); dbEvent.begin(); try { dbEvent.queryType = query.startsWith("SELECT") ? "SELECT" : "UPDATE"; dbEvent.tableName = table; // Simulate database work Thread.sleep(ThreadLocalRandom.current().nextInt(10, 50)); dbEvent.rowsAffected = ThreadLocalRandom.current().nextInt(1, 100); dbEvent.success = true; dbEvent.executionTimeMs = System.currentTimeMillis() - (dbEvent.startTime / 1_000_000); } catch (InterruptedException e) { Thread.currentThread().interrupt(); dbEvent.success = false; } finally { dbEvent.commit(); } } private static void cacheGet(String key) { CacheOperationEvent cacheEvent = new CacheOperationEvent(); cacheEvent.begin(); try { cacheEvent.operationType = "GET"; cacheEvent.cacheName = "application_cache"; cacheEvent.key = key; String value = cache.get(key); cacheEvent.hit = (value != null); if (!cacheEvent.hit) { // Simulate cache miss penalty Thread.sleep(5); } } catch (InterruptedException e) { Thread.currentThread().interrupt(); } finally { cacheEvent.executionTimeNs = System.nanoTime() - cacheEvent.startTime; cacheEvent.commit(); } } private static void cachePut(String key, String value) { CacheOperationEvent cacheEvent = new CacheOperationEvent(); cacheEvent.begin(); try { cacheEvent.operationType = "PUT"; cacheEvent.cacheName = "application_cache"; cacheEvent.key = key; cache.put(key, value); cacheEvent.hit = false; } finally { cacheEvent.executionTimeNs = System.nanoTime() - cacheEvent.startTime; cacheEvent.commit(); } } private static void stopAllRecordings() { for (Recording recording : FlightRecorder.getFlightRecorder().getRecordings()) { if (recording.isRunning()) { recording.stop(); } } } } Advanced JFR Configuration for Native Images
Example 3: Programmatic JFR Configuration
import jdk.jfr.*; import java.lang.management.*; import java.nio.file.*; import java.time.*; import java.util.*; public class AdvancedJFRConfiguration { private Recording activeRecording; public void startConfiguredRecording(String name, Path destination, Duration duration) { try { // Create custom configuration Map<String, String> settings = new HashMap<>(); // CPU and Memory events settings.put("jdk.CPULoad#period", "2 s"); settings.put("jdk.GarbageCollection#enabled", "true"); settings.put("jdk.JavaMonitorEnter#threshold", "10 ms"); settings.put("jdk.ThreadPark#threshold", "10 ms"); // Custom events settings.put("com.example.BusinessTransaction#enabled", "true"); settings.put("com.example.DatabaseQuery#enabled", "true"); settings.put("com.example.CacheOperation#enabled", "true"); // Create configuration Configuration customConfig = Configuration.create(settings); // Start recording activeRecording = new Recording(customConfig); activeRecording.setName(name); activeRecording.setDestination(destination); activeRecording.setDumpOnExit(true); activeRecording.setToDisk(true); if (!duration.isZero()) { activeRecording.setDuration(duration); activeRecording.setShouldWriteRecordingWhenStopped(true); } activeRecording.start(); System.out.printf("Started recording: %s -> %s%n", name, destination); } catch (Exception e) { System.err.println("Failed to start configured recording: " + e.getMessage()); } } public void startProfilingRecording(Path destination, Duration duration) { try { // Configuration for performance profiling Map<String, String> settings = new HashMap<>(); // High-frequency events for profiling settings.put("jdk.ExecutionSample#enabled", "true"); settings.put("jdk.ExecutionSample#period", "10 ms"); settings.put("jdk.NativeMethodSample#enabled", "true"); settings.put("jdk.NativeMethodSample#period", "10 ms"); settings.put("jdk.ThreadDump#enabled", "true"); settings.put("jdk.ThreadDump#period", "5 s"); // Memory profiling settings.put("jdk.ObjectAllocationInNewTLAB#enabled", "true"); settings.put("jdk.ObjectAllocationOutsideTLAB#enabled", "true"); settings.put("jdk.GCHeapSummary#period", "2 s"); Configuration profilingConfig = Configuration.create(settings); Recording profilingRecording = new Recording(profilingConfig); profilingRecording.setName("Performance Profiling"); profilingRecording.setDestination(destination); profilingRecording.setDumpOnExit(true); profilingRecording.setToDisk(true); profilingRecording.setDuration(duration); profilingRecording.start(); System.out.printf("Started profiling recording for %s%n", duration); } catch (Exception e) { System.err.println("Failed to start profiling recording: " + e.getMessage()); } } public void snapshotRecording(Path snapshotPath) { try { if (activeRecording != null && activeRecording.isRunning()) { activeRecording.dump(snapshotPath); System.out.println("Recording snapshot saved to: " + snapshotPath); } } catch (Exception e) { System.err.println("Failed to create snapshot: " + e.getMessage()); } } public void stopRecording() { if (activeRecording != null && activeRecording.isRunning()) { activeRecording.stop(); System.out.println("Stopped recording: " + activeRecording.getName()); } } public static void main(String[] args) throws Exception { AdvancedJFRConfiguration jfrManager = new AdvancedJFRConfiguration(); // Start different types of recordings Path mainRecording = Paths.get("main-recording.jfr"); Path profilingRecording = Paths.get("profiling.jfr"); Path snapshotPath = Paths.get("snapshot.jfr"); // Start main recording jfrManager.startConfiguredRecording("Main Application", mainRecording, Duration.ofMinutes(5)); // Start short profiling recording jfrManager.startProfilingRecording(profilingRecording, Duration.ofSeconds(30)); // Simulate application work simulateWorkWithCustomEvents(); // Take a snapshot Thread.sleep(5000); jfrManager.snapshotRecording(snapshotPath); // Continue working Thread.sleep(5000); // Stop recordings jfrManager.stopRecording(); System.out.println("All recordings completed"); } private static void simulateWorkWithCustomEvents() { Thread worker = new Thread(() -> { Random random = new Random(); for (int i = 0; i < 100; i++) { // Emit custom events emitCustomEvent("TASK_" + i, random.nextDouble() * 1000, random.nextBoolean()); try { Thread.sleep(random.nextInt(100)); } catch (InterruptedException e) { Thread.currentThread().interrupt(); break; } } }); worker.start(); try { worker.join(); } catch (InterruptedException e) { Thread.currentThread().interrupt(); } } private static void emitCustomEvent(String name, double value, boolean success) { BusinessTransactionEvent event = new BusinessTransactionEvent(); event.begin(); event.transactionId = name; event.transactionType = "BACKGROUND_TASK"; event.amount = value; event.success = success; event.commit(); } } Native Image Configuration for JFR
Example 4: Native Image Build Configuration
// META-INF/native-image/jfr-config.json { "resources": { "includes": [ { "pattern": "jdk/jfr/*" }, { "pattern": "META-INF/services/jdk.jfr.*" } ] }, "jni": { "classes": [ { "name": "jdk.jfr.internal.JVM" }, { "name": "jdk.jfr.internal.PlatformRecording" } ] }, "reflect": { "classes": [ { "name": "jdk.jfr.Configuration", "methods": [ {"name": "getConfigurations", "parameterTypes": [] }, {"name": "getConfiguration", "parameterTypes": ["java.lang.String"] } ] }, { "name": "jdk.jfr.Recording", "methods": [ {"name": "<init>", "parameterTypes": ["jdk.jfr.Configuration"] }, {"name": "start", "parameterTypes": [] }, {"name": "stop", "parameterTypes": [] } ] }, { "name": "com.example.BusinessTransactionEvent", "methods": [ {"name": "<init>", "parameterTypes": [] } ] } ] } } Native Image Build Arguments
# Build command with JFR support native-image \ --enable-monitoring=jfr \ --initialize-at-build-time=jdk.jfr \ -H:ConfigurationFileDirectories=META-INF/native-image \ -H:+AllowFoldMethods \ -jar application.jar
JFR Event Streaming in Native Images
Example 5: Real-time JFR Event Processing
import jdk.jfr.*; import jdk.jfr.consumer.*; import java.nio.file.*; import java.util.concurrent.*; import java.util.function.Consumer; public class JFRStreamingDemo { private volatile boolean running = true; public void startEventStreaming(Consumer<RecordedEvent> eventHandler) { try { // Create in-memory recording for streaming Recording streamingRecording = new Recording(); streamingRecording.setToDisk(false); // Keep in memory streamingRecording.setName("Streaming Recording"); // Enable events for streaming streamingRecording.enable("jdk.CPULoad").withPeriod(Duration.ofSeconds(1)); streamingRecording.enable("jdk.GarbageCollection"); streamingRecording.enable("jdk.JavaMonitorEnter").withThreshold(Duration.ofMillis(10)); streamingRecording.enable(BusinessTransactionEvent.class); streamingRecording.start(); // Start event processing thread Thread processorThread = new Thread(() -> { processEvents(streamingRecording, eventHandler); }); processorThread.setDaemon(true); processorThread.start(); System.out.println("JFR event streaming started"); } catch (Exception e) { System.err.println("Failed to start event streaming: " + e.getMessage()); } } private void processEvents(Recording recording, Consumer<RecordedEvent> eventHandler) { try (RecordingStream stream = new RecordingStream()) { // Enable the same events as the recording stream.enable("jdk.CPULoad").withPeriod(Duration.ofSeconds(1)); stream.enable("jdk.GarbageCollection"); stream.enable("jdk.JavaMonitorEnter").withThreshold(Duration.ofMillis(10)); stream.enable(BusinessTransactionEvent.class); // Set up event handlers stream.onEvent("jdk.CPULoad", event -> { handleCPULoad(event); eventHandler.accept(event); }); stream.onEvent("jdk.GarbageCollection", event -> { handleGarbageCollection(event); eventHandler.accept(event); }); stream.onEvent("com.example.BusinessTransaction", event -> { handleBusinessTransaction(event); eventHandler.accept(event); }); // Start streaming stream.start(); } catch (Exception e) { System.err.println("Event streaming error: " + e.getMessage()); } } private void handleCPULoad(RecordedEvent event) { Double jvmUser = event.getDouble("jvmUser"); Double jvmSystem = event.getDouble("jvmSystem"); Double machineTotal = event.getDouble("machineTotal"); if (jvmUser != null && machineTotal != null) { double cpuUsage = (jvmUser + (jvmSystem != null ? jvmSystem : 0)) / machineTotal * 100; System.out.printf("CPU Usage: %.1f%% (JVM: %.1f%%)%n", machineTotal * 100, cpuUsage); } } private void handleGarbageCollection(RecordedEvent event) { String gcName = event.getString("name"); Long gcId = event.getLong("id"); Long duration = event.getLong("duration"); if (duration != null) { System.out.printf("GC Event: %s (ID: %d) took %d ms%n", gcName, gcId, duration.toMillis()); } } private void handleBusinessTransaction(RecordedEvent event) { String transactionId = event.getString("transactionId"); Boolean success = event.getBoolean("success"); Double amount = event.getDouble("amount"); System.out.printf("Business Transaction: %s, Amount: %.2f, Success: %b%n", transactionId, amount, success); } public void stop() { running = false; } public static void main(String[] args) throws Exception { JFRStreamingDemo streamingDemo = new JFRStreamingDemo(); // Start event streaming streamingDemo.startEventStreaming(event -> { // Custom event processing if (event.getEventType().getName().contains("BusinessTransaction")) { // Additional business logic here } }); // Generate some events generateTestEvents(); // Keep running for a while Thread.sleep(30000); streamingDemo.stop(); System.out.println("JFR streaming demo completed"); } private static void generateTestEvents() { Thread eventGenerator = new Thread(() -> { Random random = new Random(); int eventCount = 0; while (eventCount < 50) { BusinessTransactionEvent event = new BusinessTransactionEvent(); event.begin(); event.transactionId = "STREAM-TXN-" + eventCount; event.transactionType = "STREAMING"; event.amount = random.nextDouble() * 1000; event.success = random.nextBoolean(); event.commit(); eventCount++; try { Thread.sleep(random.nextInt(500)); } catch (InterruptedException e) { Thread.currentThread().interrupt(); break; } } }); eventGenerator.setDaemon(true); eventGenerator.start(); } } Performance Monitoring with JFR
Example 6: Application Performance Monitoring
import jdk.jfr.*; import java.util.concurrent.atomic.*; public class ApplicationPerformanceMonitor { private static final AtomicLong totalTransactions = new AtomicLong(); private static final AtomicLong failedTransactions = new AtomicLong(); private static final AtomicLong totalProcessingTime = new AtomicLong(); @Name("com.example.ApplicationMetrics") @Label("Application Performance Metrics") @Category("Performance") @Description("Application-level performance metrics") static class ApplicationMetricsEvent extends Event { @Label("Total Transactions") public long totalTransactions; @Label("Failed Transactions") public long failedTransactions; @Label("Success Rate") public double successRate; @Label("Average Processing Time") public double avgProcessingTimeMs; @Label("Transactions Per Second") public double transactionsPerSecond; } public static void recordTransaction(boolean success, long processingTimeMs) { totalTransactions.incrementAndGet(); if (!success) { failedTransactions.incrementAndGet(); } totalProcessingTime.addAndGet(processingTimeMs); } public static void emitPerformanceMetrics() { ApplicationMetricsEvent metrics = new ApplicationMetricsEvent(); metrics.begin(); long total = totalTransactions.get(); long failed = failedTransactions.get(); long totalTime = totalProcessingTime.get(); metrics.totalTransactions = total; metrics.failedTransactions = failed; metrics.successRate = total > 0 ? (double)(total - failed) / total * 100 : 100.0; metrics.avgProcessingTimeMs = total > 0 ? (double)totalTime / total : 0.0; // This would typically be calculated based on time window metrics.transactionsPerSecond = Math.random() * 100; // Simulated metrics.commit(); System.out.printf("Performance Metrics - Total: %d, Failed: %d, Success Rate: %.1f%%%n", total, failed, metrics.successRate); } public static void startPeriodicMetricsEmitter() { Thread emitterThread = new Thread(() -> { while (!Thread.currentThread().isInterrupted()) { try { emitPerformanceMetrics(); Thread.sleep(5000); // Emit every 5 seconds } catch (InterruptedException e) { Thread.currentThread().interrupt(); break; } } }); emitterThread.setDaemon(true); emitterThread.start(); } public static void main(String[] args) throws Exception { // Start JFR recording Recording recording = new Recording(); recording.setName("Performance Monitoring"); recording.setDestination(Paths.get("performance-monitoring.jfr")); recording.setDumpOnExit(true); recording.enable(ApplicationMetricsEvent.class); recording.enable("jdk.CPULoad").withPeriod(Duration.ofSeconds(2)); recording.enable("jdk.GarbageCollection"); recording.start(); // Start metrics emitter startPeriodicMetricsEmitter(); // Simulate application workload simulateWorkload(); // Stop recording Thread.sleep(30000); recording.stop(); System.out.println("Performance monitoring completed"); } private static void simulateWorkload() { Thread workloadThread = new Thread(() -> { Random random = new Random(); int transactionCount = 0; while (transactionCount < 100) { long startTime = System.currentTimeMillis(); // Simulate transaction processing boolean success = random.nextDouble() > 0.1; // 90% success rate long processingTime = random.nextInt(100) + 50; // 50-150ms try { Thread.sleep(processingTime); } catch (InterruptedException e) { Thread.currentThread().interrupt(); break; } long actualProcessingTime = System.currentTimeMillis() - startTime; recordTransaction(success, actualProcessingTime); transactionCount++; // Random delay between transactions try { Thread.sleep(random.nextInt(200)); } catch (InterruptedException e) { Thread.currentThread().interrupt(); break; } } }); workloadThread.start(); try { workloadThread.join(); } catch (InterruptedException e) { Thread.currentThread().interrupt(); } } } Best Practices for JFR in Native Images
- Event Design: Keep custom events simple and focused
- Event Volume: Be mindful of event generation rate
- Configuration: Use appropriate recording configurations
- Resource Management: Properly manage recording lifecycle
- Testing: Test JFR functionality in both JVM and native image
- Monitoring: Monitor the impact of JFR on application performance
Limitations and Considerations
- Reduced Event Set: Fewer events available compared to JVM
- Build Time Configuration: Some JFR features require build-time setup
- Performance Impact: Still has some overhead, though reduced
- Tooling Support: JMC and other tools may have limited native image support
Conclusion
JFR in Native Images provides powerful performance monitoring capabilities:
Key Benefits:
- Production Ready: Low-overhead profiling in production
- Custom Events: Business-specific monitoring
- Integration: Works with existing JFR ecosystem
- Native Performance: Optimized for AOT compilation
Use Cases:
- Performance monitoring in production
- Business transaction tracking
- Resource utilization analysis
- Latency and throughput measurement
- Capacity planning and optimization
Best Practices:
- Design focused, meaningful events
- Use appropriate recording configurations
- Monitor JFR overhead
- Integrate with existing monitoring systems
JFR in Native Images bridges the gap between traditional JVM monitoring and native performance, enabling comprehensive observability for GraalVM native applications.
Next Steps: Experiment with custom events for your specific use case, integrate JFR monitoring into your production deployment, and use JMC to analyze the recorded data for performance insights.