Decision Trees in Weka: Complete Java Implementation Guide

Weka (Waikato Environment for Knowledge Analysis) is a popular machine learning library written in Java. Decision trees are one of the most interpretable and widely used machine learning algorithms. Here's a comprehensive guide to implementing decision trees using Weka in Java.

Weka Setup and Dependencies

Maven Dependencies

<dependency> <groupId>nz.ac.waikato.cms.weka</groupId> <artifactId>weka-stable</artifactId> <version>3.8.6</version> </dependency> <!-- Or for the latest version --> <dependency> <groupId>nz.ac.waikato.cms.weka</groupId> <artifactId>weka-dev</artifactId> <version>3.9.7</version> </dependency>

Manual JAR Download

Download Weka JAR from Weka Website and add to classpath.

Basic Decision Tree Implementation

Example 1: Basic J48 Decision Tree

import weka.classifiers.trees.J48; import weka.core.Instances; import weka.core.converters.ConverterUtils.DataSource; import weka.core.SerializationHelper; import java.io.File; public class BasicDecisionTree { public static void main(String[] args) { try { // Load dataset DataSource source = new DataSource("data/weather.nominal.arff"); Instances data = source.getDataSet(); // Set the class index (the attribute we want to predict) if (data.classIndex() == -1) { data.setClassIndex(data.numAttributes() - 1); } System.out.println("Dataset loaded successfully:"); System.out.println("Number of instances: " + data.numInstances()); System.out.println("Number of attributes: " + data.numAttributes()); System.out.println("Class attribute: " + data.classAttribute().name()); // Build J48 decision tree classifier J48 tree = new J48(); // Set options (similar to command-line options) String[] options = {"-U"}; // Unpruned tree tree.setOptions(options); // Build classifier tree.buildClassifier(data); // Print the decision tree System.out.println("\n=== Decision Tree ==="); System.out.println(tree); // Evaluate the model evaluateModel(tree, data); } catch (Exception e) { e.printStackTrace(); } } private static void evaluateModel(J48 tree, Instances data) throws Exception { System.out.println("\n=== Model Evaluation ==="); // Cross-validation evaluation weka.classifiers.Evaluation eval = new weka.classifiers.Evaluation(data); eval.crossValidateModel(tree, data, 10, new java.util.Random(1)); // Print evaluation results System.out.println("Correctly Classified Instances: " + eval.correct() + " (" + eval.pctCorrect() + "%)"); System.out.println("Incorrectly Classified Instances: " + eval.incorrect() + " (" + eval.pctIncorrect() + "%)"); System.out.println("Kappa statistic: " + eval.kappa()); System.out.println("Mean absolute error: " + eval.meanAbsoluteError()); System.out.println("Root mean squared error: " + eval.rootMeanSquaredError()); System.out.println("Relative absolute error: " + eval.relativeAbsoluteError() + "%"); System.out.println("Root relative squared error: " + eval.rootRelativeSquaredError() + "%"); System.out.println("Total Number of Instances: " + eval.numInstances()); // Detailed accuracy by class System.out.println("\n=== Detailed Accuracy By Class ==="); System.out.println(eval.toClassDetailsString()); // Confusion matrix System.out.println("=== Confusion Matrix ==="); double[][] confusionMatrix = eval.confusionMatrix(); for (double[] row : confusionMatrix) { for (double val : row) { System.out.print((int) val + "\t"); } System.out.println(); } } }

Example 2: Advanced Decision Tree with Custom Configuration

import weka.classifiers.trees.J48; import weka.core.Instances; import weka.core.converters.ConverterUtils.DataSource; import weka.filters.Filter; import weka.filters.unsupervised.attribute.Remove; import java.util.Random; public class AdvancedDecisionTree { public static void main(String[] args) { try { // Load dataset DataSource source = new DataSource("data/iris.arff"); Instances data = source.getDataSet(); // Set class index (last attribute for iris dataset) data.setClassIndex(data.numAttributes() - 1); System.out.println("Dataset: " + data.relationName()); System.out.println("Number of instances: " + data.numInstances()); System.out.println("Number of attributes: " + data.numAttributes()); System.out.println("Class distribution: " + data.attributeStats(data.classIndex()).toString()); // Create and configure J48 classifier with various options J48 tree = new J48(); // Set multiple options String[] options = { "-C", "0.25", // Confidence factor for pruning (default 0.25) "-M", "2", // Minimum number of instances per leaf (default 2) "-R", // Use reduced error pruning "-N", "3", // Number of folds for reduced error pruning "-B", // Use binary splits for nominal attributes "-S", // Don't perform subtree raising "-L", // Do not clean up after the tree has been built "-A", // Laplace smoothing for predicted probabilities "-Q", "1" // Seed for random data shuffling }; tree.setOptions(options); // Alternative: Set options individually tree.setConfidenceFactor(0.1f); tree.setMinNumObj(5); tree.setUnpruned(false); tree.setUseLaplace(true); // Build classifier tree.buildClassifier(data); // Print the tree System.out.println("\n=== Configured Decision Tree ==="); System.out.println(tree); // Print tree configuration System.out.println("\n=== Tree Configuration ==="); System.out.println("Confidence factor: " + tree.getConfidenceFactor()); System.out.println("Minimum instances per leaf: " + tree.getMinNumObj()); System.out.println("Unpruned: " + tree.getUnpruned()); System.out.println("Use Laplace: " + tree.getUseLaplace()); System.out.println("Reduced error pruning: " + tree.getReducedErrorPruning()); // Advanced evaluation performAdvancedEvaluation(tree, data); } catch (Exception e) { e.printStackTrace(); } } private static void performAdvancedEvaluation(J48 tree, Instances data) throws Exception { System.out.println("\n=== Advanced Evaluation ==="); weka.classifiers.Evaluation eval = new weka.classifiers.Evaluation(data); // 10-fold cross-validation eval.crossValidateModel(tree, data, 10, new Random(42)); // Print comprehensive evaluation metrics System.out.println("Summary Statistics:"); System.out.println("Correctly classified: " + String.format("%.2f", eval.pctCorrect()) + "%"); System.out.println("Incorrectly classified: " + String.format("%.2f", eval.pctIncorrect()) + "%"); System.out.println("Kappa: " + String.format("%.3f", eval.kappa())); System.out.println("MAE: " + String.format("%.3f", eval.meanAbsoluteError())); System.out.println("RMSE: " + String.format("%.3f", eval.rootMeanSquaredError())); System.out.println("RAE: " + String.format("%.2f", eval.relativeAbsoluteError()) + "%"); System.out.println("RRSE: " + String.format("%.2f", eval.rootRelativeSquaredError()) + "%"); // Precision, Recall, F-measure for each class System.out.println("\n=== Per-Class Statistics ==="); String[] classLabels = new String[data.classAttribute().numValues()]; for (int i = 0; i < classLabels.length; i++) { classLabels[i] = data.classAttribute().value(i); } System.out.printf("%-15s %-10s %-10s %-10s %-10s%n", "Class", "Precision", "Recall", "F-Measure", "ROC Area"); for (int i = 0; i < classLabels.length; i++) { System.out.printf("%-15s %-10.3f %-10.3f %-10.3f %-10.3f%n", classLabels[i], eval.precision(i), eval.recall(i), eval.fMeasure(i), eval.areaUnderROC(i)); } // Weighted averages System.out.printf("%-15s %-10.3f %-10.3f %-10.3f %-10.3f%n", "Weighted Avg.", eval.weightedPrecision(), eval.weightedRecall(), eval.weightedFMeasure(), eval.weightedAreaUnderROC()); } }

Working with Different Data Sources

Example 3: Loading Data from Various Formats

import weka.core.Instances; import weka.core.converters.ConverterUtils.DataSource; import weka.core.converters.CSVSaver; import weka.core.converters.ArffSaver; import weka.classifiers.trees.J48; import weka.filters.Filter; import weka.filters.unsupervised.attribute.NumericToNominal; import java.io.File; public class DataLoadingExamples { public static void main(String[] args) { try { // Example 1: Load from ARFF file System.out.println("=== Loading ARFF File ==="); loadARFFData("data/weather.nominal.arff"); // Example 2: Load from CSV file System.out.println("\n=== Loading CSV File ==="); loadCSVData("data/iris.csv"); // Example 3: Create dataset programmatically System.out.println("\n=== Creating Dataset Programmatically ==="); createProgrammaticDataset(); } catch (Exception e) { e.printStackTrace(); } } private static void loadARFFData(String filePath) throws Exception { DataSource source = new DataSource(filePath); Instances data = source.getDataSet(); // Set class index (assuming last attribute) if (data.classIndex() == -1) { data.setClassIndex(data.numAttributes() - 1); } printDatasetInfo(data); buildAndEvaluateTree(data); } private static void loadCSVData(String filePath) throws Exception { DataSource source = new DataSource(filePath); Instances data = source.getDataSet(); // For CSV, we might need to set types and class index System.out.println("Original data: " + data.numInstances() + " instances, " + data.numAttributes() + " attributes"); // Convert numeric class to nominal if needed if (data.classIndex() != -1 && data.classAttribute().isNumeric()) { NumericToNominal filter = new NumericToNominal(); filter.setAttributeIndices("last"); filter.setInputFormat(data); data = Filter.useFilter(data, filter); } // Set class index to last attribute data.setClassIndex(data.numAttributes() - 1); printDatasetInfo(data); buildAndEvaluateTree(data); // Save as ARFF for future use saveAsARFF(data, "data/converted_iris.arff"); } private static void createProgrammaticDataset() throws Exception { // Create attributes java.util.ArrayList<weka.core.Attribute> attributes = new java.util.ArrayList<>(); // Add numeric attributes attributes.add(new weka.core.Attribute("age")); attributes.add(new weka.core.Attribute("income")); // Add nominal attribute java.util.ArrayList<String> genderValues = new java.util.ArrayList<>(); genderValues.add("Male"); genderValues.add("Female"); attributes.add(new weka.core.Attribute("gender", genderValues)); // Add class attribute java.util.ArrayList<String> classValues = new java.util.ArrayList<>(); classValues.add("Yes"); classValues.add("No"); attributes.add(new weka.core.Attribute("buys_computer", classValues)); // Create dataset Instances data = new Instances("ComputerBuying", attributes, 0); data.setClassIndex(data.numAttributes() - 1); // Add instances addInstance(data, new double[]{25, 35000, 0, 0}); // 0=Male, 0=Yes addInstance(data, new double[]{35, 45000, 1, 1}); // 1=Female, 1=No addInstance(data, new double[]{45, 55000, 0, 0}); addInstance(data, new double[]{20, 25000, 1, 1}); addInstance(data, new double[]{55, 65000, 0, 0}); addInstance(data, new double[]{30, 30000, 1, 0}); printDatasetInfo(data); buildAndEvaluateTree(data); } private static void addInstance(Instances data, double[] values) { data.add(new weka.core.DenseInstance(1.0, values)); } private static void printDatasetInfo(Instances data) { System.out.println("Dataset: " + data.relationName()); System.out.println("Number of instances: " + data.numInstances()); System.out.println("Number of attributes: " + data.numAttributes()); System.out.println("Class attribute: " + data.classAttribute().name()); // Print attribute information System.out.println("Attributes:"); for (int i = 0; i < data.numAttributes(); i++) { weka.core.Attribute attr = data.attribute(i); System.out.println(" " + attr.name() + " (" + (attr.isNumeric() ? "Numeric" : "Nominal") + ")"); } } private static void buildAndEvaluateTree(Instances data) throws Exception { J48 tree = new J48(); tree.buildClassifier(data); weka.classifiers.Evaluation eval = new weka.classifiers.Evaluation(data); eval.crossValidateModel(tree, data, 5, new java.util.Random(1)); System.out.println("Accuracy: " + String.format("%.2f", eval.pctCorrect()) + "%"); System.out.println("Tree size: " + tree.measureTreeSize()); System.out.println("Number of leaves: " + tree.measureNumLeaves()); } private static void saveAsARFF(Instances data, String filePath) throws Exception { ArffSaver saver = new ArffSaver(); saver.setInstances(data); saver.setFile(new File(filePath)); saver.writeBatch(); System.out.println("Dataset saved as: " + filePath); } }

Model Persistence and Prediction

Example 4: Saving and Loading Models

import weka.classifiers.trees.J48; import weka.core.Instances; import weka.core.converters.ConverterUtils.DataSource; import weka.core.SerializationHelper; import java.io.File; public class ModelPersistence { public static void main(String[] args) { try { String datasetPath = "data/weather.nominal.arff"; String modelPath = "models/decision_tree.model"; // Train and save model J48 trainedModel = trainAndSaveModel(datasetPath, modelPath); // Load model and make predictions loadAndPredict(modelPath, datasetPath); // Compare models compareWithNewModel(datasetPath, modelPath); } catch (Exception e) { e.printStackTrace(); } } private static J48 trainAndSaveModel(String datasetPath, String modelPath) throws Exception { System.out.println("=== Training and Saving Model ==="); // Load data DataSource source = new DataSource(datasetPath); Instances data = source.getDataSet(); data.setClassIndex(data.numAttributes() - 1); // Train model J48 tree = new J48(); tree.buildClassifier(data); // Save model SerializationHelper.write(modelPath, tree); System.out.println("Model saved to: " + modelPath); // Print model info System.out.println("Tree size: " + tree.measureTreeSize()); System.out.println("Number of leaves: " + tree.measureNumLeaves()); return tree; } private static void loadAndPredict(String modelPath, String datasetPath) throws Exception { System.out.println("\n=== Loading Model and Making Predictions ==="); // Load model J48 loadedTree = (J48) SerializationHelper.read(modelPath); System.out.println("Model loaded from: " + modelPath); // Load data for prediction DataSource source = new DataSource(datasetPath); Instances data = source.getDataSet(); data.setClassIndex(data.numAttributes() - 1); System.out.println("Making predictions on " + data.numInstances() + " instances:"); System.out.printf("%-5s %-15s %-15s %-10s%n", "Inst#", "Actual", "Predicted", "Confidence"); // Make predictions for (int i = 0; i < data.numInstances(); i++) { double actual = data.instance(i).classValue(); String actualClass = data.classAttribute().value((int) actual); double prediction = loadedTree.classifyInstance(data.instance(i)); String predictedClass = data.classAttribute().value((int) prediction); // Get prediction distribution (confidence) double[] distribution = loadedTree.distributionForInstance(data.instance(i)); double confidence = distribution[(int) prediction]; System.out.printf("%-5d %-15s %-15s %-10.3f%n", i + 1, actualClass, predictedClass, confidence); // For demonstration, only show first 10 predictions if (i >= 9) { System.out.println("... (showing first 10 instances)"); break; } } } private static void compareWithNewModel(String datasetPath, String savedModelPath) throws Exception { System.out.println("\n=== Comparing Saved Model with New Model ==="); // Load data DataSource source = new DataSource(datasetPath); Instances data = source.getDataSet(); data.setClassIndex(data.numAttributes() - 1); // Load saved model J48 savedTree = (J48) SerializationHelper.read(savedModelPath); // Train new model with same configuration J48 newTree = new J48(); newTree.buildClassifier(data); // Compare evaluations weka.classifiers.Evaluation savedEval = new weka.classifiers.Evaluation(data); savedEval.evaluateModel(savedTree, data); weka.classifiers.Evaluation newEval = new weka.classifiers.Evaluation(data); newEval.evaluateModel(newTree, data); System.out.printf("%-25s %-10s %-10s%n", "Metric", "Saved Model", "New Model"); System.out.printf("%-25s %-10.2f %-10.2f%n", "Accuracy (%)", savedEval.pctCorrect(), newEval.pctCorrect()); System.out.printf("%-25s %-10.3f %-10.3f%n", "Kappa", savedEval.kappa(), newEval.kappa()); System.out.printf("%-25s %-10.3f %-10.3f%n", "MAE", savedEval.meanAbsoluteError(), newEval.meanAbsoluteError()); // Compare tree structures System.out.println("\nTree Structure Comparison:"); System.out.println("Saved model - Size: " + savedTree.measureTreeSize() + ", Leaves: " + savedTree.measureNumLeaves()); System.out.println("New model - Size: " + newTree.measureTreeSize() + ", Leaves: " + newTree.measureNumLeaves()); } }

Advanced Decision Tree Techniques

Example 5: Different Tree Algorithms and Ensemble Methods

import weka.classifiers.trees.J48; import weka.classifiers.trees.RandomForest; import weka.classifiers.trees.RandomTree; import weka.classifiers.trees.REPTree; import weka.classifiers.meta.Bagging; import weka.classifiers.meta.AdaBoostM1; import weka.core.Instances; import weka.core.converters.ConverterUtils.DataSource; import java.util.Random; public class AdvancedTreeAlgorithms { public static void main(String[] args) { try { String datasetPath = "data/iris.arff"; DataSource source = new DataSource(datasetPath); Instances data = source.getDataSet(); data.setClassIndex(data.numAttributes() - 1); System.out.println("=== Comparing Different Tree Algorithms ==="); // Compare various tree algorithms compareAlgorithms(data); // Ensemble methods System.out.println("\n=== Ensemble Methods ==="); testEnsembleMethods(data); } catch (Exception e) { e.printStackTrace(); } } private static void compareAlgorithms(Instances data) throws Exception { // J48 (C4.5 implementation) J48 j48 = new J48(); j48.setConfidenceFactor(0.25f); j48.setMinNumObj(2); // REPTree (Reduced Error Pruning Tree) REPTree repTree = new REPTree(); repTree.setMaxDepth(10); repTree.setMinNum(2.0); repTree.setNoPruning(false); // RandomTree RandomTree randomTree = new RandomTree(); randomTree.setKValue(0); randomTree.setMaxDepth(0); // 0 = unlimited // Test all algorithms ClassifierResult j48Result = evaluateClassifier(j48, "J48", data); ClassifierResult repTreeResult = evaluateClassifier(repTree, "REPTree", data); ClassifierResult randomTreeResult = evaluateClassifier(randomTree, "RandomTree", data); // Print comparison System.out.printf("%-15s %-10s %-10s %-10s %-10s%n", "Algorithm", "Accuracy", "Kappa", "Tree Size", "Leaves"); printResult(j48Result); printResult(repTreeResult); printResult(randomTreeResult); } private static void testEnsembleMethods(Instances data) throws Exception { // Bagging with J48 Bagging bagging = new Bagging(); bagging.setClassifier(new J48()); bagging.setNumIterations(10); // AdaBoost with Decision Stump AdaBoostM1 adaboost = new AdaBoostM1(); adaboost.setClassifier(new weka.classifiers.trees.DecisionStump()); adaboost.setNumIterations(10); // Random Forest RandomForest randomForest = new RandomForest(); randomForest.setNumIterations(100); randomForest.setMaxDepth(0); // Evaluate ensemble methods ClassifierResult baggingResult = evaluateClassifier(bagging, "Bagging(J48)", data); ClassifierResult adaboostResult = evaluateClassifier(adaboost, "AdaBoost", data); ClassifierResult rfResult = evaluateClassifier(randomForest, "RandomForest", data); System.out.printf("%-15s %-10s %-10s %-15s%n", "Ensemble", "Accuracy", "Kappa", "Description"); System.out.printf("%-15s %-10.2f %-10.3f %-15s%n", baggingResult.name, baggingResult.accuracy, baggingResult.kappa, "10 iterations"); System.out.printf("%-15s %-10.2f %-10.3f %-15s%n", adaboostResult.name, adaboostResult.accuracy, adaboostResult.kappa, "10 iterations"); System.out.printf("%-15s %-10.2f %-10.3f %-15s%n", rfResult.name, rfResult.accuracy, rfResult.kappa, "100 trees"); } private static ClassifierResult evaluateClassifier(weka.classifiers.Classifier classifier, String name, Instances data) throws Exception { weka.classifiers.Evaluation eval = new weka.classifiers.Evaluation(data); eval.crossValidateModel(classifier, data, 10, new Random(1)); ClassifierResult result = new ClassifierResult(); result.name = name; result.accuracy = eval.pctCorrect(); result.kappa = eval.kappa(); result.mae = eval.meanAbsoluteError(); result.rmse = eval.rootMeanSquaredError(); // For tree-specific metrics if (classifier instanceof J48) { J48 tree = (J48) classifier; tree.buildClassifier(data); // Need to build to get tree metrics result.treeSize = tree.measureTreeSize(); result.numLeaves = tree.measureNumLeaves(); } else if (classifier instanceof weka.classifiers.trees.REPTree) { weka.classifiers.trees.REPTree tree = (weka.classifiers.trees.REPTree) classifier; tree.buildClassifier(data); result.treeSize = tree.measureTreeSize(); result.numLeaves = tree.measureNumLeaves(); } return result; } private static void printResult(ClassifierResult result) { if (result.treeSize > 0) { System.out.printf("%-15s %-10.2f %-10.3f %-10.1f %-10.1f%n", result.name, result.accuracy, result.kappa, result.treeSize, result.numLeaves); } else { System.out.printf("%-15s %-10.2f %-10.3f %-10s %-10s%n", result.name, result.accuracy, result.kappa, "N/A", "N/A"); } } static class ClassifierResult { String name; double accuracy; double kappa; double mae; double rmse; double treeSize; double numLeaves; } }

Feature Selection and Preprocessing

Example 6: Preprocessing and Feature Selection

import weka.classifiers.trees.J48; import weka.core.Instances; import weka.core.converters.ConverterUtils.DataSource; import weka.filters.Filter; import weka.filters.unsupervised.attribute.Normalize; import weka.filters.unsupervised.attribute.Standardize; import weka.filters.unsupervised.attribute.Remove; import weka.filters.supervised.attribute.AttributeSelection; import weka.attributeSelection.CfsSubsetEval; import weka.attributeSelection.GreedyStepwise; import weka.attributeSelection.InfoGainAttributeEval; import weka.attributeSelection.Ranker; public class PreprocessingAndFeatureSelection { public static void main(String[] args) { try { String datasetPath = "data/iris.arff"; DataSource source = new DataSource(datasetPath); Instances data = source.getDataSet(); data.setClassIndex(data.numAttributes() - 1); System.out.println("=== Original Dataset ==="); printDatasetSummary(data); // Test different preprocessing techniques testNormalization(data); testStandardization(data); testFeatureSelection(data); testCombinedPreprocessing(data); } catch (Exception e) { e.printStackTrace(); } } private static void testNormalization(Instances data) throws Exception { System.out.println("\n=== Testing Normalization ==="); // Create normalized dataset Normalize normalizeFilter = new Normalize(); normalizeFilter.setInputFormat(data); Instances normalizedData = Filter.useFilter(data, normalizeFilter); normalizedData.setClassIndex(normalizedData.numAttributes() - 1); System.out.println("After normalization:"); printDatasetSummary(normalizedData); // Compare performance comparePerformance(data, normalizedData, "Original", "Normalized"); } private static void testStandardization(Instances data) throws Exception { System.out.println("\n=== Testing Standardization ==="); // Create standardized dataset Standardize standardizeFilter = new Standardize(); standardizeFilter.setInputFormat(data); Instances standardizedData = Filter.useFilter(data, standardizeFilter); standardizedData.setClassIndex(standardizedData.numAttributes() - 1); System.out.println("After standardization:"); printDatasetSummary(standardizedData); // Compare performance comparePerformance(data, standardizedData, "Original", "Standardized"); } private static void testFeatureSelection(Instances data) throws Exception { System.out.println("\n=== Testing Feature Selection ==="); // Method 1: Correlation-based Feature Selection AttributeSelection cfsFilter = new AttributeSelection(); CfsSubsetEval cfsEval = new CfsSubsetEval(); GreedyStepwise search = new GreedyStepwise(); search.setSearchBackwards(true); cfsFilter.setEvaluator(cfsEval); cfsFilter.setSearch(search); cfsFilter.setInputFormat(data); Instances cfsData = Filter.useFilter(data, cfsFilter); cfsData.setClassIndex(cfsData.numAttributes() - 1); System.out.println("After CFS feature selection:"); printDatasetSummary(cfsData); // Method 2: Information Gain ranking AttributeSelection igFilter = new AttributeSelection(); InfoGainAttributeEval igEval = new InfoGainAttributeEval(); Ranker ranker = new Ranker(); ranker.setNumToSelect(2); // Select top 2 features igFilter.setEvaluator(igEval); igFilter.setSearch(ranker); igFilter.setInputFormat(data); Instances igData = Filter.useFilter(data, igFilter); igData.setClassIndex(igData.numAttributes() - 1); System.out.println("After Information Gain feature selection:"); printDatasetSummary(igData); // Compare performance comparePerformance(data, cfsData, "Original", "CFS Selected"); comparePerformance(data, igData, "Original", "IG Selected"); } private static void testCombinedPreprocessing(Instances data) throws Exception { System.out.println("\n=== Testing Combined Preprocessing ==="); // Step 1: Feature selection AttributeSelection featureFilter = new AttributeSelection(); InfoGainAttributeEval eval = new InfoGainAttributeEval(); Ranker ranker = new Ranker(); ranker.setNumToSelect(3); featureFilter.setEvaluator(eval); featureFilter.setSearch(ranker); featureFilter.setInputFormat(data); Instances selectedData = Filter.useFilter(data, featureFilter); // Step 2: Standardization Standardize standardizeFilter = new Standardize(); standardizeFilter.setInputFormat(selectedData); Instances processedData = Filter.useFilter(selectedData, standardizeFilter); processedData.setClassIndex(processedData.numAttributes() - 1); System.out.println("After combined preprocessing:"); printDatasetSummary(processedData); comparePerformance(data, processedData, "Original", "Preprocessed"); } private static void printDatasetSummary(Instances data) { System.out.println("Instances: " + data.numInstances() + ", Attributes: " + data.numAttributes()); System.out.print("Attributes: "); for (int i = 0; i < data.numAttributes(); i++) { if (i == data.classIndex()) continue; System.out.print(data.attribute(i).name() + " "); } System.out.println(); // Print basic statistics for numeric attributes for (int i = 0; i < data.numAttributes(); i++) { if (data.attribute(i).isNumeric() && i != data.classIndex()) { System.out.printf(" %s: min=%.2f, max=%.2f, mean=%.2f%n", data.attribute(i).name(), data.attributeStats(i).numericStats.min, data.attributeStats(i).numericStats.max, data.attributeStats(i).numericStats.mean); } } } private static void comparePerformance(Instances originalData, Instances processedData, String originalName, String processedName) throws Exception { J48 originalTree = new J48(); J48 processedTree = new J48(); weka.classifiers.Evaluation originalEval = new weka.classifiers.Evaluation(originalData); originalEval.crossValidateModel(originalTree, originalData, 10, new java.util.Random(1)); weka.classifiers.Evaluation processedEval = new weka.classifiers.Evaluation(processedData); processedEval.crossValidateModel(processedTree, processedData, 10, new java.util.Random(1)); System.out.printf("Performance: %s=%.2f%%, %s=%.2f%%%n", originalName, originalEval.pctCorrect(), processedName, processedEval.pctCorrect()); } }

Practical Tips and Best Practices

Data Preparation: Always preprocess data and handle missing values
Cross-Validation: Use cross-validation for reliable performance estimates
Parameter Tuning: Experiment with different tree parameters
Feature Selection: Remove irrelevant features to improve performance
Model Interpretation: Use tree visualization for interpretability
Ensemble Methods: Consider Random Forest or Bagging for better performance

Common Decision Tree Parameters

Confidence Factor: Controls pruning aggressiveness (lower = more pruning)
Minimum Instances per Leaf: Prevents overfitting by requiring minimum samples
Maximum Depth: Limits tree depth to prevent overfitting
Number of Folds: For reduced error pruning

Conclusion

Weka provides a comprehensive implementation of decision trees with the J48 algorithm (C4.5 implementation). Key advantages include:

Easy to use with simple API
Comprehensive evaluation metrics
Model persistence for reuse
Multiple algorithms and ensemble methods
Extensive preprocessing capabilities

For most applications, start with J48 with default parameters and then experiment with preprocessing and parameter tuning based on your specific dataset and requirements.