Black Duck is a comprehensive security and license compliance solution that helps organizations identify and manage open source security vulnerabilities, license compliance issues, and operational risks in their software projects.
What is Black Duck?
Black Duck provides:
- Open Source Detection: Automatically discovers open source components
- Vulnerability Assessment: Identifies known security vulnerabilities
- License Compliance: Analyzes license obligations and conflicts
- Risk Management: Prioritizes and tracks remediation efforts
- Policy Enforcement: Enforces organizational security policies
Black Duck Architecture
Source Code/Binaries → Black Duck Scanner → Black Duck Hub → Reports & Dashboards ↑ ↑ ↑ ↑ Application Detection Engine Central Database Security Teams Artifacts Vulnerability DB License DB Development Teams
Getting Started with Black Duck
Installation and Setup
# Download Black Duck CLI (Detect) curl -s https://detect.synopsys.com/detect.sh | bash # Or use Docker image docker pull blackducksoftware/blackduck-detect:latest # Set environment variables export BLACKDUCK_URL=https://your-blackduck-instance.com export BLACKDUCK_API_TOKEN=your-api-token export BLACKDUCK_TRUST_CERT=true
Black Duck Scanning Methods
Method 1: Basic Java Project Scan
# For Maven projects java -jar synopsys-detect-8.0.0.jar \ --blackduck.url=$BLACKDUCK_URL \ --blackduck.api.token=$BLACKDUCK_API_TOKEN \ --detect.project.name="MyJavaApp" \ --detect.project.version="1.0.0" \ --detect.tools=DETECTOR \ --detect.maven.build.command="clean compile" # For Gradle projects java -jar synopsys-detect-8.0.0.jar \ --blackduck.url=$BLACKDUCK_URL \ --blackduck.api.token=$BLACKDUCK_API_TOKEN \ --detect.project.name="MyJavaApp" \ --detect.project.version="1.0.0" \ --detect.tools=DETECTOR \ --detect.gradle.build.command="build"
Method 2: Docker-based Scanning
# Scan using Docker container docker run -it --rm \ -v $(pwd):/source \ -e BLACKDUCK_URL=$BLACKDUCK_URL \ -e BLACKDUCK_API_TOKEN=$BLACKDUCK_API_TOKEN \ blackducksoftware/blackduck-detect:latest \ --detect.project.name="MyJavaApp" \ --detect.project.version="1.0.0" \ --detect.source.path="/source"
Integration with Build Tools
Example 1: Maven Integration
<!-- pom.xml - Maven configuration -->
<project>
<build>
<plugins>
<!-- Black Duck Detect Maven Plugin -->
<plugin>
<groupId>com.synopsys.integration</groupId>
<artifactId>blackduck-maven-plugin</artifactId>
<version>2.0.0</version>
<configuration>
<blackduckUrl>https://your-blackduck-instance.com</blackduckUrl>
<blackduckApiToken>${blackduck.api.token}</blackduckApiToken>
<projectName>${project.name}</projectName>
<projectVersion>${project.version}</projectVersion>
<failOnPolicyViolation>true</failOnPolicyViolation>
<outputDirectory>${project.build.directory}/blackduck</outputDirectory>
</configuration>
<executions>
<execution>
<goals>
<goal>run</goal>
</goals>
<phase>verify</phase>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Example 2: Gradle Integration
// build.gradle - Gradle configuration
plugins {
id 'com.synopsys.integration.blackduck' version '2.0.0'
}
blackduck {
url = 'https://your-blackduck-instance.com'
apiToken = System.getenv('BLACKDUCK_API_TOKEN')
projectName = project.name
projectVersion = project.version
failOnPolicyViolation = true
outputDirectory = "${buildDir}/blackduck"
scan {
paths = ["${projectDir}"]
}
}
Advanced Scanning Configuration
Example 3: Comprehensive Detect Configuration
#!/bin/bash # comprehensive-blackduck-scan.sh java -jar synopsys-detect-8.0.0.jar \ --blackduck.url=$BLACKDUCK_URL \ --blackduck.api.token=$BLACKDUCK_API_TOKEN \ --blackduck.trust.cert=true \ # Project Information --detect.project.name="my-java-application" \ --detect.project.version="1.0.0-$BUILD_NUMBER" \ --detect.project.tier="PRODUCTION" \ # Scanning Configuration --detect.tools=DETECTOR,BINARY_SCAN \ --detect.detector.search.depth=10 \ --detect.excluded.directories="test,node_modules,.git" \ # Vulnerability Management --detect.policy.check.fail.on.severities="BLOCKER,CRITICAL,HIGH" \ --detect.risk.check.fail.on.severities="HIGH" \ # Reporting --detect.output.path="./blackduck-reports" \ --detect.report.format=JSON,HTML \ # Notifications --detect.notification.email.addresses="[email protected]" \ --detect.notification.email.subject="Black Duck Scan Results" \ # Advanced Options --detect.parallel.processors=4 \ --detect.timeout=600 \ --detect.cleanup=true
CI/CD Integration
Example 4: Jenkins Pipeline Integration
// Jenkinsfile
pipeline {
agent any
environment {
BLACKDUCK_URL = 'https://your-blackduck-instance.com'
BLACKDUCK_API_TOKEN = credentials('blackduck-api-token')
}
stages {
stage('Build') {
steps {
sh 'mvn clean compile -DskipTests'
}
}
stage('Black Duck Scan') {
steps {
script {
// Download Detect
sh 'curl -s https://detect.synopsys.com/detect.sh | bash'
// Run Black Duck Scan
sh """
java -jar synopsys-detect-*.jar \
--blackduck.url=${BLACKDUCK_URL} \
--blackduck.api.token=${BLACKDUCK_API_TOKEN} \
--detect.project.name="${env.JOB_NAME}" \
--detect.project.version="${env.BUILD_NUMBER}" \
--detect.policy.check.fail.on.severities="BLOCKER,CRITICAL" \
--detect.risk.check.fail.on.severities="HIGH" \
--detect.report.format=JSON,HTML
"""
}
}
post {
always {
// Archive reports
archiveArtifacts artifacts: 'blackduck-reports/*', allowEmptyArchive: true
// Publish HTML reports
publishHTML([
allowMissing: true,
alwaysLinkToLastBuild: true,
keepAll: true,
reportDir: 'blackduck-reports',
reportFiles: 'report.html',
reportName: 'Black Duck Security Report'
])
}
}
}
stage('Security Gate') {
steps {
script {
// Check for policy violations
def scanResult = readJSON file: 'blackduck-reports/scan_result.json'
if (scanResult.policyViolations > 0) {
error "Black Duck policy violations detected: ${scanResult.policyViolations}"
}
}
}
}
}
}
Example 5: GitHub Actions Integration
# .github/workflows/blackduck-scan.yml
name: Black Duck Security Scan
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main ]
schedule:
- cron: '0 0 * * 0' # Weekly scan
jobs:
security-scan:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Set up Java
uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'temurin'
- name: Build project
run: mvn clean compile -DskipTests
- name: Run Black Duck Scan
run: |
curl -s https://detect.synopsys.com/detect.sh | bash
java -jar synopsys-detect-*.jar \
--blackduck.url=${{ secrets.BLACKDUCK_URL }} \
--blackduck.api.token=${{ secrets.BLACKDUCK_API_TOKEN }} \
--detect.project.name="${{ github.event.repository.name }}" \
--detect.project.version="${{ github.sha }}" \
--detect.policy.check.fail.on.severities="BLOCKER,CRITICAL" \
--detect.report.format=JSON
- name: Upload Security Report
uses: actions/upload-artifact@v3
with:
name: blackduck-report
path: blackduck-reports/
- name: Check for Critical Vulnerabilities
run: |
if [ -f blackduck-reports/scan_result.json ]; then
VIOLATIONS=$(jq '.policyViolations' blackduck-reports/scan_result.json)
if [ "$VIOLATIONS" -gt 0 ]; then
echo "Critical vulnerabilities found: $VIOLATIONS"
exit 1
fi
fi
Custom Policy Configuration
Example 6: Black Duck Policy Rules
{
"name": "Java Security Policy",
"description": "Security policy for Java applications",
"overallConditions": {
"operator": "OR",
"expressions": [
{
"name": "Critical Vulnerabilities",
"conditionType": "VULNERABILITY",
"severityLevels": ["CRITICAL"],
"vulnerabilityCountThreshold": 0
},
{
"name": "High Risk Licenses",
"conditionType": "LICENSE",
"licenseRiskProfile": {
"highRiskCountThreshold": 0
}
}
]
},
"componentConditions": [
{
"name": "No Banned Components",
"conditionType": "COMPONENT",
"bannedComponents": [
"log4j:log4j:1.2.17",
"commons-collections:commons-collections:3.1"
]
},
{
"name": "Vulnerability Age",
"conditionType": "VULNERABILITY",
"vulnerabilityAgeThreshold": 90,
"severityLevels": ["HIGH", "CRITICAL"]
}
]
}
Remediation Workflow
Example 7: Automated Remediation Script
#!/bin/bash # blackduck-remediation.sh PROJECT_NAME="my-java-app" PROJECT_VERSION="1.0.0" BLACKDUCK_URL="https://your-blackduck-instance.com" API_TOKEN=$BLACKDUCK_API_TOKEN # Get vulnerability report curl -X GET \ -H "Authorization: Bearer $API_TOKEN" \ "$BLACKDUCK_URL/api/projects?q=name:$PROJECT_NAME" > project_list.json PROJECT_ID=$(jq -r '.items[0]._meta.href' project_list.json | sed 's|.*/||') # Get components with vulnerabilities curl -X GET \ -H "Authorization: Bearer $API_TOKEN" \ "$BLACKDUCK_URL/api/projects/$PROJECT_ID/versions/$PROJECT_VERSION/components" > components.json # Process vulnerable components jq -r '.items[] | select(.securityRiskProfile != null) | "\(.componentName):\(.componentVersionName) - \(.securityRiskProfile)"' components.json | while read component; do echo "Processing: $component" # Extract component details COMPONENT_NAME=$(echo $component | cut -d':' -f1) COMPONENT_VERSION=$(echo $component | cut -d':' -f2 | cut -d' ' -f1) # Check for available upgrades curl -X GET \ -H "Authorization: Bearer $API_TOKEN" \ "$BLACKDUCK_URL/api/components?q=name:$COMPONENT_NAME" > component_versions.json LATEST_VERSION=$(jq -r '.items[0].versionName' component_versions.json) if [ "$COMPONENT_VERSION" != "$LATEST_VERSION" ]; then echo "Upgrade available: $COMPONENT_NAME from $COMPONENT_VERSION to $LATEST_VERSION" # Auto-update in pom.xml (for Maven) sed -i "s/<$COMPONENT_NAME.version>$COMPONENT_VERSION<\/$COMPONENT_NAME.version>/<$COMPONENT_NAME.version>$LATEST_VERSION<\/$COMPONENT_NAME.version>/" pom.xml fi done # Generate remediation report echo "Remediation actions completed. Please review and commit changes."
Reporting and Dashboards
Example 8: Custom Reporting Script
#!/usr/bin/env python3
# blackduck-report-generator.py
import requests
import json
import csv
from datetime import datetime
class BlackDuckReporter:
def __init__(self, base_url, token):
self.base_url = base_url
self.headers = {
'Authorization': f'Bearer {token}',
'Content-Type': 'application/json'
}
def get_project_vulnerabilities(self, project_name, version_name):
# Get project ID
projects_url = f"{self.base_url}/api/projects"
response = requests.get(projects_url, headers=self.headers)
projects = response.json()
project_id = None
for project in projects['items']:
if project['name'] == project_name:
project_id = project['_meta']['href'].split('/')[-1]
break
if not project_id:
raise Exception(f"Project {project_name} not found")
# Get version ID
versions_url = f"{self.base_url}/api/projects/{project_id}/versions"
response = requests.get(versions_url, headers=self.headers)
versions = response.json()
version_id = None
for version in versions['items']:
if version['versionName'] == version_name:
version_id = version['_meta']['href'].split('/')[-1]
break
if not version_id:
raise Exception(f"Version {version_name} not found")
# Get vulnerability report
vuln_url = f"{self.base_url}/api/projects/{project_id}/versions/{version_id}/vulnerabilities"
response = requests.get(vuln_url, headers=self.headers)
return response.json()
def generate_csv_report(self, vulnerabilities, output_file):
with open(output_file, 'w', newline='') as csvfile:
fieldnames = ['Component', 'Version', 'Vulnerability', 'Severity', 'CVSS', 'Published', 'Description']
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
writer.writeheader()
for vuln in vulnerabilities.get('items', []):
for affected_component in vuln.get('affectedComponents', []):
writer.writerow({
'Component': affected_component.get('componentName', ''),
'Version': affected_component.get('componentVersionName', ''),
'Vulnerability': vuln.get('vulnerabilityName', ''),
'Severity': vuln.get('severity', ''),
'CVSS': vuln.get('cvssScore', ''),
'Published': vuln.get('publishedDate', ''),
'Description': vuln.get('description', '')
})
# Usage
if __name__ == "__main__":
reporter = BlackDuckReporter(
base_url="https://your-blackduck-instance.com",
token="your-api-token"
)
vulnerabilities = reporter.get_project_vulnerabilities(
project_name="my-java-application",
version_name="1.0.0"
)
reporter.generate_csv_report(vulnerabilities, "security-report.csv")
print("Security report generated: security-report.csv")
Best Practices for Black Duck Scanning
1. Comprehensive Scanning Strategy
#!/bin/bash # comprehensive-scan-strategy.sh # 1. Pre-scan: Dependency check echo "Running pre-scan dependency analysis..." mvn dependency:tree > dependency-tree.txt # 2. Main Black Duck scan echo "Running Black Duck security scan..." java -jar synopsys-detect-*.jar \ --blackduck.url=$BLACKDUCK_URL \ --blackduck.api.token=$BLACKDUCK_API_TOKEN \ --detect.project.name="$PROJECT_NAME" \ --detect.project.version="$VERSION" \ --detect.tools=DETECTOR,BINARY_SCAN,IMPACT_ANALYSIS \ --detect.policy.check.fail.on.severities="BLOCKER,CRITICAL" \ --detect.risk.check.fail.on.severities="HIGH" \ --detect.report.format=JSON,HTML,CSV # 3. Post-scan: Generate remediation plan echo "Generating remediation plan..." python3 generate-remediation-plan.py blackduck-reports/scan_result.json
2. Security Gates in CI/CD
# Security gate configuration security_gates: vulnerability_severity: blocker: 0 critical: 0 high: 5 medium: 20 low: 100 license_risk: high: 0 medium: 3 low: 10 operational_risk: high: 0 medium: 5 low: 15
Common Java Vulnerabilities Detected
Black Duck effectively identifies:
- Log4j Vulnerabilities (CVE-2021-44228, etc.)
- Apache Commons Vulnerabilities
- Spring Framework Security Issues
- Jackson Deserialization Vulnerabilities
- Tomcat Security Flaws
- Hibernate Security Issues
- JUnit Vulnerabilities
- SLF4J Security Problems
Conclusion
Black Duck provides comprehensive security scanning for Java applications:
Key Benefits:
- Comprehensive Detection: Identifies all open source components
- Vulnerability Intelligence: Leverages extensive vulnerability database
- License Compliance: Manages open source license obligations
- Risk Prioritization: Helps focus on critical issues
- Integration Friendly: Works with existing development workflows
Implementation Strategy:
- Initial Setup: Configure Black Duck instance and API access
- CI/CD Integration: Embed scanning in build pipelines
- Policy Definition: Establish security and license policies
- Remediation Workflow: Create processes for addressing findings
- Continuous Monitoring: Regular scanning and reporting
Best Practices:
- Scan early and often in development lifecycle
- Fail builds on critical security violations
- Maintain updated vulnerability databases
- Train development teams on interpreting results
- Establish clear remediation workflows
By implementing Black Duck security scanning, organizations can significantly improve their software security posture, ensure license compliance, and reduce the risk of deploying vulnerable applications to production.
Next Steps: Start with a pilot project, establish baseline security policies, integrate into your CI/CD pipeline, and gradually expand coverage across all Java applications in your organization.