Threat Hunting with Zeek – Log Types and Use cases

Zeek (formerly Bro) is a powerful open-source network monitoring and intrusion detection system that generates detailed logs about network traffic. Zeek logs contain valuable information about network activity, which can be analyzed to detect anomalies, threats, and trends. Here’s a detailed guide to analyzing Zeek logs effectively:

Zeek Log Types

Zeek generates a variety of logs, each focused on a specific type of network activity. The most common logs include:

a. Conn.log

• Purpose: Records all network connections.
• Key Fields:
  • id.orig_h / id.resp_h: Source and destination IP addresses.
  • id.orig_p / id.resp_p: Source and destination ports.
  • proto: Protocol (e.g., TCP, UDP).
  • service: Detected service (e.g., HTTP, DNS).
  • duration: Connection duration.
  • missed_bytes: Bytes missed during capture.
• Use Case: Baseline network activity, identify unusual connections, detect large volumes of data transfer.

b. Dns.log

• Purpose: Records DNS query activity.
• Key Fields:
  • query: The queried domain.
  • qtype_name: Query type (e.g., A, AAAA).
  • answers: Returned IP addresses.
  • rcode_name: Response code (e.g., NXDOMAIN for nonexistent domains).
• Use Case: Detect suspicious DNS requests, domain generation algorithms (DGA), or tunneling.

c. Http.log

• Purpose: Records HTTP traffic details.
• Key Fields:
  • host: Hostname of the request.
  • uri: Requested URI.
  • method: HTTP method (e.g., GET, POST).
  • status_code: HTTP response status code.
• Use Case: Identify unusual HTTP traffic, detect malware callbacks, analyze browsing activity.

d. Ssl.log

• Purpose: Records SSL/TLS activity.
• Key Fields:
  • version: TLS version (e.g., TLSv1.2).
  • cipher: Cipher suite used.
  • issuer: SSL certificate issuer.
• Use Case: Detect unencrypted traffic, outdated protocols used by malwares, or self-signed certificates.

e. Files.log

• Purpose: Tracks file transfers over the network.
• Key Fields:
  • filename: Name of the file (if available).
  • mime_type: File type.
  • md5: MD5 hash of the file.
• Use Case: Detect suspicious or malicious file transfers.

f. Notice.log

• Purpose: Summarizes key security-relevant events.
• Key Fields:
  • note: Type of notice (e.g., SSH brute force, DDoS detection).
  • msg: Detailed message about the event.
• Use Case: High-priority events, intrusion detection.

More log types check here: https://f.hubspotusercontent00.net/hubfs/8645105/Corelight_May2021/Pdf/002_CORELIGHT_080420_ZEEK_LOGS_US_ONLINE.pdf

Common Analysis Scenarios

a. Detect Anomalies in Network Traffic

• Identify connections with unusually long durations or large data transfers (conn.log).
• Look for high missed_bytes values, indicating potential evasion attempts.

b. Identify Malware Activity

• Look for uncommon DNS queries to suspicious domains (dns.log).
• Check for HTTP traffic with unusual User-Agent strings (http.log).

c. Detect Unauthorized Access

• Review authentication logs (ssh.log, kerberos.log) for failed login attempts.
• Identify unusual SSH connections or brute-force attempts (notice.log).

d. Spot Data Exfiltration

• Monitor large outbound file transfers (files.log).
• Look for unusual protocols used for large data volumes (conn.log).

Techniques for Effective Analysis

a. Baseline Normal Behavior

• Understand the usual traffic patterns, services, and domains in your network.
• Compare current data against historical baselines.

b. Correlate Across Logs

• Use the uid field to correlate events across different logs (e.g., conn.log ↔ http.log).

c. Automate and Script

• Write scripts (e.g., in Python) to automate repetitive tasks like filtering logs or detecting patterns.

d. Visualize Data

• Use tools like Kibana, Grafana, or Brim to create dashboards for better insights.

Prominent use cases:

1. Intrusion Detection

Zeek is widely used as a network intrusion detection system (NIDS), analyzing network traffic to identify malicious activity.

Examples:

• Detecting Brute Force Attacks: Use Zeek’s notice.log or ssh.log to identify repeated failed login attempts.
• Detecting Scans and Probes: Monitor conn.log for unusual patterns, like a single IP attempting to connect to many ports or hosts.
• Malware C2 Communication: Analyze http.log or dns.log for suspicious domains or irregular traffic patterns.

2. Threat Hunting

Zeek logs provide deep visibility into network traffic, making it ideal for proactive threat hunting.

Examples:

• Analyzing DNS Queries: Hunt for domain generation algorithm (DGA)-like patterns or known bad domains using dns.log.
• Suspicious File Transfers: Use files.log to detect potentially malicious files based on hash values or file types.
• Beaconing Detection: Look for periodic traffic in conn.log, which may indicate malware beaconing to a command-and-control server.

3. Network Forensics

Zeek captures and logs detailed network activity, helping investigators reconstruct events post-incident.

Examples:

• Incident Reconstruction: Use correlated logs (uid field) to trace the flow of an attack, e.g., a connection in conn.log leading to a file download in http.log.
• Data Exfiltration: Examine conn.log and files.log to identify large outbound data transfers or unusual file movements.
• Packet-Level Analysis: Combine Zeek with tools like Wireshark for deeper inspection of captured packets.

4. Monitoring and Anomaly Detection

Zeek helps baseline normal network behavior and identify deviations.

Examples:

• Service Identification: Use conn.log and service fields to map active services and detect unauthorized ones.
• Protocol Anomalies: Analyze logs like ssl.log for outdated or insecure protocol usage (e.g., TLS 1.0).
• Unusual Traffic Patterns: Detect large, unexpected data transfers or connections to uncommon regions.

5. Compliance and Auditing

Zeek logs can be used to ensure compliance with regulations like GDPR, PCI DSS, or HIPAA.

Examples:

• Logging for Compliance: Maintain detailed records of network activity for auditing purposes.
• Sensitive Data Protection: Monitor for unencrypted data transfers (ssl.log) or accidental exposure of sensitive information.

6. File Analysis

Zeek’s files.log and related capabilities allow in-depth file transfer monitoring.

Examples:

• File Hashing: Generate MD5, SHA1, or SHA256 hashes of transferred files for malware scanning.
• File Extraction: Automatically extract files for further analysis using external tools (e.g., YARA).

7. Detection of Insider Threats

Zeek helps monitor internal traffic to detect unusual or unauthorized activities by employees or compromised devices.

Examples:

• Unauthorized Access: Use conn.log and auth.log to detect unusual login patterns.
• Lateral Movement: Trace connections between internal hosts to identify unauthorized data access.

8. Detecting Phishing and Social Engineering

Zeek can analyze email and HTTP traffic to spot indicators of phishing.

Examples:

• Malicious Attachments: Detect suspicious file downloads through http.log and files.log.
• Suspicious Domains: Monitor dns.log for domains with low reputations or resembling popular ones (e.g., typosquatting).

9. TLS/SSL Monitoring

Zeek’s ssl.log provides visibility into encrypted traffic without decrypting it.

Examples:

• Certificate Inspection: Identify self-signed or expired certificates.
• Protocol Version Monitoring: Ensure only secure TLS versions are used.
• Unencrypted Traffic: Detect plaintext HTTP traffic where HTTPS is expected.

10. Integration with Threat Intelligence

Zeek can ingest threat intelligence feeds and correlate them with live network traffic.

Examples:

• IP Reputation Checks: Match conn.log IPs against known malicious IPs.
• Domain Reputation: Compare dns.log queries with a list of malicious domains.

11. Performance Monitoring

Zeek helps identify performance bottlenecks and anomalies in network traffic.

Examples:

• Bandwidth Analysis: Monitor traffic volumes using conn.log or stats.log.
• Application Performance: Measure latency and response times from logs like http.log.

12. Security Operations Center (SOC) Automation

Zeek’s output can feed SIEMs (e.g., Splunk, Elastic Stack) to automate alerting and response.

Examples:

• Alert Enrichment: Enhance alerts with contextual data from Zeek logs.
• Custom Alerting Rules: Create automated rules based on patterns in notice.log.

13. Research and Development

Zeek is often used by researchers to study network behaviors or develop novel detection techniques.

Examples:

• Protocol Analysis: Analyze how new or proprietary protocols behave.
• Attack Simulation: Simulate attacks and capture logs for training and testing.

14. IoT Device Monitoring

Zeek is useful for monitoring the behavior of IoT devices, which are often vulnerable to attacks.

Examples:

• Device Behavior Baseline: Understand normal communication patterns of IoT devices.
• Anomalous Traffic: Detect devices connecting to unusual IPs or regions.

Zeek (formerly Bro) is a premier network analysis tool and has become indispensable for threat hunters and cybersecurity professionals to monitor, log, and analyze network traffic in detail makes it a cornerstone of many security operations. Happy Hunting !!!