Python: Build an IOC Extractor
MediumDevOps30-45 minThe SOC pastes suspicious logs into chat and someone greps them by hand for addresses, domains and hashes. Replace that with a real tool. You are logged in as 'student' (no sudo); your case files are in ~/case.
Write /home/student/ioc.py so that python3 ~/ioc.py LOGFILE prints every indicator of compromise in LOGFILE, one per line, as ip <addr>, domain <name> or sha256 <hash>. The full spec is in ~/case/brief.txt; the short version:
- IPs: any IPv4 address in a line, including a URL host. Drop invalid ones (octet over 255) and our own 10.0.0.0/8 and 127.0.0.0/8 (use the ipaddress module, not is_private).
- Domains: only from DNS query lines and http(s) URL hosts, without the :port, lower-cased, never an IP, and never corp.example or its subdomains.
- Hashes: SHA-256 only (exactly 64 hex characters on their own), lower-cased. MD5, SHA-1 and SHA-512 are not SHA-256.
- Each IOC once. Some lines contain bytes that are not valid UTF-8; don't crash. A missing file gets a one-line error on stderr and a non-zero exit, never a traceback.
Self-test against ~/case/sample.log and ~/case/sample.expected, then run it on ~/case/incident.log.
Grading runs your script, as you, on a different incident log you never see, with different IPs, domains and hashes. A script that prints the answers for your own log scores nothing.
- Pull IPv4 addresses, domains and SHA-256 hashes out of mixed logs with the re module
- Validate and filter addresses with ipaddress (invalid octets, your own networks)
- Normalise and deduplicate indicators with sets and lower-casing
- Make a command-line tool that survives bad bytes and fails cleanly with argparse and try/except
Hands-on lab: you work on a real Linux box in your browser, graded on the machine's actual state.
