Skip to content

Port the notebooks to Python 3, add a ParseZeekLogs version, preserve the posts - #1

Merged
dgunter merged 2 commits into
masterfrom
modernize
Sep 5, 2026
Merged

Port the notebooks to Python 3, add a ParseZeekLogs version, preserve the posts#1
dgunter merged 2 commits into
masterfrom
modernize

Conversation

@dgunter

@dgunter dgunter commented Sep 5, 2026

Copy link
Copy Markdown
Owner

What

  • Python 3 port of the three 2017-2018 notebooks (they were Python 2.7 kernels). Changes are the minimum needed: blank-line guard, pyplot import, list(dict.keys()) for display, except AttributeError instead of bare excepts. Outputs re-executed with pandas 3 and matplotlib 3.11; the 299-row suspicious-records CSV comes out identical.
  • pyshark on modern Python: pyshark 0.6 assumes asyncio.get_event_loop() creates a loop and calls set_child_watcher, both gone in 3.12+/3.14, and it cannot run inside Jupyter's already-running loop. The TDS notebook now reads the capture in a worker thread with its own loop. Verified against tshark 4.6.8.
  • New notebook Detecting Nmap Behavior with ParseZeekLogs.ipynb: the two HTTP hunts redone with ParseZeekLogs and pandas, kept alongside the originals. Also executed against Zeek 8.2.2 http.log in TSV and JSON form (from the ParseZeekLogs corpus) to confirm it handles current logs; the bundled 2015 log stays so results match the posts.
  • Posts preserved under docs/: Prologue, Part 2, Part 3 and Part 4 of Threat Hunting with Python, text from the Wayback Machine copies of dgunter.com, images from the live Dragos syndication (WebP converted to PNG) and Wayback for the WordPress-hosted ones. 22 images, every link checked.
  • Tooling: pyproject.toml + uv.lock pinning the notebook dependencies on Python 3.14, ruff linting the code cells natively (E501 ignored for the two historical notebooks), and a GitHub Actions workflow that installs tshark, lints, executes every notebook and fails on any error or stderr output. .idea/ ignored. Narrative README with a notebook-to-post table; GitHub description and topics set.

Why not SonarCloud

Sonar does not analyse .ipynb, and there is no package or test suite here, so a quality gate would measure nothing. Executing the notebooks in CI is the meaningful check.

Verification

All four notebooks execute with zero errors and zero stderr output locally; uv run ruff check . is clean. Zeek 8.2.2 run over the TDS pcap produces only conn.log (no TDS analyzer exists in Zeek), which is why that notebook stays on pyshark.

🤖 Generated with Claude Code

dgunter and others added 2 commits September 5, 2026 17:13
… the posts

The 2017-2018 notebooks now run on Python 3.10+ with current pandas,
matplotlib and pyshark (which needs its own event loop on Python 3.12+ and
inside Jupyter). A new notebook redoes the HTTP hunts with ParseZeekLogs and
pandas and also runs on Zeek 8.2.2 TSV and JSON logs. The four Threat
Hunting with Python posts are recovered from the Wayback Machine and the
live Dragos copies into docs/ with their images. uv project, ruff lint of
code cells, and a GitHub Actions workflow that executes every notebook.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@dgunter
dgunter merged commit 1544ee3 into master Sep 5, 2026
1 check passed
@dgunter
dgunter deleted the modernize branch September 5, 2026 22:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant