Malcolm v25.01.0

[mmguero]

Malcolm v25.01.0 contains quite a few UI/UX improvements; new parsers; a bevy of component version updates including to Arkime, Zeek, NetBox; and several bug fixes.

v24.12.0...v25.01.0

• ✨ Features and enhancements
  • integrate Omron FINS parser and added corresponding dashboard (integrate omron fins parser #554)
  • integrate PostgreSQL parser (added in Zeek v7.1.0) and added corresponding dashboard (update Zeek to feature release v7.1.0 #553)
  • normalize Winlogbeat with Fluent Bit's winlog/winevtlog event and evtx event schemas (normalize winlogbeats with fluent bit winlog/winevtlog #356)
    • Winlogbeat seems to parse more fields from Windows events than Fluent Bit's winevtlog or winlog do, so users forwarding Windows event logs to Malcolm using Fluent Bit may want to evaluate Winlogbeat as an alternative.
  • support syslog ingestion over UDP and/or TCP (enhance support for syslog ingestion #354)
  • clicking field values in Dashboards tables will now pivot to Arkime or NetBox (URL pivot links from dashboards to arkime #551)
  • add navigation pane to all non-network dashboards (add navigation pane to non-network dashboards #543)
• ✅ Component version updates
  • Arkime to v5.6.0
  • beats to v8.17.0
  • elasticsearch-dsl Python library to v8.17.1
  • Jinja Python library to v3.1.5 (security fix release)
  • Logstash to v8.17.0
  • NetBox to v4.1.11
  • osd_transform_vis (Dashboards visualization library) to v2.18.0
  • yq to v4.45.1
  • Zeek to v7.1.0 (update Zeek to feature release v7.1.0 #553)
• 🐛 Bug fixes
  • Extracted File Downloads interface not working with some filenames (extracted_files_http_server.py not working with some filenames #524)
  • user-defined custom field formats for index patterns are overwritten (user-defined custom field formats for index patterns can get overwritten by Malcolm #542)
  • port numbers should not be shown with commas in Dashboards (port numbers should not be shown with commas in Dashboards #540)
  • pivoting between Arkime and Dashboards doesn't work when Malcolm is behind a reverse proxy (e.g., traefik) (pivoting between Arkime and Dashboards doesn't work when Malcolm is behind a reverse proxy (e.g., traefik) #552)
  • opensearch.keystore not created when running in Hedgehog run profile (opensearch.keystore not created when running in Hedgehog profile #533)
  • ensure all conn.log entries are tagged ics for OT protocols (ensure all conn.log entries are tagged "ics" for OT protocols #541)
• 📄 Configuration changes (in environment variables in ./config/) for Malcolm and in control_vars.conf for Hedgehog Linux
  • The following variables in ./config/filebeat.env configure Malcolm's ability to accept syslog messages:
    • FILEBEAT_SYSLOG_TCP_LISTEN and FILEBEAT_SYSLOG_UDP_LISTEN - if set to true, Malcolm will accept syslog messages over TCP and/or UDP, respectively
    • FILEBEAT_SYSLOG_TCP_PORT and FILEBEAT_SYSLOG_UDP_PORT - the port on which Malcolm will accept syslog messages over TCP and/or UDP, respectively
    • FILEBEAT_SYSLOG_TCP_FORMAT and FILEBEAT_SYSLOG_UDP_FORMAT - one of auto, rfc3164, or rfc5424, to specify the allowed format for syslog messages over TCP and/or UDP, respectively (default auto)
    • FILEBEAT_SYSLOG_TCP_MAX_MESSAGE_SIZE and FILEBEAT_SYSLOG_UDP_MAX_MESSAGE_SIZE - defines the maximum message size of the message received over TCP and/or UDP, respectively (default: 10KiB for UDP, 20MiB for TCP)
    • FILEBEAT_SYSLOG_TCP_MAX_CONNECTIONS - specifies the maximum current number of TCP connections for syslog messages
    • FILEBEAT_SYSLOG_TCP_SSL - if set to true, syslog messages over TCP will require the use of TLS. When ./scripts/auth_setup is run, self-signed certificates are generated which may be used by remote log forwarders. Located in Malcolm's ./filebeat/certs/ directory, the certificate authority and client certificate and key files should be copied to the host on which the forwarder is running and used when defining its settings for connecting to Malcolm.
  • The following variables in ./config/zeek.env for Malcolm and control_vars.conf for Hedgehog Linux pertain to the new Omron FINS protocol parser:
    • ZEEK_DISABLE_ICS_OMRON_FINS - if set to true, the Omron FINS parser will be disabled
    • ZEEK_OMRON_FINS_DETAILED - if set to true, a verbose Omron FINS details log (omron_fins_detail.log) will be created
• 🧹 Code and project maintenance
  • Changed ⓒ year to 2025

Malcolm is a powerful, easily deployable network 🖧 traffic analysis tool suite for network security monitoring 🕵🏻‍♀️.

Malcolm operates as a cluster of containers 📦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker 🐋, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

Alternatively, dedicated official ISO installer images 💿 for Malcolm and Hedgehog Linux 🦔 can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split 🪓 into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell 🪟 (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

As always, join us on the Malcolm discussions board 💬 to engage with the community, or pop some corn 🍿 and watch a video 📼.

---

This discussion was created from the release Merge pull request #558 from cisagov/v25.01.0_merge_cisagov.