v0.4.3 - EventID Query Table Inference

What's Changed

• add: mapped GrandParentImage with InitiatingProcessParentFileName by @0xFustang in #30
• add: EventID to query_table mappings by @slincoln-aiq in #32
  • Previously, the query_table was selected by logsource.category, or provided by the user via a pipeline or pipeline arg. query_table is required in order for fieldmappings and valid query tables in the final query. This new feature also allows query_table to be set if an EventID is present in any selection detection sections and logsource.category is missing and the query_table is not supplied by the user. This allows for more rules to be translated to KQL queries
• fix: SigmaNumber conversion errors when in a grouped as-in expression of mixed types by @slincoln-aiq in #32
  • This fixes #29
• Minor formatting with black/ruff

New Contributors

• @0xFustang made their first contribution in #30

Full Changelog: v0.4.2...v0.4.3

v0.4.2 - Minor bug fix

🚀 Release Notes

🐛 Bug Fixes

• query_table param fix: Resolved issue where query_table would not persist when provided in another pipeline merged by sigma-cli

v0.4.1 - Bug Fixes

🚀 Release Notes

🐛 Bug Fixes

• Keyword Detection Items: Resolved an issue where "Invalid SigmaDetectionItem field name encountered: None" was incorrectly raised for keyword detection items. (Fixes #27 )

• Table Name Prepending: Fixed a bug where table names were only prepended to queries in SigmaCollection objects and not SigmaRule objects. This was addressed by implementing a postprocessing item on each rule instead of using a finalizer. (Fixes #28 )

🧪 Testing Improvements

• Comprehensive Test Coverage: Added tests to cover both SigmaCollection and SigmaRule objects for all pipelines, ensuring more robust functionality across different use cases.

📚 Documentation

• README Update: The README file has been updated to reflect recent changes and provide more accurate information.

🛠 Maintenance

• Code Formatting: Applied minor formatting updates.

• Development Dependencies: Updated the dev dependency group in pyproject.toml.

v0.4.0 - Azure Monitor Pipeline & Code Refactoring

🚀 Release Notes

🌟 Major Changes

🛡️ Microsoft XDR Pipeline (formerly Microsoft 365 Defender)

• 🔄 Microsoft 365 Defender pipeline renamed to Microsoft XDR
• ⚠️ Users should migrate to the new Microsoft XDR pipeline

🆕 Azure Monitor Pipeline (NEW!)

• 🧪 New Azure Monitor pipeline introduced (alpha status)
• 🗃️ Supports field mappings for SecurityEvents and SigninLogs tables
• 📊 All 698 Azure Monitor tables supported in final queries

🔍 Enhanced Sentinel ASIM Pipeline (Beta)

• 🔑 Additional field mappings added (beta status)

📈 Expanded Table Support

• Microsoft XDR: 38 tables
• Sentinel ASIM: 8 tables
• Azure Monitor: 698 tables

🏗️ Codebase Refactoring

• 🧱 Improved organization and structure
• 🔄 Better sharing of components across pipelines

✨ New Features

🎛️ Custom Table Name Support

• 🆕 Set custom table names with query_table parameter
• 🐍 Configurable via YAML or Python

🔀 Flexible Rule Category Handling

• 🚫 "Unsupported rule category" error suppressed when the following conditions are met:
  • Rule category is absent or category not in mappings.py for each pippeline
• A valid table is supplied via query_table param

🛠️ Technical Improvements

📜 Table Generation Scripts

• 🤖 New scripts in utils folder
• 🔄 Auto-populate valid tables and field schema in tables.py for each pipeline

🗺️ Field Mappings

• 🔨 Ongoing improvements for all pipelines

📊 Rule-to-Table Mapping

• 🚧 Work in progress on advanced mapping methods

📚 Documentation

• 📝 Updated README with query_table usage
• 💡 New examples for YAML and Python implementations
• FAQ/Troubleshooting section

⚠️ Deprecation Notices

• 🚫 Microsoft 365 Defender pipeline is deprecated
• 🔜 Users should migrate to Microsoft XDR pipeline

🔮 Future Work

• 🔍 Expanding field mappings across pipelines
• 🧠 Developing sophisticated rule-to-table mapping
• 🔧 Refining Azure Monitor and Sentinel ASIM pipelines

📘 Please refer to the updated README for detailed usage instructions and examples of the new features.

v0.3.3 - SentinelASIM Pipeline Bug Fixes

🚀 Release Notes: pySigma Kusto Backend v0.3.2

🐛 Bug Fixes

🛡️ Sentinel ASIM Pipeline

• Resolved issues with the sentinelasim pipeline, improving its stability and reliability.
  • Fixes #25

🧪 Testing Improvements

📊 Sentinel ASIM Pipeline

• Added basic test coverage for the sentinelasim pipeline, enhancing our ability to catch potential issues early.

🔮 Ongoing Development

We're actively working on expanding and refining our pipeline support:

• The sentinelasim pipeline and other pipelines are under active development.
• We're continuously adding more tables and features to these pipelines.
• We're also working on refactoring the codebase to make it more organized and easier to contribute to
• Your patience and feedback during this process are greatly appreciated!

v0.3.1 - Now with more Kusto!

🚀 Release Notes: pySigma Kusto Backend v0.3.1

🏷️ Repository Rename

We've renamed the repository from pySigma-backend-microsoft365defender to pySigma-backend-kusto. This change reflects our expanded focus on supporting the Kusto Query Language (KQL) as a backend for various platforms.

🆕 New Features

📊 Sentinel ASIM Pipeline

• Introduced the sentinel_asim pipeline, providing initial beta support for the Sentinel Advanced Security Information Model (ASIM) tables. (Thanks @adonm!)

🛠️ Backend Enhancements

• Renamed Microsoft365DefenderBackend to KustoBackend for clarity and broader applicability.
• The microsoft_365_defender_pipeline is no longer automatically applied by the backend, allowing for more flexible configurations.

🔧 Microsoft 365 Defender Pipeline Improvements

• Table renaming is now performed within the pipeline rather than the backend.
• Added the ability to override Sigma Rule category -> table name mappings with a custom query_table parameter.

📘 Usage Example

For details on using custom table names, see the README.

🔮 Future Plans

We're actively working on expanding support for additional KQL-based platforms. Stay tuned for updates on Microsoft XDR and other integrations!

🙏 Acknowledgements

Special thanks to the SigmaHQ Discord community for their valuable input on the repository direction.

v0.2.6

What's Changed

• Update microsoft365defender.py by @adonm in #19
  • Ignores the "Initiated" field in network connections as for endpoints almost all events are outbound
• Increased pinned certifi dependency version

New Contributors

• @adonm made their first contribution in #19

Full Changelog: v0.2.4...v0.2.5

v0.2.4: Wildcard escaping fixes

• Fixed issue #13 where '*' character was being escaped incorrectly CommandLine strings
• Fixed issue #14 where Sigma schema wildcards ('*', '?') in the middle of a string would create nonsense queries
  • Since KQL does not use wildcards, anytime a wildcard value is seen inside a string (not at the beginning or end) from the Sigma Rule, we now split it by the wildcard and use a contains for each substring.
  • Example: a CommandLine field with a value of advfirewall firewall add rule name=Dropbox dir=in action=allow "program=?:\Program Files (x86)\Dropbox\Client\Dropbox.exe" enable=yes profile=Any will be converted to (ProcessCommandLine contains "advfirewall firewall add rule name=Dropbox dir=in action=allow \"program=" and ProcessCommandLine contains ":\\Program Files (x86)\\Dropbox\\Client\\Dropbox.exe\" enable=yes profile=Any")

v0.2.3

• Fixed issue #11 by adding more verbose error handling and hash algorithm parsing in 'Hashes' field
• Loosened pySigma pinned version to allow compatibility with pySigma >= 0.11.0

v0.2.2

• Increased supported pySigma version to >= 0.9.0, <= 0.10.6