[LukasKarner]

Task

Description

This PR introduces an abstract base class for alert aggregation based on density-clustering, which will be used subsequently for the implementation of alert aggregation by the time-delta and AlertBERT methods.

How Has This Been Tested?

appropriate tests have been added and run successfully

Checklist

• This Pull-Request goes to the development branch.
• I have successfully run prek locally.
• I have added tests to cover my changes.
• I have linked the issue-id to the task-description.
• I have performed a self-review of my own code.

---

^{Need help on this PR? Tag @codesmith-bot with what you need.}

[LukasKarner]

The config field documentation still needs to be compiled, otherwise ready for merging. @ipmach

[ipmach]

There are a few things that has to be done before merging

[ipmach]

Unittests are failing:

=================================== FAILURES ===================================
__ TestDensityClusteringAggregation.test_default_hooks_raise_not_implemented ___

self = <test_density_clustering.TestDensityClusteringAggregation object at 0x734983146000>

    def test_default_hooks_raise_not_implemented(self) -> None:
        aggregator = DensityClusteringAggregation(
            config=DensityClusteringAggregationConfig(delta=1.0)
        )
        alert = make_alert("a", 0.0)
    
>       with pytest.raises(NotImplementedError):
E       Failed: DID NOT RAISE NotImplementedError

tests/test_alert_aggregators/test_density_clustering.py:128: Failed
=========================== short test summary info ============================
FAILED tests/test_alert_aggregators/test_density_clustering.py::TestDensityClusteringAggregation::test_default_hooks_raise_not_implemented

And

=================================== FAILURES ===================================
_______________________ test_doc_example_runs[example1] ________________________

example = PosixPath('/home/pepe/Documents/code/remove_later/DetectMateLibrary/docs/examples/alert_aggregators/density_cluster.py')

    @pytest.mark.parametrize("example", example_files)
    def test_doc_example_runs(example):
>       runpy.run_path(str(example), run_name="__main__")

tests/test_docs/test_doc_examples.py:21: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
<frozen runpy>:286: in run_path
    ???
<frozen runpy>:98: in _run_module_code
    ???
<frozen runpy>:88: in _run_code
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    # --8<-- [start:example_1]
    import yaml
    
    from detectmatelibrary.alert_aggregation.density_clustering import (
        DensityClusteringAggregation,
    )
    from detectmatelibrary.schemas import AggregateSchema, DetectorSchema
    
    with open("docs/examples/alert_aggregators/density_cluster.yaml") as f:
        config = yaml.safe_load(f)
    
    
    def make_alert(alert_id: str, score: float) -> DetectorSchema:
        return DetectorSchema(
            {
                "detectorID": "detector",
                "detectorType": "dummy",
                "alertID": alert_id,
                "detectedTimestamp": 0,
                "logIDs": [f"log-{alert_id}"],
                "score": score,
                "logTimestamps": [0],
                "description": "test alert",
                "receivedTimestamp": 0,
                "alertDetails": {},
            }
        )
    
    
    class ConcreteDensityClusteringAggregation(DensityClusteringAggregation):
        def single_metric(
            self,
            alert1: DetectorSchema,
            alert2: DetectorSchema
        ) -> float:
            return abs(alert1["score"] - alert2["score"])
    
        def discard_alert_fn(self, alerts: list[DetectorSchema]) -> list[bool]:
            return [False for alert in alerts]
    
    
    aggregator = DensityClusteringAggregation(config)
    first = make_alert("a", 0.0)
    second = make_alert("b", 6.0)
    bridge = make_alert("c", 3.0)
    
    first_output = AggregateSchema()
    aggregator.aggregate_alerts(first, first_output)
    
    second_output = AggregateSchema()
    aggregator.aggregate_alerts(second, second_output)
    
    bridge_output = AggregateSchema()
    aggregator.aggregate_alerts(bridge, bridge_output)
    
    print(first_output["alertDetails"]["a"] == "0")  # True
    print(second_output["alertDetails"]["b"] == "1")  # True
    print(bridge_output["alertDetails"]["c"] == "0")  # True
    
    # group labels 0 and 1 are merged into a single group
>   print(aggregator.active_groups[1] == {0, 1})  # True
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
E   KeyError: 1

docs/examples/alert_aggregators/density_cluster.py:61: KeyError
----------------------------- Captured stdout call -----------------------------
True
True
False