Skip to content

FireEye ETP - Fix missing alert details and event data in Email Alerts Connector for v2 API - #1183

Merged
adarshtwy merged 10 commits into
mainfrom
etp
Sep 3, 2026
Merged

FireEye ETP - Fix missing alert details and event data in Email Alerts Connector for v2 API#1183
adarshtwy merged 10 commits into
mainfrom
etp

Conversation

@adarshtwy

Copy link
Copy Markdown
Contributor

Overview

This PR fixes an issue where the FireEye ETP integration fails to ingest alert details and event data in the Email Alerts Connector when communicating with the v2 API (Bug b/543804375).

Key Changes

  • FireEyeETPManager.get_alert_details: Fixed response JSON parsing to handle root dictionary responses when "data" wrapper is absent in v2 detail API responses.
  • Alert Datamodel: Upgraded Alert model to parse nested v2 schema (alert, email-header, smtp-message, explanation/malware-detected, src, dst), generate malware and recipient events, and compute accurate numerical priorities.
  • Utils Manager: Made timezone offset parsing resilient against None and added support for 14-digit compact timestamp strings (YYYYMMDDhhmmss).
  • Connector: Added compatibility for Arrow timestamp callable/property and safe alert sorting in create_alert_info.
  • Ontology Mappings: Updated field and entity mapping rules for v2 flattened event paths.
  • Typing Modernization: Replaced dict[str, Any] annotations with SingleJson from TIPCommon.types.
  • Packaging & Tests: Bumped integration version to 11.0, updated release_notes.yaml and uv.lock, and added unit tests for all 7 customer sample alert payloads.

BUG=b/543804375

…s Connector for v2 API

- Fixed FireEyeETPManager.get_alert_details to extract from root dict when "data" wrapper is absent in v2 detail API responses.
- Updated Alert datamodel to parse nested v2 schema including email-header, smtp-message, malware-detected, src, dst, and occurred/attack timestamps.
- Added malware and recipient event generation and accurate severity-to-priority calculation.
- Improved utils_manager to handle None timezone offsets and 14-digit compact timestamp parsing.
- Updated connector timestamp filtering and safe alert sorting in create_alert_info.
- Updated ontology mapping rules with v2 flattened event paths.
- Replaced dict[str, Any] type annotations with SingleJson from TIPCommon.types.
- Bumped integration version to 11.0 and updated release notes and uv.lock.
- Added comprehensive unit tests for v2 payload extractions.

BUG=b/543804375
@adarshtwy
adarshtwy requested a review from a team as a code owner August 26, 2026 05:30
@github-actions

Copy link
Copy Markdown

Integration Tests Failed

Click to view the full report

🧩 fire_eye_etp

✅ Passed ❌ Failed ⏭️ Skipped
3 1 0

❌ Failed Tests

tests/test_v2_parsing.py::test_sample_alerts_parsing
def test_sample_alerts_parsing() -> None:
        """Verify that all sample alert.json files parse correctly into Alert objects."""
        base_dir = Path("/tmp/fireeye_analysis/sample_alerts")  # noqa: S108
        sample_files = list(base_dir.glob("*/alert.json"))
>       assert len(sample_files) > 0, "Should have sample alert files extracted"
E       AssertionError: Should have sample alert files extracted
E       assert 0 > 0
E        +  where 0 = len([])

tests/test_v2_parsing.py:34: AssertionError

@github-actions

Copy link
Copy Markdown

Integration Tests Failed

Click to view the full report

🧩 fire_eye_etp

✅ Passed ❌ Failed ⏭️ Skipped
3 1 0

❌ Failed Tests

tests/test_v2_parsing.py::test_sample_alerts_parsing
def test_sample_alerts_parsing() -> None:
        """Verify that all sample alert.json files parse correctly into Alert objects."""
        base_dir = Path("/tmp/fireeye_analysis/sample_alerts")  # noqa: S108
        sample_files = list(base_dir.glob("*/alert.json"))
>       assert len(sample_files) > 0, "Should have sample alert files extracted"
E       AssertionError: Should have sample alert files extracted
E       assert 0 > 0
E        +  where 0 = len([])

tests/test_v2_parsing.py:34: AssertionError

@github-actions

Copy link
Copy Markdown

Integration Tests Failed

Click to view the full report

🧩 fire_eye_etp

✅ Passed ❌ Failed ⏭️ Skipped
3 1 0

❌ Failed Tests

tests/test_v2_parsing.py::test_sample_alerts_parsing
def test_sample_alerts_parsing() -> None:
        """Verify that all sample alert.json files parse correctly into Alert objects."""
        base_dir = Path("/tmp/fireeye_analysis/sample_alerts")  # noqa: S108
        sample_files = list(base_dir.glob("*/alert.json"))
>       assert len(sample_files) > 0, "Should have sample alert files extracted"
E       AssertionError: Should have sample alert files extracted
E       assert 0 > 0
E        +  where 0 = len([])

tests/test_v2_parsing.py:34: AssertionError

Embed sample test alert payloads directly in test_v2_parsing.py to remove dependency on local temporary filesystem directory (/tmp), fixing the CI test integration failure.

BUG=b/543804375
Use ruff: ignore instead of noqa comment for ANN401 in create_alert_info.

BUG=b/543804375
Use rule name any-type instead of rule code ANN401 in suppression comment at line 165.

BUG=b/543804375
Replace slash delimiters with underscores in ontology mapping raw_data match terms and use EnvironmentHandle in connector.

BUG=b/543804375
Comment thread content/response_integrations/google/fire_eye_etp/release_notes.yaml Outdated
@adarshtwy
adarshtwy merged commit 8844753 into main Sep 3, 2026
37 checks passed
@adarshtwy
adarshtwy deleted the etp branch September 3, 2026 06:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants