waf challenge: add threshold support

[blotus]

Add new helpers to allow scoring of a request:

• AddRequestScore(points, reason): add points for the given reason. Can be negative (eg, because something in the request is trustworthy)
• RequestScore()
• RequestScoreReasons()
• RequestScoreFor(reason)

This allows to write more granular appsec configs and to use weaker signals to make a decision whether an IP is coming from a bot or not (eg, utc_timezone is set. It's very unusual, but blocking based only on this would be too harsh):

inband:
  on_challenge_submit:
    - filter: EvaluateMismatches().Has("cdp")
      apply:
        - AddRequestScore(100, "cdp")
    - filter: EvaluateMismatches().Has("webdriver")
      apply:
        - AddRequestScore(100, "webdriver")
    - filter: EvaluateMismatches().Has("webdriver_writable")
      apply:
        - AddRequestScore(100, "webdriver_writable")
     - filter: EvaluateMismatches().Has("headless_screen_resolution")
      apply:
        - AddRequestScore(50, "headless_screen_resolution")
     - filter: EvaluateMismatches().Has("gpu_mismatch")
      apply:
        - AddRequestScore(30, "gpu_mismatch")
    - filter: EvaluateMismatches().Has("high_cpu_count")
      apply:
        - AddRequestScore(30, "high_cpu_count")
    - filter: RequestScore() >= 75
      apply:
        - 'RejectSubmission("request score " + string(RequestScore()) + ": " + join(RequestScoreReasons(), ","), "verbose")'

While currently targeted towards the challenge mode, it can also be used to score any request:

 - filter: req.Header.Get("Accept-Language") == ""
     apply:
       - AddRequestScore(15, "no_accept_language")

[blotus]

/area appsec
/kind enhancement

[github-actions]

@blotus: There are no 'kind' label on this PR. You need a 'kind' label to generate the release automatically.

• /kind feature
• /kind enhancement
• /kind refactoring
• /kind fix
• /kind chore
• /kind dependencies

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[github-actions]

@blotus: There are no area labels on this PR. You can add as many areas as you see fit.

• /area agent
• /area local-api
• /area cscli
• /area appsec
• /area security
• /area configuration

Details

I am a bot created to help the crowdsecurity developers manage community feedback and contributions. You can check out my manifest file to understand my behavior and what I can do. If you want to use this for your project, you can check out the BirthdayResearch/oss-governance-bot repository.

[codecov]

Codecov Report

❌ Patch coverage is 74.73684% with 24 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.03%. Comparing base (803377e) to head (bed6e66).
⚠️ Report is 1 commits behind head on master.

Files with missing lines	Patch %	Lines
pkg/appsec/waf_helpers.go	50.00%	15 Missing ⚠️
pkg/appsec/appsec.go	79.16%	5 Missing ⚠️
pkg/appsec/score.go	89.47%	2 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4593      +/-   ##
==========================================
- Coverage   64.38%   64.03%   -0.35%     
==========================================
  Files         499      518      +19     
  Lines       38576    39460     +884     
==========================================
+ Hits        24837    25269     +432     
- Misses      11409    11814     +405     
- Partials     2330     2377      +47     

Flag	Coverage Δ
bats	41.54% <0.00%> (-0.07%)	⬇️
unit-linux	40.80% <74.73%> (+0.18%)	⬆️
unit-windows	29.43% <70.52%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.