hub: False positive http-crawl-non_statics with Nextcloud desktop client

What happened?

When setting up the Nextcloud desktop client it will sync a lot of files with the Nextcloud server. This quickly triggers crowdsecurity/http-crawl-non_statics and blocks the client.

What did you expect to happen?

Allow the sync to run through without triggering the crowdsecurity/http-crawl-non_statics

How can we reproduce it (as minimally and precisely as possible)?

Install the Nextcloud desktop client from https://nextcloud.com/install/ Put a couple of files in the sync directory and start syncing.

Anything else we need to know?

No response

Crowdsec version

$ cscli version
2023/03/03 10:21:01 version: v1.4.6-debian-pragmatic-5f71037b40c498045e1b59923504469e2b8d0140
2023/03/03 10:21:01 Codename: alphaga
2023/03/03 10:21:01 BuildDate: 2023-02-09_14:34:10
2023/03/03 10:21:01 GoVersion: 1.19.2
2023/03/03 10:21:01 Platform: linux
2023/03/03 10:21:01 Constraint_parser: >= 1.0, <= 2.0
2023/03/03 10:21:01 Constraint_scenario: >= 1.0, < 3.0
2023/03/03 10:21:01 Constraint_api: v1
2023/03/03 10:21:01 Constraint_acquis: >= 1.0, < 2.0

OS version

# On Linux:
$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

$ uname -a
Linux tron 5.10.0-21-amd64 crowdsecurity/crowdsec#1 SMP Debian 5.10.162-1 (2023-01-21) x86_64 GNU/Linux

Enabled collections and parsers

$ cscli hub list -o raw
crowdsecurity/apache2,enabled,0.1,apache2 support : parser and generic http scenarios ,collections
crowdsecurity/base-http-scenarios,enabled,0.6,http common : scanners detection,collections
crowdsecurity/dovecot,enabled,0.1,dovecot support : parser and spammer detection,collections
crowdsecurity/http-cve,enabled,1.9,,collections
crowdsecurity/linux,enabled,0.2,core linux support : syslog+geoip+ssh,collections
crowdsecurity/nextcloud,enabled,0.3,Nextcloud support : parser and brute-force detection,collections
crowdsecurity/postfix,enabled,0.2,postfix support : parser and spammer detection,collections
crowdsecurity/sshd,enabled,0.2,sshd support : parser and brute-force detection,collections
crowdsecurity/apache2-logs,enabled,1.3,Parse Apache2 access and error logs,parsers
crowdsecurity/dateparse-enrich,enabled,0.2,,parsers
crowdsecurity/dovecot-logs,enabled,0.6,Parse dovecot logs,parsers
crowdsecurity/geoip-enrich,enabled,0.2,"Populate event with geoloc info : as, country, coords, source range.",parsers
crowdsecurity/http-logs,enabled,1.1,"Parse more Specifically HTTP logs, such as HTTP Code, HTTP path, HTTP args and if its a static ressource",parsers
crowdsecurity/nextcloud-logs,enabled,0.2,Parse nextcloud logs,parsers
crowdsecurity/nextcloud-whitelist,enabled,0.3,Whitelist events from nextcloud,parsers
crowdsecurity/postfix-logs,enabled,0.4,Parse postfix logs,parsers
crowdsecurity/postscreen-logs,enabled,0.2,Parse postscreen logs,parsers
crowdsecurity/sshd-logs,enabled,2.0,Parse openSSH logs,parsers
crowdsecurity/syslog-logs,enabled,0.8,,parsers
crowdsecurity/whitelists,enabled,0.2,Whitelist events from private ipv4 addresses,parsers
crowdsecurity/CVE-2022-26134,enabled,0.1,Detect CVE-2022-26134 exploits,scenarios
crowdsecurity/CVE-2022-35914,enabled,0.1,Detect CVE-2022-35914 exploits,scenarios
crowdsecurity/CVE-2022-37042,enabled,0.1,Detect CVE-2022-37042 exploits,scenarios
crowdsecurity/CVE-2022-40684,enabled,0.2,Detect cve-2022-40684 exploitation attempts,scenarios
crowdsecurity/CVE-2022-41082,enabled,0.3,Detect CVE-2022-41082 exploits,scenarios
crowdsecurity/CVE-2022-41697,enabled,0.1,Detect CVE-2022-41697 enumeration,scenarios
crowdsecurity/CVE-2022-42889,enabled,0.2,Detect CVE-2022-42889 exploits (Text4Shell),scenarios
crowdsecurity/CVE-2022-44877,enabled,0.2,Detect CVE-2022-44877 exploits,scenarios
crowdsecurity/CVE-2022-46169,enabled,0.1,Detect CVE-2022-46169 brute forcing,scenarios
crowdsecurity/apache_log4j2_cve-2021-44228,enabled,0.4,Detect cve-2021-44228 exploitation attemps,scenarios
crowdsecurity/dovecot-spam,enabled,0.3,detect errors on dovecot,scenarios
crowdsecurity/f5-big-ip-cve-2020-5902,enabled,0.1,Detect cve-2020-5902 exploitation attemps,scenarios
crowdsecurity/fortinet-cve-2018-13379,enabled,0.2,Detect cve-2018-13379 exploitation attemps,scenarios
crowdsecurity/grafana-cve-2021-43798,enabled,0.1,Detect cve-2021-43798 exploitation attemps,scenarios
crowdsecurity/http-backdoors-attempts,enabled,0.3,Detect attempt to common backdoors,scenarios
crowdsecurity/http-bad-user-agent,enabled,0.7,Detect bad user-agents,scenarios
crowdsecurity/http-crawl-non_statics,enabled,0.3,Detect aggressive crawl from single ip,scenarios
crowdsecurity/http-cve-2021-41773,enabled,0.1,cve-2021-41773,scenarios
crowdsecurity/http-cve-2021-42013,enabled,0.1,cve-2021-42013,scenarios
crowdsecurity/http-generic-bf,enabled,0.4,Detect generic http brute force,scenarios
crowdsecurity/http-open-proxy,enabled,0.3,Detect scan for open proxy,scenarios
crowdsecurity/http-path-traversal-probing,enabled,0.2,Detect path traversal attempt,scenarios
crowdsecurity/http-probing,enabled,0.2,Detect site scanning/probing from a single ip,scenarios
crowdsecurity/http-sensitive-files,enabled,0.2,"Detect attempt to access to sensitive files (.log, .db ..) or folders (.git)",scenarios
crowdsecurity/http-sqli-probing,enabled,0.2,A scenario that detects SQL injection probing with minimal false positives,scenarios
crowdsecurity/http-xss-probing,enabled,0.2,A scenario that detects XSS probing with minimal false positives,scenarios
crowdsecurity/jira_cve-2021-26086,enabled,0.1,Detect Atlassian Jira CVE-2021-26086 exploitation attemps,scenarios
crowdsecurity/nextcloud-bf,enabled,0.2,Detect Nextcloud bruteforce,scenarios
crowdsecurity/postfix-spam,enabled,0.2,Detect spammers,scenarios
crowdsecurity/pulse-secure-sslvpn-cve-2019-11510,enabled,0.2,Detect cve-2019-11510 exploitation attemps,scenarios
crowdsecurity/spring4shell_cve-2022-22965,enabled,0.2,Detect cve-2022-22965 probing,scenarios
crowdsecurity/ssh-bf,enabled,0.1,Detect ssh bruteforce,scenarios
crowdsecurity/ssh-slow-bf,enabled,0.2,Detect slow ssh bruteforce,scenarios
crowdsecurity/thinkphp-cve-2018-20062,enabled,0.3,Detect ThinkPHP CVE-2018-20062 exploitation attemps,scenarios
crowdsecurity/vmware-cve-2022-22954,enabled,0.2,Detect Vmware CVE-2022-22954 exploitation attempts,scenarios
crowdsecurity/vmware-vcenter-vmsa-2021-0027,enabled,0.1,Detect VMSA-2021-0027 exploitation attemps,scenarios
ltsich/http-w00tw00t,enabled,0.1,detect w00tw00t,scenarios

Acquisition config

```console # On Linux: $ cat /etc/crowdsec/acquis.yaml /etc/crowdsec/acquis.d/* Feb 13 01:21:28 tron crowdsec-firewall-bouncer[627091]: time="2023-02-13T01:21:28+01:00" level=info msg="crowdsec-firewall-bouncer v0.0.25-debian-pragmatic-0a4fde8e9440927d02ce187d1716306af9a13780" Feb 13 01:21:28 tron crowdsec-firewall-bouncer[627091]: time="2023-02-13T01:21:28+01:00" level=info msg="config is valid" Feb 13 01:21:28 tron crowdsec-firewall-bouncer[627098]: time="2023-02-13T01:21:28+01:00" level=info msg="crowdsec-firewall-bouncer v0.0.25-debian-pragmatic-0a4fde8e9440927d02ce187d1716306af9a13780" Feb 13 01:21:28 tron systemd[1]: crowdsec-firewall-bouncer.service: Failed with result 'protocol'. ░░ The unit crowdsec-firewall-bouncer.service has entered the 'failed' state with result 'protocol'. ░░ Subject: A start job for unit crowdsec-firewall-bouncer.service has failed ░░ A start job for unit crowdsec-firewall-bouncer.service has finished with a failure. ░░ Subject: A start job for unit crowdsec-firewall-bouncer.service has begun execution ░░ A start job for unit crowdsec-firewall-bouncer.service has begun execution. Feb 13 01:23:34 tron crowdsec-firewall-bouncer[627136]: time="2023-02-13T01:23:34+01:00" level=info msg="crowdsec-firewall-bouncer v0.0.25-debian-pragmatic-0a4fde8e9440927d02ce187d1716306af9a13780" Feb 13 01:23:34 tron crowdsec-firewall-bouncer[627136]: time="2023-02-13T01:23:34+01:00" level=info msg="config is valid" Feb 13 01:23:34 tron crowdsec-firewall-bouncer[627143]: time="2023-02-13T01:23:34+01:00" level=info msg="crowdsec-firewall-bouncer v0.0.25-debian-pragmatic-0a4fde8e9440927d02ce187d1716306af9a13780" Feb 13 01:23:34 tron systemd[1]: crowdsec-firewall-bouncer.service: Failed with result 'protocol'. ░░ The unit crowdsec-firewall-bouncer.service has entered the 'failed' state with result 'protocol'. ░░ Subject: A start job for unit crowdsec-firewall-bouncer.service has failed ░░ A start job for unit crowdsec-firewall-bouncer.service has finished with a failure. ░░ Subject: A start job for unit crowdsec-firewall-bouncer.service has begun execution ░░ A start job for unit crowdsec-firewall-bouncer.service has begun execution. Feb 13 01:27:22 tron crowdsec-firewall-bouncer[627282]: time="2023-02-13T01:27:22+01:00" level=info msg="crowdsec-firewall-bouncer v0.0.25-debian-pragmatic-0a4fde8e9440927d02ce187d1716306af9a13780" Feb 13 01:27:22 tron crowdsec-firewall-bouncer[627282]: time="2023-02-13T01:27:22+01:00" level=info msg="config is valid" Feb 13 01:27:22 tron crowdsec-firewall-bouncer[627289]: time="2023-02-13T01:27:22+01:00" level=info msg="crowdsec-firewall-bouncer v0.0.25-debian-pragmatic-0a4fde8e9440927d02ce187d1716306af9a13780" Feb 13 01:27:22 tron systemd[1]: crowdsec-firewall-bouncer.service: Failed with result 'protocol'. ░░ The unit crowdsec-firewall-bouncer.service has entered the 'failed' state with result 'protocol'. ░░ Subject: A start job for unit crowdsec-firewall-bouncer.service has failed ░░ A start job for unit crowdsec-firewall-bouncer.service has finished with a failure.

Config show

$ cscli config show
Global:
   - Configuration Folder   : /etc/crowdsec
   - Data Folder            : /var/lib/crowdsec/data
   - Hub Folder             : /etc/crowdsec/hub
   - Simulation File        : /etc/crowdsec/simulation.yaml
   - Log Folder             : /var/log/
   - Log level              : info
   - Log Media              : file
Crowdsec:
  - Acquisition File        : /etc/crowdsec/acquis.yaml
  - Parsers routines        : 1
  - Acquisition Folder      : /etc/crowdsec/acquis.d
cscli:
  - Output                  : human
  - Hub Branch              : 
  - Hub Folder              : /etc/crowdsec/hub
Local API Server:
  - Listen URL              : 127.0.0.1:8080
  - Profile File            : /etc/crowdsec/profiles.yaml
  - Trusted IPs: 
      - 127.0.0.1
      - ::1
  - Database:
      - Type                : sqlite
      - Path                : /var/lib/crowdsec/data/crowdsec.db
      - Flush age           : 7d
      - Flush size          : 5000

Prometheus metrics

$ cscli metrics
[empty]

Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Comments: 15 (5 by maintainers)

Commits related to this issue

Most upvoted comments

I suspect that the whitelist does not cover response code 200 while the desktop client syncs its files.

Yeah that is the issue here is the line

- evt.Meta.http_status == '404' && evt.Meta.http_verb in ['PROPFIND', 'GET'] && evt.Meta.http_path matches '^/remote.php/(web)?dav/'

We can update it to

- evt.Meta.http_status in ['404', '200'] && evt.Meta.http_verb in ['PROPFIND', 'GET'] && evt.Meta.http_path matches '^/remote.php/(web)?dav/'

No I didn´t sorry I missed it. So ignore my message.

I’m afraid I’m not the person from Discord. 😃