hub: False positive http-crawl-non_statics with Nextcloud desktop client
What happened?
When setting up the Nextcloud desktop client it will sync a lot of files with the Nextcloud server. This quickly triggers crowdsecurity/http-crawl-non_statics and blocks the client.
What did you expect to happen?
Allow the sync to run through without triggering the crowdsecurity/http-crawl-non_statics
How can we reproduce it (as minimally and precisely as possible)?
Install the Nextcloud desktop client from https://nextcloud.com/install/ Put a couple of files in the sync directory and start syncing.
Anything else we need to know?
No response
Crowdsec version
$ cscli version
2023/03/03 10:21:01 version: v1.4.6-debian-pragmatic-5f71037b40c498045e1b59923504469e2b8d0140
2023/03/03 10:21:01 Codename: alphaga
2023/03/03 10:21:01 BuildDate: 2023-02-09_14:34:10
2023/03/03 10:21:01 GoVersion: 1.19.2
2023/03/03 10:21:01 Platform: linux
2023/03/03 10:21:01 Constraint_parser: >= 1.0, <= 2.0
2023/03/03 10:21:01 Constraint_scenario: >= 1.0, < 3.0
2023/03/03 10:21:01 Constraint_api: v1
2023/03/03 10:21:01 Constraint_acquis: >= 1.0, < 2.0
OS version
# On Linux:
$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
$ uname -a
Linux tron 5.10.0-21-amd64 crowdsecurity/crowdsec#1 SMP Debian 5.10.162-1 (2023-01-21) x86_64 GNU/Linux
Enabled collections and parsers
$ cscli hub list -o raw
crowdsecurity/apache2,enabled,0.1,apache2 support : parser and generic http scenarios ,collections
crowdsecurity/base-http-scenarios,enabled,0.6,http common : scanners detection,collections
crowdsecurity/dovecot,enabled,0.1,dovecot support : parser and spammer detection,collections
crowdsecurity/http-cve,enabled,1.9,,collections
crowdsecurity/linux,enabled,0.2,core linux support : syslog+geoip+ssh,collections
crowdsecurity/nextcloud,enabled,0.3,Nextcloud support : parser and brute-force detection,collections
crowdsecurity/postfix,enabled,0.2,postfix support : parser and spammer detection,collections
crowdsecurity/sshd,enabled,0.2,sshd support : parser and brute-force detection,collections
crowdsecurity/apache2-logs,enabled,1.3,Parse Apache2 access and error logs,parsers
crowdsecurity/dateparse-enrich,enabled,0.2,,parsers
crowdsecurity/dovecot-logs,enabled,0.6,Parse dovecot logs,parsers
crowdsecurity/geoip-enrich,enabled,0.2,"Populate event with geoloc info : as, country, coords, source range.",parsers
crowdsecurity/http-logs,enabled,1.1,"Parse more Specifically HTTP logs, such as HTTP Code, HTTP path, HTTP args and if its a static ressource",parsers
crowdsecurity/nextcloud-logs,enabled,0.2,Parse nextcloud logs,parsers
crowdsecurity/nextcloud-whitelist,enabled,0.3,Whitelist events from nextcloud,parsers
crowdsecurity/postfix-logs,enabled,0.4,Parse postfix logs,parsers
crowdsecurity/postscreen-logs,enabled,0.2,Parse postscreen logs,parsers
crowdsecurity/sshd-logs,enabled,2.0,Parse openSSH logs,parsers
crowdsecurity/syslog-logs,enabled,0.8,,parsers
crowdsecurity/whitelists,enabled,0.2,Whitelist events from private ipv4 addresses,parsers
crowdsecurity/CVE-2022-26134,enabled,0.1,Detect CVE-2022-26134 exploits,scenarios
crowdsecurity/CVE-2022-35914,enabled,0.1,Detect CVE-2022-35914 exploits,scenarios
crowdsecurity/CVE-2022-37042,enabled,0.1,Detect CVE-2022-37042 exploits,scenarios
crowdsecurity/CVE-2022-40684,enabled,0.2,Detect cve-2022-40684 exploitation attempts,scenarios
crowdsecurity/CVE-2022-41082,enabled,0.3,Detect CVE-2022-41082 exploits,scenarios
crowdsecurity/CVE-2022-41697,enabled,0.1,Detect CVE-2022-41697 enumeration,scenarios
crowdsecurity/CVE-2022-42889,enabled,0.2,Detect CVE-2022-42889 exploits (Text4Shell),scenarios
crowdsecurity/CVE-2022-44877,enabled,0.2,Detect CVE-2022-44877 exploits,scenarios
crowdsecurity/CVE-2022-46169,enabled,0.1,Detect CVE-2022-46169 brute forcing,scenarios
crowdsecurity/apache_log4j2_cve-2021-44228,enabled,0.4,Detect cve-2021-44228 exploitation attemps,scenarios
crowdsecurity/dovecot-spam,enabled,0.3,detect errors on dovecot,scenarios
crowdsecurity/f5-big-ip-cve-2020-5902,enabled,0.1,Detect cve-2020-5902 exploitation attemps,scenarios
crowdsecurity/fortinet-cve-2018-13379,enabled,0.2,Detect cve-2018-13379 exploitation attemps,scenarios
crowdsecurity/grafana-cve-2021-43798,enabled,0.1,Detect cve-2021-43798 exploitation attemps,scenarios
crowdsecurity/http-backdoors-attempts,enabled,0.3,Detect attempt to common backdoors,scenarios
crowdsecurity/http-bad-user-agent,enabled,0.7,Detect bad user-agents,scenarios
crowdsecurity/http-crawl-non_statics,enabled,0.3,Detect aggressive crawl from single ip,scenarios
crowdsecurity/http-cve-2021-41773,enabled,0.1,cve-2021-41773,scenarios
crowdsecurity/http-cve-2021-42013,enabled,0.1,cve-2021-42013,scenarios
crowdsecurity/http-generic-bf,enabled,0.4,Detect generic http brute force,scenarios
crowdsecurity/http-open-proxy,enabled,0.3,Detect scan for open proxy,scenarios
crowdsecurity/http-path-traversal-probing,enabled,0.2,Detect path traversal attempt,scenarios
crowdsecurity/http-probing,enabled,0.2,Detect site scanning/probing from a single ip,scenarios
crowdsecurity/http-sensitive-files,enabled,0.2,"Detect attempt to access to sensitive files (.log, .db ..) or folders (.git)",scenarios
crowdsecurity/http-sqli-probing,enabled,0.2,A scenario that detects SQL injection probing with minimal false positives,scenarios
crowdsecurity/http-xss-probing,enabled,0.2,A scenario that detects XSS probing with minimal false positives,scenarios
crowdsecurity/jira_cve-2021-26086,enabled,0.1,Detect Atlassian Jira CVE-2021-26086 exploitation attemps,scenarios
crowdsecurity/nextcloud-bf,enabled,0.2,Detect Nextcloud bruteforce,scenarios
crowdsecurity/postfix-spam,enabled,0.2,Detect spammers,scenarios
crowdsecurity/pulse-secure-sslvpn-cve-2019-11510,enabled,0.2,Detect cve-2019-11510 exploitation attemps,scenarios
crowdsecurity/spring4shell_cve-2022-22965,enabled,0.2,Detect cve-2022-22965 probing,scenarios
crowdsecurity/ssh-bf,enabled,0.1,Detect ssh bruteforce,scenarios
crowdsecurity/ssh-slow-bf,enabled,0.2,Detect slow ssh bruteforce,scenarios
crowdsecurity/thinkphp-cve-2018-20062,enabled,0.3,Detect ThinkPHP CVE-2018-20062 exploitation attemps,scenarios
crowdsecurity/vmware-cve-2022-22954,enabled,0.2,Detect Vmware CVE-2022-22954 exploitation attempts,scenarios
crowdsecurity/vmware-vcenter-vmsa-2021-0027,enabled,0.1,Detect VMSA-2021-0027 exploitation attemps,scenarios
ltsich/http-w00tw00t,enabled,0.1,detect w00tw00t,scenarios
Acquisition config
```console
# On Linux:
$ cat /etc/crowdsec/acquis.yaml /etc/crowdsec/acquis.d/*
Feb 13 01:21:28 tron crowdsec-firewall-bouncer[627091]: time="2023-02-13T01:21:28+01:00" level=info msg="crowdsec-firewall-bouncer v0.0.25-debian-pragmatic-0a4fde8e9440927d02ce187d1716306af9a13780"
Feb 13 01:21:28 tron crowdsec-firewall-bouncer[627091]: time="2023-02-13T01:21:28+01:00" level=info msg="config is valid"
Feb 13 01:21:28 tron crowdsec-firewall-bouncer[627098]: time="2023-02-13T01:21:28+01:00" level=info msg="crowdsec-firewall-bouncer v0.0.25-debian-pragmatic-0a4fde8e9440927d02ce187d1716306af9a13780"
Feb 13 01:21:28 tron systemd[1]: crowdsec-firewall-bouncer.service: Failed with result 'protocol'.
░░ The unit crowdsec-firewall-bouncer.service has entered the 'failed' state with result 'protocol'.
░░ Subject: A start job for unit crowdsec-firewall-bouncer.service has failed
░░ A start job for unit crowdsec-firewall-bouncer.service has finished with a failure.
░░ Subject: A start job for unit crowdsec-firewall-bouncer.service has begun execution
░░ A start job for unit crowdsec-firewall-bouncer.service has begun execution.
Feb 13 01:23:34 tron crowdsec-firewall-bouncer[627136]: time="2023-02-13T01:23:34+01:00" level=info msg="crowdsec-firewall-bouncer v0.0.25-debian-pragmatic-0a4fde8e9440927d02ce187d1716306af9a13780"
Feb 13 01:23:34 tron crowdsec-firewall-bouncer[627136]: time="2023-02-13T01:23:34+01:00" level=info msg="config is valid"
Feb 13 01:23:34 tron crowdsec-firewall-bouncer[627143]: time="2023-02-13T01:23:34+01:00" level=info msg="crowdsec-firewall-bouncer v0.0.25-debian-pragmatic-0a4fde8e9440927d02ce187d1716306af9a13780"
Feb 13 01:23:34 tron systemd[1]: crowdsec-firewall-bouncer.service: Failed with result 'protocol'.
░░ The unit crowdsec-firewall-bouncer.service has entered the 'failed' state with result 'protocol'.
░░ Subject: A start job for unit crowdsec-firewall-bouncer.service has failed
░░ A start job for unit crowdsec-firewall-bouncer.service has finished with a failure.
░░ Subject: A start job for unit crowdsec-firewall-bouncer.service has begun execution
░░ A start job for unit crowdsec-firewall-bouncer.service has begun execution.
Feb 13 01:27:22 tron crowdsec-firewall-bouncer[627282]: time="2023-02-13T01:27:22+01:00" level=info msg="crowdsec-firewall-bouncer v0.0.25-debian-pragmatic-0a4fde8e9440927d02ce187d1716306af9a13780"
Feb 13 01:27:22 tron crowdsec-firewall-bouncer[627282]: time="2023-02-13T01:27:22+01:00" level=info msg="config is valid"
Feb 13 01:27:22 tron crowdsec-firewall-bouncer[627289]: time="2023-02-13T01:27:22+01:00" level=info msg="crowdsec-firewall-bouncer v0.0.25-debian-pragmatic-0a4fde8e9440927d02ce187d1716306af9a13780"
Feb 13 01:27:22 tron systemd[1]: crowdsec-firewall-bouncer.service: Failed with result 'protocol'.
░░ The unit crowdsec-firewall-bouncer.service has entered the 'failed' state with result 'protocol'.
░░ Subject: A start job for unit crowdsec-firewall-bouncer.service has failed
░░ A start job for unit crowdsec-firewall-bouncer.service has finished with a failure.
Config show
$ cscli config show
Global:
- Configuration Folder : /etc/crowdsec
- Data Folder : /var/lib/crowdsec/data
- Hub Folder : /etc/crowdsec/hub
- Simulation File : /etc/crowdsec/simulation.yaml
- Log Folder : /var/log/
- Log level : info
- Log Media : file
Crowdsec:
- Acquisition File : /etc/crowdsec/acquis.yaml
- Parsers routines : 1
- Acquisition Folder : /etc/crowdsec/acquis.d
cscli:
- Output : human
- Hub Branch :
- Hub Folder : /etc/crowdsec/hub
Local API Server:
- Listen URL : 127.0.0.1:8080
- Profile File : /etc/crowdsec/profiles.yaml
- Trusted IPs:
- 127.0.0.1
- ::1
- Database:
- Type : sqlite
- Path : /var/lib/crowdsec/data/crowdsec.db
- Flush age : 7d
- Flush size : 5000
Prometheus metrics
$ cscli metrics
[empty]
Related custom configs versions (if applicable) : notification plugins, custom scenarios, parsers etc.
About this issue
- Original URL
- State: open
- Created 4 years ago
- Comments: 15 (5 by maintainers)
Commits related to this issue
- Update nextcloud-whitelist.yaml HTTP 200 needs to be whitelisted too for file-sync via webdav, as mentioned in Issue #669. — committed to UnknownHiker/hub by UnknownHiker 3 years ago
Yeah that is the issue here is the line
We can update it to
No I didn´t sorry I missed it. So ignore my message.
I’m afraid I’m not the person from Discord. 😃