elastalert: search() got an unexpected keyword argument 'doc_type'

[root@afadsfasfsad /opt/elastalert]# /usr/local/python27/bin/elastalert
/usr/local/python27/lib/python2.7/site-packages/elastalert/config.py:31: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  rule_schema = jsonschema.Draft4Validator(yaml.load(open(os.path.join(os.path.dirname(__file__), 'schema.yaml'))))
Traceback (most recent call last):
  File "/usr/local/python27/bin/elastalert", line 11, in <module>
    load_entry_point('elastalert==0.1.39', 'console_scripts', 'elastalert')()
  File "/usr/local/python27/lib/python2.7/site-packages/elastalert/elastalert.py", line 1925, in main
    client.start()
  File "/usr/local/python27/lib/python2.7/site-packages/elastalert/elastalert.py", line 1106, in start
    self.run_all_rules()
  File "/usr/local/python27/lib/python2.7/site-packages/elastalert/elastalert.py", line 1158, in run_all_rules
    self.send_pending_alerts()
  File "/usr/local/python27/lib/python2.7/site-packages/elastalert/elastalert.py", line 1534, in send_pending_alerts
    pending_alerts = self.find_recent_pending_alerts(self.alert_time_limit)
  File "/usr/local/python27/lib/python2.7/site-packages/elastalert/elastalert.py", line 1526, in find_recent_pending_alerts
    size=1000)
  File "/usr/local/python27/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 84, in _wrapped
    return func(*args, params=params, **kwargs)
TypeError: search() got an unexpected keyword argument 'doc_type'

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Reactions: 8
  • Comments: 39 (10 by maintainers)

Commits related to this issue

Most upvoted comments

try pip uninstall elasticsearch and then pip install elasticsearch==5.5.3

+43
irkartik on Apr 12, 2019

hey, I’m still getting this issue… elk 7.0.1 on dockers, lastest bitsensor/elastalert as well. any updates?

+6
stavgayer on Aug 11, 2019

And one more to help with debugging. I’ve updated my image to bitsensor/elastalert:3.0.0-beta.1 and it seems to start now, but getting a new error:

15:35:22.670Z ERROR elastalert-server:
    ProcessController:  ERROR:root:Error finding recent pending alerts: RequestError(400, u'search_phase_execution_exception', u'No mapping found for [alert_time] in order to sort on') {'sort': {'alert_time': {'order': 'asc'}}, 'query': {'bool': {'filter': {'range': {'alert_time': {'to': '2019-09-24T15:35:22.665285Z', 'from': '2019-09-22T15:35:22.665254Z'}}}, 'must': {'query_string': {'query': '!_exists_:aggregate_id AND alert_sent:false'}}}}}
    Traceback (most recent call last):
      File "/opt/elastalert/elastalert/elastalert.py", line 1528, in find_recent_pending_alerts
        res = self.writeback_es.search(index=self.writeback_index, body=query, size=1000)
      File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/client/utils.py", line 84, in _wrapped
        return func(*args, params=params, **kwargs)
      File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/client/__init__.py", line 819, in search
        "GET", _make_path(index, "_search"), params=params, body=body
      File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/transport.py", line 353, in perform_request
        timeout=timeout,
      File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/connection/http_requests.py", line 155, in perform_request
        self._raise_error(response.status_code, raw_data)
      File "/usr/lib/python2.7/site-packages/elasticsearch-7.0.2-py2.7.egg/elasticsearch/connection/base.py", line 178, in _raise_error
        status_code, error_message, additional_info
    RequestError: RequestError(400, u'search_phase_execution_exception', u'No mapping found for [alert_time] in order to sort on')

Deleting the indexes fixed this. It appears that the latest official Docker HUB image (2.0.1) needs an update as it is not running >2.

+4
Fmstrat on Sep 24, 2019

getting this issue also on 7.3.1

+3
mali30 on Sep 5, 2019

@MaximilianKaltner Did you uninstall elastalert 0.1.39 using pip uninstall? To me it seems like you are executing a dangling 0.1.39 console script.

You could always try to execute the elastalert-test rule in a similar fashion as you did with elastalert thus circumventing the script entirely: python -m elastalert.test-rule --config /etc/elastalert/config.yaml /etc/elastalert/rules/test.yaml

FYI: The test-rule in 0.2.0b2 contains some bugs that seems fixed in the master branch. Especially an assertion error caused by buggy argument parsing.

Traceback (most recent call last):
  File "/home/m/Documents/elastalert/venv/bin/elastalert-test-rule", line 11, in <module>
    load_entry_point('elastalert==0.2.0b2', 'console_scripts', 'elastalert-test-rule')()
  File "/home/m/Documents/elastalert/venv/local/lib/python2.7/site-packages/elastalert-0.2.0b2-py2.7.egg/elastalert/test_rule.py", line 441, in main
    test_instance.run_rule_test()
  File "/home/m/Documents/elastalert/venv/local/lib/python2.7/site-packages/elastalert-0.2.0b2-py2.7.egg/elastalert/test_rule.py", line 408, in run_rule_test
    conf = load_conf(args, defaults, overwrites)
  File "/home/m/Documents/elastalert/venv/local/lib/python2.7/site-packages/elastalert-0.2.0b2-py2.7.egg/elastalert/config.py", line 47, in load_conf
    configure_logging(args, conf)
  File "/home/m/Documents/elastalert/venv/local/lib/python2.7/site-packages/elastalert-0.2.0b2-py2.7.egg/elastalert/config.py", line 106, in configure_logging
    if args.verbose and args.debug:
AttributeError: 'Namespace' object has no attribute 'verbose'
+2
matsgoran on May 8, 2019

@Qmando I’m having the same issue on elastisearch 7.3.2. I am using these images in docker:

image: docker.elastic.co/elasticsearch/elasticsearch:7.3.2
image: bitsensor/elastalert:2.0.1

That bitsensor is also the latest tag.

+1
Fmstrat on Sep 24, 2019

@matsgoran I fixed the problem with ms-teams. I missed the proxy configuration in my rule!

ms_teams_proxy: By default ElastAlert will not use a network proxy to send notifications to MS Teams. Set this option using hostname:port if you need to use a proxy.

Now everything works - thanks again!

cloud@elk-stack-ubuntu-16:~/elastalert$ sudo python -m elastalert.elastalert --verbose --config /etc/elastalert/config.yaml  --rule /etc/elastalert/rules/test.yaml
elastalert/config.py:31: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  rule_schema = jsonschema.Draft4Validator(yaml.load(open(os.path.join(os.path.dirname(__file__), 'schema.yaml'))))
INFO:elastalert:Starting up
INFO:elastalert:Alert sent to MS Teams
INFO:elastalert:Alert sent to MS Teams
INFO:elastalert:Alert sent to MS Teams
INFO:elastalert:Alert sent to MS Teams
INFO:elastalert:Queried rule Test rule1 from 2019-05-09 09:49 CEST to 2019-05-09 09:50 CEST: 5 / 5 hits
INFO:elastalert:Alert sent to MS Teams
INFO:elastalert:Ran Test rule1 from 2019-05-09 09:49 CEST to 2019-05-09 09:50 CEST: 5 query hits (0 already seen), 1 matches, 5 alerts sent
INFO:elastalert:Sleeping for 56.70884 seconds
INFO:elastalert:Queried rule Test rule1 from 2019-05-09 09:49 CEST to 2019-05-09 09:51 CEST: 5 / 5 hits
INFO:elastalert:Ran Test rule1 from 2019-05-09 09:49 CEST to 2019-05-09 09:51 CEST: 5 query hits (5 already seen), 0 matches, 0 alerts sent
INFO:elastalert:Sleeping for 59.950538 seconds
+1
MaximilianKaltner on May 9, 2019

I am using the 6.7.0 elastic stack. Cannot migrate to 7.0.0 since elastalert kibana plugin is not available. I am seeing elastalert | File “/opt/elastalert/elastalert/elastalert.py”, line 1528, in send_pending_alerts elastalert | pending_alerts = self.find_recent_pending_alerts(self.alert_time_limit) elastalert | File “/opt/elastalert/elastalert/elastalert.py”, line 1520, in find_recent_pending_alerts elastalert | elastalert | 15:20:39.834Z ERROR elastalert-server: elastalert | ProcessController: size=1000) elastalert | File “/usr/lib/python2.7/site-packages/elasticsearch-7.0.0-py2.7.egg/elasticsearch/client/utils.py”, line 84, in _wrapped elastalert | return func(*args, params=params, **kwargs) elastalert | TypeError: search() got an unexpected keyword argument ‘doc_type’

Cannot go to the 5.5.3.

What are my options?

+1
cellango on Apr 14, 2019
Read more comments on GitHub
← elastalert: Not connecting to office 365 SMTP server
elastalert: Simple query doesn't work →