caldera: Atomic Plugin? Hanging the Startup process?
Good afternoon Caldera team,
I was trying to run the master Branch in a Docker container, and it never shows the “Systems Ready” message before running “sphinx” for the docs. I did not know what was happening so I CTRL+C, and I saw the following messages
2020-08-20 17:29:16 - INFO (server.py:89 <module>) Using main config from conf/local.yml
2020-08-20 17:29:16 - DEBUG (selector_events.py:54 __init__) Using selector: EpollSelector
2020-08-20 17:29:17 - DEBUG (learning_svc.py:19 __init__) Loaded 2 parsers
2020-08-20 17:29:17 - DEBUG (data_svc.py:64 restore_state) There are 0 jobs in the scheduler
2020-08-20 17:29:17 - DEBUG (contact_svc.py:35 register) Registered contact: websocket
2020-08-20 17:29:17 - DEBUG (contact_svc.py:35 register) Registered contact: gist
2020-08-20 17:29:17 - DEBUG (contact_svc.py:35 register) Registered contact: http
2020-08-20 17:29:17 - DEBUG (contact_svc.py:35 register) Registered contact: udp
2020-08-20 17:29:17 - DEBUG (contact_svc.py:35 register) Registered contact: tcp
2020-08-20 17:29:17 - DEBUG (contact_svc.py:35 register) Registered contact: html
2020-08-20 17:29:17 - DEBUG (app_svc.py:107 load) Enabled plugin: manx
2020-08-20 17:29:17 - DEBUG (app_svc.py:107 load) Enabled plugin: stockpile
2020-08-20 17:29:17 - DEBUG (sand_svc.py:93 load_sandcat_extension_modules) Loaded gocat extension module: donut
2020-08-20 17:29:17 - DEBUG (sand_svc.py:93 load_sandcat_extension_modules) Loaded gocat extension module: proxy_http
2020-08-20 17:29:17 - DEBUG (sand_svc.py:93 load_sandcat_extension_modules) Loaded gocat extension module: shared
2020-08-20 17:29:17 - DEBUG (sand_svc.py:93 load_sandcat_extension_modules) Loaded gocat extension module: shellcode
2020-08-20 17:29:17 - DEBUG (sand_svc.py:93 load_sandcat_extension_modules) Loaded gocat extension module: shells
2020-08-20 17:29:17 - DEBUG (app_svc.py:107 load) Enabled plugin: sandcat
2020-08-20 17:29:17 - DEBUG (app_svc.py:107 load) Enabled plugin: fieldmanual
2020-08-20 17:29:17 - DEBUG (app_svc.py:107 load) Enabled plugin: access
^C2020-08-20 17:34:46 - DEBUG (atomic_svc.py:74 populate_data_directory) Ingested 543 abilities (out of 560) from Atomic plugin and ran into 1 errors
2020-08-20 17:34:46 - DEBUG (app_svc.py:107 load) Enabled plugin: atomic
2020-08-20 17:34:46 - DEBUG (base_world.py:43 set_config) Configuration (agents) update, setting deployments=['356d1722-7784-40c4-822b-0cf864b0b36d', '0ab383be-b819-41bf-91b9-1bd4404d83bf', '1837b43e-4fff-46b2-a604-a602f7540469', '2f34977d-9558-4c12-abad-349716777c6b']
2020-08-20 17:34:46 - DEBUG (app_svc.py:107 load) Enabled plugin: response
2020-08-20 17:34:46 - DEBUG (app_svc.py:107 load) Enabled plugin: compass
2020-08-20 17:34:46 - DEBUG (app_svc.py:107 load) Enabled plugin: training
2020-08-20 17:34:46 - DEBUG (app_svc.py:107 load) Enabled plugin: gameboard
2020-08-20 17:34:49 - DEBUG (auth_svc.py:58 apply) Created authentication group: blue
2020-08-20 17:34:49 - DEBUG (auth_svc.py:58 apply) Created authentication group: red
2020-08-20 17:34:49 - INFO (server.py:57 run_tasks) All systems ready.
It seems that the Atomic plugin is hanging the process after it runs into an error? If the startup process hanging is the plugin then I can open an issue in the Atomic plugin. However, I believe even if the plugin fails or it has errors, the startup process should handle it and not hang right?
How to replicate it? I added the main steps below to the following DOCKERFILE
git clone --recursive https://github.com/mitre/caldera.git /opt/caldera
cd /opt/caldera && git submodule update --recursive --remote
pip3 install --no-cache-dir -r requirements.txt
python3 server.py
I do not use --insecure parameter because I pass my on local.yml file
Thank you in avance!
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 15 (13 by maintainers)
Hey @wbooth thank you ! It works fine now 😃
@wbooth , not sure if you’ve dug into this at all yet, but I (accidentally)was able to reproduce it and pointed a profiler at it while it hanging. There’s really high cpu usage too.
I’m guessing that this conditional (https://github.com/mitre/atomic/blob/master/app/atomic_svc.py#L158 ) isn’t firing in the loop so
string_to_analyzeisn’t getting modified. It looks like the function is writing changes out to the file system which could explain why it’s not an issue when caldera is restarted. I’ll try to carve out some time to work on it if you don’t beat me to it 😃A couple of data points to add. I can duplicate the Atomic Plugin issue consistently during a Caldera install in an Ubuntu 20.04 container. Per Cyb3rWard0g, removing the Atomic Plugin from the conf file fixes Caldera’s startup in the container. However, I have not been able to reproduce the issue during Caldera installations on Ubuntu 20.04 or Fedora 32 running as bare-metal OSs. Definitely odd.
I enabled the atomic plugin on Ubuntu 20 with v2.8 and it didn’t hang the startup process but I also haven’t tried to use the atomic plugin either.