timesketch: Plaso upload fails if Timesketch's L2T version is older than 6 months

TL;DR - If you need a fix now and don’t want to follow all this haggling over this being a bug or not:

Change line 649 of file /usr/local/lib/python3.10/dist-packages/timesketch/lib/tasks.py
- from: storage_counters = json.loads(re.sub(r"^{, ", r"{", storage_counters_json))
- to: storage_counters = json.loads(re.sub(r"^{, ", r"{", re.sub(r"^.*\n{", r"{", storage_counters_json,flags=re.DOTALL)))
Restart your Timesketch instance.

Last week, I had to do some analysis on an older case (not too old - from July/August) and analyze some triage collections that were created at that time, but not used so far. So I started the VM for that case again (no config changes whatsoever) and - the upload failed. I tired other collections that had already been imported successfully in July/August - but they now failed as well.

After quite some trying, testing and debugging, I found out that pinfo.py is the culprit: When triggered by tasks.py/run_plaso, it throws the following warning (which decode.py cannot handle, resulting in a crash):

[WARNING] (MainProcess) PID:14237 <tools> This version of plaso is more than 6 months old.
WARNING: the version of plaso you are using is more than 6 months old. We strongly recommend to update it.

The L2T version installed at that time was 20230311, i.e. the warning is probably generated on any run of pinfo.py on or after Sep 11 (9/11 - yet again…)

Note: This is not a case of L2T versions being out of sync between creation of plaso file and upload of the resulting plaso file to Timesketch or of using a very old plaso version in general (I’ve seen other issues on this). The plaso file has been created with the exact same L2T version as used during upload and still the upload fails

To Reproduce Steps to reproduce the behavior:

Install Log2Timeline version 20230311 in your Timesketch environment
Create a plaso file from some image or triage collection using this L2T version
Try to upload it to Timesketch (either via Web GUI or from CLI using timesketch_importer)

Expected behavior Make the parser reading the pinfo.py output more robust by just ignoring any warnings generated by pinfo.py.

For sake of isolation and reproducibility, I usually clone separate VMs for each case and I do not want to change any configs in the VM while I’m working on that case. In this case, it’s especially annoying, since with L2T version 20230717, the fields _parser_chain and _event_values_hash have been removed from the OpenSearch output, i.e. after uploading the latest collections with the new L2T version, I cannot use the same queries across all timelines anymore.

I think it’s important to have some leeway regarding the L2T version - as long as the same version is used both for parsing and ingesting to Timesketch - to avoid being forced to switch L2T versions in the middle of a case.

Screenshots No screenshots (the timeline “Info” pop-up of the Web GUI unfortunately does not provide any Error Details)

Content from worker.log: [2023-09-15 12:53:19,262] celery.worker.strategy/INFO Task timesketch.lib.tasks.run_plaso[e0e8d913dc4449489b892fab98387254] received [2023-09-15 12:53:19,269] timesketch.tasks/INFO Index timeline [base-admin-triage20230914t153402z-triage] to index [44c185b3afe842d5a5bc7ae66602d17e] (source: plaso) [2023-09-15 12:53:20,028] timesketch.tasks/ERROR Error: Expecting value: line 1 column 1 (char 0) Traceback (most recent call last): File "/usr/local/lib/python3.10/dist-packages/timesketch/lib/tasks.py", line 649, in run_plaso storage_counters = json.loads(re.sub(r"^{, ", r"{", storage_counters_json)) File "/usr/lib/python3.10/json/__init__.py", line 346, in loads return _default_decoder.decode(s) File "/usr/lib/python3.10/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.10/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Console output of timesketch_import: $ timesketch_importer -u admin -p admin --host http://127.0.0.1 --timeline_name <import-name> --sketch_id 1 </path/to/import.plaso> [2023-09-15 11:10:39,753] timesketch_importer.importer_frontend/INFO Using cached credentials. [2023-09-15 11:11:16,383] timesketch_importer.importer_frontend/INFO Creating a client. [2023-09-15 11:11:31,802] timesketch_importer.importer_frontend/INFO Client created. [2023-09-15 11:11:33,419] timesketch_importer.importer_frontend/INFO Saving TS config. [2023-09-15 11:13:00,555] timesketch_importer.importer_frontend/INFO Uploading file. [2023-09-15 11:13:49,050] timesketch_importer.importer_frontend/INFO About to upload file. [2023-09-15 11:14:08,891] timesketch_importer.importer_frontend/INFO File upload completed. Checking file upload status: [FAIL]

Desktop (please complete the following information):

OS: Linux Mint 21.1
Browser: not relevant (same behavior in browser and on CLI)
Version: L2T: 20230311 / 20230717; Timesketch: Latest (is there any way to find out a version number from a running Timesketch instance?)

About this issue

Original URL
State: closed
Created 9 months ago
Comments: 16 (6 by maintainers)

Most upvoted comments

@blueteam0ps : Yes - you’ll find the description at the top of this thread - below the “TL;DR” headline. BTW: Thanks for updating your tags.yaml so quickly!

bit4n6 on Oct 2, 2023

But reproducibility is a core requirement in forensics and therefore, once a case has been started with an (at that point in time) current toolset, it should be ensured that the case can be completed with it (and then start the next case with the then latest version available).

which would be the responsibility of the analyst, given that more factors play into this outside the control of any tool author, e.g. the system and its configuration on which the tool runs.

Well - in this case it’s out of the control of the analyst - because I, as an analyst, cannot stop the time from ticking on. If I do not change anything in the setup - nothing should change in the output (especially not to the effect that there’s no output).

I think I’ve provided all the analysis for the bug so fixing it should really be a very minor effort and it makes timesketch_importer just a bit more robust.

bit4n6 on Sep 17, 2023

Anyway - since you chimed in to this discussion: Maybe you could add a “suppress warnings on stdout”-option to pinfo.py?

Given the tooling and dependencies are continues evolving it is not a good idea to use older versions.

I totally agree with you that we should keep your tools up-to-date. But reproducibility is a core requirement in forensics and therefore, once a case has been started with an (at that point in time) current toolset, it should be ensured that the case can be completed with it (and then start the next case with the then latest version available).
A forensic tool that worked fine at the beginning of a case should not fail from one day to another just because the clock ticked on.

The folks from blueteam0ps also used _parser_chain quite heavily in the rules of their tags.yaml file (see my issue there:

parser_chain is not the way to do this, use data_type

I guess that’s my key takeaway from this issue.

bit4n6 on Sep 17, 2023