plantcv: invalid json
Finally catching up to development. I initialized a test project to get pipelines going. I can run a PSII pipeline with one image pair. I ran
scripts/psii.py -i1 "data\psII\A1-SILK_dark-2018-12-10 20_01_50-PSII0_1_0_0.tif" -i2 "data\psII\A1-SILK_dark-2018-12-10 20_01_50-PSII0_2_0_0.tif" -o debug -D print -writeimg True -r results.json
which spit out results.json, but when i try to convert it to a csv i get an error:
(test-environment)
Dominik Schneider@CPPC1 MINGW64 ~/Documents/phenomics/testpipeline (master)
$ plantcv-utils.py json2csv -j results.json -c result-table
Traceback (most recent call last):
File "C:/Users/dominikschneider/Miniconda3/envs/test-environment/Scripts/plantcv-utils.py", line 4, in <module>
__import__('pkg_resources').run_script('plantcv==3.5.0+16.g5b9908c.dirty', 'plantcv-utils.py')
File "C:\Users\dominikschneider\Miniconda3\envs\test-environment\lib\site-packages\pkg_resources\__init__.py", line 666, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "C:\Users\dominikschneider\Miniconda3\envs\test-environment\lib\site-packages\pkg_resources\__init__.py", line 1453, in run_script
exec(code, namespace, namespace)
File "c:\users\dominikschneider\miniconda3\envs\test-environment\lib\site-packages\plantcv-3.5.0+16.g5b9908c.dirty-py3.7.egg\EGG-INFO\scripts\plantcv-utils.py", line 74, in <module>
main()
File "c:\users\dominikschneider\miniconda3\envs\test-environment\lib\site-packages\plantcv-3.5.0+16.g5b9908c.dirty-py3.7.egg\EGG-INFO\scripts\plantcv-utils.py", line 69, in main
options()
File "c:\users\dominikschneider\miniconda3\envs\test-environment\lib\site-packages\plantcv-3.5.0+16.g5b9908c.dirty-py3.7.egg\EGG-INFO\scripts\plantcv-utils.py", line 44, in options
args.func(args)
File "c:\users\dominikschneider\miniconda3\envs\test-environment\lib\site-packages\plantcv-3.5.0+16.g5b9908c.dirty-py3.7.egg\EGG-INFO\scripts\plantcv-utils.py", line 52, in run_json2csv plantcv.utils.json2csv(json_file=args.json, csv_file=args.csv)
File "C:\Users\dominikschneider\Miniconda3\envs\test-environment\lib\site-packages\plantcv-3.5.0+16.g5b9908c.dirty-py3.7.egg\plantcv\utils\converters.py", line 12, in json2csv raise ValueError("Invalid JSON file: {0}".format(json_file))ValueError: Invalid JSON file: results.json
I took a look at the parallel version for flat files and of course I’ve been labeling my files from lemnabase differently from you. I am using format code {0}-{3}-{1}-{6} which corresponds to plantid-measurement label-timestamp-camera label whereas you have it setup with measurement label and timestamp flipped.
Anyway to work on making this more flexible?
Is this going to iterate through every timestamp?
here is my repo: https://github.com/dschneiderch/testpipeline
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 20 (11 by maintainers)
I agree, it would be easier to have temp files saved as
.jsonduring parallelization. On my branch I’m working on I hadif 'text/plain' in mimetypes.guess_type(filename) or 'application/json' in mimetypes.guess_type(filename):but this can certainly be simplified and ensure that.pyscripts, git files, or anything else gets identified as plain text.Looks like we should name the files with the
.jsonextension so that we can do:That should avoid the overlap with other text files at least.
Ok I figured it out. I had
results.txtin the top directory. os.walk() returns everything underjob_dir. in my case my .py scripts and git files identified as plaintext and the script was trying to process them. I placed results.txt intooutput/and it worked as expected.Would it be a good idea to add a header line to the json files so they are easy to identify by their first line? Alternatively, you could just check that they are a valid json file.
Hm my
presults.jsonfile is as follows:I got
{"variables": {}, "entities": []}before renaming the file with a .txt file extension but after renaming I got the above output so I’m not sure if it’s a Windows thing or what. This reformatted file works withjson2csv.