front-end-monorepo: Unexpected classifier v2 classificaiton annotation format(s) for dropdown tasks
I’m investigating a broken classification data export in the API and have found 4 classifications (all by the same user) submitted by the FEM classifier (classifier_version: '2.0') but that are for taskType: 'dropdown' not dropdown-simple (see the sample data below)
Based on my reading of the FEM code for registered task types https://github.com/zooniverse/front-end-monorepo/blob/813f0334eb9543a9163ac5c312c7654722f3458f/packages/lib-classifier/src/plugins/tasks/index.js#L13-L25
and the simple dropdown ADR https://github.com/zooniverse/front-end-monorepo/blob/master/docs/arch/adr-30.md#consequences, it seems that FEM annotations should not have the taskType: 'dropdown' value as that isn’t a registered tasks type in FEM land, is this correct?
I believe the main issue at play here is these problematic classifications annotations are from the v2 classifier and have a mixture of PFE and FEM annotation formats, specifically I have these questions:
- Are these valid FEM annotations?
- Can the v2 (FEM) classifier submit annotations for dropdowns that have more than 1 select?
- If so then I need guidance to figure out which select the submitted value aligns to.
- Am I incorrectly interpreting the annotation format from the v2 classifier?
I found this issue when working on a classifier v2 annotation format exporter in panoptes, which handles the different annotation for dropdown-simple tasks which is working fine, https://github.com/zooniverse/panoptes/pull/3593. Once we figure out what’s up with these annotations I can proceed to fix the API classification data exporters for the FEM v2 classification annotations formats.
Here is a sample of the classification data that is causing issues:
#<Classification:
id: 320640562,
project_id: 11429,
user_id: redacted,
workflow_id: 18118,
annotations:
[{"task"=>"T1", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T2", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T3", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T4", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T5", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T7", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T8", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T9", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T10", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T11", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T4", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T5", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T7", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T8", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T9", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T10", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T11", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T13", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T14", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T15", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T16", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T17", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T19", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T20", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T21", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T22", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T23", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T25", "value"=>nil, "taskType"=>"dropdown"}, # this is the problem task annotation right here...
{"task"=>"T26", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T27", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T28", "value"=>nil, "taskType"=>"dropdown-simple"},
{"task"=>"T29", "value"=>nil, "taskType"=>"dropdown-simple"}],
created_at: Tue, 30 Mar 2021 16:49:23 UTC +00:00,
updated_at: Tue, 30 Mar 2021 16:49:23 UTC +00:00,
workflow_version: "2.11"
metadata:
{"classifier_version"=>"2.0"} # I redacted the rest
and the associated workflow task below (this is the 2.11 version of the workflow as used when exporting the annotation above)
# problematic workflow Task - note 2 selects (dropdowns)
# in my understanding having more than 1 select should break the wiring of the classifier and no classification should be submitted
{"help"=>"T25.help",
"type"=>"dropdown",
"selects"=>
[{"id"=>"cd1c1c2ffc297",
"title"=>"Select Circumstance #21",
"options"=>
{"*"=>
[{"label"=>"T25.selects.0.options.*.0.label", "value"=>"2db17b653c0cf"},
{"label"=>"T25.selects.0.options.*.1.label", "value"=>"58d7d68f3a22c"},
{"label"=>"T25.selects.0.options.*.2.label", "value"=>"f9368bbaa19c7"},
{"label"=>"T25.selects.0.options.*.3.label", "value"=>"243905a9b1b51"},
{"label"=>"T25.selects.0.options.*.4.label", "value"=>"2d1b73e02926f"},
{"label"=>"T25.selects.0.options.*.5.label", "value"=>"ed34cc05001ca"},
{"label"=>"T25.selects.0.options.*.6.label", "value"=>"d96161a08626e"},
{"label"=>"T25.selects.0.options.*.7.label", "value"=>"b4af1681aafd9"},
{"label"=>"T25.selects.0.options.*.8.label", "value"=>"d78f16c4d8ad"},
{"label"=>"T25.selects.0.options.*.9.label", "value"=>"d39597d25ca0f"},
{"label"=>"T25.selects.0.options.*.10.label", "value"=>"75753ce2df7f9"},
{"label"=>"T25.selects.0.options.*.11.label", "value"=>"fed66fdd3fcdd"},
{"label"=>"T25.selects.0.options.*.12.label", "value"=>"2976671f2d3e8"},
{"label"=>"T25.selects.0.options.*.13.label", "value"=>"d9b71212f6b3b"},
{"label"=>"T25.selects.0.options.*.14.label", "value"=>"53e7641fc823d"},
{"label"=>"T25.selects.0.options.*.15.label", "value"=>"17474857fd73f"},
{"label"=>"T25.selects.0.options.*.16.label", "value"=>"7097b605c85e1"},
{"label"=>"T25.selects.0.options.*.17.label", "value"=>"63a98408f2935"},
{"label"=>"T25.selects.0.options.*.18.label", "value"=>"9d628f1cb9692"}]},
"required"=>true,
"allowCreate"=>false},
{"id"=>"605671ee9452f", "title"=>"Other (Please transcribe)", "options"=>{}, "required"=>false, "condition"=>"cd1c1c2ffc297", "allowCreate"=>true}],
"instruction"=>"T25.instruction"}
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 18 (18 by maintainers)
I’ve got a test workflow set up on staging (project 908, workflow 3489) with a dropdown task that has two menus. It throws errors in the console, as expected.
and later