front-end-monorepo: Unexpected classifier v2 classificaiton annotation format(s) for dropdown tasks

I’m investigating a broken classification data export in the API and have found 4 classifications (all by the same user) submitted by the FEM classifier (classifier_version: '2.0') but that are for taskType: 'dropdown' not dropdown-simple (see the sample data below)

Based on my reading of the FEM code for registered task types https://github.com/zooniverse/front-end-monorepo/blob/813f0334eb9543a9163ac5c312c7654722f3458f/packages/lib-classifier/src/plugins/tasks/index.js#L13-L25 and the simple dropdown ADR https://github.com/zooniverse/front-end-monorepo/blob/master/docs/arch/adr-30.md#consequences, it seems that FEM annotations should not have the taskType: 'dropdown' value as that isn’t a registered tasks type in FEM land, is this correct?

I believe the main issue at play here is these problematic classifications annotations are from the v2 classifier and have a mixture of PFE and FEM annotation formats, specifically I have these questions:

  1. Are these valid FEM annotations?
  2. Can the v2 (FEM) classifier submit annotations for dropdowns that have more than 1 select?
    • If so then I need guidance to figure out which select the submitted value aligns to.
  3. Am I incorrectly interpreting the annotation format from the v2 classifier?

I found this issue when working on a classifier v2 annotation format exporter in panoptes, which handles the different annotation for dropdown-simple tasks which is working fine, https://github.com/zooniverse/panoptes/pull/3593. Once we figure out what’s up with these annotations I can proceed to fix the API classification data exporters for the FEM v2 classification annotations formats.

Here is a sample of the classification data that is causing issues:

#<Classification:
 id: 320640562,
 project_id: 11429,
 user_id: redacted,
 workflow_id: 18118,
 annotations:
  [{"task"=>"T1", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T2", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T3", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T4", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T5", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T7", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T8", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T9", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T10", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T11", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T4", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T5", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T7", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T8", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T9", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T10", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T11", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T13", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T14", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T15", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T16", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T17", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T19", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T20", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T21", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T22", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T23", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T25", "value"=>nil, "taskType"=>"dropdown"}, # this is the problem task annotation right here...
   {"task"=>"T26", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T27", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T28", "value"=>nil, "taskType"=>"dropdown-simple"},
   {"task"=>"T29", "value"=>nil, "taskType"=>"dropdown-simple"}],
 created_at: Tue, 30 Mar 2021 16:49:23 UTC +00:00,
 updated_at: Tue, 30 Mar 2021 16:49:23 UTC +00:00,
 workflow_version: "2.11"
 metadata: 
    {"classifier_version"=>"2.0"} # I redacted the rest

and the associated workflow task below (this is the 2.11 version of the workflow as used when exporting the annotation above)

# problematic workflow Task - note 2 selects (dropdowns)
# in my understanding having more than 1 select should break the wiring of the classifier and no classification should be submitted
{"help"=>"T25.help",
 "type"=>"dropdown",
 "selects"=>
  [{"id"=>"cd1c1c2ffc297",
    "title"=>"Select Circumstance #21",
    "options"=>
     {"*"=>
       [{"label"=>"T25.selects.0.options.*.0.label", "value"=>"2db17b653c0cf"},
        {"label"=>"T25.selects.0.options.*.1.label", "value"=>"58d7d68f3a22c"},
        {"label"=>"T25.selects.0.options.*.2.label", "value"=>"f9368bbaa19c7"},
        {"label"=>"T25.selects.0.options.*.3.label", "value"=>"243905a9b1b51"},
        {"label"=>"T25.selects.0.options.*.4.label", "value"=>"2d1b73e02926f"},
        {"label"=>"T25.selects.0.options.*.5.label", "value"=>"ed34cc05001ca"},
        {"label"=>"T25.selects.0.options.*.6.label", "value"=>"d96161a08626e"},
        {"label"=>"T25.selects.0.options.*.7.label", "value"=>"b4af1681aafd9"},
        {"label"=>"T25.selects.0.options.*.8.label", "value"=>"d78f16c4d8ad"},
        {"label"=>"T25.selects.0.options.*.9.label", "value"=>"d39597d25ca0f"},
        {"label"=>"T25.selects.0.options.*.10.label", "value"=>"75753ce2df7f9"},
        {"label"=>"T25.selects.0.options.*.11.label", "value"=>"fed66fdd3fcdd"},
        {"label"=>"T25.selects.0.options.*.12.label", "value"=>"2976671f2d3e8"},
        {"label"=>"T25.selects.0.options.*.13.label", "value"=>"d9b71212f6b3b"},
        {"label"=>"T25.selects.0.options.*.14.label", "value"=>"53e7641fc823d"},
        {"label"=>"T25.selects.0.options.*.15.label", "value"=>"17474857fd73f"},
        {"label"=>"T25.selects.0.options.*.16.label", "value"=>"7097b605c85e1"},
        {"label"=>"T25.selects.0.options.*.17.label", "value"=>"63a98408f2935"},
        {"label"=>"T25.selects.0.options.*.18.label", "value"=>"9d628f1cb9692"}]},
    "required"=>true,
    "allowCreate"=>false},
   {"id"=>"605671ee9452f", "title"=>"Other (Please transcribe)", "options"=>{}, "required"=>false, "condition"=>"cd1c1c2ffc297", "allowCreate"=>true}],
 "instruction"=>"T25.instruction"}

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 18 (18 by maintainers)

Most upvoted comments

I’ve got a test workflow set up on staging (project 908, workflow 3489) with a dropdown task that has two menus. It throws errors in the console, as expected.

T0 dropdown is not a supported task type

and later

at path "/type" value `"dropdown"` is not assignable to type: `"dropdown-simple"`