inference: can't find model "tokenization"

language models -> bert -> accuracy-squad.py

import numpy as np 
import six
import tokenization
from transformers import BertTokenizer
from create_squad_data import read_squad_examples, convert_examples_to_features

I follow the steps in the readme file and start a docker container to run Bert evaluation. However, I got the above error, and I even can’t find a third module (tokenization) that satisfied the API in the script (accuracy-squad.py).

Can anyone tell me where I can find this module (tokenization)? Thank you very much.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 19 (13 by maintainers)

Most upvoted comments

do we still plan to upstream (one of) our fix(es) to BERT accuracy mode?

@LukeIreland1 It would be great if you can provide a fix and file a PR so that we can merge it into upstream? Thanks

@nsircombe, do we still plan to upstream (one of) our fix(es) to BERT accuracy mode?