layout-parser: conflicting dependencies

@bertsky, nice to virtually meet you. Found your Leipzig institutional affiliation via search engine sleuthing. 😃 So wish you were on Twitter so you could be engaged in the fine-grained research chatter related to the intersection of Digital Humanities and AI/ML wrt digitization pipelines. As an independent Citizen Scientist, that communication channel is invaluable to me.

You must be buddies with Clemens Neudecker and the OCR-D network given your recent involvement in that mega project. I know them through involvement in the Time Machine EU project, NewsEye, DATeCH, and Andreas Maier’s Pattern Recognition lab at FAU etc. First noticed your GitHub comment about LAREX’s PAGEgts reading order issue. I resonated with that as magazines and newspapers have complex document structures where within page reading order is indeterminate while being determinant at the issue level.

It is not surprising to see you here kicking the tires of Zejiang @lolipopshock Layout-Parser. The PDF annotation tool Zejiang alludes to is #PAWLS from @allenai lab here: https://github.com/allenai/pawls. Looks awesome and could be very helpful to my #MAGAZINEgts research. Would corresponding to you via your Leipzig email address be okay? Or do you have a preferred way to chat about shared interests?

Thanks @lolipopshock , I successfully installed and used library. Dependency problem solved for me.

In #18, the dependency on fvcore has been removed. And I plan to fix the Detectron2 version as v0.3 in the installation as well. Could you try again with the current following installation scripts and see if it can fix your issues? @edisongustavo @CrisChir @maxml7331 @ermissa

pip install -U 'git+https://github.com/layout-parser/layout-parser.git#egg=layoutparser'
pip install -U 'git+https://github.com/facebookresearch/detectron2.git@v0.3#egg=detectron2' 

Also Kudos to @edisongustavo for the help in #18.

When trying to install the latest version of detectron2 (0.4) from this URL (https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.8/detectron2-0.4%2Bcu111-cp39-cp39-linux_x86_64.whl) I get this error:

  SolverProblemError

  Because detectron2 (0.4+cu111) depends on pycocotools (>=2.0.2)
   and layoutparser (0.1.3) depends on pycocotools (2.0.1), detectron2 (0.4+cu111) is incompatible with layoutparser (0.1.3).
  And because no versions of layoutparser match >0.1.3,<0.2.0, detectron2 (0.4+cu111) is incompatible with layoutparser (>=0.1.3,<0.2.0).
  So, because german-tools depends on both layoutparser (^0.1.3) and detectron2 (0.4+cu111), version solving failed.

The message looks a bit differently than the other reports here because I’m using Poetry, but the problem is the same.

When installing from master, this is the error I get:

Because detectron2 (0.4+cu111) depends on fvcore (>=0.1.3,<0.1.4)
   and layoutparser (rev master) depends on fvcore (0.1.1.post20200623), detectron2 (0.4+cu111) is incompatible with layoutparser (rev master).
  So, because german-tools depends on both layoutparser (branch master) and detectron2 (0.4+cu111), version solving failed.

I’m trying to install it on Manjaro Linux with Python 3.9.

Thanks for reporting the issue for installing pycocotools on win10 - Let me try to fix that.

Speaking on the next major update, we are currently working on more features and it should be a huge update! Please stay tuned and we will release it very soon!

I noticed that I fed images in BGR instead of RGB (the cv2.imread example in the README does not explain this, should have read the other documentation first). But it does not make a difference. (Also, strangely, all models have input.format: BGR in their configuration…)

Then I tried to do binarization, deskewing and cropping externally. This did help somewhat:

• on primalayout Mask-RCNN model:
• on publaynet (full) Mask-RCNN model (non-empty now):

I wonder what the masks would look like. Is there a way to query / plot them?

Also, getting back to vertical text and in-domain data, here’s a different set of examples:

original	on HJDataset model

Is that expected output, too? Why do I get the feeling I am making some user-side mistake?

@lolipopshock

And the document is significantly different from the training datasets, so it’s expected to generate the non-sense outputs.

I see. I had my hopes up a little for the HJDataset model, because they are equally historic and are similar in visual appearance. But of course, vertical text in horizontal columns and horizontal text in vertical columns are still very different kinds of layouts. Anyway, for the sake of completeness, here’s the result of applying that model to my sample after 90° reflection:

So I guess I should really start training my own models now. But I am wondering: Do you have any experience how well mixing different datasets (in the hope to increase generality) or transferring weights from a larger model like the full PubLayNet to a small domain model works?

BTW @bertsky, given my OCDish bent, I actually spent a looonnnggg day getting Layout-Parser installed and working on my Windows 10 development box. I told Zejiang @lolipopshock I’d contribute an installation note to that effect once I confirmed the procedure. So far I have not been successful rewalking that torturous process. 😕 Will likely wait for a Docker release so as not to need to knock myself out with future updates. 😃

Many thanks for your fast response!

Good to know you are already onto these dependency conflicts.

So am I correct in inferring these warnings don’t actually do any harm?

1. it generates no outputs because it does not detect any boxes more than the specified confidence thresholds. You could try adding the extra_config when initializing the models to lower the threshold and try to see some outputs:

   lp.Detectron2LayoutModel(
               config_path = xxx,
               extra_config=["MODEL.ROI_HEADS.SCORE_THRESH_TEST", 0.25], # reduce the values)
   

Understood. I went as far down as lp.Detectron2LayoutModel('lp://PubLayNet/mask_rcnn_X_101_32x8d_FPN_3x/config', extra_config=["MODEL.ROI_HEADS.SCORE_THRESH_TEST", 0.01]), but still got no results there.

1. I think your dataset looks significantly different from the PubLayNet dataset and the others. If possible, you could share a screenshot of your doc and I could better help you identify the best models.

With pleasure:

• original:
• on newspapernavigator Mask-RCNN model:
• on primalayout Mask-RCNN model:
• on publaynet (small) Mask-RCNN model:

But ultimately, you might need to create the annotations yourself and train you own models. Speaking of that:

• for the pre-training issue you mentioned earlier, it should be fixed within 1wk
• and we are working on an annotation tool for more efficient layout annotation. it should be released within 1mo, and please stay tuned.

Wonderful!