numba: array broadcast fails when using njit(parallel=True)

Description: When using the jit (parallel=True), numpy array broadcasting fails incorrectly.

100% reproducible

tested on: Linux Mint Conda installed numba 0.42.0 py37h962f231_0
Mac Osx Conda installed numba 0.39.0 py36h6440ff4_0
tested from both jupyter notebook and also command line ipython.

sample code: Multiply a 5,3 matrix by a 5,1 matrix using numpy broadcasting

import numba as nb
import numpy as np

@nb.njit()
def testweird1(u,f):
   print(u.shape)
   print(f.shape)
   fzz = u*f
   print (fzz.shape)

@nb.njit(parallel=True)
def testweird2(u,f):
    print(u.shape)
    print(f.shape)
    fzz = u*f
    print (fzz.shape)
        
u = np.ones((5,3),dtype=np.float32) 
f = np.ones((5,1),dtype=np.float32)
testweird1(u,f)
print ("that worked\n")
testweird2(u,f)

Output:

(5, 3)
(5, 1)
(5, 3)

that worked

(5, 3)
(5, 1)
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-2-dcdc949288c2> in <module>
      9 u = np.ones((5,3),dtype=np.float32)
     10 f = np.ones((5,1),dtype=np.float32)
---> 11 testweird2(u,f)

AssertionError: Sizes of u, f do not match on <ipython-input-2-dcdc949288c2> (6)

About this issue

Original URL
State: open
Created 5 years ago
Reactions: 5
Comments: 15 (3 by maintainers)

Most upvoted comments

This is documented in the Numba parallel docs that broadcast is not currently supported. This is not a bug since it is the intended behavior at this point. You are making a feature request. Supporting broadcasts is on my to-do list but I don’t know when I will get to it.

DrTodd13 on Aug 24, 2020

This issue is currently preventing me from parallelizing several bits of code I have, is there any update on the progress of this bug?

zachlowry on Jun 12, 2020

@lanougue thank you for asking about this. There have been no updates to this as far as I know. If you like, please subscribe to notifications on this issue to receive updates.

esc on Dec 13, 2021

Just in case someone finds it useful, it appears that while automatic broadcasting doesn’t work, manual broadcasting (with np.broadcast_arrays) does, and results in a significant speedup.

davystrong on Jul 4, 2023

Thanks for the report, I can reproduce.

First, RE formatting in GitHub issues, I fixed the formatting above, but for future reference triple backticks are needed around a code block (with optional language syntax specified), for example: ```python <your code> ```. GitHub also has a basic guide to its markdown syntax available here.

Second, to the problem in hand, given there’s no traceback in the AssertionError it’s from the generated compiled code. I think the root cause is that array analysis code is probably not accommodating this case within the broadcasting analysis. This is a bug!

CC: @DrTodd13

stuartarchibald on Feb 1, 2019