BayesianOptimization: `StopIteration: Queue is empty` error in optimizer.maximize()

I am running into the error StopIteration: Queue is empty, no more objects to retrieve when I run optimizer.maximize(...). Specifically, the traceback is as follows (with pieces referring to my code removed).

Traceback (most recent call last):
  File "/mnt/mesos/sandbox/.local/lib/python3.8/site-packages/bayes_opt/bayesian_optimization.py", line 179, in maximize
    x_probe = next(self._queue)
  File "/mnt/mesos/sandbox/.local/lib/python3.8/site-packages/bayes_opt/bayesian_optimization.py", line 25, in __next__
    raise StopIteration("Queue is empty, no more objects to retrieve.")
StopIteration: Queue is empty, no more objects to retrieve.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
[ ... ]
  File "/mnt/mesos/sandbox/.local/lib/python3.8/site-packages/bayes_opt/bayesian_optimization.py", line 182, in maximize
    x_probe = self.suggest(util)
  File "/mnt/mesos/sandbox/.local/lib/python3.8/site-packages/bayes_opt/bayesian_optimization.py", line 131, in suggest
    suggestion = acq_max(
  File "/mnt/mesos/sandbox/.local/lib/python3.8/site-packages/bayes_opt/util.py", line 55, in acq_max
    res = minimize(lambda x: -ac(x.reshape(1, -1), gp=gp, y_max=y_max),
  File "/mnt/mesos/sandbox/.local/lib/python3.8/site-packages/scipy/optimize/_minimize.py", line 617, in minimize
    return _minimize_lbfgsb(fun, x0, args, jac, bounds,
  File "/mnt/mesos/sandbox/.local/lib/python3.8/site-packages/scipy/optimize/lbfgsb.py", line 294, in _minimize_lbfgsb
    raise ValueError("LBFGSB - one of the lower bounds is greater than an upper bound.")
ValueError: LBFGSB - one of the lower bounds is greater than an upper bound.

I found another instance of this issue on StackOverflow. As a commenter mentioned, I am usually able to solve this problem by tweaking the values of pbounds, but it’s not a robust solution.

Thanks in advance!

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 5
Comments: 19

Most upvoted comments

I also faced this problem and solved this problem by downgrading scipy==1.7.3. But there were some difference. My “another exception” was:

File` "/app/docker-local-test/my_bayes_opt.py", line 885, in main
    optimizer = Bayesian_Opt(gprs=gprs, pbounds=pbounds,init_points=init_points, n_iter=n_iter, 
  File "/app/docker-local-test/my_bayes_opt.py", line 285, in Bayesian_Opt
    optimizer.maximize(init_points=init_points, n_iter=n_iter)
  File "/app/src/bayesian-optimization/bayes_opt/bayesian_optimization.py", line 268, in maximize
    x_probe = self.suggest(util)
  File "/app/src/bayesian-optimization/bayes_opt/bayesian_optimization.py", line 182, in suggest
    suggestion = acq_max(
  File "/app/src/bayesian-optimization/bayes_opt/util.py", line 65, in acq_max
    if max_acq is None or -res.fun[0] >= max_acq:
TypeError: 'float' object is not subscriptable

Tracing back to bayes-opt package, I found that minimize func from scipy.optimize package was used: see here. The latest scipy.optimize.minimize object returns fun (referred to res.fun in this line in bayes-opt pkg) as string (seen in scipy official doc) which caused my “another exception”.

Try downgrading scipy==1.7.3 might help.

lingxiaolyu191231 on Mar 2, 2022

Ugh, and now the problem is back. No idea why.

noa-codes on Feb 10, 2021

Thanks @daeh - the root cause of that error is #300 - which we will hopefully get fixed in the coming weeks.

bwheelz36 on May 16, 2022

I also faced this problem. But here are some differences. My “another exception” is "ValueError: Complex data not supported“. After checking my optimization function, I find its result is a complex number. When I adjust the function, the optimization procedure works properly.

KeshawnHou on Aug 27, 2021

@noa-codes FYI, not sure if this will help, but I was getting the Queue is empty error and managed to fix it by downgrading scikit-learn to 0.22 (pip install scikit-learn==0.22) based on thread #243.

That worked perfectly, thank you @kasimte!

noa-codes on Feb 8, 2021