xgboost: XGBoost 2.0 doesn't support Optuna Pruning Callback

@hanhanwu Thank you for providing the Python snippet. I was able to reproduce the error.

Diagnosis The current version of Optuna has a bug that made XGBoostPruningCallback incompatible with XGBoost 2.0. The bug was fixed in this pull request: https://github.com/optuna/optuna/pull/4921. Unfortunately, the fix is not yet part of the latest release of Optuna (3.3.0).

Solution For now, you can install the development version of Optuna in your Python environment:

pip install git+https://github.com/optuna/optuna.git

This should fix the error.

@hanhanwu Thank you for providing the Python snippet. I was able to reproduce the error.

Diagnosis The current version of Optuna has a bug that made XGBoostPruningCallback incompatible with XGBoost 2.0. The bug was fixed in this pull request: optuna/optuna#4921. Unfortunately, the fix is not yet part of the latest release of Optuna (3.3.0).

Solution For now, you can install the development version of Optuna in your Python environment:

pip install git+https://github.com/optuna/optuna.git

This should fix the error.

pip install git+https://github.com/optuna/optuna.git

Thank you guys!

You guys are still very young, having lots of future potentials. I honestly think what you’re doing here has more meaning than you think now. Don’t plan to change you opinions, as a gift, want to share 2 stories:

• This is one of the examples when I was interacting with FLAML (another open source tool). I asked them many questions, sometimes, even got their answers during the weekends and holidays… they even ask me feedbacks (you don’t need to work this hard). I just feel, they always made me believed they want to make this open source tool better and better.
• This is how ZenML cofounder helped me solve a problem. Who can be busier than a cofounder… Finally he helped me found a great solution.

These people all inspired me a lot in work. Sometimes, it’s just this type of small things, Q&A made a difference. I wonder 5 years later, how are you guys going to think 😃 Anyway, thank you all!

Apologies for closing the issue prematurely. Next time, we won’t close issues until they are resolved. Also I am sorry that I rushed my 2nd response (https://github.com/dmlc/xgboost/issues/9608#issuecomment-1734600728). It was technically correct but not very helpful. I will try to do better next time.

Since finally you still spent the effort to dig out the reason, why it didn’t happen earlier?

Mostly because we are limited in our time. Like other open source projects, we try to provide support at the best effort basis. (Of course, this does not excuse closing this issue prematurely. I am just providing a context here.)

Sure, apologies for the inconvenience. Sometimes I’m too used to these types of issues and assume that people will get the answer by looking into some of the code, hence the referenced link to optuna. There’s definitely room for improvement.

It’s mostly open source project’s issues, you see, it’s not a product, and we are not customer service. I started contribution as a student in school. We expect the community to cooperate and help given that xgboost is a library and users are developers themselves, at least try to give some quick debugging to the issues at hand, it’s not unusual for me to ask others do they want to open a PR.

Feel free to reopen the issue if the answer is unsatisfying, we can provide further assistance at best effort.

You need the latest optuna: https://github.com/optuna/optuna/blob/a920a91e4e5a32ade707f3e9819fff0f96cc7946/optuna/integration/xgboost.py#L16