ray: [rllib] Weights & Biases logger cannot handle objects in configuration

What is the problem?

The Weights & Biases logger cannot handle object references in RLlib configurations, for example in the callback API.

Process _WandbLoggingProcess-1:
Traceback (most recent call last):
  File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "[...]/ray/tune/integration/wandb.py", line 127, in run
    wandb.init(*self.args, **self.kwargs)
  File "[...]/wandb/__init__.py", line 1303, in init
    as_defaults=not allow_val_change)
  File "[...]/wandb/wandb_config.py", line 333, in _update
    self.persist()
  File "[...]/wandb/wandb_config.py", line 238, in persist
    conf_file.write(str(self))
  File "[...]/wandb/wandb_config.py", line 374, in __str__
    allow_unicode=True, encoding='utf-8')
  File "[...]/yaml/__init__.py", line 290, in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
  File "[...]/yaml/__init__.py", line 278, in dump_all
    dumper.represent(data)
  File "[...]/yaml/representer.py", line 27, in represent
    node = self.represent_data(data)
  File "[...]/yaml/representer.py", line 48, in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
  File "[...]/yaml/representer.py", line 207, in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
  File "[...]/yaml/representer.py", line 118, in represent_mapping
    node_value = self.represent_data(item_value)
  File "[...]/yaml/representer.py", line 48, in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
  File "[...]/yaml/representer.py", line 207, in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
  File "[...]/yaml/representer.py", line 118, in represent_mapping
    node_value = self.represent_data(item_value)
  File "[...]/yaml/representer.py", line 58, in represent_data
    node = self.yaml_representers[None](self, data)
  File "[...]/yaml/representer.py", line 231, in represent_undefined
    raise RepresenterError("cannot represent an object", data)
yaml.representer.RepresenterError: ('cannot represent an object', <class '__main__.MyCallbacks'>)

Ray version and other system information (Python version, TensorFlow version, OS):

• Ray 0.8.7
• Ubuntu 18.04
• Python 3.7

Reproduction (REQUIRED)

Please provide a script that can be run to reproduce the issue. The script should have no external library dependencies (i.e., use fake or mock data / environments):

from ray import tune
from ray.rllib.agents.ppo import PPOTrainer
from ray.rllib.agents.callbacks import DefaultCallbacks
from ray.tune.integration.wandb import WandbLogger

class MyCallbacks(DefaultCallbacks):
    def on_episode_end(self, worker, base_env, policies, episode, **kwargs):
        print("Episode ended")

tune.run(
    PPOTrainer,
    checkpoint_freq=1,
    config={
        "framework": "torch",
        "num_workers": 8,
        "num_gpus": 1,
        "env": "CartPole-v0",
        "callbacks": MyCallbacks,
        "logger_config": {
          "wandb": {
              "project": "test",
              "api_key_file": "./wandb_api_key_file",
          }
        }
    },
    stop={
        "training_iteration":10
    },
    loggers=[WandbLogger]
)

If we cannot run your script, we cannot fix your issue.

• I have verified my script runs in a clean environment and reproduces the issue.
• I have verified the issue also occurs with the latest wheels.