trl: 'NoneType' object is not subscriptable when using phi-1.5

I’m running the same code from https://github.com/huggingface/trl/blob/main/examples/scripts/ppo.py, with the only difference being that I replaced gpt-2 with phi-1.5 (https://huggingface.co/microsoft/phi-1_5). Now I’m getting the following error when trying to run the code:

'NoneType' object is not subscriptable

It appears that the problem lies with AutoModelForCausalLMWithValueHead, specifically trying to call the forward function of the model instantiated using AutoModelForCausalLMWithValueHead using the TRL library, even though model.generate still works. So basically my questions are:

• Is phi-1.5 not supported? Why is that?
• What can I do to solve this problem? I really would like to use phi-1.5 for my training.