tensorrtllm_backend: The stop_words does not work with codeLlama?

I use codeLlama-7b model, and use ensemble of “https://github.com/triton-inference-server/tensorrtllm_backend/tree/e514b4af5ec87477b095d3ba6fe63cc7b797055f/all_models/inflight_batcher_llm”.

The prompt is “def quickSort”, the stop_words is “quickSort,quickSort(right)”, and I print the result of “_to_word_list_format” stop_words = self._to_word_list_format(stop_words_dict) self.logger.log_info(f"================== preprocessing execute stop_words: {stop_words}")

I get the following result: model.py:137] ================== preprocessing execute stop_words: [[[ 1 4996 13685 1 4996 13685 29898 1266 29897] [ 3 9 -1 -1 -1 -1 -1 -1 -1]]]

However, the code completion result is: def quickSort(arr):\n if len(arr) \u003c= 1:\n return arr\n else:\n pivot = arr[0]\n left = []\n right = []\n for i in arr:\n if i \u003c pivot:\n left.append(i))\n else:\n right.append(i))\n return quickSort(left)) + [pivot]) + quickSort(right))\n\n\n\n\n\n\n

As we can see, infer does not stop early when it meets the “quickSort” or “quickSort(right)”, this is weird. Is there something wrong with where I am using it?

Thank u.