midi-model: RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

我更新了代码下来,生成的时候大概率会碰到这个错误。 Scores 会传入nan的情况

CUDA Version: 11.8 torch==2.0.1cu118 transformers==4.33.1 pytorch-lightning==2.0.7

具体:

打印sample_top_p_k参数,model.sample_top_p_k(scores, top_p, top_k) def sample_top_p_k(self, probs, p, k): # Debugging code print(“probs:”, probs) print(“Contains inf:”, torch.isinf(probs).any()) print(“Contains nan:”, torch.isnan(probs).any()) print(“Contains negative:”, (probs < 0).any())

probs: tensor([[[nan, nan, nan, …, nan, nan, nan]]], device=‘cuda:0’) Contains inf: tensor(False, device=‘cuda:0’) Contains nan: tensor(True, device=‘cuda:0’) Contains negative: tensor(False, device=‘cuda:0’) generating: 77%|███████▋ | 1149/1493 [01:55<00:34, 9.94it/s] Traceback (most recent call last): File “C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\routes.py”, line 427, in run_predict output = await app.get_blocks().process_api( File “C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py”, line 1323, in process_api result = await self.call_function( File “C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\blocks.py”, line 1067, in call_function prediction = await utils.async_iteration(iterator) File “C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\utils.py”, line 336, in async_iteration return await iterator.anext() File “C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\utils.py”, line 329, in anext return await anyio.to_thread.run_sync( File “C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio\to_thread.py”, line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File “C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio_backends_asyncio.py”, line 937, in run_sync_in_worker_thread return await future File “C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\anyio_backends_asyncio.py”, line 867, in run result = context.run(func, *args) File “C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\gradio\utils.py”, line 312, in run_sync_iterator_async return next(iterator) File “D:\Data\Git Repository\midi-model\app.py”, line 128, in run for i, token_seq in enumerate(generator): File “C:\Users\HOLA\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\utils_contextlib.py”, line 56, in generator_context response = gen.send(request) File “D:\Data\Git Repository\midi-model\app.py”, line 62, in generate sample = model.sample_top_p_k(scores, top_p, top_k) File “D:\Data\Git Repository\midi-model\midi_model.py”, line 72, in sample_top_p_k next_token = torch.multinomial(probs_sort.reshape(-1, shape[-1]), num_samples=1).reshape(*shape[:-1], 1) RuntimeError: probability tensor contains either inf, nan or element < 0

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 18 (7 by maintainers)

Most upvoted comments

这就奇怪了,我之前用的版本代码好好的,现在推理也不行