rancher: Any idea what these errors are after upgrade to 1.2.0 : "State [purged] is not valid for process" ?

Rancher Version: 1.2.0, 1.2.0-rc*

Docker Version: 1.12.3

OS and where are the hosts located? (cloud, bare metal, etc): bare metal, Ubuntu 16.04

Setup Details: (single node rancher vs. HA rancher, internal DB vs. external DB) 3 nodes, but not HA.

Environment Type: (Cattle/Kubernetes/Swarm/Mesos) Cattle

Steps to Reproduce: upgrade to 1.2.x and started getting these errors in docker logs -f rancher

Results:

2016-12-01 23:53:03,271 ERROR [:] [] [] [] [ecutorService-2] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [mount.create:613801] on [17807], canceled by [State [purged] is not valid for process [volume.activate:957592] on resource [2217]]
2016-12-01 23:53:03,276 ERROR [:] [] [] [] [cutorService-15] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [mount.create:55730] on [5387], canceled by [State [purged] is not valid for process [volume.activate:957604] on resource [2217]]
2016-12-01 23:53:03,276 ERROR [:] [] [] [] [ecutorService-5] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [mount.create:685234] on [19631], canceled by [State [purged] is not valid for process [volume.activate:957600] on resource [2214]]
2016-12-01 23:53:03,276 ERROR [:] [] [] [] [cutorService-14] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [mount.create:55731] on [5388], canceled by [State [purged] is not valid for process [volume.activate:957599] on resource [2227]]
2016-12-01 23:53:03,276 ERROR [:] [] [] [] [cutorService-16] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [mount.create:613802] on [17808], canceled by [State [purged] is not valid for process [volume.activate:957596] on resource [2226]]
2016-12-01 23:53:03,276 ERROR [:] [] [] [] [ecutorService-1] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [mount.create:55733] on [5390], canceled by [State [purged] is not valid for process [volume.activate:957603] on resource [2225]]
2016-12-01 23:53:03,276 ERROR [:] [] [] [] [cutorService-13] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [mount.create:55735] on [5392], canceled by [State [purged] is not valid for process [volume.activate:957605] on resource [2226]]
2016-12-01 23:53:03,276 ERROR [:] [] [] [] [cutorService-10] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [mount.create:613804] on [17810], canceled by [State [purged] is not valid for process [volume.activate:957598] on resource [2225]]
2016-12-01 23:53:03,280 ERROR [:] [] [] [] [cutorService-11] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [mount.create:55760] on [5395], canceled by [State [purged] is not valid for process [volume.activate:957597] on resource [2223]]
2016-12-01 23:53:03,280 ERROR [:] [] [] [] [ecutorService-7] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [mount.create:613803] on [17809], canceled by [State [purged] is not valid for process [volume.activate:957593] on resource [2227]]
2016-12-01 23:53:03,280 ERROR [:] [] [] [] [ecutorService-3] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [mount.create:685235] on [19632], canceled by [State [purged] is not valid for process [volume.activate:957594] on resource [2215]]
2016-12-01 23:53:03,280 ERROR [:] [] [] [] [cutorService-12] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [mount.create:55719] on [5384], canceled by [State [purged] is not valid for process [volume.activate:957602] on resource [2226]]
2016-12-01 23:53:03,280 ERROR [:] [] [] [] [ecutorService-4] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [mount.create:55721] on [5386], canceled by [State [purged] is not valid for process [volume.activate:957595] on resource [2227]]
2016-12-01 23:53:03,280 ERROR [:] [] [] [] [ecutorService-8] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [mount.create:55718] on [5383], canceled by [State [purged] is not valid for process [volume.activate:957601] on resource [2225]]

Expected: I guess expected either more descriptive message or no error?

About this issue

  • Original URL
  • State: closed
  • Created 8 years ago
  • Reactions: 2
  • Comments: 17 (1 by maintainers)

Most upvoted comments

The same issue is with 1.5.1.

I got exactly the same issue: 2017-03-09 15:14:09,048 ERROR [:] [] [] [] [ecutorService-3] [.e.s.i.ProcessInstanceDispatcherImpl] Unknown exception running process [service.remove:24696] on [8], canceled by [State [purged] is not valid for process [instance.stop:25209] on resource [14]]

Now the service is stuk in being removed status and I can’t force to delete it, or do anything. I was only able to delete the containers inside it…

Any ideas on how to delete this zombie service? I had the problem in 1.4.0 (never used any RC in production) and now I expected that the upgrade to 1.5.1 would might fix the issue…