open-liberty: J2CA0030E: Method enlist caught java.lang.IllegalStateException: Illegal attempt to enlist 2PC XAResource without recovery information
Hi there,
one of our applications (pretty legacy one) using XAtransactions with DB2 started to fail with below error. After switching from 23.0.0.1 to 23.0.0.6. No code or java version changes except Liberty.
I’ve reviewed history of related classes changes and seems like indeed something was done. https://github.com/OpenLiberty/open-liberty/commits/cada2b83d9ca4a79f2e57a0366dc1d89fb14f938/dev/com.ibm.ws.jdbc/src/com/ibm/ws/rsadapter/impl https://github.com/OpenLiberty/open-liberty/commits/cada2b83d9ca4a79f2e57a0366dc1d89fb14f938/dev/com.ibm.ws.jca.cm/src/com/ibm/ejs/j2c https://github.com/OpenLiberty/open-liberty/commits/cada2b83d9ca4a79f2e57a0366dc1d89fb14f938/dev/com.ibm.ws.transaction/src/com/ibm/ws/transaction/services https://github.com/OpenLiberty/open-liberty/commits/cada2b83d9ca4a79f2e57a0366dc1d89fb14f938/dev/com.ibm.tx.jta/src/com/ibm/tx/jta/impl
@nmittles, @jmstephensgit, @tabishop @neilgsyoung, @jonhawkes, @dazavala could you please take a look? You have contributed recently 🥇
[8/3/23 11:09:13:317 UTC] 00000059 com.ibm.ejs.j2c.XATransactionWrapper E J2CA0030E: Method enlist caught java.lang.IllegalStateException: Illegal attempt to enlist 2PC XAResource without recovery information
at com.ibm.tx.jta.impl.TransactionImpl.enlistResource(TransactionImpl.java:2077)
at com.ibm.tx.jta.embeddable.impl.EmbeddableTranManagerSet.enlist(EmbeddableTranManagerSet.java:141)
at com.ibm.ws.transaction.services.TransactionManagerService.enlist(TransactionManagerService.java:419)
at com.ibm.ejs.j2c.XATransactionWrapper.enlist(XATransactionWrapper.java:460)
at com.ibm.ejs.j2c.ConnectionManager.lazyEnlist(ConnectionManager.java:1429)
at com.ibm.ws.rsadapter.impl.WSRdbManagedConnectionImpl.lazyEnlistInGlobalTran(WSRdbManagedConnectionImpl.java:1831)
at com.ibm.ws.rsadapter.jdbc.WSJdbcConnection.beginTransactionIfNecessary(WSJdbcConnection.java:391)
at com.ibm.ws.rsadapter.jdbc.WSJdbcConnection.getMetaData(WSJdbcConnection.java:1616)
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:70)
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:35)
at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:101)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:263)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:237)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:214)
at org.hibernate.id.factory.internal.DefaultIdentifierGeneratorFactory.injectServices(DefaultIdentifierGeneratorFactory.java:152)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.injectDependencies(AbstractServiceRegistryImpl.java:286)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.initializeService(AbstractServiceRegistryImpl.java:243)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.getService(AbstractServiceRegistryImpl.java:214)
at org.hibernate.boot.internal.InFlightMetadataCollectorImpl.<init>(InFlightMetadataCollectorImpl.java:175)
at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.complete(MetadataBuildingProcess.java:118)
at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.build(MetadataBuildingProcess.java:83)
at org.hibernate.boot.internal.MetadataBuilderImpl.build(MetadataBuilderImpl.java:473)
at org.hibernate.boot.internal.MetadataBuilderImpl.build(MetadataBuilderImpl.java:84)
at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:689)
at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:724)
at org.springframework.orm.hibernate5.LocalSessionFactoryBean.buildSessionFactory(LocalSessionFactoryBean.java:616)
at org.springframework.orm.hibernate5.LocalSessionFactoryBean.afterPropertiesSet(LocalSessionFactoryBean.java:600)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1863)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1800)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:620)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:330)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:113)
at org.springframework.beans.factory.support.ConstructorResolver.resolveConstructorArguments(ConstructorResolver.java:707)
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:198)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:953)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:144)
at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:95)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:211)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:117)
at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:311)
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:296)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:953)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:144)
at org.springframework.context.support.ClassPathXmlApplicationContext.<init>(ClassPathXmlApplicationContext.java:85)
at com.*.as1.common5.spring.util.AS1SpringContainers.doRegisterContainer(AS1SpringContainers.java:133)
at com.*.as1.common5.spring.util.AS1SpringContainers.registerMainContainer(AS1SpringContainers.java:115)
at com.*.as1.common5.spring.util.AS1SpringContainers.registerMainContainer(AS1SpringContainers.java:82)
at com.*.a14.ejb.A14ApplicationInitialization.initializeApplication(A14ApplicationInitialization.java:47)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.ibm.ejs.container.interceptors.InterceptorProxy.invokeInterceptor(InterceptorProxy.java:202)
at com.ibm.ejs.container.interceptors.InvocationContextImpl.proceed(InvocationContextImpl.java:587)
at com.ibm.ws.cdi.ejb.impl.InterceptorChain.proceed(InterceptorChain.java:121)
at com.ibm.ws.cdi.ejb.impl.EJBCDIInterceptorWrapper.invokeInterceptors(EJBCDIInterceptorWrapper.java:133)
at com.ibm.ws.cdi.ejb.impl.EJBCDIInterceptorWrapper.postConstruct(EJBCDIInterceptorWrapper.java:69)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.ibm.ejs.container.interceptors.InterceptorProxy.invokeInterceptor(InterceptorProxy.java:191)
at com.ibm.ejs.container.interceptors.InvocationContextImpl.proceed(InvocationContextImpl.java:579)
at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:73)
at com.ibm.ws.cdi.ejb.interceptor.WeldSessionBeanInterceptorWrapper.aroundInvoke(WeldSessionBeanInterceptorWrapper.java:60)
at com.ibm.ws.cdi.ejb.interceptor.WeldSessionBeanInterceptorWrapper.postConstruct(WeldSessionBeanInterceptorWrapper.java:69)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.ibm.ejs.container.interceptors.InterceptorProxy.invokeInterceptor(InterceptorProxy.java:191)
at com.ibm.ejs.container.interceptors.InvocationContextImpl.proceed(InvocationContextImpl.java:579)
at com.ibm.ejs.container.interceptors.InvocationContextImpl.doLifeCycle(InvocationContextImpl.java:328)
at com.ibm.ejs.container.SingletonBeanO.callTransactionalLifecycleInterceptors(SingletonBeanO.java:223)
at com.ibm.ejs.container.SingletonBeanO.initialize(SingletonBeanO.java:309)
at com.ibm.ejs.container.BeanOFactory.create(BeanOFactory.java:105)
at com.ibm.ejs.container.EJSHome.createSingletonBeanO(EJSHome.java:3478)
at com.ibm.ejs.csi.EJBApplicationMetaData.createStartupBeans(EJBApplicationMetaData.java:1005)
at com.ibm.ejs.csi.EJBApplicationMetaData.startedModule(EJBApplicationMetaData.java:739)
at com.ibm.ws.ejbcontainer.osgi.internal.EJBRuntimeImpl.start(EJBRuntimeImpl.java:1123)
at com.ibm.ws.ejbcontainer.osgi.internal.EJBModuleRuntimeContainerImpl.startModule(EJBModuleRuntimeContainerImpl.java:166)
at com.ibm.ws.app.manager.module.internal.ModuleHandlerBase.deployModule(ModuleHandlerBase.java:101)
at com.ibm.ws.app.manager.module.internal.DeployedModuleInfoImpl.installModule(DeployedModuleInfoImpl.java:51)
at com.ibm.ws.app.manager.module.internal.SimpleDeployedAppInfoBase.deployModules(SimpleDeployedAppInfoBase.java:599)
at com.ibm.ws.app.manager.module.internal.SimpleDeployedAppInfoBase.installApp(SimpleDeployedAppInfoBase.java:513)
at com.ibm.ws.app.manager.module.internal.DeployedAppInfoBase.deployApp(DeployedAppInfoBase.java:351)
at com.ibm.ws.app.manager.ear.internal.EARApplicationHandlerImpl.install(EARApplicationHandlerImpl.java:79)
at com.ibm.ws.app.manager.internal.statemachine.StartAction.execute(StartAction.java:184)
at com.ibm.ws.app.manager.internal.statemachine.ApplicationStateMachineImpl.enterState(ApplicationStateMachineImpl.java:1369)
at com.ibm.ws.app.manager.internal.statemachine.ApplicationStateMachineImpl.run(ApplicationStateMachineImpl.java:912)
at com.ibm.ws.threading.internal.ExecutorServiceImpl$RunnableWrapper.run(ExecutorServiceImpl.java:247)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:826)
while trying to enlist resources from DataSource jdbc/A14 with the Transaction Manager for the current transaction, and threw a ResourceException.
[8/3/23 11:09:13:377 UTC] 00000059 com.ibm.ws.logging.internal.impl.IncidentImpl I FFDC1015I: An FFDC Incident has been created: "javax.resource.ResourceException: enlist: caught Exception com.ibm.ws.rsadapter.jdbc.WSJdbcConnection.beginTransactionIfNecessary 324" at ffdc_23.08.03_13.09.13.2.log
[8/3/23 11:09:13:385 UTC] 00000059 com.ibm.ws.logging.internal.impl.IncidentImpl I FFDC1015I: An FFDC Incident has been created: "java.sql.SQLException: enlist: caught Exception com.ibm.ws.rsadapter.jdbc.WSJdbcConnection.getMetaData 922" at ffdc_23.08.03_13.09.13.3.log
Best regards, K.M.
About this issue
- Original URL
- State: open
- Created 3 years ago
- Comments: 16 (6 by maintainers)
@tschurins I take it you’re referring to the original issue here - the IllegalStateException. We’ll need additional info… Please supply a “system dump” and trace (as per https://www.ibm.com/support/pages/set-trace-and-get-full-dump-websphere-liberty) using trace string “*=info:RRA=all:WAS.j2c=all:com.ibm.ws.jdbc.=all:Transaction=all”
Trace should be taken so that it includes app server startup as well as the failure occuring. There’s a two-step process here. The enlist/enlistResource is the second step - it passes in a recoveryIndex parameter (-1 in earlier trace snippet) - that -1 indicates a failure in the first step where the recoveryIndex is generated by a call to registerResourceInfo which will default to -1 if there’s a failure. Looking at the code, it’s possible that there some timing window that that method is called before the transaction service is ready that the current tests don’t run into. It doesn’t look like the code will either wait, nor retry in the event of this condition which I find surprising.
As Jim noted previously “If you have support, please open a skill case for this issue. It’s likely we will need to request trace and do more extensive work to figure out this problem.”