che: Multi-user Che fails to start on k8s using helm

Description

multiuser fails to complete installation when using minikube, and the chectl operator.

Reproduction Steps

chectl server:start --multiuser …

    > Che pod bootstrap
      √ scheduling...done.
      √ downloading images...done.
      × starting
        → ERR_TIMEOUT: Timeout set to pod ready timeout 130000
      Retrieving Che Server URL
      Che status check
Error: ERR_TIMEOUT: Timeout set to pod ready timeout 130000
    at KubeHelper.<anonymous> (C:/snapshot/chectl/lib/api/kube.js:0:0)
    at Generator.next (<anonymous>)
    at fulfilled (C:/snapshot/chectl/node_modules/tslib/tslib.js:107:62)

OS and version:
Tried on both centos, and windows. I’m running latest. chectl/0.0.2-33a6fa1 win32-x64 node-v10.4.1

Diagnostics: Error on the che pod

Error on the che pod logs
Using embedded assembly...
2019-06-25 14:05:41,894[main]             [INFO ] [o.a.c.s.VersionLoggerListener 89]    - Server version:        Apache Tomcat/8.5.35
2019-06-25 14:05:41,896[main]             [INFO ] [o.a.c.s.VersionLoggerListener 91]    - Server built:          Nov 3 2018 17:39:20 UTC
2019-06-25 14:05:41,897[main]             [INFO ] [o.a.c.s.VersionLoggerListener 93]    - Server number:         8.5.35.0
2019-06-25 14:05:41,897[main]             [INFO ] [o.a.c.s.VersionLoggerListener 95]    - OS Name:               Linux
2019-06-25 14:05:41,897[main]             [INFO ] [o.a.c.s.VersionLoggerListener 97]    - OS Version:            4.15.0
2019-06-25 14:05:41,897[main]             [INFO ] [o.a.c.s.VersionLoggerListener 99]    - Architecture:          amd64
2019-06-25 14:05:41,898[main]             [INFO ] [o.a.c.s.VersionLoggerListener 101]   - Java Home:             /usr/lib/jvm/java-1.8-openjdk/jre
2019-06-25 14:05:41,898[main]             [INFO ] [o.a.c.s.VersionLoggerListener 103]   - JVM Version:           1.8.0_191-b12
2019-06-25 14:05:41,898[main]             [INFO ] [o.a.c.s.VersionLoggerListener 105]   - JVM Vendor:            Oracle Corporation
2019-06-25 14:05:41,898[main]             [INFO ] [o.a.c.s.VersionLoggerListener 107]   - CATALINA_BASE:         /home/user/eclipse-che/tomcat
2019-06-25 14:05:41,899[main]             [INFO ] [o.a.c.s.VersionLoggerListener 109]   - CATALINA_HOME:         /home/user/eclipse-che/tomcat
2019-06-25 14:05:41,899[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Djava.util.logging.config.file=/home/user/eclipse-che//tomcat/conf/logging.properties
2019-06-25 14:05:41,899[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
2019-06-25 14:05:41,899[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -XX:MaxRAMFraction=2
2019-06-25 14:05:41,899[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -XX:+UseParallelGC
2019-06-25 14:05:41,900[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -XX:MinHeapFreeRatio=10
2019-06-25 14:05:41,900[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -XX:MaxHeapFreeRatio=20
2019-06-25 14:05:41,900[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -XX:GCTimeRatio=4
2019-06-25 14:05:41,900[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -XX:AdaptiveSizePolicyWeight=90
2019-06-25 14:05:41,900[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -XX:+UnlockExperimentalVMOptions
2019-06-25 14:05:41,901[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -XX:+UseCGroupMemoryLimitForHeap
2019-06-25 14:05:41,901[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dsun.zip.disableMemoryMapping=true
2019-06-25 14:05:41,901[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Xms20m
2019-06-25 14:05:41,901[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dche.docker.network=bridge
2019-06-25 14:05:41,901[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dport.http=8080
2019-06-25 14:05:41,901[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dche.home=/home/user/eclipse-che/
2019-06-25 14:05:41,902[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dche.logs.dir=/logs/
2019-06-25 14:05:41,902[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dche.logs.level=INFO
2019-06-25 14:05:41,902[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Djuli-logback.configurationFile=file:/home/user/eclipse-che//tomcat/conf/tomcat-logger.xml
2019-06-25 14:05:41,902[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Djdk.tls.ephemeralDHKeySize=2048
2019-06-25 14:05:41,902[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
2019-06-25 14:05:41,903[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0022
2019-06-25 14:05:41,903[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dcom.sun.management.jmxremote
2019-06-25 14:05:41,903[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dcom.sun.management.jmxremote.ssl=false
2019-06-25 14:05:41,904[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dcom.sun.management.jmxremote.authenticate=false
2019-06-25 14:05:41,904[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dche.local.conf.dir=/home/user/eclipse-che//tomcat/conf/
2019-06-25 14:05:41,904[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dignore.endorsed.dirs=
2019-06-25 14:05:41,904[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dcatalina.base=/home/user/eclipse-che//tomcat
2019-06-25 14:05:41,905[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Dcatalina.home=/home/user/eclipse-che//tomcat
2019-06-25 14:05:41,905[main]             [INFO ] [o.a.c.s.VersionLoggerListener 115]   - Command line argument: -Djava.io.tmpdir=/home/user/eclipse-che//tomcat/temp
2019-06-25 14:05:41,989[main]             [INFO ] [o.a.c.http11.Http11NioProtocol 560]  - Initializing ProtocolHandler ["http-nio-8080"]
2019-06-25 14:05:41,997[main]             [INFO ] [o.a.t.util.net.NioSelectorPool 67]   - Using a shared selector for servlet write/read
2019-06-25 14:05:42,005[main]             [INFO ] [o.a.catalina.startup.Catalina 649]   - Initialization processed in 321 ms
2019-06-25 14:05:42,017[main]             [INFO ] [c.m.JmxRemoteLifecycleListener 336]  - The JMX Remote Listener has configured the registry on port [32001] and the server on port [32101] for the [Platform] server
2019-06-25 14:05:42,017[main]             [INFO ] [o.a.c.core.StandardService 416]      - Starting service [Catalina]
2019-06-25 14:05:42,017[main]             [INFO ] [o.a.c.core.StandardEngine 259]       - Starting Servlet Engine: Apache Tomcat/8.5.35
2019-06-25 14:05:42,449[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 957]        - Deploying web application archive [/home/user/eclipse-che/tomcat/webapps/swagger.war]
2019-06-25 14:05:42,539[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 1020]       - Deployment of web application archive [/home/user/eclipse-che/tomcat/webapps/swagger.war] has finished in [89] ms
2019-06-25 14:05:42,540[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 957]        - Deploying web application archive [/home/user/eclipse-che/tomcat/webapps/api.war]
2019-06-25 14:05:43,417[ost-startStop-1]  [INFO ] [.e.c.c.d.JNDIDataSourceFactory 63]   - This=org.eclipse.che.core.db.postgresql.PostgreSQLJndiDataSourceFactory@5724fcab obj=ResourceRef[className=javax.sql.DataSource,factoryClassLocation=null,factoryClassName=org.apache.naming.factory.ResourceFactory,{type=scope,content=Shareable},{type=auth,content=Container},{type=singleton,content=true},{type=factory,content=org.eclipse.che.api.CommonJndiDataSourceFactory}] name=che Context=org.apache.naming.NamingContext@6a839d1d environment={}
2019-06-25 14:05:45,620[ost-startStop-1]  [ERROR] [o.a.c.c.C.[.[localhost].[/api] 4798] - Exception sending context initialized event to listener instance of class [org.eclipse.che.inject.CheBootstrap]
com.google.inject.CreationException: Unable to create injector, see the following errors:

1) No implementation for java.util.Map<java.lang.String, java.util.Set<java.lang.String>> annotated with @com.google.inject.name.Named(value=allowedEnvironmentTypeUpgrades) was bound.
  while locating java.util.Map<java.lang.String, java.util.Set<java.lang.String>> annotated with @com.google.inject.name.Named(value=allowedEnvironmentTypeUpgrades)
    for the 2nd parameter of org.eclipse.che.workspace.infrastructure.kubernetes.devfile.KubernetesEnvironmentProvisioner.<init>(KubernetesEnvironmentProvisioner.java:59)
  while locating org.eclipse.che.workspace.infrastructure.kubernetes.devfile.KubernetesEnvironmentProvisioner
    for the 2nd parameter of org.eclipse.che.workspace.infrastructure.kubernetes.devfile.KubernetesComponentToWorkspaceApplier.<init>(KubernetesComponentToWorkspaceApplier.java:66)
  at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInfraModule.lambda$configure$1(KubernetesInfraModule.java:194) (via modules: org.eclipse.che.api.deploy.WsMasterModule -> org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInfraModule)

2) No implementation for java.util.Map<java.lang.String, java.util.Set<java.lang.String>> annotated with @com.google.inject.name.Named(value=allowedEnvironmentTypeUpgrades) was bound.
  while locating java.util.Map<java.lang.String, java.util.Set<java.lang.String>> annotated with @com.google.inject.name.Named(value=allowedEnvironmentTypeUpgrades)
    for the 2nd parameter of org.eclipse.che.workspace.infrastructure.kubernetes.devfile.KubernetesEnvironmentProvisioner.<init>(KubernetesEnvironmentProvisioner.java:59)
  while locating org.eclipse.che.workspace.infrastructure.kubernetes.devfile.KubernetesEnvironmentProvisioner
    for the 2nd parameter of org.eclipse.che.workspace.infrastructure.kubernetes.devfile.DockerimageComponentToWorkspaceApplier.<init>(DockerimageComponentToWorkspaceApplier.java:78)
  at org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInfraModule.lambda$configure$1(KubernetesInfraModule.java:197) (via modules: org.eclipse.che.api.deploy.WsMasterModule -> org.eclipse.che.workspace.infrastructure.kubernetes.KubernetesInfraModule)

2 errors
        at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:543)
        at com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java:159)
        at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:106)
        at com.google.inject.Guice.createInjector(Guice.java:87)
        at org.everrest.guice.servlet.EverrestGuiceContextListener.getInjector(EverrestGuiceContextListener.java:140)
        at com.google.inject.servlet.GuiceServletContextListener.contextInitialized(GuiceServletContextListener.java:45)
        at org.everrest.guice.servlet.EverrestGuiceContextListener.contextInitialized(EverrestGuiceContextListener.java:85)
        at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4792)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5256)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
        at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:754)
        at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:730)
        at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:734)
        at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:985)
        at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1857)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
2019-06-25 14:05:45,620[ost-startStop-1]  [ERROR] [o.a.c.core.StandardContext 5257]     - One or more listeners failed to start. Full details will be found in the appropriate container log file
2019-06-25 14:05:45,621[ost-startStop-1]  [ERROR] [o.a.c.core.StandardContext 5308]     - Context [/api] startup failed due to previous errors
2019-06-25 14:05:45,637[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 1020]       - Deployment of web application archive [/home/user/eclipse-che/tomcat/webapps/api.war] has finished in [3,097] ms
2019-06-25 14:05:45,637[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 957]        - Deploying web application archive [/home/user/eclipse-che/tomcat/webapps/docs.war]
2019-06-25 14:05:45,990[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 1020]       - Deployment of web application archive [/home/user/eclipse-che/tomcat/webapps/docs.war] has finished in [353] ms
2019-06-25 14:05:45,990[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 957]        - Deploying web application archive [/home/user/eclipse-che/tomcat/webapps/workspace-loader.war]
2019-06-25 14:05:46,044[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 1020]       - Deployment of web application archive [/home/user/eclipse-che/tomcat/webapps/workspace-loader.war] has finished in [54] ms
2019-06-25 14:05:46,045[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 957]        - Deploying web application archive [/home/user/eclipse-che/tomcat/webapps/ROOT.war]
2019-06-25 14:05:46,868[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 1020]       - Deployment of web application archive [/home/user/eclipse-che/tomcat/webapps/ROOT.war] has finished in [823] ms
2019-06-25 14:05:46,869[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 957]        - Deploying web application archive [/home/user/eclipse-che/tomcat/webapps/dashboard.war]
2019-06-25 14:05:47,096[ost-startStop-1]  [INFO ] [o.a.c.startup.HostConfig 1020]       - Deployment of web application archive [/home/user/eclipse-che/tomcat/webapps/dashboard.war] has finished in [227] ms
2019-06-25 14:05:47,101[main]             [INFO ] [o.a.c.http11.Http11NioProtocol 588]  - Starting ProtocolHandler ["http-nio-8080"]
2019-06-25 14:05:47,113[main]             [INFO ] [o.a.catalina.startup.Catalina 700]   - Server startup in 5107 ms

Received SIGTERM
Stopping Che server running on localhost:8080
2019-06-25 14:07:01,223[main]             [INFO ] [o.a.c.core.StandardServer 524]       - A valid shutdown command was received via the shutdown port. Stopping the Server instance.
2019-06-25 14:07:01,224[main]             [INFO ] [o.a.c.http11.Http11NioProtocol 609]  - Pausing ProtocolHandler ["http-nio-8080"]
2019-06-25 14:07:01,242[main]             [INFO ] [o.a.c.core.StandardService 480]      - Stopping service [Catalina]
2019-06-25 14:07:01,285[main]             [INFO ] [o.a.c.http11.Http11NioProtocol 629]  - Stopping ProtocolHandler ["http-nio-8080"]
2019-06-25 14:07:01,289[main]             [INFO ] [o.a.c.http11.Http11NioProtocol 643]  - Destroying ProtocolHandler ["http-nio-8080"]

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 23 (14 by maintainers)

Most upvoted comments

@gito0o the PR hasn’t been merged yet. We are working on it but it’s not ready yet. I will provide an update when the PR is merged and a new version of chectl is released.

@gito0o If that works for you that’s great. Otherwise, until the PR get merged there are 2 options:

  1. use helm with multi-user support:
chectl server:start --installer=helm --multiuser
  1. clone eclipse/che-operator repository and run the deploy script as described here https://github.com/eclipse/che-operator/#how-to-deploy