cloudstack: NPE on simulator launch

Now simulator works for me as well. Thanks everyone for engagement 😄

Okay, thanks everyone for your help. Let’s settle this down. I’ll make a new build and try things out. Keep in touch

@tampler I built a new docker image from the 4.16 branch and observed the same behavior as reported before - I noticed the NPE, but MS is up and running

[17:30:41] pearl@pearl-XPS-15-7590:~/lab/cloudstack$ docker image ls
REPOSITORY                                                TAG       IMAGE ID       CREATED          SIZE
cloudstack/simulator                                      latest    d8640fffd454   31 minutes ago   13GB


[17:32:16] pearl@pearl-XPS-15-7590:~/lab/cloudstack$ docker ps
CONTAINER ID   IMAGE                         COMMAND                  CREATED          STATUS          PORTS                                                           NAMES
a8b95f3944f7   cloudstack/simulator:latest   "/usr/bin/supervisord"   10 minutes ago   Up 10 minutes   8080/tcp, 8096/tcp, 0.0.0.0:8080->5050/tcp, :::8080->5050/tcp   sim


### Grepping for the NPE & the subsequent logs inside the container
root@a8b95f3944f7:~# cat vmops.log | grep NullPointerException -A30
java.lang.NullPointerException
	at com.cloud.upgrade.SystemVmTemplateRegistration.registerTemplate(SystemVmTemplateRegistration.java:676)
	at com.cloud.storage.StorageManagerImpl$3.doInTransactionWithoutResult(StorageManagerImpl.java:2845)
	at com.cloud.utils.db.TransactionCallbackNoReturn.doInTransaction(TransactionCallbackNoReturn.java:25)
	at com.cloud.utils.db.Transaction$2.doInTransaction(Transaction.java:50)
	at com.cloud.utils.db.Transaction.execute(Transaction.java:40)
	at com.cloud.utils.db.Transaction.execute(Transaction.java:47)
	at com.cloud.storage.StorageManagerImpl.registerSystemVmTemplateOnFirstNfsStore(StorageManagerImpl.java:2802)
	at com.cloud.storage.StorageManagerImpl.discoverImageStore(StorageManagerImpl.java:2786)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
	at com.sun.proxy.$Proxy114.discoverImageStore(Unknown Source)
	at org.apache.cloudstack.api.command.admin.storage.AddImageStoreCmd.execute(AddImageStoreCmd.java:135)
	at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:156)
	at com.cloud.api.ApiServer.queueCommand(ApiServer.java:772)
	at com.cloud.api.ApiServer.handleRequest(ApiServer.java:596)
	at com.cloud.api.ApiServer.handle(ApiServer.java:502)
	at org.apache.http.protocol.HttpService.doService(HttpService.java:437)
	at org.apache.http.protocol.HttpService.handleRequest(HttpService.java:342)
	at com.cloud.api.ApiServer$WorkerTask.runInContext(ApiServer.java:1326)
	at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:48)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:55)
	at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:102)
--
Caused by: java.lang.NullPointerException
	at com.cloud.upgrade.SystemVmTemplateRegistration.registerTemplate(SystemVmTemplateRegistration.java:676)
	at com.cloud.storage.StorageManagerImpl$3.doInTransactionWithoutResult(StorageManagerImpl.java:2845)
	at com.cloud.utils.db.TransactionCallbackNoReturn.doInTransaction(TransactionCallbackNoReturn.java:25)
	... 32 more
2022-02-07 08:56:24,131 INFO  [c.c.u.SystemVmTemplateRegistration] (ApiServer-8:ctx-68a2e5b0 ctx-e497c9bd) (logid:b8cb6485) Unmounting store
2022-02-07 08:56:24,131 DEBUG [c.c.u.s.Script] (ApiServer-8:ctx-68a2e5b0 ctx-e497c9bd) (logid:b8cb6485) Executing: /bin/bash -c sudo umount /tmp/tmp10742134008486500295 
2022-02-07 08:56:24,132 DEBUG [c.c.u.s.Script] (ApiServer-8:ctx-68a2e5b0 ctx-e497c9bd) (logid:b8cb6485) Executing while with timeout : 3600000
2022-02-07 08:56:24,139 DEBUG [c.c.u.s.Script] (ApiServer-8:ctx-68a2e5b0 ctx-e497c9bd) (logid:b8cb6485) Exit value is 32
2022-02-07 08:56:24,139 DEBUG [c.c.u.s.Script] (ApiServer-8:ctx-68a2e5b0 ctx-e497c9bd) (logid:b8cb6485) umount: /tmp/tmp10742134008486500295: no mount point specified.
2022-02-07 08:56:24,172 INFO  [c.c.c.ConfigurationManagerImpl] (ApiServer-9:ctx-3056b89e ctx-70b4389a) (logid:8d178580) No storage traffic type was specified by admin, create default storage traffic on physical network 200 with same configure of management traffic type
2022-02-07 08:56:24,415 DEBUG [c.c.a.ApiServlet] (qtp1665366772-287:ctx-e669acd4) (logid:2f8314ef) ===START===  127.0.0.1 -- POST  
2022-02-07 08:56:24,430 DEBUG [c.c.u.AccountManagerImpl] (qtp1665366772-287:ctx-e669acd4) (logid:2f8314ef) Attempting to log in user: admin in domain 1
2022-02-07 08:56:24,432 DEBUG [o.a.c.s.a.PBKDF2UserAuthenticator] (qtp1665366772-287:ctx-e669acd4) (logid:2f8314ef) Retrieving user: admin
2022-02-07 08:56:24,708 DEBUG [c.c.u.AccountManagerImpl] (qtp1665366772-287:ctx-e669acd4) (logid:2f8314ef) CIDRs from which account 'Acct[149f6018-87c4-11ec-81c5-50eb7122da94-admin] -- Account {"id": 2, "name": "admin", "uuid": "149f6018-87c4-11ec-81c5-50eb7122da94"}' is allowed to perform API calls: 0.0.0.0/0,::/0

And I can access the UI:

Yes, using the deployDatacenter.py tool with the advancedsg configuration. You should start the UI separately with npm, but API access should be there, immediately.

@tampler Hi, Suresh is the release manager of cloudstack 4.16.1.0. He is working hard on the 4.16.1.0 release. Normally when we merge a PR which fix an issue, we close the original issue for better release tracking. If you still have the same issue, please feel free to open the issue.

good @Pearl1594

api endpoint works in my testing.

root@e974dad4bb09:~# curl http://localhost:8080/client/api 
<?xml version="1.0" encoding="UTF-8"?><errorresponse cloud-stack-version="4.17.0.0-SNAPSHOT"><errorcode>401</errorcode><errortext>unable to verify user credentials and/or request signature</errortext></errorresponse>root@e974dad4bb09:~#

@tampler I tried deploying an env with your image, and the logs indicate that the management server is running. However the service isn’t running on port 8080, but I was able to reach it via port 8096. I noticed the Dockerfile in the container, it’s slightly different from the upstream one…

$ docker exec -it  sim-tampler /bin/bash
root@e0cece3025f5:~# ps -aef
UID          PID    PPID  C STIME TTY          TIME CMD
root           1       0  0 08:31 ?        00:00:01 /usr/bin/python3 /usr/bin/supervisord
root          10       1 22 08:31 ?        00:04:06 /usr/lib/jvm/java-11-openjdk-amd64/bin/java -classpath /opt/apache-maven-3.8.4/boot/plexus-classworlds-2.6.0.jar -Dclassworlds.conf=/opt/apache-maven-3
root          12       1  0 08:31 ?        00:00:00 /bin/sh /usr/bin/mysqld_safe
mysql        198      12  3 08:31 ?        00:00:41 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/mysql/plugin --user=mysql --log-error=/var/log/mysql/error.log --pid-fil
root        2660       0  1 08:49 pts/0    00:00:00 /bin/bash
root        2672    2660  0 08:49 pts/0    00:00:00 ps -aef
root@e0cece3025f5:~# 
root@e0cece3025f5:~# 


root@e0cece3025f5:~# curl localhost:8096?command=listZones
<?xml version="1.0" encoding="UTF-8"?><listzonesresponse cloud-stack-version="4.16.1.0-SNAPSHOT"><count>1</count><zone><id>f2e6d1c8-fb62-47fd-846e-126f16095e8c</id><name>Sandbox-simulator</name><dns1>10.147.28.6</dns1><internaldns1>10.147.28.6</internaldns1><guestcidraddress>10.1.1.0/24</guestcidraddress><networktype>Advanced</networktype><securitygroupsenabled>false</securitygroupsenabled><allocationstate>Enabled</allocationstate><zonetoken>a26b0b9f-4a09-3016-8d3f-5a0bff27311b</zonetoken><dhcpprovider>VirtualRouter</dhcpprovider><localstorageenabled>false</localstorageenabled><hasannotations>false</hasannotations></zone></listzonesresponse>root@e0cece3025f5:~# 

following is the output of the same on an image built locally from the 4.16 branch:

[14:25:22] pearl@pearl-XPS-15-7590:~$ docker ps 
CONTAINER ID   IMAGE                         COMMAND                  CREATED          STATUS          PORTS                                                                NAMES
e0cece3025f5   tampler/acs-sim:lat           "/usr/bin/supervisord"   23 minutes ago   Up 23 minutes   8080-8085/tcp, 8096/tcp, 0.0.0.0:8081->5050/tcp, :::8081->5050/tcp   sim-tampler
a3e0173ad5fe   cloudstack/simulator:latest   "/usr/bin/supervisord"   25 minutes ago   Up 25 minutes   8080/tcp, 8096/tcp, 0.0.0.0:8080->5050/tcp, :::8080->5050/tcp        sim1

[14:25:24] pearl@pearl-XPS-15-7590:~$ docker image ls
REPOSITORY                                                TAG       IMAGE ID       CREATED         SIZE
cloudstack/simulator                                      latest    1a80778b0bb7   2 hours ago     3.95GB
tampler/acs-sim                                           lat       ac458281e7ef   39 hours ago    3.43GB

pearl@pearl-XPS-15-7590:~$ docker exec -it sim1 /bin/bash
root@a3e0173ad5fe:~# ps -aef
UID          PID    PPID  C STIME TTY          TIME CMD
root           1       0  0 08:30 ?        00:00:02 /usr/bin/python3 /usr/bin/supervisord
root           8       1 16 08:30 ?        00:04:11 /usr/bin/java -classpath /usr/share/maven/boot/plexus-classworlds-2.x.jar -Dclassworlds.conf=/usr/share/maven/bin/m2.conf -Dmaven.home=/usr/share/maven
root           9       1  0 08:30 ?        00:00:00 npm
root          10       1  0 08:30 ?        00:00:00 /bin/sh /usr/bin/mysqld_safe
mysql        205      10  2 08:30 ?        00:00:35 /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --plugin-dir=/usr/lib/mysql/plugin --user=mysql --log-error=/var/log/mysql/error.log --pid-fil
root         210       9  0 08:30 ?        00:00:00 sh -c vue-cli-service lint --no-fix && vue-cli-service serve
root         302     210 11 08:30 ?        00:02:58 node /root/ui/node_modules/.bin/vue-cli-service serve
root         618       0  0 08:56 pts/0    00:00:00 /bin/bash
root         631     618  0 08:56 pts/0    00:00:00 ps -aef

root@a3e0173ad5fe:~# curl localhost:5050
<!--
     Licensed to the Apache Software Foundation (ASF) under one
     or more contributor license agreements.  See the NOTICE file
     distributed with this work for additional information
     regarding copyright ownership.  The ASF licenses this file
     to you under the Apache License, Version 2.0 (the
     "License"); you may not use this file except in compliance
     with the License.  You may obtain a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

     Unless required by applicable law or agreed to in writing,
     software distributed under the License is distributed on an
.....

@tampler my apologies, I must have missed the comment that it worked about 2 weeks ago… I’ll build an image from the latest 4.16 branch and will get back to you. Thanks