Repository navigation
Management Server fails to start on new install #11588
Description
problem
Followed the instructions to create a new management server with a separate mysql server. Like many others, the server fails to start. When I try to go to http::8080, I get:
HTTP ERROR 503 Service Unavailable
| URI: | /client/ |
|---|---|
| 503 | |
| Service Unavailable |
When I look at the logs, I see multiple errors, although the two that stick out for me are:
management-server.out: lots of
2025-09-05 22:24:28,011 main ERROR TcpSocketManager (TCP:localhost:4560) caught exception and will continue: java.io.IOException: Unable to create socket for localhost at port 4560 using ip address 127.0.0.1 and port 4560
management-server.log:
Unhandled configuration exception: The script injectkeys.sh failed to run successfully : Files /var/lib/cloudstack/management/.ssh/id_rsa and /usr/share/cloudstack-common/scripts/vm/systemvm/id_rsa.cloud differsudo: a password is required
2025-09-05 22:26:17,543 DEBUG [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:[]) (logid:) module start failure of module [core] was due to: org.springframework.context.ApplicationContextException: Failed to start bean 'cloudStackLifeCycle'; nested exception is com.cloud.utils.exception.CloudRuntimeException: Unhandled configuration exception
tons of module load errors for various modules such as:
2025-09-05 22:26:17,604 WARN [o.a.c.s.m.m.i.DefaultModuleDefinitionSet] (main:[]) (logid:) Failed to start module [allocator] due to: [Error creating bean with name 'org.springframework.beans.factory.config.PropertyPlaceholderConfigurer#0' defined in URL [jar:file:/usr/share/cloudstack-management/lib/cloudstack-4.21.0.0.jar!/META-INF/cloudstack/bootstrap/spring-bootstrap-context-inheritable.xml]: Cannot resolve reference to bean 'DefaultConfigResources' while setting bean property 'locations'; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'DefaultConfigResources' available].
This appears to be the same problem someone else had, but it doesn't look like it went anywhere. #11534 (reply in thread)
versions
ACS: v4.21
mysql: 8.0.43-0ubuntu0.22.04.1
Running on a Ubuntu 22.04 under VMWare Cloud.
1 nic
The steps to reproduce the bug
- Install the server as per instructions
What to do about it?
Cloudstack is currently unusable out of the box.
I have attached:
-all the logs generated by the server
-a screen capture of the terminal
-the script I use to do the installation of cloudstack
Activity
rastopato commentedon Sep 8, 2025
I have exact same problem. I have multiple environments. It happened on new install, but also on upgrade of production one. Only way back was restore database and ACS management server.
The same error: Cannot resolve reference to bean 'DefaultConfigResources' while setting bean property 'locations'.
It looks like upgrade from 4.20.1 to 4.21 cause some error in database.
I did upgrades multiple times successfully on the same environment from 4.19.1. This one causing the issue.
iloving commentedon Sep 8, 2025
I don't think deployment onto Ubuntu was tested at all, because as a test I set up a Rocky 9 instance and was able to install Cloudstack without issue on that.
rastopato commentedon Sep 8, 2025
I have it running on RHEL 9.6.
Database Mysql 8.x on separate RHEL 9.6.
weizhouapache commentedon Sep 8, 2025
@rastopato
the issue is caused by
can you update sudoers file to allow
clouduser to run commands without password prompt ?for example
or update the file
/etc/sudoers.d/cloudstack-managementrastopato commentedon Sep 8, 2025
ok, very first error is:
2025-09-08 17:33:19,365 ERROR [c.c.u.d.ScriptRunner] (main:[]) (logid:) Error executing: ALTER TABLE
cloud.kubernetes_clusterADD CONSTRAINTfk_cluster__control_node_service_offering_idFOREIGN KEYfk_cluster__control_node_service_offering_id(control_node_service_offering_id) REFERENCESservice_offering(id) ON DELETE CASCADE2025-09-08 17:33:19,369 ERROR [c.c.u.d.ScriptRunner] (main:[]) (logid:) java.sql.SQLException: Duplicate foreign key constraint name 'fk_cluster__control_node_service_offering_id'
2025-09-08 17:33:19,373 ERROR [c.c.u.DatabaseUpgradeChecker] (main:[]) (logid:) Unable to execute upgrade script java.sql.SQLException: Duplicate foreign key constraint name 'fk_cluster__control_node_service_offering_id'
iloving commentedon Sep 8, 2025
The missing sudoers entry appears to have been the blocker for me. The management server now installs and runs.
I guess it got missed in the ubuntu install packaging.
weizhouapache commentedon Sep 8, 2025
@rastopato
I guess it is not the first try, right ?
when you try to upgrade again, you need to restore the db to the original state before upgrade
rastopato commentedon Sep 8, 2025
It was from restored to original working state. I don't have time to try it again now, but later I will try and collect all logs. But I didn't have success 3 times on 2 different environments :(