[DB] Add force recreate parameter to cloudstack-setup-databases script by nvazquez · Pull Request #11239 · apache/cloudstack

nvazquez · 2025-07-17T17:05:59Z

Description

This PR adds a new parameter --force-recreate to the cloudstack-setup-databases script, to prevent the recreation of existing databases and prevent accidental data loss.
Fixes: #11202

Documentation PR: apache/cloudstack-documentation#534

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)
build/CI
test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Tested without passing the --force-recreate parameter having already defined cloud and cloud_usage databases:

# cloudstack-setup-databases cloud:cloud@10.0.34.102 --deploy-as=root:XXX -i 10.0.34.102
Mysql user name:cloud                                                           [ OK ]
Mysql user password:******                                                      [ OK ]
Mysql server ip:10.0.34.102                                                     [ OK ]
Mysql server port:3306                                                          [ OK ]
Mysql root user name:root                                                       [ OK ]
Mysql root user password:******                                                 [ OK ]
Using specified cluster management server node IP 10.0.34.102                   [ OK ]
Checking Cloud database files ...                                               [ OK ]
Checking local machine hostname ...                                             [ OK ]
Checking SELinux setup ...                                                      [ OK ]
Preparing /etc/cloudstack/management/db.properties                              [ OK ]


We apologize for below error:
***************************************************************
Aborting script as the databases (cloud, cloud_usage) already exist.
Please use the --force-recreation parameter if you want to recreate the schemas.
***************************************************************
Please run:

    cloudstack-setup-databases -h

for full help

Tested passing the --force-recreate parameter having already defined cloud and cloud_usage databases:

# cloudstack-setup-databases cloud:cloud@10.0.34.102 --deploy-as=root:XXX -i 10.0.34.102 --force-recreate
Mysql user name:cloud                                                           [ OK ]
Mysql user password:******                                                      [ OK ]
Mysql server ip:10.0.34.102                                                     [ OK ]
Mysql server port:3306                                                          [ OK ]
Mysql root user name:root                                                       [ OK ]
Mysql root user password:******                                                 [ OK ]
Using specified cluster management server node IP 10.0.34.102                   [ OK ]
Checking Cloud database files ...                                               [ OK ]
Checking local machine hostname ...                                             [ OK ]
Checking SELinux setup ...                                                      [ OK ]
Preparing /etc/cloudstack/management/db.properties                              [ OK ]
Applying /usr/share/cloudstack-management/setup/create-database.sql             [ OK ]
Applying /usr/share/cloudstack-management/setup/create-schema.sql               [ OK ]
Applying /usr/share/cloudstack-management/setup/create-database-premium.sql     [ OK ]
Applying /usr/share/cloudstack-management/setup/create-schema-premium.sql       [ OK ]
Applying /usr/share/cloudstack-management/setup/server-setup.sql                [ OK ]
Applying /usr/share/cloudstack-management/setup/templates.sql                   [ OK ]
Processing encryption ...                                                       [ OK ]
Finalizing setup ...                                                            [ OK ]

CloudStack has successfully initialized database, you can check your database configuration in /etc/cloudstack/management/db.properties

Dropped cloud database manually
Invoked the script with and without the parameter:

# cloudstack-setup-databases cloud:cloud@10.0.34.102 --deploy-as=root:XXX -i 10.0.34.102
Mysql user name:cloud                                                           [ OK ]
Mysql user password:******                                                      [ OK ]
Mysql server ip:10.0.34.102                                                     [ OK ]
Mysql server port:3306                                                          [ OK ]
Mysql root user name:root                                                       [ OK ]
Mysql root user password:******                                                 [ OK ]
Using specified cluster management server node IP 10.0.34.102                   [ OK ]
Checking Cloud database files ...                                               [ OK ]
Checking local machine hostname ...                                             [ OK ]
Checking SELinux setup ...                                                      [ OK ]
Preparing /etc/cloudstack/management/db.properties                              [ OK ]


We apologize for below error:
***************************************************************
Aborting script as the databases (cloud, cloud_usage) already exist.
Please use the --force-recreation parameter if you want to recreate the schemas.
***************************************************************
Please run:

    cloudstack-setup-databases -h

for full help
# cloudstack-setup-databases cloud:cloud@10.0.34.102 --deploy-as=root:XXX -i 10.0.34.102 --force-recreate
Mysql user name:cloud                                                           [ OK ]
Mysql user password:******                                                      [ OK ]
Mysql server ip:10.0.34.102                                                     [ OK ]
Mysql server port:3306                                                          [ OK ]
Mysql root user name:root                                                       [ OK ]
Mysql root user password:******                                                 [ OK ]
Using specified cluster management server node IP 10.0.34.102                   [ OK ]
Checking Cloud database files ...                                               [ OK ]
Checking local machine hostname ...                                             [ OK ]
Checking SELinux setup ...                                                      [ OK ]
Preparing /etc/cloudstack/management/db.properties                              [ OK ]
Applying /usr/share/cloudstack-management/setup/create-database.sql             [ OK ]
Applying /usr/share/cloudstack-management/setup/create-schema.sql               [ OK ]
Applying /usr/share/cloudstack-management/setup/create-database-premium.sql     [ OK ]
Applying /usr/share/cloudstack-management/setup/create-schema-premium.sql       [ OK ]
Applying /usr/share/cloudstack-management/setup/server-setup.sql                [ OK ]
Applying /usr/share/cloudstack-management/setup/templates.sql                   [ OK ]
Processing encryption ...                                                       [ OK ]
Finalizing setup ...                                                            [ OK ]

CloudStack has successfully initialized database, you can check your database configuration in /etc/cloudstack/management/db.properties

How did you try to break this feature and the system with this change?

nvazquez · 2025-07-17T17:06:30Z

@blueorangutan package

blueorangutan · 2025-07-17T17:08:04Z

@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

codecov · 2025-07-17T17:14:05Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 16.57%. Comparing base (b57994e) to head (5d5a70b).
⚠️ Report is 165 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #11239      +/-   ##
============================================
- Coverage     16.57%   16.57%   -0.01%     
+ Complexity    14059    14058       -1     
============================================
  Files          5772     5772              
  Lines        512938   512938              
  Branches      62304    62304              
============================================
- Hits          85026    85025       -1     
- Misses       418431   418432       +1     
  Partials       9481     9481

Flag	Coverage Δ
uitests	`3.89% <ø> (ø)`
unittests	`17.47% <ø> (-0.01%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

blueorangutan · 2025-07-17T18:17:45Z

Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 14240

nvazquez · 2025-07-17T18:22:05Z

@blueorangutan test

blueorangutan · 2025-07-17T18:24:04Z

@nvazquez a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

DaanHoogland

clgtm

blueorangutan · 2025-07-18T11:07:04Z

[SF] Trillian test result (tid-13816)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 57004 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr11239-t13816-kvm-ol8.zip
Smoke tests completed. 142 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test	Result	Time (s)	Test File

DaanHoogland · 2025-07-23T13:15:39Z

@adietrich-ussignal , can you review and test this?

sureshanaparti · 2025-07-28T08:56:21Z

Verfied cloudstack-setup-databases without cloud & cloud_usage databases, and with cloud / cloud_usage databases (using --force-recreate).

cloudstack-setup-databases with cloud database, no --force-recreate option.

[root@ol8 ~]# cloudstack-setup-databases cloud:cloud@localhost --deploy-as=root:*********
Mysql user name:cloud                                                           [ OK ]
Mysql user password:******                                                      [ OK ]
Mysql server ip:localhost                                                       [ OK ]
Mysql server port:3306                                                          [ OK ]
Mysql root user name:root                                                       [ OK ]
Mysql root user password:******                                                 [ OK ]
Checking Cloud database files ...                                               [ OK ]
Checking local machine hostname ...                                             [ OK ]
Checking SELinux setup ...                                                      [ OK ]
Detected local IP address as 10.0.34.59, will use as cluster management server node IP[ OK ]
Preparing /etc/cloudstack/management/db.properties                              [ OK ]


We apologize for below error:
***************************************************************
Aborting script as the databases (cloud, cloud_usage) already exist.
Please use the --force-recreate parameter if you want to recreate the schemas.
***************************************************************
Please run:

    cloudstack-setup-databases -h

for full help
[root@ol8 ~]#

cloudstack-setup-databases with cloud database, using --force-recreate option.

[root@ol8 ~]# cloudstack-setup-databases cloud:cloud@localhost --deploy-as=root:********* --force-recreate
Mysql user name:cloud                                                           [ OK ]
Mysql user password:******                                                      [ OK ]
Mysql server ip:localhost                                                       [ OK ]
Mysql server port:3306                                                          [ OK ]
Mysql root user name:root                                                       [ OK ]
Mysql root user password:******                                                 [ OK ]
Checking Cloud database files ...                                               [ OK ]
Checking local machine hostname ...                                             [ OK ]
Checking SELinux setup ...                                                      [ OK ]
Detected local IP address as 10.0.34.59, will use as cluster management server node IP[ OK ]
Preparing /etc/cloudstack/management/db.properties                              [ OK ]
Applying /usr/share/cloudstack-management/setup/create-database.sql             [ OK ]
Applying /usr/share/cloudstack-management/setup/create-schema.sql               [ OK ]
Applying /usr/share/cloudstack-management/setup/create-database-premium.sql     [ OK ]
Applying /usr/share/cloudstack-management/setup/create-schema-premium.sql       [ OK ]
Applying /usr/share/cloudstack-management/setup/server-setup.sql                [ OK ]
Applying /usr/share/cloudstack-management/setup/templates.sql                   [ OK ]
Processing encryption ...                                                       [ OK ]
Finalizing setup ...                                                            [ OK ]

CloudStack has successfully initialized database, you can check your database configuration in /etc/cloudstack/management/db.properties

[root@ol8 ~]#

cloudstack-setup-databases after dropping cloud & cloud_usage databases

mysql> drop database cloud;
Query OK, 136 rows affected (1.54 sec)

mysql> drop database cloud_usage;
Query OK, 15 rows affected (0.13 sec)

mysql> exit
Bye
[root@ol8 ~]# cloudstack-setup-databases cloud:cloud@localhost --deploy-as=root:*********
Mysql user name:cloud                                                           [ OK ]
Mysql user password:******                                                      [ OK ]
Mysql server ip:localhost                                                       [ OK ]
Mysql server port:3306                                                          [ OK ]
Mysql root user name:root                                                       [ OK ]
Mysql root user password:******                                                 [ OK ]
Checking Cloud database files ...                                               [ OK ]
Checking local machine hostname ...                                             [ OK ]
Checking SELinux setup ...                                                      [ OK ]
Detected local IP address as 10.0.34.59, will use as cluster management server node IP[ OK ]
Preparing /etc/cloudstack/management/db.properties                              [ OK ]
Applying /usr/share/cloudstack-management/setup/create-database.sql             [ OK ]
Applying /usr/share/cloudstack-management/setup/create-schema.sql               [ OK ]
Applying /usr/share/cloudstack-management/setup/create-database-premium.sql     [ OK ]
Applying /usr/share/cloudstack-management/setup/create-schema-premium.sql       [ OK ]
Applying /usr/share/cloudstack-management/setup/server-setup.sql                [ OK ]
Applying /usr/share/cloudstack-management/setup/templates.sql                   [ OK ]
Processing encryption ...                                                       [ OK ]
Finalizing setup ...                                                            [ OK ]

CloudStack has successfully initialized database, you can check your database configuration in /etc/cloudstack/management/db.properties

[root@ol8 ~]#

sureshanaparti

LGTM

apache#11239)

[DB] Add force recreate parameter to cloudstack-setup-databases script

5d5a70b

nvazquez added this to the 4.21.0 milestone Jul 17, 2025

nvazquez requested review from DaanHoogland, Pearl1594, shwstppr and weizhouapache July 17, 2025 17:05

nvazquez added the component:database label Jul 17, 2025

DaanHoogland approved these changes Jul 18, 2025

View reviewed changes

sureshanaparti added the status:needs-testing label Jul 18, 2025

sureshanaparti added this to Apache CloudStack 4.21.0 Jul 18, 2025

sureshanaparti moved this to In Progress in Apache CloudStack 4.21.0 Jul 18, 2025

DaanHoogland mentioned this pull request Jul 25, 2025

when creating a DB the script should refuse to create if the schemas already exist. #11202

Closed

sureshanaparti approved these changes Jul 28, 2025

View reviewed changes

sureshanaparti removed the status:needs-testing label Jul 28, 2025

sureshanaparti merged commit 21dde2b into apache:main Jul 28, 2025
25 checks passed

github-project-automation bot moved this from In Progress to Done in Apache CloudStack 4.21.0 Jul 28, 2025

DaanHoogland deleted the 421-fix-db-setup-script branch July 28, 2025 08:57

nvazquez mentioned this pull request Jul 28, 2025

Add force recreate parameter on setup DB script apache/cloudstack-documentation#534

Merged

dhslove pushed a commit to ablecloud-team/ablestack-cloud that referenced this pull request Aug 6, 2025

[DB] Add force recreate parameter to cloudstack-setup-databases script (

be1a2b0

apache#11239)

sureshanaparti added the type:improvement label Aug 20, 2025

bradh352 mentioned this pull request Sep 10, 2025

Regression 4.21.0: cloud-setup-databases errors out on --schema-only #11609

Closed

14 tasks

sureshanaparti mentioned this pull request Nov 6, 2025

DB setup: support db schema creation (with --schema-only) without force recreate option #12004

Merged

14 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DB] Add force recreate parameter to cloudstack-setup-databases script#11239

[DB] Add force recreate parameter to cloudstack-setup-databases script#11239
sureshanaparti merged 1 commit intoapache:mainfrom
shapeblue:421-fix-db-setup-script

nvazquez commented Jul 17, 2025 •

edited by sureshanaparti

Loading

Uh oh!

nvazquez commented Jul 17, 2025

Uh oh!

blueorangutan commented Jul 17, 2025

Uh oh!

codecov bot commented Jul 17, 2025 •

edited

Loading

Uh oh!

blueorangutan commented Jul 17, 2025

Uh oh!

nvazquez commented Jul 17, 2025

Uh oh!

blueorangutan commented Jul 17, 2025

Uh oh!

DaanHoogland left a comment

Uh oh!

blueorangutan commented Jul 18, 2025

Uh oh!

DaanHoogland commented Jul 23, 2025

Uh oh!

sureshanaparti commented Jul 28, 2025

Uh oh!

sureshanaparti left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

nvazquez commented Jul 17, 2025 • edited by sureshanaparti Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

Uh oh!

nvazquez commented Jul 17, 2025

Uh oh!

blueorangutan commented Jul 17, 2025

Uh oh!

codecov bot commented Jul 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

blueorangutan commented Jul 17, 2025

Uh oh!

nvazquez commented Jul 17, 2025

Uh oh!

blueorangutan commented Jul 17, 2025

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

blueorangutan commented Jul 18, 2025

Uh oh!

DaanHoogland commented Jul 23, 2025

Uh oh!

sureshanaparti commented Jul 28, 2025

Uh oh!

sureshanaparti left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

nvazquez commented Jul 17, 2025 •

edited by sureshanaparti

Loading

codecov bot commented Jul 17, 2025 •

edited

Loading