agent: Detect existing hosts with UEFI support by davidjumani · Pull Request #6139 · apache/cloudstack

davidjumani · 2022-03-18T10:58:39Z

Description

Fixes #6087
Adds support to detect existing hosts with UEFI capability

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale

Major
Minor

How Has This Been Tested?

Bring up a 4.13 env with VMware 6.5 (anything that supports UEFI)
Upgrade to latest main
Try to deploy a VM with UEFI
Deployment now passes Previously failed to find a suitable host as none were UEFI compatible

Before :

MariaDB [cloud]> select * from host_details where name = 'host.uefi.enable' \G
Empty set (0.00 sec)

After :

MariaDB [cloud]> select * from host_details where name = 'host.uefi.enable' \G
*************************** 1. row ***************************
     id: 34
host_id: 1
   name: host.uefi.enable
  value: true
*************************** 2. row ***************************
     id: 40
host_id: 2
   name: host.uefi.enable
  value: true
2 rows in set (0.00 sec)

davidjumani · 2022-03-18T10:58:50Z

@blueorangutan package

blueorangutan · 2022-03-18T10:59:03Z

@davidjumani a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

nvazquez · 2022-03-18T11:08:10Z

                details.put("url", hostMo.getHostName());
                details.put("username", username);
                details.put("password", password);
-                boolean uefiLegacySupported = hostMo.isUefiLegacySupported();


I think these could be kept, same for other discoverers

It would be redundant as the code to check the UEFI support is subsequently called while adding a host

davidjumani · 2022-03-18T11:38:23Z

@blueorangutan package

blueorangutan · 2022-03-18T11:39:02Z

@davidjumani a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2022-03-18T12:17:08Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 2930

nvazquez · 2022-03-18T12:57:30Z

@blueorangutan test

blueorangutan · 2022-03-18T12:58:03Z

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

DaanHoogland

clgtm

weizhouapache

tested on centos7, works fine

weizhouapache · 2022-03-18T16:35:51Z

@blueorangutan test matrix

blueorangutan · 2022-03-18T16:36:03Z

@weizhouapache a Trillian-Jenkins matrix job (centos7 mgmt + xs71, centos7 mgmt + vmware65, centos7 mgmt + kvmcentos7) has been kicked to run smoke tests

blueorangutan · 2022-03-18T22:03:55Z

Trillian test result (tid-3664)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 31374 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6139-t3664-kvm-centos7.zip
Smoke tests completed. 92 look OK, 0 have errors
Only failed tests results shown below:

Test	Result	Time (s)	Test File

blueorangutan · 2022-03-19T02:00:29Z

Trillian test result (tid-3671)
Environment: vmware-65u2 (x2), Advanced Networking with Mgmt server 7
Total time taken: 32668 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6139-t3671-vmware-65u2.zip
Intermittent failure detected: /marvin/tests/smoke/test_usage.py
Intermittent failure detected: /marvin/tests/smoke/test_iso.py

davidjumani · 2022-03-22T05:53:23Z

@blueorangutan help

davidjumani · 2022-03-22T05:56:04Z

@blueorangutan help

blueorangutan · 2022-03-22T05:57:02Z

@davidjumani I understand these words: "help", "hello", "thanks", "package", "test"
Test command usage: test [mgmt os] [hypervisor] [keepEnv]
Mgmt OS options: ['suse15', 'centos7', 'centos6', 'alma8', 'ubuntu18', 'ubuntu20', 'rocky8']
Hypervisor options: ['kvm-centos6', 'kvm-centos7', 'kvm-rocky8', 'kvm-alma8', 'kvm-ubuntu18', 'kvm-ubuntu20', 'kvm-suse15', 'vmware-55u3', 'vmware-60u2', 'vmware-65u2', 'vmware-67u3', 'vmware-70u1', 'vmware-70u2', 'vmware-70u3', 'xenserver-65sp1', 'xenserver-71', 'xenserver-74', 'xcpng74', 'xcpng76', 'xcpng80', 'xcpng81', 'xcpng82']
Note: when keepEnv is passed, you need to specify mgmt server os and hypervisor or use the matrix command.
Package command usage: package [all(default value),kvm,xen,vmware,hyperv,ovm] - a comma separated list can be passed with package command to bundle the required hypervisor's systemVM templates. Not passing any argument will bundle all - kvm,xen and vmware templates.

Blessed contributors for kicking Trillian test jobs: ['rohityadavcloud', 'nvazquez', 'borisstoyanov', 'DaanHoogland', 'shwstppr', 'andrijapanicsb', 'Pearl1594', 'davidjumani', 'harikrishna-patnala', 'vladimirpetrov', 'sureshanaparti', 'weizhouapache', 'NuxRo']

davidjumani · 2022-03-22T05:58:05Z

@blueorangutan test centos7 vmware-67u3

blueorangutan · 2022-03-22T05:59:03Z

@davidjumani a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

davidjumani · 2022-03-22T06:00:36Z

@blueorangutan test centos7 xcpng82

blueorangutan · 2022-03-22T06:01:03Z

@davidjumani a Trillian-Jenkins test job (centos7 mgmt + xcpng82) has been kicked to run smoke tests

davidjumani · 2022-03-22T06:05:13Z

@blueorangutan test centos7 xenserver-71

blueorangutan · 2022-03-22T06:06:03Z

@davidjumani a Trillian-Jenkins test job (centos7 mgmt + xenserver-71) has been kicked to run smoke tests

blueorangutan · 2022-03-22T14:06:44Z

Trillian test result (tid-3691)
Environment: xcpng82 (x2), Advanced Networking with Mgmt server 7
Total time taken: 27824 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6139-t3691-xcpng82.zip
Intermittent failure detected: /marvin/tests/smoke/test_volumes.py
Intermittent failure detected: /marvin/tests/smoke/test_metrics_api.py

nvazquez · 2022-03-22T14:07:32Z

@blueorangutan test centos7 xcpng82

blueorangutan · 2022-03-22T14:08:03Z

@nvazquez a Trillian-Jenkins test job (centos7 mgmt + xcpng82) has been kicked to run smoke tests

blueorangutan · 2022-03-22T14:08:57Z

Trillian test result (tid-3690)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 28196 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6139-t3690-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py

blueorangutan · 2022-03-22T14:08:57Z

Trillian test result (tid-3692)
Environment: xenserver-71 (x2), Advanced Networking with Mgmt server 7
Total time taken: 27656 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6139-t3692-xenserver-71.zip
Intermittent failure detected: /marvin/tests/smoke/test_diagnostics.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py

blueorangutan · 2022-03-23T02:24:34Z

Trillian test result (tid-3698)
Environment: xcpng82 (x2), Advanced Networking with Mgmt server 7
Total time taken: 42933 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6139-t3698-xcpng82.zip
Smoke tests completed. 91 look OK, 1 have errors
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_12_resize_volume_with_only_size_parameter	`Error`	1.06	test_volumes.py

weizhouapache · 2022-03-29T07:58:05Z

@blueorangutan package

blueorangutan · 2022-03-29T07:59:03Z

@weizhouapache a Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

blueorangutan · 2022-03-29T08:43:10Z

Packaging result: ✔️ el7 ✔️ el8 ✔️ debian ✔️ suse15. SL-JID 3004

weizhouapache · 2022-03-29T08:48:36Z

@blueorangutan test

blueorangutan · 2022-03-29T08:49:03Z

@weizhouapache a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2022-03-29T18:02:56Z

Trillian test result (tid-3748)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 31855 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr6139-t3748-kvm-centos7.zip
Smoke tests completed. 92 look OK, 0 have errors
Only failed tests results shown below:

Test	Result	Time (s)	Test File

nvazquez

LGTM - manually tested upgrading a 4.13 Vmware env with 2 hosts to 4.17:

Before upgrading:

MariaDB [cloud]> select * from host_details where name like '%uefi%';
Empty set (0.00 sec)

After upgrade:

MariaDB [cloud]> select * from host_details where name like '%uefi%';
+----+---------+------------------+-------+
| id | host_id | name             | value |
+----+---------+------------------+-------+
| 24 |       2 | host.uefi.enable | true  |
| 30 |       1 | host.uefi.enable | true  |
+----+---------+------------------+-------+
2 rows in set (0.00 sec)

Hudratronium · 2022-08-17T21:29:40Z

Will the check take place on every "upgrade" from now on?

yadvr · 2023-01-12T06:12:22Z

+    }
+
+    private boolean hostSupportsUefi() {
+        String cmd = "rpm -qa | grep -i ovmf";


cc @davidjumani @weizhouapache @DaanHoogland we may need to fix this as we support both EL and Ubuntu, rpm isn't installed by default on Ubuntu. Probably just check if /sys/firmware/efi using File::exists?

Hudratronium · 2023-01-15T10:39:58Z

Just as an idea:
Maybe checking via 'dpkg' should work for ubuntu - and even better - as well as for other debian derivate.

… hosts (apache#6423) (apache#221) * Do not allow UEFI deployments on non UEFI enabled hosts * Fix UEFI detection on KVM * Refactor * Improvement agent: Detect existing hosts with UEFI support (apache#6139) * agent: Pass uefi enabled status as part of ready command * Cleanup * Fix checkstyle * Save uefi status if different Co-authored-by: Marcus Sorensen <mls@apple.com>

davidjumani added 2 commits March 18, 2022 14:37

agent: Pass uefi enabled status as part of ready command

26b738a

Cleanup

11fdae3

nvazquez added this to the 4.17.0.0 milestone Mar 18, 2022

nvazquez reviewed Mar 18, 2022

View reviewed changes

nvazquez added the type:enhancement label Mar 18, 2022

davidjumani added 2 commits March 18, 2022 16:57

Fix checkstyle

c7da0fc

Save uefi status if different

ccbd188

davidjumani mentioned this pull request Mar 18, 2022

UEFI hosts capability not detected after upgrade #6087

Closed

DaanHoogland approved these changes Mar 18, 2022

View reviewed changes

weizhouapache approved these changes Mar 18, 2022

View reviewed changes

DaanHoogland mentioned this pull request Mar 21, 2022

Set UefiCapabilty for all hypervisors in hostresponse #6140

Merged

10 tasks

davidjumani changed the title ~~agent: Pass uefi support status as part of ready command~~ agent: Detect existing hosts with UEFI support Mar 22, 2022

nvazquez marked this pull request as draft March 22, 2022 14:07

nvazquez marked this pull request as ready for review April 1, 2022 15:37

nvazquez approved these changes Apr 5, 2022

View reviewed changes

nvazquez merged commit 5ab4309 into apache:main Apr 5, 2022

yadvr reviewed Jan 12, 2023

View reviewed changes

Conversation

davidjumani commented Mar 18, 2022

Description

Types of changes

Feature/Enhancement Scale

How Has This Been Tested?

Before :

After :

Uh oh!

davidjumani commented Mar 18, 2022

Uh oh!

blueorangutan commented Mar 18, 2022

Uh oh!

Uh oh!

nvazquez Mar 18, 2022

Choose a reason for hiding this comment

Uh oh!

davidjumani Mar 18, 2022

Choose a reason for hiding this comment

Uh oh!

davidjumani commented Mar 18, 2022

Uh oh!

blueorangutan commented Mar 18, 2022

Uh oh!

blueorangutan commented Mar 18, 2022

Uh oh!

nvazquez commented Mar 18, 2022

Uh oh!

blueorangutan commented Mar 18, 2022

Uh oh!

DaanHoogland left a comment

Choose a reason for hiding this comment

Uh oh!

weizhouapache left a comment

Choose a reason for hiding this comment

Uh oh!

weizhouapache commented Mar 18, 2022

Uh oh!

blueorangutan commented Mar 18, 2022

Uh oh!

blueorangutan commented Mar 18, 2022

Uh oh!

blueorangutan commented Mar 19, 2022

Uh oh!

davidjumani commented Mar 22, 2022

Uh oh!

davidjumani commented Mar 22, 2022

Uh oh!

blueorangutan commented Mar 22, 2022

Uh oh!

davidjumani commented Mar 22, 2022

Uh oh!

blueorangutan commented Mar 22, 2022

Uh oh!

davidjumani commented Mar 22, 2022

Uh oh!

blueorangutan commented Mar 22, 2022

Uh oh!

davidjumani commented Mar 22, 2022

Uh oh!

blueorangutan commented Mar 22, 2022

Uh oh!

blueorangutan commented Mar 22, 2022

Uh oh!

nvazquez commented Mar 22, 2022

Uh oh!

blueorangutan commented Mar 22, 2022

Uh oh!

blueorangutan commented Mar 22, 2022

Uh oh!

blueorangutan commented Mar 22, 2022

Uh oh!

blueorangutan commented Mar 23, 2022

Uh oh!

weizhouapache commented Mar 29, 2022

Uh oh!

blueorangutan commented Mar 29, 2022

Uh oh!

blueorangutan commented Mar 29, 2022

Uh oh!