marvin - Fix k8s test failures on VMware by Pearl1594 · Pull Request #4896 · apache/cloudstack

Pearl1594 · 2021-04-05T12:02:00Z

Description

This PR fixes the k8s test failures noticed on vmware.

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Ran the smoke test - test_kubernetes_clusters.py

[root@ref-trl-715-v-M7-pearl-dsilva-marvin marvin]# nosetests --with-xunit --xunit-file=results.xml --with-marvin --marvin-config=ref-trl-715-v-M7-pearl-dsilva-advanced-cfg -s -a tags=advanced --hypervisor=vmware /marvin/tests/smoke/test_kubernetes_clusters.py 
/usr/lib/python2.7/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.
  from cryptography.hazmat.backends import default_backend

==== Marvin Init Started ====

=== Marvin Parse Config Successful ===

=== Marvin Setting TestData Successful===

==== Log Folder Path: /marvin/MarvinLogs/Apr_05_2021_09_37_43_KQKSTB. All logs will be available here ====

=== Marvin Init Logging Successful===

==== Marvin Init Successful ====
=== TestName: test_01_deploy_kubernetes_cluster | Status : SUCCESS ===

=== TestName: test_02_invalid_upgrade_kubernetes_cluster | Status : SUCCESS ===

=== TestName: test_03_deploy_and_upgrade_kubernetes_cluster | Status : SUCCESS ===

=== TestName: test_04_deploy_and_scale_kubernetes_cluster | Status : SUCCESS ===

=== TestName: test_05_delete_kubernetes_cluster | Status : SUCCESS ===

=== TestName: test_06_deploy_invalid_kubernetes_ha_cluster | Status : SUCCESS ===

=== TestName: test_07_deploy_kubernetes_ha_cluster | Status : SUCCESS ===

=== TestName: test_08_deploy_and_upgrade_kubernetes_ha_cluster | Status : SUCCESS ===

=== TestName: test_09_delete_kubernetes_ha_cluster | Status : SUCCESS ===

Pearl1594 · 2021-04-05T12:02:14Z

@blueorangutan package

blueorangutan · 2021-04-05T12:03:03Z

@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-04-05T12:27:27Z

Packaging result: ✔️ centos7 ✔️ centos8 ✔️ debian. SL-JID 329

Pearl1594 · 2021-04-05T12:28:58Z

@blueorangutan test centos7 vmware-67u3

blueorangutan · 2021-04-05T12:30:02Z

@Pearl1594 a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

blueorangutan · 2021-04-06T01:13:49Z

Trillian test result (tid-352)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 44710 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4896-t352-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_diagnostics.py
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Smoke tests completed. 84 look OK, 2 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_03_deploy_and_upgrade_kubernetes_cluster	`Error`	3757.68	test_kubernetes_clusters.py
ContextSuite context=TestKubernetesCluster>:teardown	`Error`	90.86	test_kubernetes_clusters.py
ContextSuite context=TestVAppsVM>:setup	`Error`	43.47	test_vm_life_cycle.py

DaanHoogland · 2021-04-06T09:45:08Z

+            if cls.hypervisor.lower() == 'vmware':
+                cls.create_full_clone = Configurations.list(cls.apiclient, name="vmware.create.full.clone")[0].value
+                if cls.create_full_clone not in ["true", True]:
+                    Configurations.update(cls.apiclient,
+                                          "vmware.create.full.clone",
+                                          "true")
+                allStoragePools = StoragePool.list(
+                    cls.apiclient
+                )
+                for pool in allStoragePools:
+                    Configurations.update(cls.apiclient,
+                                          storageid=pool.id,
+                                          name="vmware.create.full.clone",
+                                          value="true")


I see the same code below, can you externalise this in a separate method?

Will do @DaanHoogland

…k8s-test-vmware

yadvr · 2021-04-06T10:57:05Z

            Field f = startVm.getClass().getDeclaredField("id");
            f.setAccessible(true);
            f.set(startVm, vm.getId());
-            resizeNodeVolume(vm);


Would that cause an issue (i.e. not resizing node on start?)

the resizing happens before this is invoked, having this effectively meant we we attempting to resize twice. Removed it because it is redundant

Checked, resize call is present in provisionKubernetesClusterNodeVms at line 331

Pearl1594 · 2021-04-06T14:01:10Z

@blueorangutan package

blueorangutan · 2021-04-06T14:02:03Z

@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-04-06T14:26:14Z

Packaging result: ✔️ centos7 ✔️ centos8 ✔️ debian. SL-JID 341

Pearl1594 · 2021-04-06T14:27:38Z

@blueorangutan test centos7 vmware-67u3

blueorangutan · 2021-04-06T14:28:03Z

@Pearl1594 a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

blueorangutan · 2021-04-07T00:42:46Z

Trillian test result (tid-366)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 35728 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4896-t366-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_diagnostics.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Smoke tests completed. 85 look OK, 2 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_10_traceroute_in_vr	`Failure`	61.36	test_diagnostics.py
ContextSuite context=TestVAppsVM>:setup	`Error`	42.53	test_vm_life_cycle.py

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

Pearl1594 · 2021-04-07T10:19:29Z

@blueorangutan package

blueorangutan · 2021-04-07T10:20:02Z

@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-04-07T10:44:36Z

Packaging result: ✔️ centos7 ✔️ centos8 ✔️ debian. SL-JID 347

Pearl1594 · 2021-04-07T10:45:17Z

@blueorangutan test centos7 vmware-67u3

blueorangutan · 2021-04-07T10:46:02Z

@Pearl1594 a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

blueorangutan · 2021-04-07T22:08:24Z

Trillian test result (tid-369)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 39804 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4896-t369-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Intermittent failure detected: /marvin/tests/smoke/test_vpc_vpn.py
Smoke tests completed. 86 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
ContextSuite context=TestVAppsVM>:setup	`Error`	46.43	test_vm_life_cycle.py

davidjumani · 2021-04-08T04:15:49Z

@blueorangutan package

blueorangutan · 2021-04-08T04:17:02Z

@davidjumani a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-04-08T04:38:40Z

Packaging result: ✔️ centos7 ✔️ centos8 ✔️ debian. SL-JID 357

davidjumani · 2021-04-08T06:07:25Z

@blueorangutan test centos7 vmware-67u3

blueorangutan · 2021-04-08T06:08:03Z

@davidjumani a Trillian-Jenkins test job (centos7 mgmt + vmware-67u3) has been kicked to run smoke tests

blueorangutan · 2021-04-08T16:13:35Z

Trillian test result (tid-381)
Environment: vmware-67u3 (x2), Advanced Networking with Mgmt server 7
Total time taken: 35195 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4896-t381-vmware-67u3.zip
Intermittent failure detected: /marvin/tests/smoke/test_diagnostics.py
Intermittent failure detected: /marvin/tests/smoke/test_vm_life_cycle.py
Smoke tests completed. 85 look OK, 2 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_10_traceroute_in_vr	`Failure`	61.14	test_diagnostics.py
ContextSuite context=TestVAppsVM>:setup	`Error`	42.32	test_vm_life_cycle.py

yadvr · 2021-04-09T06:23:12Z

@Pearl1594 LGTM, can you look at test_10_traceroute_in_vr failure which seems something new

shwstppr

LGTM

shwstppr · 2021-04-09T06:46:58Z

            Field f = startVm.getClass().getDeclaredField("id");
            f.setAccessible(true);
            f.set(startVm, vm.getId());
-            resizeNodeVolume(vm);


Checked, resize call is present in provisionKubernetesClusterNodeVms at line 331

shwstppr · 2021-04-09T06:50:38Z

test_10_traceroute_in_vr failure doesn't seem relevant to this PR. Failed in another PR as well #4758 (comment)

Fix k8s tests - enable full clone in case of resize of volumes

0c880bb

Pearl1594 changed the title ~~marvin - Fix k8s test failures for VMware~~ marvin - Fix k8s test failures on VMware Apr 5, 2021

DaanHoogland reviewed Apr 6, 2021

View reviewed changes

Merge branch '4.15' of https://github.com/apache/cloudstack into fix-…

77c1dd0

…k8s-test-vmware

yadvr reviewed Apr 6, 2021

View reviewed changes

yadvr added this to the 4.15.1.0 milestone Apr 6, 2021

yadvr assigned davidjumani Apr 6, 2021

extract code

a4204e1

shwstppr and others added 2 commits April 7, 2021 10:19

precent deleting already registered template

0bca841

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>

re-arrange test cases for faster run

8a5d523

Pearl1594 marked this pull request as ready for review April 7, 2021 10:19

davidjumani reviewed Apr 7, 2021

View reviewed changes

Comment thread test/integration/smoke/test_kubernetes_clusters.py Outdated

Address comment

120e4d2

shwstppr approved these changes Apr 9, 2021

View reviewed changes

yadvr approved these changes Apr 9, 2021

View reviewed changes

yadvr merged commit cd38519 into apache:4.15 Apr 9, 2021

Conversation

Pearl1594 commented Apr 5, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Types of changes

Feature/Enhancement Scale or Bug Severity

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Uh oh!

Pearl1594 commented Apr 5, 2021

Uh oh!

blueorangutan commented Apr 5, 2021

Uh oh!

blueorangutan commented Apr 5, 2021

Uh oh!

Pearl1594 commented Apr 5, 2021

Uh oh!

blueorangutan commented Apr 5, 2021

Uh oh!

blueorangutan commented Apr 6, 2021

Uh oh!

DaanHoogland Apr 6, 2021

Choose a reason for hiding this comment

Uh oh!

Pearl1594 Apr 6, 2021

Choose a reason for hiding this comment

Uh oh!

yadvr Apr 6, 2021

Choose a reason for hiding this comment

Uh oh!

Pearl1594 Apr 6, 2021

Choose a reason for hiding this comment

Uh oh!

shwstppr Apr 9, 2021

Choose a reason for hiding this comment

Uh oh!

Pearl1594 commented Apr 6, 2021

Uh oh!

blueorangutan commented Apr 6, 2021

Uh oh!

blueorangutan commented Apr 6, 2021

Uh oh!

Pearl1594 commented Apr 6, 2021

Uh oh!

blueorangutan commented Apr 6, 2021

Uh oh!

blueorangutan commented Apr 7, 2021

Uh oh!

Pearl1594 commented Apr 7, 2021

Uh oh!

blueorangutan commented Apr 7, 2021

Uh oh!

blueorangutan commented Apr 7, 2021

Uh oh!

Pearl1594 commented Apr 7, 2021

Uh oh!

blueorangutan commented Apr 7, 2021

Uh oh!

Uh oh!

blueorangutan commented Apr 7, 2021

Uh oh!

davidjumani commented Apr 8, 2021

Uh oh!

blueorangutan commented Apr 8, 2021

Uh oh!

blueorangutan commented Apr 8, 2021

Uh oh!

davidjumani commented Apr 8, 2021

Uh oh!

blueorangutan commented Apr 8, 2021

Uh oh!

blueorangutan commented Apr 8, 2021

Uh oh!

yadvr commented Apr 9, 2021

Uh oh!

shwstppr left a comment

Choose a reason for hiding this comment

Uh oh!

shwstppr Apr 9, 2021

Pearl1594 commented Apr 5, 2021 •

edited

Loading