-
Notifications
You must be signed in to change notification settings - Fork 1.3k
SystemVM optimizations #5831
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
SystemVM optimizations #5831
Changes from all commits
Commits
Show all changes
78 commits
Select commit
Hold shift + click to select a range
522c542
Support for live patching systemVMs and deprecating systemVM.iso. Inc…
Pearl1594 50c59bc
The following commit:
Pearl1594 5cb090d
- Support to patch SystemVMs - VMWare
Pearl1594 86bc3bf
Commit comprises of:
Pearl1594 5a27cb7
Add ssh to k8s nodes details in the Access tab on the UI
Pearl1594 9c182db
test
Pearl1594 8f1e3a3
Refactor ca/cert patching logic
Pearl1594 9fef181
Commit comprises of the following changes:
Pearl1594 bdaa364
remove all references of systemvm.iso
Pearl1594 b923ed2
Fix keystore-cert-import invocation + refactor cert timeout in CP/SS VMs
Pearl1594 a380f68
fix script timeout
Pearl1594 447ae42
Refactor cert patching for systemVMs + update keystore-cert-import sc…
Pearl1594 c7e79ec
remove commented code + change core user to cloud for cks nodes
Pearl1594 e3d0051
Update ownership of ssh directory
Pearl1594 84f31ac
NEED TO DISCUSS - add on the fly template conversion as an ExecStartP…
Pearl1594 da5bba8
Add UI changes + move changes from patch file to runcmd
Pearl1594 0ccf346
test: validate performance for template modification during seeding
Pearl1594 b07f20d
create vms folder in cloudstack-commons directory - debian rules
Pearl1594 9137261
remove logic for on the fly template convert + update k8s test
Pearl1594 554e7c8
fix syntax issue - causing issue with shared network tests
Pearl1594 9781668
Code cleanup
Pearl1594 6244ca4
refactor patching logic - certs
Pearl1594 0196929
move logic of fixing rootdiskcontroller from upgrade to kubernetes se…
Pearl1594 21f0cf5
add livepatch option to restart network & vpc
Pearl1594 92c71c0
smooth upgrade of cks clusters
Pearl1594 c7fd6de
Support for live patching systemVMs and deprecating systemVM.iso. Inc…
Pearl1594 65a8762
The following commit:
Pearl1594 ce92de4
- Support to patch SystemVMs - VMWare
Pearl1594 f1e71aa
Commit comprises of:
Pearl1594 abc5732
Add ssh to k8s nodes details in the Access tab on the UI
Pearl1594 2960d58
test
Pearl1594 87f1b73
Refactor ca/cert patching logic
Pearl1594 10f6d33
Commit comprises of the following changes:
Pearl1594 da7db39
remove all references of systemvm.iso
Pearl1594 ec5d082
Fix keystore-cert-import invocation + refactor cert timeout in CP/SS VMs
Pearl1594 6c190d1
fix script timeout
Pearl1594 3a27954
Refactor cert patching for systemVMs + update keystore-cert-import sc…
Pearl1594 e678b46
remove commented code + change core user to cloud for cks nodes
Pearl1594 0293ab7
Update ownership of ssh directory
Pearl1594 f37d74f
NEED TO DISCUSS - add on the fly template conversion as an ExecStartP…
Pearl1594 ff7d4ed
Add UI changes + move changes from patch file to runcmd
Pearl1594 e6f9c29
test: validate performance for template modification during seeding
Pearl1594 ebfd912
create vms folder in cloudstack-commons directory - debian rules
Pearl1594 f130b25
remove logic for on the fly template convert + update k8s test
Pearl1594 8937867
fix syntax issue - causing issue with shared network tests
Pearl1594 d306404
Code cleanup
Pearl1594 0174903
Merge branch 'sysvm-optimizations-v2' of github.com:shapeblue/cloudst…
Pearl1594 a1968e3
Merge branch 'main' of https://github.com/apache/cloudstack into sysv…
Pearl1594 420f54a
add cgroup config for containerd
Pearl1594 62c274f
add systemd config for kubelet
Pearl1594 79792e3
add additional info during image registry config
Pearl1594 e42b5cf
address comments
Pearl1594 c6e7dc7
Merge branch 'main' of https://github.com/apache/cloudstack into sysv…
Pearl1594 ed18685
add temp links of download.cloudstack.org
Pearl1594 ff386ac
address part of the comments
Pearl1594 c6412a8
Merge branch 'main' of https://github.com/apache/cloudstack into sysv…
Pearl1594 fb1678c
address comments
Pearl1594 bc961f7
Merge branch 'main' of https://github.com/apache/cloudstack into sysv…
Pearl1594 cb47ea4
update containerd config - as version has upgraded to 1.5 from 1.4.12…
Pearl1594 0589ef8
address comments - simplify
Pearl1594 e40625e
fix vue3 related icon changes
Pearl1594 3d1ec29
allow network commands when router template version is lower but is p…
Pearl1594 971eac5
add internal LB to the list of routers to be patched on network resta…
Pearl1594 df9355f
Merge branch 'main' of https://github.com/apache/cloudstack into sysv…
Pearl1594 41e6738
add unit tests for API param validations and new helper utilities - f…
Pearl1594 06791f4
perform patching only for non-user i.e., system VMs
Pearl1594 a12789d
add test to validate params
Pearl1594 f10d925
Merge branch 'main' of https://github.com/apache/cloudstack into sysv…
Pearl1594 e35d51d
remove unused import
Pearl1594 a44ef85
Merge branch 'main' of https://github.com/apache/cloudstack into sysv…
Pearl1594 8e9175b
add column to domain_router to display software version and support n…
Pearl1594 e409227
Merge branch 'main' of https://github.com/apache/cloudstack into sysv…
Pearl1594 809ea91
Merge branch 'main' into sysvm-optimizations-v2
Pearl1594 30c00e6
Requires upgrade column to consider package (cloud-scripts) checksum …
Pearl1594 47693fd
use router software version instead of checksum
Pearl1594 75b46b2
show N/A if no software version reported i.e., in upgraded envs
Pearl1594 cec9c82
fix deb failure
Pearl1594 de0ce15
update pom to official links of systemVM template
Pearl1594 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -52,4 +52,6 @@ public enum RedundantState { | |
| Long getVpcId(); | ||
|
|
||
| String getTemplateVersion(); | ||
|
|
||
| String getScriptsVersion(); | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
108 changes: 108 additions & 0 deletions
108
api/src/main/java/org/apache/cloudstack/api/command/admin/systemvm/PatchSystemVMCmd.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,108 @@ | ||
| // Licensed to the Apache Software Foundation (ASF) under one | ||
| // or more contributor license agreements. See the NOTICE file | ||
| // distributed with this work for additional information | ||
| // regarding copyright ownership. The ASF licenses this file | ||
| // to you under the Apache License, Version 2.0 (the | ||
| // "License"); you may not use this file except in compliance | ||
| // with the License. You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, | ||
| // software distributed under the License is distributed on an | ||
| // "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| // KIND, either express or implied. See the License for the | ||
| // specific language governing permissions and limitations | ||
| // under the License. | ||
| package org.apache.cloudstack.api.command.admin.systemvm; | ||
|
|
||
| import com.cloud.event.EventTypes; | ||
| import com.cloud.user.Account; | ||
| import com.cloud.utils.Pair; | ||
| import com.cloud.vm.VirtualMachine; | ||
| import org.apache.cloudstack.acl.RoleType; | ||
| import org.apache.cloudstack.api.APICommand; | ||
| import org.apache.cloudstack.api.ApiConstants; | ||
| import org.apache.cloudstack.api.ApiErrorCode; | ||
| import org.apache.cloudstack.api.BaseAsyncCmd; | ||
| import org.apache.cloudstack.api.BaseCmd; | ||
| import org.apache.cloudstack.api.Parameter; | ||
| import org.apache.cloudstack.api.ServerApiException; | ||
| import org.apache.cloudstack.api.response.SuccessResponse; | ||
| import org.apache.cloudstack.api.response.SystemVmResponse; | ||
| import org.apache.cloudstack.context.CallContext; | ||
| import org.apache.log4j.Logger; | ||
|
|
||
| @APICommand(name = PatchSystemVMCmd.APINAME, description = "Attempts to live patch systemVMs - CPVM, SSVM ", | ||
| responseObject = SuccessResponse.class, requestHasSensitiveInfo = false, | ||
| responseHasSensitiveInfo = false, authorized = { RoleType.Admin }, since = "4.17.0") | ||
| public class PatchSystemVMCmd extends BaseAsyncCmd { | ||
| public static final Logger s_logger = Logger.getLogger(PatchSystemVMCmd.class.getName()); | ||
| public static final String APINAME = "patchSystemVm"; | ||
|
|
||
| ///////////////////////////////////////////////////// | ||
| //////////////// API parameters ///////////////////// | ||
| ///////////////////////////////////////////////////// | ||
| @Parameter(name = ApiConstants.ID, type = CommandType.UUID, entityType = SystemVmResponse.class, | ||
| description = "patches systemVM - CPVM/SSVM with the specified ID") | ||
| private Long id; | ||
|
|
||
| @Parameter(name = ApiConstants.FORCED, type = CommandType.BOOLEAN, | ||
| description = "If true, initiates copy of scripts and restart of the agent, even if the scripts version matches." + | ||
| "To be used with ID parameter only") | ||
| private Boolean force; | ||
|
|
||
| ///////////////////////////////////////////////////// | ||
| /////////////////// Accessors /////////////////////// | ||
| ///////////////////////////////////////////////////// | ||
|
|
||
|
|
||
| public Long getId() { | ||
| return id; | ||
| } | ||
|
|
||
| public boolean isForced() { | ||
| return force != null && force; | ||
| } | ||
|
|
||
| ///////////////////////////////////////////////////// | ||
| /////////////// API Implementation/////////////////// | ||
| ///////////////////////////////////////////////////// | ||
|
|
||
| @Override | ||
| public String getEventType() { | ||
| return EventTypes.EVENT_LIVE_PATCH_SYSTEMVM; | ||
| } | ||
|
|
||
| @Override | ||
| public String getEventDescription() { | ||
| return String.format("Attempting to live patch System VM with Id: %s ", this._uuidMgr.getUuid(VirtualMachine.class, getId())); | ||
| } | ||
|
|
||
| @Override | ||
| public String getCommandName() { | ||
| return APINAME.toLowerCase() + BaseCmd.RESPONSE_SUFFIX; | ||
| } | ||
|
|
||
| @Override | ||
| public long getEntityOwnerId() { | ||
| Account account = CallContext.current().getCallingAccount(); | ||
| if (account != null) { | ||
| return account.getId(); | ||
| } | ||
|
|
||
| return Account.ACCOUNT_ID_SYSTEM; | ||
| } | ||
|
|
||
| @Override | ||
| public void execute() { | ||
| Pair<Boolean, String> patched = _mgr.patchSystemVM(this); | ||
| if (patched.first()) { | ||
| final SuccessResponse response = new SuccessResponse(getCommandName()); | ||
| response.setDisplayText(patched.second()); | ||
| setResponseObject(response); | ||
| } else { | ||
| throw new ServerApiException(ApiErrorCode.INTERNAL_ERROR, patched.second()); | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
92 changes: 92 additions & 0 deletions
92
api/src/test/java/org/apache/cloudstack/api/command/admin/systemvm/PatchSystemVMCmdTest.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| // Licensed to the Apache Software Foundation (ASF) under one | ||
| // or more contributor license agreements. See the NOTICE file | ||
| // distributed with this work for additional information | ||
| // regarding copyright ownership. The ASF licenses this file | ||
| // to you under the Apache License, Version 2.0 (the | ||
| // "License"); you may not use this file except in compliance | ||
| // with the License. You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, | ||
| // software distributed under the License is distributed on an | ||
| // "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| // KIND, either express or implied. See the License for the | ||
| // specific language governing permissions and limitations | ||
| // under the License.A | ||
| package org.apache.cloudstack.api.command.admin.systemvm; | ||
|
|
||
| import com.cloud.server.ManagementService; | ||
| import com.cloud.user.Account; | ||
| import com.cloud.utils.Pair; | ||
| import org.apache.cloudstack.context.CallContext; | ||
| import org.junit.Assert; | ||
| import org.junit.Before; | ||
| import org.junit.Test; | ||
| import org.junit.runner.RunWith; | ||
| import org.mockito.InjectMocks; | ||
| import org.mockito.Mock; | ||
| import org.mockito.Mockito; | ||
| import org.mockito.MockitoAnnotations; | ||
| import org.powermock.api.mockito.PowerMockito; | ||
| import org.powermock.core.classloader.annotations.PowerMockIgnore; | ||
| import org.powermock.core.classloader.annotations.PrepareForTest; | ||
| import org.powermock.modules.junit4.PowerMockRunner; | ||
| import org.springframework.test.util.ReflectionTestUtils; | ||
|
|
||
| @RunWith(PowerMockRunner.class) | ||
| @PrepareForTest(CallContext.class) | ||
| @PowerMockIgnore({"javax.xml.*", "org.w3c.dom.*", "org.apache.xerces.*", "org.xml.*"}) | ||
| public class PatchSystemVMCmdTest { | ||
|
|
||
| @Mock | ||
| private ManagementService _mgr; | ||
|
|
||
| @InjectMocks | ||
| PatchSystemVMCmd cmd = new PatchSystemVMCmd(); | ||
|
|
||
| @Before | ||
| public void setUp() throws Exception { | ||
| MockitoAnnotations.initMocks(this); | ||
| } | ||
|
|
||
| @Test | ||
| public void patchValidSystemVM() { | ||
| ReflectionTestUtils.setField(cmd, "id", 1L); | ||
| Pair successResponse = new Pair<>(true, ""); | ||
| Mockito.doReturn(successResponse).when(_mgr).patchSystemVM(cmd); | ||
| try { | ||
| cmd.execute(); | ||
| } catch (Exception e) { | ||
| e.printStackTrace(); | ||
| } | ||
| } | ||
|
|
||
| @Test | ||
| public void patchInvalidSystemVM() { | ||
| ReflectionTestUtils.setField(cmd, "id", null); | ||
| Pair<Boolean, String> failureResponse = new Pair<>(false, "Please provide a valid ID of a system VM to be patched"); | ||
| Mockito.doReturn(failureResponse).when(_mgr).patchSystemVM(cmd); | ||
| try { | ||
| cmd.execute(); | ||
| } catch (Exception e) { | ||
| Assert.assertEquals(failureResponse.second(), e.getMessage()); | ||
| } | ||
| } | ||
|
|
||
| @Test | ||
| public void validateArgsForPatchSystemVMApi() { | ||
| PowerMockito.mockStatic(CallContext.class); | ||
| CallContext callContextMock = PowerMockito.mock(CallContext.class); | ||
| PowerMockito.when(CallContext.current()).thenReturn(callContextMock); | ||
| Account accountMock = PowerMockito.mock(Account.class); | ||
| PowerMockito.when(callContextMock.getCallingAccount()).thenReturn(accountMock); | ||
| Mockito.when(accountMock.getId()).thenReturn(2L); | ||
| ReflectionTestUtils.setField(cmd, "id", 1L); | ||
| Assert.assertEquals((long)cmd.getId(), 1L); | ||
| Assert.assertFalse(cmd.isForced()); | ||
| Assert.assertEquals(cmd.getEntityOwnerId(), 2L); | ||
|
|
||
|
|
||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -890,32 +890,6 @@ | |
| </dependency> | ||
| </dependencies> | ||
| <build> | ||
| <plugins> | ||
| <plugin> | ||
| <artifactId>maven-antrun-plugin</artifactId> | ||
| <version>1.7</version> | ||
| <executions> | ||
| <!-- Copy the systemvm in the package phase as it is generated by console-proxy in the package | ||
| phase. --> | ||
| <execution> | ||
| <id>copy-systemvm</id> | ||
| <phase>process-resources</phase> | ||
| <goals> | ||
| <goal>run</goal> | ||
| </goals> | ||
| <configuration> | ||
| <target> | ||
| <copy todir="${basedir}/target/common/vms"> | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. :D so good to see old/archaic code gone |
||
| <fileset dir="${basedir}/../systemvm/dist"> | ||
| <include name="systemvm.iso" /> | ||
| </fileset> | ||
| </copy> | ||
| </target> | ||
| </configuration> | ||
| </execution> | ||
| </executions> | ||
| </plugin> | ||
| </plugins> | ||
| </build> | ||
| </profile> | ||
| <profile> | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same comment as above, see comment for restart network cmd.