Skip to content

CLOUDSTACK-10177: Only pass IPv6 address to Security Group Python scr…#2355

Merged
yadvr merged 1 commit into
apache:masterfrom
wido:CLOUDSTACK-10177
Dec 11, 2017
Merged

CLOUDSTACK-10177: Only pass IPv6 address to Security Group Python scr…#2355
yadvr merged 1 commit into
apache:masterfrom
wido:CLOUDSTACK-10177

Conversation

@wido
Copy link
Copy Markdown
Contributor

@wido wido commented Dec 6, 2017

…ipt if present

Otherwise we send down a 'null' to a ProcessBuilder in Java instead of a String and this
causes a NPE.

We should check first if the Instance has a IPv6 address before sending it there.

Signed-off-by: Wido den Hollander wido@widodh.nl

…ipt if present

Otherwise we send down a 'null' to a ProcessBuilder in Java instead of a String and this
causes a NPE.

We should check first if the Instance has a IPv6 address before sending it there.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
@wido wido added this to the 4.11 milestone Dec 6, 2017
Copy link
Copy Markdown
Member

@GabrielBrascher GabrielBrascher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Wido. Code LGTM.

@bwsw
Copy link
Copy Markdown
Contributor

bwsw commented Dec 6, 2017

@wido
@GabrielBrascher

Hi, take a look here. Massive fixes to make 4.10 work. Includes this fix and other ipv6 mods.
#2320

@GabrielBrascher
Copy link
Copy Markdown
Member

Thanks for pointing out PR #2320 @bwsw.

@bwsw
Copy link
Copy Markdown
Contributor

bwsw commented Dec 6, 2017

@GabrielBrascher @wido
Also, this fix already introduced somewhere in previous PRs in broader edition, because I cherry picked it to #2320 from somewhere. Also, please take a look at security_groups.py because there are still cases when ICMPv6 can lead to exceptions without proper handling. E.g. passing -1,-1 for icmp v6 SG.

cmd.add("--vmid", vmId);
cmd.add("--vmip", guestIP);
cmd.add("--vmip6", guestIP6);
if (StringUtils.isNotBlank(guestIP6)) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wido Can you use !String.isNullOrEmpty() instead?

@yadvr
Copy link
Copy Markdown
Member

yadvr commented Dec 6, 2017

LGTM.
@blueorangutan package

@blueorangutan
Copy link
Copy Markdown

@rhtyd a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

@blueorangutan
Copy link
Copy Markdown

Packaging result: ✔centos6 ✔centos7 ✔debian. JID-1350

@yadvr
Copy link
Copy Markdown
Member

yadvr commented Dec 6, 2017

@blueorangutan test

@blueorangutan
Copy link
Copy Markdown

@rhtyd a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

@blueorangutan
Copy link
Copy Markdown

Trillian test result (tid-1740)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 29843 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr2355-t1740-kvm-centos7.zip
Test /marvin/tests/smoke/test_accounts.py took 1285 seconds
Test /marvin/tests/smoke/test_affinity_groups_projects.py took 200 seconds
Test /marvin/tests/smoke/test_affinity_groups.py took 100 seconds
Test /marvin/tests/smoke/test_certauthority_root.py took 43 seconds
Test /marvin/tests/smoke/test_deploy_vgpu_enabled_vm.py took 4 seconds
Test /marvin/tests/smoke/test_deploy_virtio_scsi_vm.py took 1516 seconds
Test /marvin/tests/smoke/test_deploy_vm_iso.py took 85 seconds
Test /marvin/tests/smoke/test_deploy_vm_root_resize.py took 120 seconds
Test /marvin/tests/smoke/test_deploy_vms_with_varied_deploymentplanners.py took 180 seconds
Test /marvin/tests/smoke/test_deploy_vm_with_userdata.py took 90 seconds
Test /marvin/tests/smoke/test_disk_offerings.py took 4 seconds
Test /marvin/tests/smoke/test_dynamicroles.py took 107 seconds
Test /marvin/tests/smoke/test_global_settings.py took 4 seconds
Test /marvin/tests/smoke/test_guest_vlan_range.py took 25 seconds
Test /marvin/tests/smoke/test_host_annotations.py took 13 seconds
Test /marvin/tests/smoke/test_hostha_simulator.py took 5 seconds
Test /marvin/tests/smoke/test_host_maintenance.py took 295 seconds
Test /marvin/tests/smoke/test_hosts.py took 5 seconds
Test /marvin/tests/smoke/test_internal_lb.py took 1516 seconds
Test /marvin/tests/smoke/test_iso.py took 242 seconds
Test /marvin/tests/smoke/test_list_ids_parameter.py took 640 seconds
Test /marvin/tests/smoke/test_loadbalance.py took 666 seconds
Test /marvin/tests/smoke/test_login.py took 21 seconds
Test /marvin/tests/smoke/test_metrics_api.py took 112 seconds
Test /marvin/tests/smoke/test_multipleips_per_nic.py took 136 seconds
Test /marvin/tests/smoke/test_nested_virtualization.py took 9 seconds
Test /marvin/tests/smoke/test_network_acl.py took 81 seconds
Test /marvin/tests/smoke/test_network.py took 1398 seconds
Test /marvin/tests/smoke/test_nic_adapter_type.py took 9 seconds
Test /marvin/tests/smoke/test_nic.py took 462 seconds
Test /marvin/tests/smoke/test_non_contigiousvlan.py took 19 seconds
Test /marvin/tests/smoke/test_outofbandmanagement_nestedplugin.py took 90 seconds
Test /marvin/tests/smoke/test_outofbandmanagement.py took 235 seconds
Test /marvin/tests/smoke/test_over_provisioning.py took 4 seconds
Test /marvin/tests/smoke/test_password_server.py took 265 seconds
Test /marvin/tests/smoke/test_portable_publicip.py took 50 seconds
Test /marvin/tests/smoke/test_portforwardingrules.py took 116 seconds
Test /marvin/tests/smoke/test_primary_storage.py took 546 seconds
Test /marvin/tests/smoke/test_privategw_acl.py took 864 seconds
Test /marvin/tests/smoke/test_projects.py took 602 seconds
Test /marvin/tests/smoke/test_public_ip_range.py took 10 seconds
Test /marvin/tests/smoke/test_pvlan.py took 9 seconds
Test /marvin/tests/smoke/test_regions.py took 5 seconds
Test /marvin/tests/smoke/test_reset_vm_on_reboot.py took 256 seconds
Test /marvin/tests/smoke/test_resource_detail.py took 14 seconds
Test /marvin/tests/smoke/test_router_dhcphosts.py took 421 seconds
Test /marvin/tests/smoke/test_router_dns.py took 197 seconds
Test /marvin/tests/smoke/test_router_dnsservice.py took 174 seconds
Test /marvin/tests/smoke/test_routers_iptables_default_policy.py took 320 seconds
Test /marvin/tests/smoke/test_routers_network_ops.py took 950 seconds
Test /marvin/tests/smoke/test_routers.py took 269 seconds
Test /marvin/tests/smoke/test_scale_vm.py took 3 seconds
Test /marvin/tests/smoke/test_secondary_storage.py took 4 seconds
Test /marvin/tests/smoke/test_service_offerings.py took 370 seconds
Test /marvin/tests/smoke/test_snapshots.py took 208 seconds
Test /marvin/tests/smoke/test_ssvm.py took 886 seconds
Test /marvin/tests/smoke/test_staticroles.py took 3 seconds
Test /marvin/tests/smoke/test_templates.py took 1211 seconds
Test /marvin/tests/smoke/test_usage_events.py took 4 seconds
Test /marvin/tests/smoke/test_usage.py took 1579 seconds
Test /marvin/tests/smoke/test_vm_life_cycle.py took 1394 seconds
Test /marvin/tests/smoke/test_vm_snapshots.py took 634 seconds
Test /marvin/tests/smoke/test_volumes.py took 2212 seconds
Test /marvin/tests/smoke/test_vpc_redundant.py took 2728 seconds
Test /marvin/tests/smoke/test_vpc_router_nics.py took 674 seconds
Test /marvin/tests/smoke/test_vpc_vpn.py took 696 seconds
Test /marvin/tests/smoke/test_hostha_kvm.py took 833 seconds
Test completed. 61 look OK, 6 have error(s)

Test Result Time (s) Test File
ContextSuite context=TestDeployVirtioSCSIVM>:setup Error 0.00 test_deploy_virtio_scsi_vm.py
ContextSuite context=TestInternalLb>:setup Error 0.00 test_internal_lb.py
test_01_vpc_privategw_acl Failure 56.15 test_privategw_acl.py
test_02_vpc_privategw_static_routes Failure 157.12 test_privategw_acl.py
test_03_vpc_privategw_restart_vpc_cleanup Failure 152.27 test_privategw_acl.py
test_04_rvpc_privategw_static_routes Failure 277.91 test_privategw_acl.py
test_10_attachAndDetach_iso Failure 673.68 test_vm_life_cycle.py
test_01_create_redundant_VPC_2tiers_4VMs_4IPs_4PF_ACL Failure 387.61 test_vpc_redundant.py
test_01_vpc_remote_access_vpn Failure 60.71 test_vpc_vpn.py

@wido
Copy link
Copy Markdown
Contributor Author

wido commented Dec 7, 2017

@bwsw @GabrielBrascher @rhtyd : I am closing this one in favor of #2320

I will review that one

@bwsw
Copy link
Copy Markdown
Contributor

bwsw commented Dec 7, 2017

@wido I found your commit eaffe3a in wido/ipv6-icmpv6all. Together with current PR, I believe, they make ipv6 in 4.10 functional, unfortunately your PR wido/ipv6-icmpv6all wasn't included in 4.10 by unknown reason.

I suppose it's wise to keep current PR, because together:

  1. this PR (to merge to 4.11)
  2. wido/ipv6-icmpv6all (merged to 4.11)
  3. CLOUDSTACK-10138 (merged to 4.11)

they do all necessary IPv6+sg work. But I still feel that in security_group.py code should include try-except:

        for ip in rule['ipv4']:
            try:
                if protocol == 'all':
                    execute('iptables -I ' + vmchain + ' -m state --state NEW ' + direction + ' ' + ip + ' -j ' + action)
                elif protocol != 'icmp':
                    execute('iptables -I ' + vmchain + ' -p ' + protocol + ' -m ' + protocol + ' --dport ' + range + ' -m state --state NEW ' + direction + ' ' + ip + ' -j ' + action)
                else:
                    execute('iptables -I ' + vmchain + ' -p icmp --icmp-type ' + range + ' ' + direction + ' ' + ip + ' -j ' + action)
            except:
                pass

        for ip in rule['ipv6']:
            try:
                if protocol == 'all':
                    execute('ip6tables -I ' + vmchain + ' -m state --state NEW ' + direction + ' ' + ip + ' -j ' + action)
                elif 'icmp' != protocol:
                    execute('ip6tables -I ' + vmchain + ' -p ' + protocol + ' -m ' + protocol + ' --dport ' + range + ' -m state --state NEW ' + direction + ' ' + ip + ' -j ' + action)
                else:
                    if range == 'any':
                        execute('ip6tables -I ' + vmchain + ' -p icmpv6 ' + direction + ' ' + ip + ' -j ' + action)
                    else:
                        execute('ip6tables -I ' + vmchain + ' -p icmpv6 --icmpv6-type ' + range + ' ' + direction + ' ' + ip + ' -j ' + action)
            except:
                pass

at least because user can pass wrong icmp type/code and we should handle it at least skipping but not failing all security group configuration. So if you could improve current PR with try-catch I think it's more reasonable than diving into: #2320 as it includes several cherry-picks which already in master and it is quite messy and I believe it shouldn't be merged to master. Just for those who willing to make 4.10 work.

@yadvr
Copy link
Copy Markdown
Member

yadvr commented Dec 8, 2017

@wido this is still open, are you closing this?

@wido
Copy link
Copy Markdown
Contributor Author

wido commented Dec 8, 2017

Hmm @rhtyd and @bwsw: I think this one can stay open and merged.

For the try/except I can submit a new PR against master. But I'm not to happy with fixes going into 4.10 branch @bwsw and not going into master.

We should probably backport my fixes to 4.10 since that keeps the code-path clean.

@bwsw
Copy link
Copy Markdown
Contributor

bwsw commented Dec 8, 2017

@wido I agree with you and don't mind at all. I published that PR not for inclusion but to help people who would like to get 4.10 up and running asap.

@wido
Copy link
Copy Markdown
Contributor Author

wido commented Dec 8, 2017

Ok @bwsw! We should then merge this one and have a few separate PRs which go into master to fix it for 4.11. How does that sound?

I will send another PR for the try/except

@bwsw
Copy link
Copy Markdown
Contributor

bwsw commented Dec 11, 2017

@wido Great!

@yadvr
Copy link
Copy Markdown
Member

yadvr commented Dec 11, 2017

LGTM, merging. All errors seen are intermittent and env related.

@yadvr yadvr merged commit be3a39e into apache:master Dec 11, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants