Create XML file in vRealize Orchestrator for NSX Automation

NSX API uses XML format for API communication. To automate NSX in VMware vRealize Orchestror, it is always required to create a XML file with javascript  as vRO workflow supports javascript only.Here i only shows you an example to how to do it.

The target here is to create a security group and add a simple firewall rule in this newly created security group.

Note: this vRO workflow has 2 inputs:
securityGroupName, description
And 2 properties:
nsxManagerRestHost, realtime(equal to sgID in Step1)

Step1: create a security group

var xmlbody = new XML('<securitygroup />');
xmlbody.objectId = " ";
xmlbody.type.typeName = " ";
xmlbody.description = description;
xmlbody.name = securityGroupName;
xmlbody.revision = 0;
xmlbody.objectTypeName = " ";
System.log(xmlbody);
var request = nsxManagerRestHost.createRequest("POST", "/2.0/services/securitygroup/bulk/globalroot-0", xmlbody.toString());
request.contentType = "application/xml";
System.log("Creating a SecurityGroup " + securityGroupName);
System.log("POST Request URL: " + request.fullUrl);
var response = request.execute();
if (response.statusCode == 201) {
	System.debug("Successfully created Security Group " + securityGroupName);
	}
else {
	throw("Failed to SecurityGroup " + securityGroupName);
	}
sgID = response.getAllHeaders().get("Location").split('/').pop();
realtime=sgID

Step2: add a section in DFW and add a firewall rules

//create XML object for DFW source;
var rulesources = new XML('<sources excluded="false" />');
rulesources.source.name = " ";
rulesources.source.value = "10.47.161.23";
rulesources.source.type = "Ipv4Address";
rulesources.source.isValid = 'true';
System.log("Source: "+rulesources);

//create XML object for DFW destination;
var ruledestionations = new XML('<destinations excluded="false" />');
ruledestionations.destination.name = " ";
ruledestionations.destination.value = "10.47.161.24";
ruledestionations.destination.type = "Ipv4Address";
ruledestionations.destination.isValid = 'true';
System.log("Destination: "+ruledestionations);

//create XML object for DFW service
var ruleservices = new XML('<services />');
ruleservices.service.destinationPort = "80";
ruleservices.service.protocol = "6";
ruleservices.service.subProtocol = "6";
ruleservices.service.isValid = 'true';
System.log("Service: "+ruleservices);

//create XML object for the whole rule
var xmlbodyrule = new XML('<rule disabled="false" logged="true" />');
xmlbodyrule.name = "vro created rule";
xmlbodyrule.action = "allow";
xmlbodyrule.notes = " ";
xmlbodyrule.appliedToList.appliedTo.name = securityGroupName;
xmlbodyrule.appliedToList.appliedTo.value = realtime;
xmlbodyrule.appliedToList.appliedTo.type = 'SecurityGroup';
xmlbodyrule.appliedToList.appliedTo.isValid = 'true';
xmlbodyrule.sectionId = " ";
xmlbodyrule.sources = rulesources;
xmlbodyrule.destinations = ruledestionations;
xmlbodyrule.services = ruleservices;

//create XML object for DFW section
var xmlbody = new XML(
<section name ={securityGroupName} />);
//xmlbody.rule = 'disabled="false" logged="true" />';
xmlbody.rule=xmlbodyrule;
System.log("XML file for new rules: "+xmlbody);

var request = nsxManagerRestHost.createRequest("POST", "/4.0/firewall/globalroot-0/config/layer3sections", xmlbody.toString());
request.contentType = "application/xml";
var response = request.execute();
if (response.statusCode == 201) {
	System.debug("Successfully created Security Group Section" + securityGroupName);
	}
else {
	throw("Failed to SecurityGroup Section" + securityGroupName);
	}

Below is the output of XML file for creating a security group:

<securitygroup>
  <objectId></objectId>
  <type>
    <typeName></typeName>
  </type>
  <description>nsx1001test</description>
  <name>nsx1001test</name>
  <revision>0</revision>
  <objectTypeName></objectTypeName>
</securitygroup>

XML file for creating a NSX DFW section and adding a new simple firewall rules:

<section name="nsx1001test">
  <rule disabled="false" logged="true">
    <name>vro created rule</name>
    <action>allow</action>
    <notes></notes>
    <appliedToList>
      <appliedTo>
        <name>nsx1001test</name>
        <value>securitygroup-947</value>
        <type>SecurityGroup</type>
        <isValid>true</isValid>
      </appliedTo>
    </appliedToList>
    <sectionId></sectionId>
    <sources excluded="false">
      <source>
        <name></name>
        <value>10.47.161.23</value>
        <type>Ipv4Address</type>
        <isValid>true</isValid>
      </source>
    </sources>
    <destinations excluded="false">
      <destination>
        <name></name>
        <value>10.47.161.24</value>
        <type>Ipv4Address</type>
        <isValid>true</isValid>
      </destination>
    </destinations>
    <services>
      <service>
        <destinationPort>80</destinationPort>
        <protocol>6</protocol>
        <subProtocol>6</subProtocol>
        <isValid>true</isValid>
      </service>
    </services>
  </rule>
</section>

NSX Load Balancer Qucik Summary

Recently, I was asked a lot of questions around the capability of NSX load balancer by team and customers. So I put a quick summary of NSX load balancer to ease my life.

NSX can perform L4 or L7 load balancing:

  • L4 Load Balancing (packet-based load balancing) : support TCP and UDP load balancing, which is based on Linux Virtual Server.
  • L7 Load Balancing (socket-based load balancing): Support TCP and TCP-based application (e.g. HTTPs_ load balancing, which is based on HAProxy.

Regarding SSL load balancing, it requests L7 load balancing.

3 options for SSL load balancing:

  • SSL Passthrough:
    • NSX load balancer won’t terminate the client session and only pass through the SSL traffic;
    • Session persistence: SSL session id or source IP
  • SSL Offload:
    • client SSL session will be terminated on NSX load balancer and a clear-text (e.g. HTTP) session will be initiated from NSX load balancer to backend server;
    • Session persistence: cookie, SSL session id or source IP
  • SSL end to end:
    • client SSL session will be terminated on NSX load balancer and a new SSL session will be initiated from NSX load balancer to backend server;
    • Session persistence: cookie, SSL session id or source IP

Tips:

  1. L4 and L7 virtual server can co-exist on the same NSX load balancer;
  2. NSX load balancer can use 1 or multiple security groups as pool member, which means Virtual machines will be added into the load balancing pool automatically if they are added into right security group; This feature is especially useful when your Cloud VM is re-provisioned and its IP is changed;
  3. Transparent mode load balancing is not recommended due to the complexity and potential performance issue;
  4. In proxy mode, you can try to use HTTP x-forwarded-for to maintain the source IP information in the request;

Limitation and Constraints:

  1. Don’t supprt the integration with HSM;
  2. As NSX load balancer use the secondary IPs of vNIC, the size of virtual IP can’t scale up well;
  3. Lack of fine security control for traffic to virtual server;
  4. NSX can’t provide good service monitoring like F5 BIGIP or Citrix Netscaler;

 

vRA7.3 and NSX Integration: Network Security Data Collection Failure

We are building vRA 7.3 . We added vCenter and NSX manager as endpoint in vRA. And associate NSX manager with vCenter. All of computing resource data collection works well but not NSX (network and security):

So in vRA reservation, we only can see vSphere cluster, vDS port-group/logical switch but not Transport zone, security group/tags

When check the log, we see the following:

Workflow ‘vSphereVCNSInventory’ failed with the following exception:

One or more errors occurred.

Inner Exception: An error occurred while sending the request.

at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)

at DynamicOps.VCNSModel.Interface.NSXClient.GetDatacenters()

at DynamicOps.VCNSModel.Activities.CollectDatacenters.Execute(CodeActivityContext context)

at System.Activities.CodeActivity.InternalExecute(ActivityInstance instance, ActivityExecutor executor, BookmarkManager bookmarkManager)

at System.Activities.Runtime.ActivityExecutor.ExecuteActivityWorkItem.ExecuteBody(ActivityExecutor executor, BookmarkManager bookmarkManager, Location resultLocation)

Inner Exception:

VCNS Workflow failure

I tried to delete NSX end point and recreate from vRA but no luck. I raised the issue in vmware community but can’t get any real valuable feedback.

After a few hours investigation, finally I find a fix:

run the “create a NSX endpoint” workflow in vRO as the below

2017-07-26_184701

Then I re-start network & security data collection in vRA. Everything works and I can see all defined NSX Transport Zone, security groups and DLR in vRA network reservations.

Hope this fix can help others who have the same issue.

Perform Packet Capture on VMware ESXi Host for NSX Troubleshooting

VMware offers a great and powerful tool pktcap-uw to perform packet capture on ESXi host.

Pktcap-uw offers a lot of options for packet capture.

https://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2051814

Here I show most common used in my daily life here for your reference. I normally perform a packet based on vSwitch port ID or DV filter (NSX DFW)

To do that, I firstly need to find the vSwitch port ID and DV filter ID on ESXi host so that I can refer them in your packet capture. I normally use “summarize-dvfilter” CLI to find the requested information.

[root@esx4005:/tmp] summarize-dvfilter | grep -C 10 1314
slowPathID: none
filter source: Dynamic Filter Creation
vNic slot 1
name: nic-18417802-eth0-dvfilter-generic-vmware-swsec.1
agentName: dvfilter-generic-vmware-swsec
state: IOChain Attached
vmState: Detached
failurePolicy: failClosed
slowPathID: none
filter source: Alternate Opaque Channel
world 18444553 vmm0:auslslnxsd1314-113585a5-f6ed-4eb3-abd2-12083901e942 vcUuid:’11 35 85 a5 f6 ed 4e b3-ab d2 12 08 39 01 e9 42′
port 33554558 (vSwitch PortID) auslslnxsd1314-113585a5-f6ed-4eb3-abd2-12083901e942.eth0
vNic slot 2
name: nic-18444553-eth0-vmware-sfw.2 (DV Filter ID)
agentName: vmware-sfw
state: IOChain Attached
vmState: Detached
failurePolicy: failClosed
slowPathID: none
filter source: Dynamic Filter Creation
vNic slot 1
name: nic-18444553-eth0-dvfilter-generic-vmware-swsec.1

 

After I have the vSwitch port ID and DV filter ID, I can start my packet capture.

  • Packet capture to a VM based on vSwitch PortID

pktcap-uw –switchport 33554558 —dir 0 -o /tmp/from1314.pcap

  • Packet capture from a VM based on vSwitch PortID

pktcap-uw –switchport 33554558 —dir 1 -o /tmp/to1314.pcap

  • Packet capture from a VM based on DV filter

pktcap-uw –capture PreDVFilter –dvfilter nic-18444553-eth0-vmware-sfw.2 -o /tmp/1314v3.pcap

Below is a brief explanation of the parameters which we use in the above.

-o (output): save the capture as a packet capture file;

-dir (direction): 0 for traffic to VM and 1 for traffic from VM;

-PreDVFilter: perform packet capture before DFW rules are applied;

-PostDVFilter: perform packet capture after DFW rules are applied;

In addition, you can add filter as well for your capture:

pktcap-uw –switchport 33554558 –tcpport 9000 –dir 1 -o /tmp/from1314.pcap

I list all available filter options here for your reference:

–srcmac
The Ethernet source MAC address.
–dstmac
The Ethernet destination MAC address.
–mac
The Ethernet MAC address(src or dst).
–ethtype
The Ethernet type. HEX format.
–vlan
The Ethernet VLAN ID.
–srcip
The source IP address.
–dstip
The destination IP address.
–ip
The IP address(src or dst).
–proto 0x
The IP protocol.
–srcport
The TCP source port.
–dstport
The TCP destination port.
–tcpport
The TCP port(src or dst).
–vxlan
The vxlan id of flow.

Update:

Start 2 capture at the same time:

pktcap-uw –switchport 50331665 -o /tmp/50331665.pcap & pktcap-uw –uplink vmnic2 -o /tmp/vmnic2.pcap &

Stop all packet capture:

kill $(lsof |grep pktcap-uw |awk ‘{print $1}’| sort -u)

Of course, you can perform some basic packet capture in NSX manager via Central CLI. If you are interested in, please refer my another blog:

https://davidwzhang.com/2017/01/07/limitation-of-nsx-central-cli-packet-capture/

NSX IPSec Throughput in IBM Softlayer

To understand the real throughput capacity of NSX IPSec in Softlayer, I built a quick IPSec performance testing environment.

Below are the network topology of my testing environment:

NSX_IPSec_Performance_Topology

NSX version: 6.2.4
NSX Edge: X-Large (6 vCPUs and 8G Memory), which is the largest size NSX offers. All of Edges in this testing enviroment reside in the same vSphere cluster which include 3 ESXi hosts. Each ESXi host has 64GB DDR4 Memory and 2 processors (2.4GHz Intel Xeon-Haswell (E5-2620-V3-HexCore))
IPerf Client: Redhat 7.1 (2 vCPUs and 4GB Memory)
IPerf Server: Redhat 7.1 (2 vCPUs and 4GB Memory)
IPerf version: IPerf3

2 IPsec tunnels are built as the above diagram. IPSec setting is:

  • Encryption: AES-GCM
  • Diff-Hellman Group: DH5
  • PFS(Perfect forward secrecy): Enabled
  • AESNI: Enabled
I include 3 test cases in my testing:
Test1_Bandwidth_Utilisation
  • Test Case 2: 2 IPerf Clients (172.16.31.0/24) to 2 IPerf Servers (172.16.38.0/24) via 1 IPsec Tunnel. Result: around 1.6-2.3Gbit/s in total
Test2_Bandwidth_Utilisation
Test3_Bandwidth_Utilisation
Please note:
  1. Firewall function on NSX Edge is disabled in all test cases.
  2. TCP traffic is used in all 3 test cases. 10 parallel streams are used to push the performance test to the max on each IPerf Client.
  3. I didn’t see any CPU or Memory contention in all test cases: the CPU utilisation of NSX Edge was  less than 40% and memory utilisation is nearly zero.

CPU_Mem

Simple Python Script Creating a Dynamic Membership Security Group

In this blog, I developed a very simple Python scripts to create NSX security group whose membership is based on Security Tag. Please note this script is to show you the basic, which has not been ready for a production environment.

Two Python functions are includes in this script:

  1. create_tag is used to create a NSX security tag;
  2. create_sg is used to create a security group and define a criterion to add all virtual machines tagged with the specified security tag into this newly created security group;
import requests
from base64 import b64encode
import getpass
username=raw_input('Enter Your NSXManager Username: ')
yourpass = getpass.getpass('Enter Your NSXManager Password: ')
sg_name=raw_input('Enter Security Group Name: ')
vm_tag=raw_input('Enter Tag Name: ')
userandpass=username+":"+yourpass
userpass = b64encode(userandpass).decode("ascii")
auth ="Basic " + userpass
payload_tag="<securityTag>\r\n<objectTypeName>SecurityTag</objectTypeName>\r\n<type>\r\n<typeName>SecurityTag</typeName>\r\n</type>\r\n<name>"+vm_tag+"</name>\r\n<isUniversal>false</isUniversal>\r\n<description>This tage is created by API</description>\r\n<extendedAttributes></extendedAttributes>\r\n</securityTag>"
payload_sg= "<securitygroup>\r\n <objectId></objectId>\r\n <objectTypeName>SecurityGroup</objectTypeName>\r\n <type>\r\n <typeName>SecurityGroup</typeName>\r\n </type>\r\n <description></description>\r\n <name>"+sg_name+"</name>\r\n <revision>0</revision>\r\n<dynamicMemberDefinition>\r\n <dynamicSet>\r\n <operator>OR</operator>\r\n <dynamicCriteria>\r\n <operator>OR</operator>\r\n <key>VM.SECURITY_TAG</key>\r\n <criteria>contains</criteria>\r\n <value>"+vm_tag+"</value>\r\n </dynamicCriteria>\r\n </dynamicSet>\r\n</dynamicMemberDefinition>\r\n</securitygroup>"

def create_tag():
        try:
                response = requests.post(
                url="https://NSX-Manager-IP/api/2.0/services/securitytags/tag",
                verify=False,
                headers={
                        "Authorization": auth,
                        "Content-Type": "application/xml",
                    },
                data=payload_tag
                    )
                print('Response HTTP Status Code: {status_code}'.format(status_code=response.status_code))
                #print('Response HTTP Response Body: {content}'.format(content=response.content))
                if response.status_code == 403:
                        print "***********************************************************************"
                        print "WARNING: your username or password is wrong, please retry again!"
                        print "***********************************************************************"
                if  response.status_code == 201:
                        print "***********************************************************************"
                        print('Response HTTP Response Body: {content}'.format(content=response.content))
                api_response=response.text
                print api_response
        except requests.exceptions.RequestException:
                print('HTTP Request failed')

def create_sg():
        try:
                response = requests.post(
                url="https://NSX-Manager-IP/api/2.0/services/securitygroup/bulk/globalroot-0",
                verify=False,
                headers={
                        "Authorization": auth,
                        "Content-Type": "application/xml",
                    },
                data=payload_sg
                    )
                print('Response HTTP Status Code: {status_code}'.format(status_code=response.status_code))
                #print('Response HTTP Response Body: {content}'.format(content=response.content))
                if response.status_code == 403:
                        print "***********************************************************************"
                        print "WARNING: your username or password is wrong, please retry again!"
                        print "***********************************************************************"
                if  response.status_code == 201:
                        print "***********************************************************************"
                        print('Response HTTP Response Body: {content}'.format(content=response.content))
                api_response=response.text
                print api_response
        except requests.exceptions.RequestException:
                print('HTTP Request failed')

Run this script in our O-Dev:

[root]$ python create_sg_dynamic_member_20170429.py

Enter Your NSXManager UserName: admin

Enter Your NSXManager Passowrd:

Enter Security Group Name: sg_app1_web

Enter Tag Name: tag_app1_web

Response HTTP Status Code: 201

***********************************************************************

Response HTTP Response Body: securitytag-14

securitytag-14

Response HTTP Status Code: 201

***********************************************************************

Response HTTP Response Body: securitygroup-485

securitygroup-485

In NSX manager, we can see a securtiy group sg_app1_web is created as below:

2017-04-30_140657

And its dynamic membeship criterion is:

2017-04-30_140729

NSX-v DLR OSPF Adjacencies Configuration Maximums

In one of NSX doc, the below is suggested around DLR OSPF Adjacencies configuration maximum:

OSPF Adjacencies per DLR 10

This maximum applies to NSX 6.1, 6.2 and 6.3.

In OSPF , OSPF optimizes the LSA flooding process on multiaccess network by using DR (designated rourer) and BDR (backup DR). Routers that are not DR or BDR are called DRother routers. All DRother routers only form full adjacency with DR and BDR. Among DRother routers, they will stay in 2Way state and forms OSPF neighborship but not full adjacency.

To clarify if  “adjacencies” in VMWare NSX configuration maximums doc means “full adjacency” or “neighbor/2way state”, I raise a SR with VMWare GSS. The response from VMware GSS is:

  • their “Adjacencies” mean “neighborship” not “full adjacency”.
  • the 2WAY state also will be included  in the configuration limit of the 10 OSPF adjacencies per DLR