Category: Network Automation

Export NSX-v DFW Rules as CSV File

On By insidepacketIn Network Automation, NSX, PythonLeave a comment

From NSX-v version 6.4.0, NSX API begins to support JSON format for its response not like before only XML format. From my own expereince, I prefer to use JSON format than XML format as it is easier to decode and encode JSON data than XML data. So I took 1 weekend to re-write my old Python code. Now this code can get Json format NSX-V DFW rules from NSX manager and then place into a CSV file so that you can view and search your DFW rules easily.

Below is a sample of CSV file which is generated by my Python code.

dfw-csv

I have put the source code in Github:

https://github.com/insidepacket/NSX-Toolkit/blob/master/export_dfw_to_csv.py

Feel free to enjoy.

 

Install PowerCLI and PowerNSX Offline on RHEL7

On By insidepacketIn Network Automation, NSX, VirtualizationLeave a comment

With the release of PowerCLI 10.0.0, VMware adds support for Mac OS and Linux! Now you can install PowerCLI and PowerNSX on Linux System including RHEL, Centos, Unbuntu and Mac OS. To complete installation of VMware PowerCLI 10 and PowerNSX, firstly you need to install Powershell Core 6.0.

In most of enterprise environments, we won’t be so lucky to have Internet access for all your Redhat RHEL systems. In this blog, I will show you how to install Powershell, PowerCLI and PowerNSX offline on Red Hat Enterprise Linux Server.

Software version:

Red Hat Enterprise Linux Server release 7.5 (Maipo)

PowerShell v6.0.2

VMware PowerCLI 10.1.1

VMware PowerNSX 3.0.1110

Step 0: Prerequisite

You have another Windows workstation/server or Linux which have Internet access and Powershell installed so that we can download all required packages.

In addition, make sure that your RHEL meet the following prerequisites:

  • openssl devel (version 1.0.2k and above) package installed

[root@localhost Powershell]# rpm -qa | grep openssl
openssl-1.0.2k-12.el7.x86_64
xmlsec1-openssl-1.2.20-7.el7_4.x86_64
openssl-libs-1.0.2k-12.el7.x86_64
openssl-devel-1.0.2k-12.el7.x86_64

  • “Development tools” packages installed

You can find out which packages are included in the “Development Tools” packages by CLI: yum group info “Development Tools”

Step 1: Install PowerShell v6.0.2

Go to website https://packages.microsoft.com/rhel/7/prod/ to download the required packages including dotnet and powershell.

dotnet

pwsh

  • Installed the following dotnet packages via “rpm -ivh”

[root@localhost yum.repos.d]# rpm -qa | grep dotn
dotnet-runtime-2.0.5-2.0.5-1.x86_64
dotnet-runtime-deps-2.1-2.1.0-1.x86_64
dotnet-hostfxr-2.0.5-2.0.5-1.x86_64
dotnet-sdk-2.1.4-2.1.4-1.x86_64
dotnet-host-2.1.0-1.x86_64

  • Install  Powershell 6.0.2

rpm -ivh powershell-6.0.2-1.rhel.7.x86_64.rpm

After you successfully installed Powershell, you need to create “Modules” directory for PowerCLI and PowerNSX modules. This “Modules” directory is under your home directory: /home/username/.local/share/powershell/Modules for current user or /usr/local/share/powershell/Modules for all users.

Step 2: Install PowerCLI Core

Since PowerCLI version 6.5, you can’t download the PowerCLI package from VMware directly any longer. You have to connect to PowerShell Gallery via Internet to install PowerCLI. As our RHEL has no Internet access. We firstly need to use “Save-Module” to download the latest PowerCLI package then upload to our RHEL system for installation.

Save-Module -Name VMware.PowerCLI -Path /root/powershell/powercli10

After uploading all sub-directories to the RHEL server, you copy all directories/files under the “Modules” directory which you created in Step 1.

[root@localhost powershell]# cd Modules/
[root@localhost Modules]# ls -al
total 4
drwxr-xr-x. 24 root root 4096 Jun 19 13:59 .
drwxr-xr-x. 5 root root 54 Jun 18 19:51 ..
drwxr-xr-x. 3 root root 27 Jun 19 08:51 VMware.DeployAutomation
drwxr-xr-x. 3 root root 27 Jun 19 08:51 VMware.ImageBuilder
drwxr-xr-x. 3 root root 28 Jun 19 08:51 VMware.PowerCLI
drwxr-xr-x. 3 root root 27 Jun 19 08:51 VMware.Vim
drwxr-xr-x. 3 root root 28 Jun 19 08:51 VMware.VimAutomation.Cis.Core
drwxr-xr-x. 3 root root 28 Jun 19 08:51 VMware.VimAutomation.Cloud
drwxr-xr-x. 3 root root 28 Jun 19 08:51 VMware.VimAutomation.Common
drwxr-xr-x. 3 root root 28 Jun 19 08:51 VMware.VimAutomation.Core
drwxr-xr-x. 3 root root 27 Jun 19 08:51 VMware.VimAutomation.HA
drwxr-xr-x. 3 root root 27 Jun 19 08:51 VMware.VimAutomation.HorizonView
drwxr-xr-x. 3 root root 28 Jun 19 08:51 VMware.VimAutomation.License
drwxr-xr-x. 3 root root 28 Jun 19 08:51 VMware.VimAutomation.Nsxt
drwxr-xr-x. 3 root root 28 Jun 19 08:51 VMware.VimAutomation.PCloud
drwxr-xr-x. 3 root root 28 Jun 19 08:51 VMware.VimAutomation.Sdk
drwxr-xr-x. 3 root root 28 Jun 19 08:52 VMware.VimAutomation.Srm
drwxr-xr-x. 3 root root 28 Jun 19 08:52 VMware.VimAutomation.Storage
drwxr-xr-x. 3 root root 21 Jun 19 08:52 VMware.VimAutomation.StorageUtility
drwxr-xr-x. 3 root root 28 Jun 19 08:52 VMware.VimAutomation.Vds
drwxr-xr-x. 3 root root 28 Jun 19 08:52 VMware.VimAutomation.Vmc
drwxr-xr-x. 3 root root 28 Jun 19 08:52 VMware.VimAutomation.vROps
drwxr-xr-x. 3 root root 27 Jun 19 08:52 VMware.VumAutomation

Now your PowerCLI is nearly ready for use.

Issue “pwsh” from bash then you start PowerShell

[root@localhost Modules]# pwsh
PowerShell v6.0.2
Copyright (c) Microsoft Corporation. All rights reserved.

https://aka.ms/pscore6-docs
Type ‘help’ to get help.

PS /root/.local/share/powershell/Modules>

As VMware PowerCLI 10 release notes, not all modules are supported with PowerShell Core 6.0 on RHEL. So before you import the PowerCLI modules, you have to change the  “VMware.PowerCLI.psd1” file to only load supported modules. The location of “VMware.PowerCLI.psd1” file  is as below

[root@localhost 10.1.1.8827524]# pwd
/root/.local/share/powershell/Modules/VMware.PowerCLI/10.1.1.8827524
[root@localhost 10.1.1.8827524]# ls -al
total 64
drwxr-xr-x. 2 root root 115 Jun 19 09:45 .
drwxr-xr-x. 3 root root 28 Jun 19 08:51 ..
-rw-r–r–. 1 root root 15196 Jun 18 21:57 PSGetModuleInfo.xml
-rw-r–r–. 1 root root 16413 Jun 14 10:36 VMware.PowerCLI.cat
-rw-r–r–. 1 root root 11603 Jun 14 10:36 VMware.PowerCLI.ps1
-rw-r–r–. 1 root root 14692 Jun 19 09:45 VMware.PowerCLI.psd1

Edit the above file like below (comment each line which include the un-supported module by adding # in the beginning)

# Modules that must be imported into the global environment prior to importing this module
RequiredModules = @(
@{“ModuleName”=”VMware.VimAutomation.Sdk”;”ModuleVersion”=”10.1.0.8342078″}
@{“ModuleName”=”VMware.VimAutomation.Common”;”ModuleVersion”=”10.1.0.8342134″}
@{“ModuleName”=”VMware.Vim”;”ModuleVersion”=”6.7.0.8343295″}
@{“ModuleName”=”VMware.VimAutomation.Core”;”ModuleVersion”=”10.1.0.8344055″}
#@{“ModuleName”=”VMware.VimAutomation.Srm”;”ModuleVersion”=”10.0.0.7893900″}
#@{“ModuleName”=”VMware.VimAutomation.License”;”ModuleVersion”=”10.0.0.7893904″}
@{“ModuleName”=”VMware.VimAutomation.Vds”;”ModuleVersion”=”10.1.0.8344219″}
@{“ModuleName”=”VMware.VimAutomation.Vmc”;”ModuleVersion”=”10.0.0.7893902″}
@{“ModuleName”=”VMware.VimAutomation.Nsxt”;”ModuleVersion”=”10.1.0.8346947″}
#@{“ModuleName”=”VMware.VimAutomation.vROps”;”ModuleVersion”=”10.0.0.7893921″}
@{“ModuleName”=”VMware.VimAutomation.Cis.Core”;”ModuleVersion”=”10.1.0.8377811″}
#@{“ModuleName”=”VMware.VimAutomation.HA”;”ModuleVersion”=”6.5.4.7567193″}
#@{“ModuleName”=”VMware.VimAutomation.HorizonView”;”ModuleVersion”=”7.5.0.8827468″}
#@{“ModuleName”=”VMware.VimAutomation.PCloud”;”ModuleVersion”=”10.0.0.7893924″}
#@{“ModuleName”=”VMware.VimAutomation.Cloud”;”ModuleVersion”=”10.0.0.7893901″}
#@{“ModuleName”=”VMware.DeployAutomation”;”ModuleVersion”=”6.7.0.8250345″}
#@{“ModuleName”=”VMware.ImageBuilder”;”ModuleVersion”=”6.7.0.8250345″}
@{“ModuleName”=”VMware.VimAutomation.Storage”;”ModuleVersion”=”10.1.0.8313015″}
@{“ModuleName”=”VMware.VimAutomation.StorageUtility”;”ModuleVersion”=”1.2.0.0″}
#@{“ModuleName”=”VMware.VumAutomation”;”ModuleVersion”=”6.5.1.7862888″}
)

If we have not removed the unsupported from the list of “Must Import Module”, we will see error like below:

Import-Module : The VMware.ImageBuilder module is not currently supported on the Core edition of PowerShell.
At line:1 char:1
+ import-module VMware.PowerCLI
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (The VMware.Imag… of PowerShell.:String) [Import-Module], RuntimeException
+ FullyQualifiedErrorId : The VMware.ImageBuilder module is not currently supported on the Core edition of PowerShell.,Microsoft.PowerShell.Commands.ImportModuleCommand 

Now you are ready to import PowerCLI modules.

PS /root/.local/share/powershell/Modules> import-module VMware.PowerCLI
$<5> Welcome to VMware PowerCLI!

Log in to a vCenter Server or ESX host: Connect-VIServer
To find out what commands are available, type: Get-VICommand
To show searchable help for all PowerCLI commands: Get-PowerCLIHelp
Once you’ve connected, display all virtual machines: Get-VM
If you need more help, visit the PowerCLI community: Get-PowerCLICommunity

Copyright (C) VMware, Inc. All rights reserved.

PS /root/.local/share/powershell/Modules>

However, when use cmdlet Connect-VIServer to connect vCenter server, you will see an error similar like this:

Error
Connect-VIServer : 06/22/18 11:22:26 AM Connect-VIServer The libcurl library in use (7.29.0) and its SSL backend (“NSS/3.21 Basic ECC”) do not support custom handling of certificates. A libcurl built with OpenSSL is required.

The cause of this error is that RHEL libcurl library is too old which doesn’t support OpenSSL. Please refer the following link which suggests how to fix the above issue by getting curl 7.52.1 installed.

https://www.opentechshed.com/powercli-core-on-centos-7/

[root@localhost ~]# curl –version
curl 7.52.1 (x86_64-pc-linux-gnu) libcurl/7.52.1 OpenSSL/1.0.2k zlib/1.2.7
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: IPv6 Largefile NTLM NTLM_WB SSL libz UnixSockets HTTPS-proxy

When we try to use “Connect-VIServer” cmdlet again, we see another error. This will happen when you connect to vCenter via IP or your RHEL think the received certificate is not valid:

Connect-VIServer : 6/21/18 11:40:16 AM Connect-VIServer Error: Invalid server certificate. Use Set-PowerCLIConfiguration to set the value for the InvalidCertificateAction option to Ignore to ignore the certificate errors for this connection.
Additional Information: Could not establish trust relationship for the SSL/TLS secure channel with authority ‘10.1.1.2’.
At line:1 char:1
+ Connect-VIServer -Server 10.1.1.2
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : SecurityError: (:) [Connect-VIServer], ViSecurityNegotiationException
+ FullyQualifiedErrorId : Client20_ConnectivityServiceImpl_Reconnect_CertificateError,VMware.VimAutomation.ViCore.Cmdlets.Commands.ConnectVIServer

We have two options here:

  1. Get valid certifcate for vCenter;
  2. Change PowerCLI configuration to disable SSL certificate verification;

Although option 2 is not good from security point of view, I still show you here so that I can go ahead for PowerNSX installation.

PS /root/.local/share/powershell/Modules> Set-PowerCLIConfiguration -InvalidCertificateAction ignore -confirm:$false

Scope ProxyPolicy DefaultVIServerMode InvalidCertificateAction DisplayDeprecationWarnings WebOperationTimeout
Seconds
—– ———– ——————- ———————— ————————– ——————-
Session UseSystemProxy Multiple Ignore True 300
User Ignore
AllUsers

Step 3: Install PowerNSX

  • Create a sub-directory called “PowerNSX” under “Modules” directory

[root@localhost powershell]# cd Modules/
[root@localhost Modules]# ls -al
total 4
drwxr-xr-x. 24 root root 4096 Jun 19 13:59 .
drwxr-xr-x. 5 root root 54 Jun 18 19:51 ..
drwxr-xr-x. 2 root root 48 Jun 19 14:01 PowerNSX

  • Download PowerNSX package from Github (https://github.com/vmware/powernsx) and upload the downloaded zip file to my RHEL server. Then unzip zip file and copy the  following 2 files into PowerNSX directory:

PowerNSX.psd1
PowerNSX.psm1

[root@localhost Modules]# ls -al PowerNSX/
total 1572
drwxr-xr-x. 2 root root 48 Jun 19 14:01 .
drwxr-xr-x. 24 root root 4096 Jun 19 13:59 ..
-rwxr-xr-x. 1 root root 15738 Jun 19 14:01 PowerNSX.psd1
-rwxr-xr-x. 1 root root 1588500 Jun 19 14:00 PowerNSX.psm1

Now you are ready to start using PowerNSX on RHEL. In my example, I query the current transport-zone and create a logical switch called PowerNSX within found NSX transport zone.

PS /root/.local/share/powershell/Modules/PowerNSX> Import-Module PowerNSX
PS /root/.local/share/powershell/Modules/PowerNSX> Get-Module

ModuleType Version Name ExportedCommands
———- ——- —- —————-
Manifest 3.1.0.0 Microsoft.PowerShell.Management {Add-Content, Clear-Content, Clear-Item, Clear-ItemProperty…}
Manifest 3.1.0.0 Microsoft.PowerShell.Utility {Add-Member, Add-Type, Clear-Variable, Compare-Object…}
Script 3.0.1110 PowerNSX {Add-NsxDynamicCriteria, Add-NsxDynamicMemberSet, Add-NsxEdgeInterfaceAddress, Add-NsxFirewallExclusionListMember…}
Script 1.2 PSReadLine {Get-PSReadlineKeyHandler, Get-PSReadlineOption, Remove-PSReadlineKeyHandler, Set-PSReadlineKeyHandler…}
Manifest 10.1.1…. VMware.PowerCLI
Script 6.7.0.8… VMware.Vim
Script 10.1.0…. VMware.VimAutomation.Cis.Core {Connect-CisServer, Disconnect-CisServer, Get-CisService}
Script 10.1.0…. VMware.VimAutomation.Common
Script 10.1.0…. VMware.VimAutomation.Core {Add-PassthroughDevice, Add-VirtualSwitchPhysicalNetworkAdapter, Add-VMHost, Add-VMHostNtpServer…}
Script 10.1.0…. VMware.VimAutomation.Nsxt {Connect-NsxtServer, Disconnect-NsxtServer, Get-NsxtService}
Script 10.1.0…. VMware.VimAutomation.Sdk {Get-InstallPath, Get-PSVersion}
Script 10.1.0…. VMware.VimAutomation.Storage {Add-KeyManagementServer, Copy-VDisk, Export-SpbmStoragePolicy, Get-KeyManagementServer…}
Script 1.2.0.0 VMware.VimAutomation.StorageUtility Update-VmfsDatastore
Script 10.1.0…. VMware.VimAutomation.Vds {Add-VDSwitchPhysicalNetworkAdapter, Add-VDSwitchVMHost, Export-VDPortGroup, Export-VDSwitch…}
Script 10.0.0…. VMware.VimAutomation.Vmc {Connect-Vmc, Disconnect-Vmc, Get-VmcService}

PS /root/.local/share/powershell/Modules/PowerNSX> Connect-NsxServer -vCenterServer 10.1.1.2

Windows PowerShell credential request
vCenter Server SSO Credentials
Password for user user1@davidwzhang.com: ***********
 
Using existing PowerCLI connection to 10.1.1.2
 
 
Version             : 6.4.0
BuildNumber         : 7564187
Credential          : System.Management.Automation.PSCredential
Server              : 10.1.1.4
Port                : 443
Protocol            : https
UriPrefix           :
ValidateCertificate : False
VIConnection        : 10.1.1.2
DebugLogging        : False
DebugLogfile        : \PowerNSXLog-user1@davidwzhang.com:@-2018_06_15_15_25_45.log
 

PS /root/.local/share/powershell/Modules/PowerNSX> Get NsxTransportZone                                                                 

objectId           : vdnscope-1
objectTypeName     : VdnScope
vsmUuid            : 42267595-0C79-1E95-35FE-E0A186F24C3B
nodeId             : 0598778a-9c46-46e7-a9c7-850beb6ac7f3
revision           : 14
type               : type
name               : transport-1
description        : transport-1
clientHandle       :
extendedAttributes :
isUniversal        : false
universalRevision  : 0
id                 : vdnscope-1
clusters           : clusters
virtualWireCount   : 59
controlPlaneMode   : UNICAST_MODE
cdoModeEnabled     : false
cdoModeState       : cdoModeState
 
PS /root/.local/share/powershell/Modules/PowerNSX> Get-NsxTransportZone  transport-1 | New-NsxLogicalSwitch -name PowerNSX
objectId              : virtualwire-65
objectTypeName        : VirtualWire
vsmUuid               : 42267595-0C79-1E95-35FE-E0A186F24C3B
nodeId                : 0598778a-9c46-46e7-a9c7-850beb6ac7f3
revision              : 2
type                  : type
name                  : PowerNSX
description           :
clientHandle          :
extendedAttributes    :
isUniversal           : false
universalRevision     : 0
tenantId              :
vdnScopeId            : vdnscope-1
vdsContextWithBacking : vdsContextWithBacking
vdnId                 : 6059
guestVlanAllowed      : false
controlPlaneMode      : UNICAST_MODE
ctrlLsUuid            : d6f2c975-8927-429c-86f7-3ae0b9ecd9fa
macLearningEnabled    : false

 

When we checked NSX manager, we can see PowerNSX logical switch is created with VXLAN-ID 6059.vxlan

 

Create XML file in vRealize Orchestrator for NSX Automation

On By insidepacketIn Network Automation, NSX, UncategorizedLeave a comment

NSX API uses XML format for API communication. To automate NSX in VMware vRealize Orchestror, it is always required to create a XML file with javascript  as vRO workflow supports javascript only.Here i only shows you an example to how to do it.

The target here is to create a security group and add a simple firewall rule in this newly created security group.

Note: this vRO workflow has 2 inputs:
securityGroupName, description
And 2 properties:
nsxManagerRestHost, realtime(equal to sgID in Step1)

Step1: create a security group

var xmlbody = new XML('<securitygroup />');
xmlbody.objectId = " ";
xmlbody.type.typeName = " ";
xmlbody.description = description;
xmlbody.name = securityGroupName;
xmlbody.revision = 0;
xmlbody.objectTypeName = " ";
System.log(xmlbody);
var request = nsxManagerRestHost.createRequest("POST", "/2.0/services/securitygroup/bulk/globalroot-0", xmlbody.toString());
request.contentType = "application/xml";
System.log("Creating a SecurityGroup " + securityGroupName);
System.log("POST Request URL: " + request.fullUrl);
var response = request.execute();
if (response.statusCode == 201) {
	System.debug("Successfully created Security Group " + securityGroupName);
	}
else {
	throw("Failed to SecurityGroup " + securityGroupName);
	}
sgID = response.getAllHeaders().get("Location").split('/').pop();
realtime=sgID

Step2: add a section in DFW and add a firewall rules

//create XML object for DFW source;
var rulesources = new XML('<sources excluded="false" />');
rulesources.source.name = " ";
rulesources.source.value = "10.47.161.23";
rulesources.source.type = "Ipv4Address";
rulesources.source.isValid = 'true';
System.log("Source: "+rulesources);

//create XML object for DFW destination;
var ruledestionations = new XML('<destinations excluded="false" />');
ruledestionations.destination.name = " ";
ruledestionations.destination.value = "10.47.161.24";
ruledestionations.destination.type = "Ipv4Address";
ruledestionations.destination.isValid = 'true';
System.log("Destination: "+ruledestionations);

//create XML object for DFW service
var ruleservices = new XML('<services />');
ruleservices.service.destinationPort = "80";
ruleservices.service.protocol = "6";
ruleservices.service.subProtocol = "6";
ruleservices.service.isValid = 'true';
System.log("Service: "+ruleservices);

//create XML object for the whole rule
var xmlbodyrule = new XML('<rule disabled="false" logged="true" />');
xmlbodyrule.name = "vro created rule";
xmlbodyrule.action = "allow";
xmlbodyrule.notes = " ";
xmlbodyrule.appliedToList.appliedTo.name = securityGroupName;
xmlbodyrule.appliedToList.appliedTo.value = realtime;
xmlbodyrule.appliedToList.appliedTo.type = 'SecurityGroup';
xmlbodyrule.appliedToList.appliedTo.isValid = 'true';
xmlbodyrule.sectionId = " ";
xmlbodyrule.sources = rulesources;
xmlbodyrule.destinations = ruledestionations;
xmlbodyrule.services = ruleservices;

//create XML object for DFW section
var xmlbody = new XML(
<section name ={securityGroupName} />);
//xmlbody.rule = 'disabled="false" logged="true" />';
xmlbody.rule=xmlbodyrule;
System.log("XML file for new rules: "+xmlbody);

var request = nsxManagerRestHost.createRequest("POST", "/4.0/firewall/globalroot-0/config/layer3sections", xmlbody.toString());
request.contentType = "application/xml";
var response = request.execute();
if (response.statusCode == 201) {
	System.debug("Successfully created Security Group Section" + securityGroupName);
	}
else {
	throw("Failed to SecurityGroup Section" + securityGroupName);
	}

Below is the output of XML file for creating a security group:

<securitygroup>
  <objectId></objectId>
  <type>
    <typeName></typeName>
  </type>
  <description>nsx1001test</description>
  <name>nsx1001test</name>
  <revision>0</revision>
  <objectTypeName></objectTypeName>
</securitygroup>

XML file for creating a NSX DFW section and adding a new simple firewall rules:

<section name="nsx1001test">
  <rule disabled="false" logged="true">
    <name>vro created rule</name>
    <action>allow</action>
    <notes></notes>
    <appliedToList>
      <appliedTo>
        <name>nsx1001test</name>
        <value>securitygroup-947</value>
        <type>SecurityGroup</type>
        <isValid>true</isValid>
      </appliedTo>
    </appliedToList>
    <sectionId></sectionId>
    <sources excluded="false">
      <source>
        <name></name>
        <value>10.47.161.23</value>
        <type>Ipv4Address</type>
        <isValid>true</isValid>
      </source>
    </sources>
    <destinations excluded="false">
      <destination>
        <name></name>
        <value>10.47.161.24</value>
        <type>Ipv4Address</type>
        <isValid>true</isValid>
      </destination>
    </destinations>
    <services>
      <service>
        <destinationPort>80</destinationPort>
        <protocol>6</protocol>
        <subProtocol>6</subProtocol>
        <isValid>true</isValid>
      </service>
    </services>
  </rule>
</section>

New Ansible F5 HTTPs Health Monitor Module

On By insidepacketIn Ansible, Network Automation, UncategorizedLeave a comment

Just got time this weekend to test the newly released dev version of Ansible F5 HTTPs health monitor. The result of testing looks good: most of common use cases have been covered properly.

Below is my first playbook for my testing:

# This version is to create a new https health monitor
---
- name: f5 config
  hosts:  lb.davidwzhang.com
  connection: local
  vars:
    ports:
      - 443
  tasks:
    - name: creat http healthmonitor
      bigip_monitor_https:
        state:  "present"
        #state: "absent"
        name: "ansible-httpshealthmonitor"
        password: "password"
        server: "10.1.1.122"
        user: "admin"
        validate_certs: "no"
        send: "Get /cgi-bin/env.sh HTTP/1.1\r\nHost:192.168.72.28\r\nConnection: Close\r\n"
        receive: "web"
        interval: "3"
        timeout: "10"
      delegate_to:  localhost

After run the playbook, I log in my F5 BIGIP VE and see the https health monitor has been created successfully.
f5 https healthmonitor

I tried to create another HTTPs health monitor, which includes basic authentication(admin/password) and customized alias address and alias service port(8443).
Playbook:

# This version is to create a new HTTP health monitor
---
- name: f5 config
  hosts:  lb.davidwzhang.com
  connection: local
  vars:
    ports:
      - 443
  tasks:
    - name: creat http healthmonitor
      bigip_monitor_https:
        state:  "present"
        #state: "absent"
        name: "ansible-httpshealthmonitor02"
        password: "password"
        server: "10.1.1.122"
        user: "admin"
        validate_certs: "no"
        ip: "192.168.72.128"
        port: "8443"
        send: "Get /cgi-bin/env.sh\r\n"
        receive: "200"
        interval: "3"
        timeout: "10"
        target_username: "admin"
        target_password: "password"
      delegate_to:  localhost

In F5, you can see the below:
f5 https healthmonitor02

In addition, you possibly noticed that I comment a line in the above 2 playbooks:

#state: "absent"

You can use it to remove the health monitor.

vRA7.3 and NSX Integration: Network Security Data Collection Failure

On By insidepacketIn Cloud, Network Automation, NSX, UncategorizedLeave a comment

We are building vRA 7.3 . We added vCenter and NSX manager as endpoint in vRA. And associate NSX manager with vCenter. All of computing resource data collection works well but not NSX (network and security):

So in vRA reservation, we only can see vSphere cluster, vDS port-group/logical switch but not Transport zone, security group/tags

When check the log, we see the following:

Workflow ‘vSphereVCNSInventory’ failed with the following exception:

One or more errors occurred.

Inner Exception: An error occurred while sending the request.

at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)

at DynamicOps.VCNSModel.Interface.NSXClient.GetDatacenters()

at DynamicOps.VCNSModel.Activities.CollectDatacenters.Execute(CodeActivityContext context)

at System.Activities.CodeActivity.InternalExecute(ActivityInstance instance, ActivityExecutor executor, BookmarkManager bookmarkManager)

at System.Activities.Runtime.ActivityExecutor.ExecuteActivityWorkItem.ExecuteBody(ActivityExecutor executor, BookmarkManager bookmarkManager, Location resultLocation)

Inner Exception:

VCNS Workflow failure

I tried to delete NSX end point and recreate from vRA but no luck. I raised the issue in vmware community but can’t get any real valuable feedback.

After a few hours investigation, finally I find a fix:

run the “create a NSX endpoint” workflow in vRO as the below

2017-07-26_184701

Then I re-start network & security data collection in vRA. Everything works and I can see all defined NSX Transport Zone, security groups and DLR in vRA network reservations.

Hope this fix can help others who have the same issue.

Simple Python Script Creating a Dynamic Membership Security Group

On By insidepacketIn Network Automation, NSXLeave a comment

In this blog, I developed a very simple Python scripts to create NSX security group whose membership is based on Security Tag. Please note this script is to show you the basic, which has not been ready for a production environment.

Two Python functions are includes in this script:

  1. create_tag is used to create a NSX security tag;
  2. create_sg is used to create a security group and define a criterion to add all virtual machines tagged with the specified security tag into this newly created security group;
import requests
from base64 import b64encode
import getpass
username=raw_input('Enter Your NSXManager Username: ')
yourpass = getpass.getpass('Enter Your NSXManager Password: ')
sg_name=raw_input('Enter Security Group Name: ')
vm_tag=raw_input('Enter Tag Name: ')
userandpass=username+":"+yourpass
userpass = b64encode(userandpass).decode("ascii")
auth ="Basic " + userpass
payload_tag="<securityTag>\r\n<objectTypeName>SecurityTag</objectTypeName>\r\n<type>\r\n<typeName>SecurityTag</typeName>\r\n</type>\r\n<name>"+vm_tag+"</name>\r\n<isUniversal>false</isUniversal>\r\n<description>This tage is created by API</description>\r\n<extendedAttributes></extendedAttributes>\r\n</securityTag>"
payload_sg= "<securitygroup>\r\n <objectId></objectId>\r\n <objectTypeName>SecurityGroup</objectTypeName>\r\n <type>\r\n <typeName>SecurityGroup</typeName>\r\n </type>\r\n <description></description>\r\n <name>"+sg_name+"</name>\r\n <revision>0</revision>\r\n<dynamicMemberDefinition>\r\n <dynamicSet>\r\n <operator>OR</operator>\r\n <dynamicCriteria>\r\n <operator>OR</operator>\r\n <key>VM.SECURITY_TAG</key>\r\n <criteria>contains</criteria>\r\n <value>"+vm_tag+"</value>\r\n </dynamicCriteria>\r\n </dynamicSet>\r\n</dynamicMemberDefinition>\r\n</securitygroup>"

def create_tag():
        try:
                response = requests.post(
                url="https://NSX-Manager-IP/api/2.0/services/securitytags/tag",
                verify=False,
                headers={
                        "Authorization": auth,
                        "Content-Type": "application/xml",
                    },
                data=payload_tag
                    )
                print('Response HTTP Status Code: {status_code}'.format(status_code=response.status_code))
                #print('Response HTTP Response Body: {content}'.format(content=response.content))
                if response.status_code == 403:
                        print "***********************************************************************"
                        print "WARNING: your username or password is wrong, please retry again!"
                        print "***********************************************************************"
                if  response.status_code == 201:
                        print "***********************************************************************"
                        print('Response HTTP Response Body: {content}'.format(content=response.content))
                api_response=response.text
                print api_response
        except requests.exceptions.RequestException:
                print('HTTP Request failed')

def create_sg():
        try:
                response = requests.post(
                url="https://NSX-Manager-IP/api/2.0/services/securitygroup/bulk/globalroot-0",
                verify=False,
                headers={
                        "Authorization": auth,
                        "Content-Type": "application/xml",
                    },
                data=payload_sg
                    )
                print('Response HTTP Status Code: {status_code}'.format(status_code=response.status_code))
                #print('Response HTTP Response Body: {content}'.format(content=response.content))
                if response.status_code == 403:
                        print "***********************************************************************"
                        print "WARNING: your username or password is wrong, please retry again!"
                        print "***********************************************************************"
                if  response.status_code == 201:
                        print "***********************************************************************"
                        print('Response HTTP Response Body: {content}'.format(content=response.content))
                api_response=response.text
                print api_response
        except requests.exceptions.RequestException:
                print('HTTP Request failed')

Run this script in our O-Dev:

[root]$ python create_sg_dynamic_member_20170429.py

Enter Your NSXManager UserName: admin

Enter Your NSXManager Passowrd:

Enter Security Group Name: sg_app1_web

Enter Tag Name: tag_app1_web

Response HTTP Status Code: 201

***********************************************************************

Response HTTP Response Body: securitytag-14

securitytag-14

Response HTTP Status Code: 201

***********************************************************************

Response HTTP Response Body: securitygroup-485

securitygroup-485

In NSX manager, we can see a securtiy group sg_app1_web is created as below:

2017-04-30_140657

And its dynamic membeship criterion is:

2017-04-30_140729

Automate F5 GSLB with Ansible

On By insidepacketIn Ansible, Network AutomationLeave a comment

F5 BIG-IP Global Traffic Manager (GTM) provides tiered global server load balancing (GSLB). BIG-IP GTM distributes DNS name resolution requests, first to the best available pool in a wide IP, and then to the best available virtual server within that pool. GTM selects the best available resource using either a static or a dynamic load balancing method. Using a static load balancing method, BIG-IP GTM selects a resource based on a pre-defined pattern. Using a dynamic load balancing method, BIG-IP GTM selects a resource based on current performance metrics collected by the big3d agents running in each data center.

So F5 GSLB configuration logic for a DNS record is as below:

  • Define a Data Center, e.g. “SL-SYD-Site1”;
  • Define a server which can be F5 LTM or any other kind of local load balancer or host;

GTM Server Type

  • Create virtual servers if you don’t use F5 BigIP LTM or you don’t “Virtual Server Discovery” feature for your F5 BigIP LTM
  • Create GTM pool/pools using virtual server as member of this newly created pool;
  • Create Wide-IP which points to the GTM pool/pools which you defined in the previous step ; Note: F5 module in Ansible 2.3 still doesn’t support the association of GTM pool with wide-ip.

Unlike F5 BigIP LTM, Ansible F5 module doesn’t support F5 BigIP GTM very well. The known limitation of automating F5 GSLB configuration with Ansible version 2.3 includes:

  1. Doesn’t support setting up a server; (Luckily, if you are using F5 BigIP LTM, this is one-off task: you only need to perform this task once for each LTM.)
  2. Doesn’t support adding pool member when you create a GTM pool;
  3. Doesn’t support adding pool when you create a wide ip;
  4. Doesn’t support health monitor when you create GTM virtual server and GTM pool;

To accommodate these above limitation, I pre-defined a F5 LTM server called “myLTM”. 

2017-04-25_164141

After running my Ansible playbook, I manually add pool member into newly created GTM pool and add the GTM pool to wideip as well.

GTMPool_Adding_Member

GTM_WideIP_Pool

My playbook YAML file:

– name: f5 config
hosts: lb.davidwzhang.com
connection: local
tasks:
– name: create a GTM DC SL-SYD-Site1
bigip_gtm_datacenter:
password: “password”
server: “10.1.1.122”
user: “admin”
name: “SL-SYD-Site1”
validate_certs: “no”
delegate_to: localhost
– name: create a virtual server myVIP
bigip_gtm_virtual_server:
password: “password”
server: “10.1.1.122”
user: “admin”
virtual_server_name: “myVIP”
virtual_server_server: “myLTM”
validate_certs: “no”
port: “80”
address: “192.168.72.199”
state: “present”
delegate_to: localhost

– name: create GTM pool: mypool
bigip_gtm_pool:
server: “10.1.1.122”
user: “admin”
password: “password”
name: “mypool”
state: “present”
type: “a”
validate_certs: “no”
delegate_to: localhost

– name: create a wideip w3.davidwzhang.com
bigip_gtm_wide_ip:
server: “10.1.1.122”
user: “admin”
password: “password”
lb_method: “round_robin”
name: “w3.davidwzhang.com”
type: “a”
state: “present”
validate_certs: “no”
delegate_to: localhost

Ansible Playbook Output:

GTM DC

GTM_DC

GTM Virtual Server

GTM_Server_VS

GTM Pool

GTM_Pool

GTM WideIP

GTM_WideIP

NSlookup for wideip: w3.davidwzhang.com

GTM_Nslookup