« Previous 1 2 3 4 Next »
Hybrid public/private cloud
Seamless
Cloud Firewall
In the next step, you create a security group that comprises host-related firewall rules for AWS. Because the firewall can protect itself, the group opens the firewall for all incoming and outgoing traffic, although this could be restricted. The next step is to create an S3 bucket that contains the starting configuration and the license for the firewall. Next, you generate the config
file for the firewall and upload it with the license. For a rented, but more expensive, firewall, this license information can also be omitted.
Now set up network interfaces for the firewall in the two subnets. Also, create a role that later allows the firewall instance to access the S3 bucket. Assign the network interfaces and the role to the firewall instance to be created, and link the subnets to the firewall. Create a routing table for the inside subnet and specify the firewall network card responsible for the inside network as the target; then, generate a public IP address and assign it to the outside network interface.
The next step is to set up a security group for the servers. To do this, first create two server instances on the inside subnet and change the inside firewall interface from a DHCP client to the static IP address that the AWS firewall has currently assigned to the card. Now set up a VPN tunnel from the local network into the AWS cloud. You need to define the rules and routes on the firewall on the local network. At the end of this configuration marathon, and assuming that all the new cloud servers can be reached, finally install and configure the Elastic stack on the two new AWS servers (Figure 3).
Cloud Shaping
In principle, Ansible would be able to perform all these tasks, but that would cause problems when cleaning up the ensemble in the cloud, at the latest. You would either have to save the information of the components created there locally, or you would have to search the Playbook for the components to be removed before the Playbook cleans them up.
A stack (similar to OpenStack) in which you describe the complete infrastructure, which can be parameterized in YAML or JSON format, is easier. Then, you build the stack with a call (also using Ansible) and clear it up with another call. The proprietary AWS technology for this is known as CloudFormation.
CloudFormation lets the stack receive a construction parameter: in this example, the IP addresses of the networks in the VPC. The author of the stack can also enter a return value, which is typically the external IP address of a generated object, so that the user of the stack knows how to use the cloud service.
Most VM images in AWS use cloud-init technology (please see the "cloud-init" box). Because CloudFormation can also provide cloud-init data to a VM, where do you draw the line between Ansible and CloudFormation? Where it is practicable and reduces the total overhead.
cloud-init
Cloud-init is a standard originally developed by Canonical and now used by all manufacturers to start instances in the cloud on the basis of operating system images. These instances set a self-defined locale, a host name, SSH keys, and temporary mount points, but also user data. Cloud-init collaborates with your configuration management solution of choice, whether Chef, Puppet, Ansible, or Salt.
Fixed and Variable
The fixed components of the target infrastructure are the VMs (the firewall and the two servers for Elastic), the network structure, the routing structure, and the logic of the security groups. All of this information should definitely be included in the CloudFormation template.
The network's IP addresses, the AWS region, and the names of the objects are variable and used as parameters in the stack definition; you have to specify them when calling the stack. The variables also include the name of the S3 bucket for the cloud-init configuration of the firewall and the public SSH key stored with AWS, which is used to enable access to the Linux VMs.
Finally, you need the internal IP addresses of the Linux VMs, the external public IP address of the firewall, and the internal private IP address of the firewall for further configuration. Accordingly, these addresses pertain to the return values of the stack.
Ansible does all the work. It fills the variables, generates the firewall configuration, which the AWS firewall receives via cloud-init, and installs the software on the Linux VMs. Cloud-init could also install the software, but Ansible will set up exactly the roles that helped to configure the local servers at the beginning.
I developed the CloudFormation template from the version by firewall manufacturer Fortinet [8]. I simplified the structure, compared with their version on GitHub, so that the template in the cloud only raises a firewall and not a cluster. Additionally, the authors of the Fortinet template used a Lambda function to modify the firewall configuration. Here, this task is done by the Playbook, which in turn uses the template.
In the CloudFormation template, the process can be static. The two Linux VMs use CentOS as their operating system and should run on the internal subnet; you simply attach them to the template and the return values. Listings 2 through 4 show excerpts from the stack definition in YAML format. The complete YAML file can be downloaded from the ADMIN anonymous FTP site [7].
Listing 2
YAML Stack Definition Part 1
01 [...] 02 Resources: 03 FortiVPC: 04 Type: AWS::EC2::VPC 05 Properties: 06 CidrBlock: 07 Ref: VPCNet 08 Tags: 09 - Key: Name 10 Value: 11 Ref: VPCName 12 13 FortiVPCFrontNet: 14 Type: AWS::EC2::Subnet 15 Properties: 16 CidrBlock: 17 Ref: VPCSubnetFront 18 MapPublicIpOnLaunch: true 19 VpcId: 20 Ref: FortiVPC 21 22 FortiVPCBackNet: 23 Type: AWS::EC2::Subnet 24 Properties: 25 CidrBlock: 26 Ref: VPCSubnetBack 27 MapPublicIpOnLaunch: false 28 AvailabilityZone: !GetAtt FortiVPCFrontNet.AvailabilityZone 29 VpcId: 30 Ref: FortiVPC 31 32 FortiSecGroup: 33 Type: AWS::EC2::SecurityGroup 34 Properties: 35 GroupDescription: Group for FG 36 GroupName: fg 37 SecurityGroupEgress: 38 - IpProtocol: -1 39 CidrIp: 0.0.0.0/0 40 SecurityGroupIngress: 41 - IpProtocol: tcp 42 FromPort: 0 43 ToPort: 65535 44 CidrIp: 0.0.0.0/0 45 - IpProtocol: udp 46 FromPort: 0 47 ToPort: 65535 48 CidrIp: 0.0.0.0/0 49 VpcId: 50 Ref: FortiVPC 51 52 InstanceProfile: 53 Properties: 54 Path: / 55 Roles: 56 - Ref: InstanceRole 57 Type: AWS::IAM::InstanceProfile 58 InstanceRole: 59 Properties: 60 AssumeRolePolicyDocument: 61 Statement: 62 - Action: 63 - sts:AssumeRole 64 Effect: Allow 65 Principal: 66 Service: 67 - ec2.amazonaws.com 68 Version: 2012-10-17 69 Path: / 70 Policies: 71 - PolicyDocument: 72 Statement: 73 - Action: 74 - ec2:Describe* 75 - ec2:AssociateAddress 76 - ec2:AssignPrivateIpAddresses 77 - ec2:UnassignPrivateIpAddresses 78 - ec2:ReplaceRoute 79 - s3:GetObject 80 Effect: Allow 81 Resource: '*' 82 Version: 2012-10-17 83 PolicyName: ApplicationPolicy 84 Type: AWS::IAM::Role
The objects of the AWS::EC2::Instance
type are the VMs designed to extend the Elastic stack (Listings 3 and 4). Because of the firewall, the VM is more complex to configure; it has to have two dedicated interface objects so that routing can point to it (Listing 3, line 11).
Listing 3
YAML Stack Definition Part 2
01 FortiInstance: 02 Type: "AWS::EC2::Instance" 03 Properties: 04 IamInstanceProfile: 05 Ref: InstanceProfile 06 ImageId: "ami-06f4dce9c3ae2c504" # for eu-west-3 paris 07 InstanceType: t2.small 08 AvailabilityZone: !GetAtt FortiVPCFrontNet.AvailabilityZone 09 KeyName: 10 Ref: KeyName 11 NetworkInterfaces: 12 - DeviceIndex: 0 13 NetworkInterfaceId: 14 Ref: fgteni1 15 - DeviceIndex: 1 16 NetworkInterfaceId: 17 Ref: fgteni2 18 UserData: 19 Fn::Base64: 20 Fn::Join: 21 - '' 22 - 23 - "{\n" 24 - '"bucket"' 25 - ' : "' 26 - Ref: S3Bucketname 27 - '"' 28 - ",\n" 29 - '"region"' 30 31 - ' : ' 32 - '"' 33 - Ref: S3Region 34 - '"' 35 - ",\n" 36 - '"license"' 37 - ' : ' 38 - '"' 39 - / 40 - Ref: LicenseFileName 41 - '"' 42 - ",\n" 43 - '"config"' 44 - ' : ' 45 - '"' 46 - /fg.txt 47 - '"' 48 - "\n" 49 - '}' 50 51 InternetGateway: 52 Type: AWS::EC2::InternetGateway 53 54 AttachGateway: 55 Properties: 56 InternetGatewayId: 57 Ref: InternetGateway 58 VpcId: 59 Ref: FortiVPC 60 Type: AWS::EC2::VPCGatewayAttachment 61 62 RouteTablePub: 63 Type: AWS::EC2::RouteTable 64 Properties: 65 VpcId: 66 Ref: FortiVPC 67 68 DefRoutePub: 69 DependsOn: AttachGateway 70 Properties: 71 DestinationCidrBlock: 0.0.0.0/0 72 GatewayId: 73 Ref: InternetGateway 74 RouteTableId: 75 Ref: RouteTablePub 76 Type: AWS::EC2::Route 77 78 RouteTablePriv: 79 [...] 80 81 DefRoutePriv: 82 [...] 83 84 SubnetRouteTableAssociationPub: 85 Properties: 86 RouteTableId: 87 Ref: RouteTablePub 88 SubnetId: 89 Ref: FortiVPCFrontNet 90 Type: AWS::EC2::SubnetRouteTableAssociation 91 92 SubnetRouteTableAssociationPriv: 93 [...]
Importantly, the firewall instance and both generated interfaces are located in the same availability zone; otherwise, the stack will fail. To this end, the VMs contain descriptions, and the second subnet contains the reference to the availability zone of the first subnet.
The UserData
part of the firewall instance (Listing 3, line 18) contains a description file that tells the VM where to find the configuration and license file previously uploaded by Ansible.
The network configuration has already been described and is defined at the top of Listing 2. The finished template can now be run at the command line with the
aws cloudformation create-stack
Listing 4
YAML Stack Definition Part 3
01 [...] 02 ServerInstance: 03 Type: "AWS::EC2::Instance" 04 Properties: 05 ImageId: "ami-0e1ab783dc9489f34" # Centos7 for paris 06 InstanceType: t3.2xlarge 07 AvailabilityZone: !GetAtt FortiVPCFrontNet.AvailabilityZone 08 KeyName: 09 Ref: KeyName 10 SubnetId: 11 Ref: FortiVPCBackNet 12 SecurityGroupIds: 13 - !Ref ServerSecGroup 14 15 Server2Instance: 16 Type: "AWS::EC2::Instance" 17 Properties: 18 ImageId: "ami-0e1ab783dc9489f34" # Centos7 for paris 19 [...]
command, which specifies the name of the YAML file created and fills the parameters at the beginning of the stack. The S3 bucket you want to pass in must already exist. Both the license and the generated configuration should be uploaded up front. All these tasks are done by the Ansible Playbook, as shown in Listings 5 through 9.
The Playbook uses multiple "plays." The first (Listing 5) creates the configuration for the firewall and, if not available, the S3 bucket (line 20) as described and uploads it together with the license.
Listing 5
Ansible Playbook Part 1
01 --- 02 - name: Create VDC in AWS with fortigate as front 03 hosts: localhost 04 connection: local 05 gather_facts: no 06 vars: 07 region: eu-west-3 08 licensefile: license.lic 09 wholenet: 10.100.0.0/16 10 frontnet: 10.100.254.0/28 11 netmaskback: 17 12 backnet: "10.100.0.0/{{ netmaskback }}" 13 lnet: 10.0.2.0/24 14 rnet: "{{ backnet }}" 15 s3name: stackdata 16 keyname: mgtkey 17 fgtpw: Firewall-Passwort 18 19 tasks: 20 - name: Create S3 Bucket for data 21 aws_s3: 22 bucket: "{{ s3name }}" 23 region: "{{ region }}" 24 mode: create 25 permission: public-read 26 register: s3bucket 27 28 - name: Upload License 29 aws_s3: 30 bucket: "{{ s3name }}" 31 region: "{{ region }}" 32 overwrite: different 33 object: "/{{ licensefile }}" 34 src: "{{ licensefile }}" 35 mode: put 36 37 - name: Generate Config 38 template: 39 src: awsforti-template.conf.j2 40 dest: fg.txt 41 42 - name: Upload Config 43 aws_s3: 44 bucket: "{{ s3name }}" 45 region: "{{ region }}" 46 overwrite: different 47 object: "/fg.txt" 48 src: "fg.txt" 49 mode: put 50 [...]
The next task creates the complete stack (Listing 6). What's new is the connection to the old Elasticsearch Playbook or Hosts file. The latter has a group named elahosts
, which adds the IP addresses of the two new servers to the Playbook so that a total of five hosts are in the list for further execution of the Playbook. However, some operations will only take place on the new hosts. Listing 6 (lines 44 and 49) creates the newhosts
group, to which it adds the two hosts.
Listing 6
Ansible Playbook Part 2
01 [...] 02 - name: Create Stack 03 cloudformation: 04 stack_name: VPCFG 05 state: present 06 region: "{{ region }}" 07 template: fortistack.yml 08 template_parameters: 09 InstanceType: c5.large 10 FGUserName: admin 11 KeyName: "{{ keyname }}" 12 VPCName: VDCVPC 13 VPCNet: "{{ wholenet }}" 14 Kubnet: "{{ lnet }}" 15 VPCSubnetFront: "{{ frontnet }}" 16 VPCSubnetBack: "{{ backnet }}" 17 S3Bucketname: "{{ s3name }}" 18 LicenseFileName: "{{ licensefile }}" 19 S3Region: "{{ region }}" 20 register: stackinfo 21 22 - name: Print Results 23 [...] 24 25 - name: Wait for VM to be up 26 [...] 27 28 - name: New Group 29 add_host: 30 groupname: fg 31 hostname: "{{ stackinfo.stack_outputs.FortiGatepubIp }}" 32 33 - name: Add ElaGroup1 34 add_host: 35 groupname: elahosts 36 hostname: "{{ stackinfo.stack_outputs.Server1Address }}" 37 38 - name: Add ElaGroup2 39 add_host: 40 groupname: elahosts 41 hostname: "{{ stackinfo.stack_outputs.Server2Address }}" 42 43 - name: Add NewGroup1 44 add_host: 45 groupname: newhosts 46 hostname: "{{ stackinfo.stack_outputs.Server1Address }}" 47 48 - name: Add NewGroup2 49 add_host: 50 groupname: newhosts 51 hostname: "{{ stackinfo.stack_outputs.Server2Address }}" 52 53 - name: Set Fact 54 set_fact: 55 netmaskback: "{{ netmaskback }}" 56 57 - name: Set Fact 58 set_fact: 59 fgtpw: "{{ fgtpw }}" 60 [...]
The next play (Listing 7) configures the firewall. In its existing configuration, the static IP address for the inside network card is missing – AWS only sets this when creating the instance. Because the data is now known, the Playbook can define the IP address.
Listing 7
Ansible Playbook Part 3
01 [...] 02 - name: ChangePW 03 hosts: fg 04 vars: 05 ansible_user: admin 06 ansible_ssh_common_args: -o StrictHostKeyChecking=no 07 ansible_ssh_pass: "{{ hostvars['localhost'].stackinfo.stack_outputs.FortiGateId }}" 08 gather_facts: no 09 10 tasks: 11 - raw: | 12 "{{ hostvars['localhost'].fgtpw }}" 13 "{{ hostvars['localhost'].fgtpw }}" 14 config system interface 15 edit port2 16 set mode static 17 set ip "{{ hostvars['localhost'].stackinfo.stack_outputs.FGIntAddress }}/17" 18 next 19 end 20 tags: pw 21 - name: Wait for License Reboot 22 pause: 23 minutes: 1 24 25 - name: Wait for VM to be up 26 wait_for: 27 host: "{{ inventory_hostname }}" 28 port: 22 29 state: started 30 delegate_to: localhost 31 [...]
When logging in to the firewall for the first time, the firewall requires a password change. You can use several methods to set up Fortigate in Ansible. However, the FortiOS network modules that have been included in the Ansible distribution for a while do not yet work properly. The raw
approach is used here (Listing 7, line 10), which pushes the commands onto the device, as on the command line.
The first two lines of the raw
task set the password, which resides on the instance ID in the AWS version. Because the license has already been installed, the firewall reboots after installation. At the end, the Ansible script in Listing 7 waits for the reboot to occur and then for it to reach the firewall again.
A play now follows that teaches the local firewall what the VPN tunnel to the firewall looks like in AWS (Listing 8). The VPN definition at the other end was in the previously uploaded configuration. Because of the described problems with the Ansible modules for FortiOS (I suspect incompatibilities between Ansible modules and the Python fosapi
), the play uses Ansible's URI method to configure the firewall. Authentication for the API requires a login process; it then returns a token that is used in the following REST calls.
Listing 8
Ansible Playbook Part 4
01 [...] 02 - name: Local Firewall Config 03 hosts: localhost 04 connection: local 05 gather_facts: no 06 vars: 07 localfw: 10.0.2.90 08 localadmin: admin 09 localpw: "" 10 vdom: root 11 lnet: 10.0.2.0/24 12 rnet: 10.100.0.0/17 13 remotefw: "{{ stackinfo.stack_outputs.FortiGatepubIp }}" 14 localinterface: port1 15 psk: "<Password>" 16 vpnname: elavpn 17 18 tasks: 19 20 - name: Get the token with uri 21 uri: 22 url: https://{{ localfw }}/logincheck 23 method: POST 24 validate_certs: no 25 body: "ajax=1&username={{ localadmin }}&password={{ localpw }}" 26 register: uriresult 27 tags: gettoken 28 29 - name: Get Token out 30 set_fact: 31 token: "{{ 32 uriresult.cookies['ccsrftoken'] | regex_replace('\"', '') }}" 33 34 - debug: msg="{{ token }}" 35 36 - name: Phase1 old Style 37 uri: 38 url: https://{{ localfw }}/api/v2/cmdb/vpn.ipsec/phase1-interface 39 validate_certs: no 40 method: POST 41 headers: 42 X-CSRFTOKEN: "{{ token }}" 43 Cookie: "{{ uriresult.set_cookie }}" 44 body: "{{ lookup('template', 'forti-phase1.j2') }}" 45 body_format: json 46 register: answer 47 tags: phase1 48 49 - name: Phase2 old style 50 uri: 51 url: https://{{ localfw }}/api/v2/cmdb/vpn.ipsec/phase2-interface 52 validate_certs: no 53 method: POST 54 headers: 55 X-CSRFTOKEN: "{{ token }}" 56 Cookie: "{{ uriresult.set_cookie }}" 57 body: "{{ lookup('template', 'forti-phase2.j2') }}" 58 body_format: json 59 register: answer 60 tags: phase2 61 62 - name: Route old style 63 [...] 64 65 - name: Local Object Old Style 66 [...] 67 68 - name: Remote Object Old Stlye 69 [...] 70 71 - name: FW-Rule-In old style 72 uri: 73 url: https://{{ localfw }}/api/v2/cmdb/firewall/policy 74 validate_certs: no 75 method: POST 76 headers: 77 Cookie: "{{ uriresult.set_cookie }}" 78 X-CSRFTOKEN: "{{ token }}" 79 body: 80 [...] 81 body_format: json 82 register: answer 83 tags: rulein 84 85 - name: FW-Rule-out old style 86 uri: 87 url: https://{{ localfw }}/api/v2/cmdb/firewall/policy 88 validate_certs: no 89 method: POST 90 headers: 91 Cookie: "{{ uriresult.set_cookie }}" 92 X-CSRFTOKEN: "{{ token }}" 93 body: 94 [...] 95 body_format: json 96 register: answer 97 tags: ruleout 98 [...]
The configuration initially consists of the key exchange phase1
and phase2
parameters. The phase1
parameter contains the password, crypto parameters, and IP address of the firewall in AWS. The phase2
parameter also provides crypto parameters and data for the local and remote networks. The configuration also provides a route (line 62) that passes the network on the AWS side to the VPN tunnel, and two firewall rules that allow traffic from and to the private network on the AWS side (lines 71 and 85).
A bit further down (Listing 9), the Playbook sets the do_ela
parameter to 1
for the new hosts so that this role will also install Elasticsearch later. It uses
as the value for masternode,
because the new hosts are data nodes. Because it usually takes some time for the VPN connection to be ready for use, the play now waits for the master node of the Elastic cluster until it can reach a new node via SSH.
Listing 9
Ansible Playbook Part 5
01 [...] 02 - name: Set Facts for new hosts 03 hosts: newhosts 04 [...] 05 masternode: 0 06 do_ela: 1 07 08 - name: Wait For VPN Tunnel 09 hosts: 10.0.2.25 10 [...] 11 12 - name: Install elastic 13 hosts: elahosts 14 vars: 15 elaversion: 6 16 eladatapath: /elkdata 17 ansible_ssh_common_args: -o StrictHostKeyChecking=no 18 19 tasks: 20 21 - ini_file: 22 path: /etc/yum.conf 23 section: main 24 option: ip_resolve 25 value: 4 26 become: yes 27 become_method: sudo 28 when: do_ela == 1 29 name: Change yum.conf 30 31 - yum: 32 name: "*" 33 state: "latest" 34 name: RHUpdates 35 become: yes 36 become_method: sudo 37 when: do_ela == 1 38 39 - include_role: 40 name: matrix.centos-elasticcluster 41 vars: 42 clustername: matrixlog 43 elaversion: 6 44 when: do_ela == 1 45 46 - name: Set Permissions for data 47 file: 48 path: "{{ eladatapath }}" 49 owner: elasticsearch 50 group: elasticsearch 51 state: directory 52 mode: "4750" 53 become: yes 54 become_method: sudo 55 when: do_ela == 1 56 57 - systemd: 58 name: elasticsearch 59 state: restarted 60 become: yes 61 become_method: sudo 62 when: do_ela == 1
The last piece of the Playbook finally installs Elasticsearch on the new node and adapts its configuration to match the existing cluster. The role takes the major version of Elasticsearch as a parameter and a path in which the Elasticsearch server can store the data, which allows you to insert a separate mount point on a data-only disk.
Within AWS, all systems are prepared for IPv6, but this does not apply to the configuration used here. Therefore, the first task forces you to switch to IPv4. The second one updates the configuration of the system. In the third task, the Elastic cluster role finally installs and configures the software.
Because Ansible only creates the Elasticsearch user to which the elkdata/
folder should belong during the installation, the script also has to tweak the permissions and restart Elasticsearch (starting in line 46). This completes the cloud expansion. If everything worked out, the Kibana console will be presented with the view from Figure 4 after a few moments.
« Previous 1 2 3 4 Next »
Buy this article as PDF
(incl. VAT)