Code Structure Walk-Through

DevOps Orchestration Platform has been designed with extendability, scalability, and flexibility in mind. With basic knowledge of JavaScript, Golang, HTML5, Ansible, Terraform, and Bash, you can extend the framework to virtually any infrastructure-as-code or Ansible use case. This section is a short walk-through of the code structure, which will be broken down directory by directory so you can understand how the project is modularized in the context of the architecture diagram given earlier.

High-Level Project Structure

The DevOps Orchestration Platform structure is shown in Listing 1.

• ansible contains everything related to Ansible playbooks and is used for temporary storage of the VM instance SSH keys.
• bootstrap in the name means the directory is dynamically generated by the tool and used for temporary storage of keys and other VM instance details.
• firewall in the name means the directory is dynamically generated by the tool and used for controlling the security groups.
• docker is a temporary storage area for scripts and commands related to bootstrapping Docker containers. This subdirectory additionally contains Docker logs in which the name contains the name of each Docker container appended.
• Logs contains the generated VM and Docker logs with the naming convention <VM instance Name>-<O/S Name>. The Logs subdirectory also contains Docker logs named for each Docker container appended.
• software contains GoTTY, an open source utility used by DevOps Orchestration Platform for browser-based SSH access.
• webserver contains all the web server Golang code, JavaScript, HTML, and templates and is described in the next section.

+-- DevOps-Orchestration-Platform
  +-- ansible
  +-- aws_bootstrap
  +-- aws_firewall
  +-- docker
  +-- gcp_bootstrap
  +-- gcp_firewall
  +-- Logs
  +-- OTHER_LICENSES
  +-- software
  +-- vm_bootstrap
  +-- webserver
    +-- ansible_handler
    +-- aws_handler
    +-- build_and_run.sh
    +-- connectivity_check
    +-- css
    +-- databases
    +-- gcp_handler
    +-- general_utility_web_handler
    +-- index.html
    +-- js
    +-- kubernetes_handler
    +-- kv_store
    +-- linux_command_line
    +-- logging
    +-- nav_frame.html
    +-- operations_frame.html
    +-- security
    +-- template_populator
    +-- templates
      +-- ansible
      +-- cloud_vm_access_and_import
      +-- docker
      +-- html
      +-- security
      +-- terraform
      +-- vagrant
    +-- title_bar.html
    +-- top_level
    +-- top_level.go
    +-- virtualbox_handler
    +-- vm_launcher

Web Server Components

Much of the webserver directory tree is self-explanatory, so only the more complex components are described.

• build_and_run.sh is a helper script that builds the Golang code and launches the Golang web server on port 6543.
• js stores most of the JavaScript. DevOps Orchestration Platform uses HTML5 iFrames, along with a lot of raw JavaScript to process frame elements. The platform makes heavy use of jQuery Ajax calls containing large JSON segments for back-end to front-end synchronization. It also uses the native Golang web server templating system for a significant portion of the back-end to front-end synchronization.
• databases contains a snapshot of the entire state of every component of the system at any one time. The database system currently employed is a simplified NoSQL scheme containing JSON strings stored on the back-end filesystem. At run time, the JSON strings are imported by the Golang code and converted into hash maps (equivalent to dict constructors in Python). Front-end actions triggered by the user result in modified hash maps being re-written to the filesystem as new JSON strings. This scheme could in future be migrated to an alternative database, such as Apache Cassandra, MongoDB, or another technology (see the later section "Extending the Framework").
• handler in the name means the directory handles web server routes for user actions intended to trigger deployment of Ansible playbooks, Terraform-based infrastructure, Vagrant, and other events triggered from the front-end web application.
• kv_store contains Golang utility functions facilitating an abstraction of the JSON-based databases and their import and export.
• connectivity_check is used for SSH connectivity utilities.
• templates, not to be confused with Ansible templates, contains not only HTML templates, but files with the .j3 extension. These are abstract template files used as templates for Terraform infrastructure bootstrap scripts, among other tasks. The .j3 files are processed within Golang template modules (e.g., to populate Terraform templates, generic Bash scripts, and temporary scripts that open and close ports on the various firewall platforms).
• templates/cloud_vm_access_and_import has the templates used to populate Ansible EC2.py and GCE.py scripts for importing existing cloud resources. The Ansible inventories are generated dynamically from within Golang code and are not themselves templated. The Ansible inventory generator code is stored at the path webserver/ansible_handler/template_generator.go. The Ansible .j2 templates themselves, stored in their respective playbook directory tree paths as usual, use variables inside the dynamically generated Ansible inventories.