ClockworkNet

Security Hardening & AD Integration

ClockworkNet was engaged by a global marine data company to deliver two projects, both of which were to embed features into a new, central data platform.

The data platform project was already live, but undergoing additional heavy development/expansion by the agile based project development team. It was therefore necessary to ensure that the work did not disturb the live services, and also adapted to the ever changing, underlying landscape. All infrastructure build and management was implemented in code and automated, therefore the delivered packages of work needed to be integrated into the existing code base.

CIS Security Hardening

The clients requirement, was for the existing infrastructure to be modified to meet the CIS Security Benchmark for Centos 7 (v2). This needed to be applied both to the systems hosting the live/test/dev services and all supporting infrastructure hosts, including:

• CI/CD tooling
• Code repository
• Host build tools
• Artifact and container repositories
• Kerberos and LDAP authentication platforms

The work was supplied as a series of additions & modifications to the clients existing Ansible code base, and completed with a scheduled reporting function that allowed an ongoing, periodic check against the platform to be carried out, ensuring continued compliance with the benchmark.

AD Integration

For the second project, the client required that all authentication and authorisation for the hosts and services covered by the security hardening project, be consolidated to use the existing company Active Directory service. Up until this point, tools and services had either been using a project specific OpenLDAP/Kerberos platform, or were simply using the tool's internal authentication and authorisation solution. Again, it was imperative that work carried out was managed, to ensure that existing, busy users were not impacted by the migration process. The project involved a number of strands:

• The initial step was to work with all relevant stakeholders, to design a suitable Active Directory object structure. This needed to allow for easy management of roles and role membership, whilst providing granular permissions throughout all in-scope hosts and services. It also needed to meet the requirements of the security & infrastructure teams, who were midway through a move from a simple single domain AD structure, to a multi-domain model.
• Each of the development and infrastructure tools then needed to be modified, and users migrated from their existing accounts to using their AD accounts.
• Finally, the data platform application would be modified and end users again migrated from their legacy accounts, to their AD accounts.

As with the security hardening project, all work was delivered and deployed primarily in the form of Ansible code, working closely with the development & infrastructure teams, to roll out in a controlled, staged fashion.

Linux Service/Infrastructure Migration & Adoption

This client, a supplier of communication services, had built their products exclusively on a Microsoft/Windows platform. Over time, as the portfolio of clients, products and geographic regions had grown, the infrastructure had expanded from a compact, bare metal platform, to a wide array of bare metal, virtualised and cloud based platforms. The client was now looking to use Linux and open source components in key areas to improve reliability & workflow, as well as add new capabilities, which lead to ClockworkNet supplying several projects. The client was also at the point of moving from hand crafting their platform configurations, to automated configuration management, and had selected Saltstack as their configuration management tool. Therefore, all supplied work was delivered as additions to their existing Saltstack codebase.

Load Balancer Migration

The clients services, made heavy use of load balancing within their multi tier applications. This had been implemented using Microsoft components but for reasons of performance, stability and manageability, the client wished to move to an open source based solution:

• HAProxy was selected after gathering requirements from all stakeholders
• A resilient configuration was developed, combining HAProxy with Keepalived, to build fault tolerant pairs
• Programatic tools were developed to allow the clients deployment tools, to control all relevant aspects of the load balancers, to faclitate the blue/green deployment model they employed
• As the public facing load balancers were responsible for HTTPS termination, tools, training and documentation were supplied, to ease the management of TLS certificates
• A review of their existing configuration and userbase was carried out, allowing simplification through the use of SNI and a tightening of TLS configuration, inline with current best practice
• Migration of all dev, test & live services to the new solution

Once complete, the move met its objectives of increasing stability together with allowing for a reduction in costs, both as a result of a reduction in required resources, plus improvements in workflow and flexibility.

Centralised Logging System

A new capability required by the client, was a centralised logging system to serve as reliable storage, but also to provide a rich interface facilitating search, alerting and monitoring. The provisioning of this tool, coincided with the clients move to a new redundant infrastructure based in multiple datacentres. The supplied solution needed to integrate with and take full advantage of this infrastructure.

• A combination of Elasticsearch, Logstash and the Beats suite of data shippers were selected. This chain of components allowed for sophisticated filtering/routing/transformation of logs, together with an intelligent caching system that minimises the loss of logs, in the event of a component failure at any layer.
• Instead of the usual Kibana being utilized for the end user interface, Graylog was supplied instead. Reasons for this choice included a shallower learning curve for both end users and operations staff, more cost effective access controls and all the necessary alerting, visualisation and security tools being supplied in the core product.
• Resilience was provided at the Logstash layer, by deploying pairs of logstash hosts employing Keepalived to manage a shared IP
• Resilience was priovided at the datacentre level, by deploying independent Elasticsearch clusters at each site, and having the logstash layers route traffic to both
• As with other packages of work for this client, this package was delivered as a series of Saltstack formulas

Cloud/Configuration Management

The client had a disparate collection of infrastructure platforms which had lead to a complicated landscape of configuration. The client was looking to move as much of that management as possible into automated processes. ClockworkNet worked to introduce some of the next generation tools to simplify this challenge. This was based around demonstrating the power of both Terraform and Packer to simplify the management and deployment of common host roles, into the various infrastructure platforms.

Deployment Process Design & Build

An international financial assistance and insurance company were embarking on a move to an Agile/DevOps workflow for their new suite of products. As such they had a requirement to build out a series of tools and platforms, to support this new way of working. Clockworknet supplied 3 projects in support of this move:

Automated Environment Provisioning

Having previously had very limited automated configuration management as part of their build processes, the client wanted to move to entirely automated environment provisioning with their change in direction, allowing for consistent builds, together with the ability to tear down and replace environments during development and testing. They had already selected Ansible as their configuration management tool of choice. ClockworkNet provided the necessary Ansible codebase to:

• Build out the application environment (Apache/JBoss/Swing)
• Build out the DB platform (Postgresql)
• Add necessary monitoring to the clients Nagios monitoring server

Automated Code Deployment Process

As well as the automation of environment provisioning, the client also wanted an easy to use method of deploying application code releases. The client were keen not to be tied to a specific application technology, and therefore needed a flexible solution that would grow with the technology requirements of their product development.

ClockworkNet designed and supplied a flexible Ansible role and inventory layout, combined with devising a deployment package standard, that met the clients needs. This comprised a modular layout, with a standardised interface, that made it simple to add both new host and application technologies into the deployment process.

Micro Service Configuration Management

The client was standardising on a micro service architecture for its new products. With many services and environments to manage, and responsibility split between developers and operations staff, the manual configuration of these services had become unwieldy and error prone.

ClockworkNet designed and developed a centralised tool, providing both a web and REST API interface, implemented with Python & Flask. This tool:

• Allowed a lot of repetitive, pattern based configuration to be automated
• Separated concerns/responsibilities so developers and operations staff had access to only the relevant configuration they required
• Facilitated the option to integrate with the code repository, allowing configuration to be collected from project files directly within the code base of each micro service