DevOps Toolsets Part 1 - CHEF

This post looks at how Chef has evolved over the last few years by adding new weaponry to their arsenal in the battle to keep up with the competition.

Continuous delivery is becoming an increasingly important part of the DevOps culture as organisations strive to improve the release cycles of their IT systems - delivering systems more frequently, with more stability and more cost effectively. This article is the first in a series of posts that look at the leading vendors of Configuration Management software, and what they have done to meet customer needs.

Chef

The latest offering from Chef was launched in July this year, and bundles Chef’s product set into a single offering called Chef Automate. It integrates a shared workflow pipeline for continuous delivery (formerly Chef Delivery and Chef Compliance) which allows customers to check that infrastructure conforms to security and regulatory requirements. In addition, Chef Automate includes an enhanced dashboard that provides access to analytics on all resources managed by Chef.


Workflow in Chef Automate

The changes to infrastructure and application code are managed using Chef Automate's workflow platform. This provides a reproducible workflow that manages change as it flows through the pipeline from the Chef workstation, through automated tests, to production roll-out.

The workflow is based on a pipeline which defines a series of automated and manual quality gates that are used validate the changes as they progress through the pipeline. The pipeline is made up of 6 stages:

  • Verify
  • Build
  • Acceptance
  • Union
  • Rehearsal
  • Delivered

Each stage consists of phases that perform a particular task - such as running some type of test. A stage has the following predefined phases and can be customised to perform whatever tests are required.

Chef Automate uses git and feature branches for handling changes before they are merged. Each pipeline has a designated target branch into which all approved changes will ultimately be merged into. It also uses a “gated master” model where changes need to be reviewed and accepted before merging a feature branch into the main branch.

The Verify and Build stages are run exclusively on the build nodes - where the necessary runtime environments are created and destroyed during the execution of the stage, typically using the Test Kitchen framework.

The Verify stage runs automatically when changes are committed to a feature branch that haven’t been approved yet, and the Lint, Syntax and Unit test phases are executed. When a change is approved the Build stage is run and the Lint, Syntax and Unit tests are repeated and additionally the Security, Quality and Publish phases run. 

The Acceptance, Union, Rehearsal and Delivered stages use additional infrastructure dedicated to that stage to perform their tests. The Acceptance stage verifies the artefacts produced in the Build stage, and is when the decision is made to roll-out the changes to production. The following four phases are executed:

  • provision infrastructure if required
  • deploy the artefacts published in the Build stage
  • execute smoke tests to verify code has been deployed
  • execute functional tests

The Union stage assesses the impact of changes made to related projects that make up the system as a whole, testing for interactions between these interdependent projects.

If all the phases of the Union stage succeed, then the Rehearsal stage is triggered automatically. The Rehearsal stage is used as a practice run to deploy the changes to infrastructure similar to that of the target production infrastructure, increasing confidence for the deployment into production.


Chef Compliance

Chef Compliance allows you to test that your infrastructure conforms to security and regulatory requirements, monitoring on an ongoing basis. Chef Compliance is based on the open source InSpec Automated Testing framework and comes with a number pre-built compliance profiles which includes basic Apache2, PostgreSQL, Linux, Windows Security and CIS Ubuntu Server Benchmarks Level 1 & 2.

The Chef Compliance Web UI allows you to run profiles on an individual node or group of nodes managed by Chef, and produces a summary report that gives an overview of all the nodes scanned, along with the number of tests that passed compliance and the number that failed.
 

...and additional details of each node showing failed compliance rules in a table ordered by severity.

This allows you to identify quickly where remedial action is needed, and by looking at the InSpec rule that failed, you get an indication of what action needs to be taken to fix the issue.


Chef Habitat

Habitat is an open source project that defines a new framework for packaging applications, and its automation as the unit of deployment. Habitat takes an app-centric approach, and doesn’t place any restrictions on any particular runtime. It defines a ‘Modern Application’ as having the following characteristics:

  • Isolated from external dependencies
  • Once built is immutable and agnostic to the environment it's deployed into
  • Deployment artefacts can be rebuilt from source with the same outcome every time
  •  Agnostic to its operating environment
  • Provides an API to configure its runtime configurable elements
  • Packaging and deployment mechanisms are easy to use and not tied to any language or execution environment
  • Supports multiple deployment patterns using the same patterns

Habitat is a full-featured runtime, made up of a packaging format and supervisor. Applications packaged with Habitat can be dynamically configured for an environment at runtime. When a service is started, the relationship with its peers can be defined - such as standalone, leader/follower or initiator. The supervisor has a number of in-built security policies that can be set up to control both packages, and what configuration policies users/peers are allowed to update.

The supervisor also understands update strategies and can be configured with the Habitat depot to check for any updates that need to be applied. The Habitat supervisor can determine the state of a service and restart if down. An HTTP API exposes information about the package as well as any configuration changes that have been made. It also provides a mechanism for monitoring systems to perform health checks on the Habitat packages.

So, there you have my overview of Chef, and in the second part of my introduction to DevOps toolsets, I'll look at how to install and configure Nagios to monitor WebLogic using Ansible.