[Turbot On] Custom CMDB Data

Import your org's custom reference data into the Turbot CMDB.

Having correct metadata for a resource is crucial for automated resolution of operations, security and compliance incidents. Teams often rely on owner-assigned resource tags to add external context to resources; however, additional deep-context can be added to resources via the Turbot CMDB.

This week, [Turbot On] will look at how to import and use custom CMDB data using the Turbot Files feature.

If you’re intrigued the idea of building custom automated governance controls, please consider registering for our talk at AWS Fest 2021 on June 22nd.

Ops as Code: Going beyond CIS with custom controls

Our CTO will be discussing the pros and cons of industry benchmarks vs. custom controls, and doing a live coding demo to show how easy it is to get started.

Traditional Workflow

Databases outside of the CMDB often contain reference data (and master data) that is important to cloud operations and security: ‘cost centers’, ‘approved project IDs’, ‘distribution lists’ and ‘data classification’ just to name a few. This type of data changes over time but its correctness can be very important when automating governance controls.

Requiring application teams to create and update tagging metadata is notoriously difficult to enforce. Even when the teams do maintain data, simple data entry mistakes, alternate spellings and capitalization mismatches are very common errors.

Turbot Files

A Turbot ‘File’ is a text-based data object that typically contains a JSON formatted string. Once the object is imported into the Turbot CMDB, the data in it can then be referenced at runtime in Calculated Policies and Stacks.

• A File resource can contain any arbitrary data. Customers will often utilize a JSON schema to make it easier to reference data inside each file.

• A File resource can be a child of the root Turbot resource or a Turbot Folder.

• The name (internally called `aka`) of the File resource is user-definable.

Get it done with Turbot

Turbot Files can be managed using standard GraphQL API or the Turbot Terraform Provider. Management can be automated in a variety of ways depending on your organization's requirements. Some examples:

• Use a trigger to update the Turbot File whenever asset data in an inventory management tool changes.

• Write a shell script to pull data from a third party API and update the File with the Turbot CLI.

• Manually update the File via Terraform.

• Write a Lambda to update the File via the GraphQL API.

In this example we will use a Terraform template to create a Turbot File in order to add application metadata to our CMDB. Once the data is imported we will use it to enhance our tagging controls.

This Terraform template contains a File resource (note the name of the resource is “dmiapps”). Applying the template adds the JSON object specified in `content` to our Turbot workspace.

Now that we have our Turbot File created we can use the metadata inside of it as a data-source for our calculated policies. For example, we can use Turbot’s tagging controls to tag resources using metadata from the “dmiapps” file. To demonstrate the approach we will use a calculated policy to tag our S3 Buckets with the correct values based on the enclosing AWS Account.

The GraphQL query (see “Step 1” below) retrieves both the bucket resource’s metadata object, and the data from the “dmiapps” file.

As seen in Step 2, above, we can then extract the AWS account ID from the resource and use it to lookup keyed values stored in the “dmiapps” file.

The provided example can easily be adjusted for any resource that can be tagged (across all supported cloud services). The best part is that whenever the Turbot File is updated (e.g. a new app is added, change in App owner, etc.), any affected tags will automatically be updated as well.

Now that our tagging template above is set via a calculated policy, we can begin enforcing the tagging control by setting the Tags policy value to `Enforce: Set tags`:

After setting this policy, Turbot will identify all resources that do not have the tags applied correctly, and then handle their remediation (i.e. set the tags).

If you are not yet ready to enforce remediation, you can still assess (and get alerts for) what resources do not have matching tags by changing the policy setting from `Enforce: Set tags` to `Check: Tags are correct`.

Make it happen

See for yourself how easy it is to manage your custom CMDB metadata configurations across all your cloud resources. A ready-to-run Terraform template is available to enable this configuration from the Turbot Development Kit (TDK). If you need any assistance please reach out to Turbot Support, and keep an eye on your inbox for another Turbot tip next week!

Cheers,

Bob