Crossplane Testing: Rendering and Validating Compositions

Platform

Community

Resources

Company

Docs

Book a demo

Get Started

Book a demo

Get Started

Steven Borrelli

Cloud Native

Aug 19, 2024

Read time: 0 mins

Crossplane Testing: Rendering and Validating Compositions

Learn essential Crossplane Composition Testing techniques. Render, validate, and ensure error-free cloud infrastructure with this guide for Platform Engineers.

Crossplane Testing: Rendering and Validating Compositions

Learn essential Crossplane Composition Testing techniques. Render, validate, and ensure error-free cloud infrastructure with this guide for Platform Engineers.

This is the first of a series on testing tools and patterns in Crossplane.

Crossplane is a Universal Control Plane that can manage almost any resource, like AWS S3 buckets or Azure Databases. Building upon Kubernetes Custom Resource Definitions (CRDs) and continually-reconciling controllers, Crossplane allows Platform Engineers to provide a cloud-like experience to end users.

While Crossplane provides an excellent foundation, Engineers still need to define APIs and configure cloud resources. Catching errors early is critically important in reducing outages and misconfigurations. This blog will cover testing during the infrastructure development cycle.

A Review of Crossplane Concepts

Crossplane has a modular control plane architecture that has multiple extensibility points and supports day 2 operations.

Providers extend Crossplane by providing schema definitions and a controller for cloud-provider APIs. For example, the AWS Provider can manage almost 1,000 separate AWS Managed Resources.

Compositions take individual Managed Resources and combine them into higher-level abstractions. An example is the EKS reference configuration, which defines a custom API and provisions IAM Roles, Nodepools, OIDC, and other AWS resources to create a fully-defined EKS cluster.

Because the real world can be complex, Composition Functions allow engineers to define resources using any programming language, like KCL or render state using text-templating engines. When rendering a Composition, Functions are run in a series of pipeline steps, with each step defining or modifying the desired state of the resources we want to create.

A Composition can contain dependencies, conditions, and loops and generate dozens or hundreds of resources, creating the risk of introducing errors and misconfiguration.

Composition developer must ensure:

The Composition code is valid and error-free
The desired state generated from the Composition matches expectations
The desired resources generated from the Composition are valid

Ideally, we’d be able to validate the Composition during development and get feedback immediately. In the next sections we’ll take a look at crossplane render and crossplane validate, utilities that emerged from Crossplane’s Developer Experience special interest group.

Rendering Functions with the Crossplane CLI

Crossplane CLI has the ability to render the output of Crossplane functions, allowing you to check and validate the desired state generated by the Composition before applying it to your Cluster.

In Crossplane 1.17, coming out in August 2024, render is graduating from beta. If you have an earlier version of the Crossplane CLI, use the crossplane beta render command for all of these examples.

Render requires three files: a resource to test, the Composition file, and a file that contains the function Docker images for crossplane render to run. We are going to test creating XTenant types:

Code for the basic example can be found here.

Using the command line, we can run:

Running this command gives us the output of the XTenant kind and all the Kubernetes Objects that provider-kubernetes uses to create the namespaces.

---
apiVersion: k8s.example.com/v1alpha1
kind: XTenant
metadata:
  name: example
spec:
  teams:
    - team-1
    - team-2
    - team-3
status:
  conditions:
    - lastTransitionTime: "2024-01-01T00:00:00Z"
      message: 'Unready resources: namespace-team-1, namespace-team-2, and namespace-team-3'
      reason: Creating
      status: "False"
      type: Ready
  namespaces:
    team-1: null
    team-2: null
    team-3: null
---
apiVersion: kubernetes.crossplane.io/v1alpha2
kind: Object
metadata:
  annotations:
    crossplane.io/composition-resource-name: namespace-team-1
  generateName: example-
  labels:
    crossplane.io/composite: example
spec:
  forProvider:
    manifest:
      apiVersion: v1
      kind: Namespace
      metadata:
        name: team-1
---
apiVersion: kubernetes.crossplane.io/v1alpha2
kind: Object

...

---
apiVersion: k8s.example.com/v1alpha1
kind: XTenant
metadata:
  name: example
spec:
  teams:
    - team-1
    - team-2
    - team-3
status:
  conditions:
    - lastTransitionTime: "2024-01-01T00:00:00Z"
      message: 'Unready resources: namespace-team-1, namespace-team-2, and namespace-team-3'
      reason: Creating
      status: "False"
      type: Ready
  namespaces:
    team-1: null
    team-2: null
    team-3: null
---
apiVersion: kubernetes.crossplane.io/v1alpha2
kind: Object
metadata:
  annotations:
    crossplane.io/composition-resource-name: namespace-team-1
  generateName: example-
  labels:
    crossplane.io/composite: example
spec:
  forProvider:
    manifest:
      apiVersion: v1
      kind: Namespace
      metadata:
        name: team-1
---
apiVersion: kubernetes.crossplane.io/v1alpha2
kind: Object

...

---
apiVersion: k8s.example.com/v1alpha1
kind: XTenant
metadata:
  name: example
spec:
  teams:
    - team-1
    - team-2
    - team-3
status:
  conditions:
    - lastTransitionTime: "2024-01-01T00:00:00Z"
      message: 'Unready resources: namespace-team-1, namespace-team-2, and namespace-team-3'
      reason: Creating
      status: "False"
      type: Ready
  namespaces:
    team-1: null
    team-2: null
    team-3: null
---
apiVersion: kubernetes.crossplane.io/v1alpha2
kind: Object
metadata:
  annotations:
    crossplane.io/composition-resource-name: namespace-team-1
  generateName: example-
  labels:
    crossplane.io/composite: example
spec:
  forProvider:
    manifest:
      apiVersion: v1
      kind: Namespace
      metadata:
        name: team-1
---
apiVersion: kubernetes.crossplane.io/v1alpha2
kind: Object

...

While this is a promising start, one problem is the status of the XTenant is not set, since we are only creating a desired state. In the next section we’ll discuss how to simulate existing resources.

More Complex Rendering Patterns

What if we want to make our test environment simulate the real world, where we have resources already provisioned?

Using the CLI, we can simulate the output of a live environment by providing more data to the render command. Observed resources allow developers to add existing resources to the context of a function.

Code for observed resources is located here.

To create observed resources, one can save the YAML output of the resource into a text file.

kubectl get cluster.eks mycluster -o

kubectl get cluster.eks mycluster -o

kubectl get cluster.eks mycluster -o

It’s important when doing this to make sure each observed resource has the right crossplane.io/composition-resource-name annotation to match it to the resource being generated in our composition.

metadata:
  annotations:
    crossplane.io/composition-resource-name

metadata:
  annotations:
    crossplane.io/composition-resource-name

metadata:
  annotations:
    crossplane.io/composition-resource-name

Now that we have stored resources in the observed directory. We update our render command to include the Observed resources:

crossplane render \
  --observed-resources observed \
  --include-full-xr

crossplane render \
  --observed-resources observed \
  --include-full-xr

crossplane render \
  --observed-resources observed \
  --include-full-xr

Finally, when we run crossplane render, the status of the XTenant is now populated with the data from the observed resources.

apiVersion: k8s.example.com/v1alpha1
kind: XTenant
metadata:
  name: example
spec:
  environment: dev
  group: webdev
  teams:
  - team-1
  - team-2
  - team-3
status:
  conditions:
  - lastTransitionTime: "2024-01-01T00:00:00Z"
    reason: Available
    status: "True"
    type: Ready
  namespaces:
    team-1:
      name: dev-teams-zjhd9
      uid: 45cd966b-c233-44e0-a11a-69a50b637b07
    team-2:
      name: dev-teams-c9fjg
      uid: 0658be34-fee1-4c15-9146-2b49a5b5c9af
    team-3:
      name: dev-teams-kwvvm
      uid

apiVersion: k8s.example.com/v1alpha1
kind: XTenant
metadata:
  name: example
spec:
  environment: dev
  group: webdev
  teams:
  - team-1
  - team-2
  - team-3
status:
  conditions:
  - lastTransitionTime: "2024-01-01T00:00:00Z"
    reason: Available
    status: "True"
    type: Ready
  namespaces:
    team-1:
      name: dev-teams-zjhd9
      uid: 45cd966b-c233-44e0-a11a-69a50b637b07
    team-2:
      name: dev-teams-c9fjg
      uid: 0658be34-fee1-4c15-9146-2b49a5b5c9af
    team-3:
      name: dev-teams-kwvvm
      uid

apiVersion: k8s.example.com/v1alpha1
kind: XTenant
metadata:
  name: example
spec:
  environment: dev
  group: webdev
  teams:
  - team-1
  - team-2
  - team-3
status:
  conditions:
  - lastTransitionTime: "2024-01-01T00:00:00Z"
    reason: Available
    status: "True"
    type: Ready
  namespaces:
    team-1:
      name: dev-teams-zjhd9
      uid: 45cd966b-c233-44e0-a11a-69a50b637b07
    team-2:
      name: dev-teams-c9fjg
      uid: 0658be34-fee1-4c15-9146-2b49a5b5c9af
    team-3:
      name: dev-teams-kwvvm
      uid

Context

To share data that is not desired state, function pipelines can contain optional context. The most usual Composition pattern is a pipeline step reads the context.

Extra Resources were introduced in Crossplane 1.15, and are a powerful feature allowing a function to search for any other Crossplane resource on a Cluster and store the data in the Function’s context. This data can be any Crossplane object from Managed Resources, other Composites. Function-environment-configs looks up Crossplane EnvironmentConfigs and loads them into the pipeline’s context.

In order to create context in render, create a directory like extra-resources, and add theYAML manifests of the resources you are trying to simulate.

The example code is located here.

Run the following command:

crossplane beta render \
  --extra-resources extra-resources \
  --observed-resources observed \
  --include-full-xr \
  --include-context

crossplane beta render \
  --extra-resources extra-resources \
  --observed-resources observed \
  --include-full-xr \
  --include-context

crossplane beta render \
  --extra-resources extra-resources \
  --observed-resources observed \
  --include-full-xr \
  --include-context

With –include-context, we can check the output to ensure that the Pipeline is selecting the correct resource. There is a dev and production EnvironmentConfig in the extra-resources directory. What happens in the Context when you change xr.yaml from dev to prod?

You may be wondering what is the difference between Observed and Context: Observed data refers to resources that are in the composition as part of the desired state. Extra resources are defined outside of the composition.

Now that we can render our Composition with Resource and External data into our function, how do we validate that the resources being generated match the schemas of the resources?

Validating Compositions

The Crossplane CLI comes with a validate command that is currently in beta status that takes a file or the output of render and checks that it matches the schema for the object.

Example code is located here.

Running the following command will render a Composition and then pass the output to validate. The validate command will look into the schemas directory for packages or YAML manifests to validate the output of render.

crossplane render \
  --observed-resources observed \
  --include-full-xr \
  xr.yaml composition.yaml functions.yaml  | crossplane beta validate schemas -

crossplane render \
  --observed-resources observed \
  --include-full-xr \
  xr.yaml composition.yaml functions.yaml  | crossplane beta validate schemas -

crossplane render \
  --observed-resources observed \
  --include-full-xr \
  xr.yaml composition.yaml functions.yaml  | crossplane beta validate schemas -

The validate command can extract Kubernetes schema definitions from the provider packages, in our example, our schema directory contains a configuration.yaml package that has dependencies:

crossplane render \
  --observed-resources observed \
  --include-full-xr \
  xr.yaml composition.yaml functions.yaml  | crossplane beta validate schemas -
package schemas does not exist, downloading:  xpkg.upbound.io/upbound/configuration-aws-network:v0.15.0
package schemas does not exist, downloading:  xpkg.upbound.io/upbound/provider-aws-ec2:v1.4.0
package schemas does not exist, downloading:  xpkg.upbound.io/crossplane-contrib/function-auto-ready:v0.2.1
package schemas does not exist, downloading:  xpkg.upbound.io/crossplane-contrib/function-go-templating:v0.5.0
[x] schema validation error aws.platform.upbound.io/v1alpha1, Kind=XNetwork, configuration-aws-network : spec.parameters.deletionPolicy: Required value
[x] schema validation error aws.platform.upbound.io/v1alpha1, Kind=XNetwork, configuration-aws-network : spec.parameters.providerConfigName: Required value
[✓] ec2.aws.upbound.io/v1beta1, Kind=InternetGateway, configuration-aws-network-w7nb4 validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=MainRouteTableAssociation, configuration-aws-network-pg888 validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=Route, configuration-aws-network-9n2x8 validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=RouteTable, configuration-aws-network-vbcgd validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=RouteTableAssociation, configuration-aws-network-stjtq validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=RouteTableAssociation, configuration-aws-network-gfb42 validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=RouteTableAssociation, configuration-aws-network-jdqj2 validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=RouteTableAssociation, configuration-aws-network-pnvdg validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=SecurityGroup, configuration-aws-network-9m99r validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=SecurityGroupRule, configuration-aws-network-s4pzf validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=SecurityGroupRule, configuration-aws-network-kq5th validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=Subnet, configuration-aws-network-52h6d validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=Subnet, configuration-aws-network-phfgs validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=Subnet, configuration-aws-network-mxxs4 validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=Subnet, configuration-aws-network-gzzrb validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=VPC, configuration-aws-network-6jtl4 validated successfully
Total 17 resources: 0 missing schemas, 16 success cases, 1 failure cases
crossplane: error: cannot validate resources: could not validate all resources
```

You'll notice two errors. `validate` will compare the Composite
Resource (XR) to the XRD schema. We can either add these values
to the [`xr.yaml`](xr.yaml) file or update our upstream Configuration
to remove the `required` field and use default values instead.

```shell
[x] schema validation error aws.platform.upbound.io/v1alpha1, Kind=XNetwork, configuration-aws-network : spec.parameters.deletionPolicy: Required value
[x] schema validation error aws.platform.upbound.io/v1alpha1, Kind=XNetwork,configuration-aws-network : spec.parameters.providerConfigName: Required value

crossplane render \
  --observed-resources observed \
  --include-full-xr \
  xr.yaml composition.yaml functions.yaml  | crossplane beta validate schemas -
package schemas does not exist, downloading:  xpkg.upbound.io/upbound/configuration-aws-network:v0.15.0
package schemas does not exist, downloading:  xpkg.upbound.io/upbound/provider-aws-ec2:v1.4.0
package schemas does not exist, downloading:  xpkg.upbound.io/crossplane-contrib/function-auto-ready:v0.2.1
package schemas does not exist, downloading:  xpkg.upbound.io/crossplane-contrib/function-go-templating:v0.5.0
[x] schema validation error aws.platform.upbound.io/v1alpha1, Kind=XNetwork, configuration-aws-network : spec.parameters.deletionPolicy: Required value
[x] schema validation error aws.platform.upbound.io/v1alpha1, Kind=XNetwork, configuration-aws-network : spec.parameters.providerConfigName: Required value
[✓] ec2.aws.upbound.io/v1beta1, Kind=InternetGateway, configuration-aws-network-w7nb4 validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=MainRouteTableAssociation, configuration-aws-network-pg888 validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=Route, configuration-aws-network-9n2x8 validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=RouteTable, configuration-aws-network-vbcgd validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=RouteTableAssociation, configuration-aws-network-stjtq validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=RouteTableAssociation, configuration-aws-network-gfb42 validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=RouteTableAssociation, configuration-aws-network-jdqj2 validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=RouteTableAssociation, configuration-aws-network-pnvdg validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=SecurityGroup, configuration-aws-network-9m99r validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=SecurityGroupRule, configuration-aws-network-s4pzf validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=SecurityGroupRule, configuration-aws-network-kq5th validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=Subnet, configuration-aws-network-52h6d validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=Subnet, configuration-aws-network-phfgs validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=Subnet, configuration-aws-network-mxxs4 validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=Subnet, configuration-aws-network-gzzrb validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=VPC, configuration-aws-network-6jtl4 validated successfully
Total 17 resources: 0 missing schemas, 16 success cases, 1 failure cases
crossplane: error: cannot validate resources: could not validate all resources
```

You'll notice two errors. `validate` will compare the Composite
Resource (XR) to the XRD schema. We can either add these values
to the [`xr.yaml`](xr.yaml) file or update our upstream Configuration
to remove the `required` field and use default values instead.

```shell
[x] schema validation error aws.platform.upbound.io/v1alpha1, Kind=XNetwork, configuration-aws-network : spec.parameters.deletionPolicy: Required value
[x] schema validation error aws.platform.upbound.io/v1alpha1, Kind=XNetwork,configuration-aws-network : spec.parameters.providerConfigName: Required value

crossplane render \
  --observed-resources observed \
  --include-full-xr \
  xr.yaml composition.yaml functions.yaml  | crossplane beta validate schemas -
package schemas does not exist, downloading:  xpkg.upbound.io/upbound/configuration-aws-network:v0.15.0
package schemas does not exist, downloading:  xpkg.upbound.io/upbound/provider-aws-ec2:v1.4.0
package schemas does not exist, downloading:  xpkg.upbound.io/crossplane-contrib/function-auto-ready:v0.2.1
package schemas does not exist, downloading:  xpkg.upbound.io/crossplane-contrib/function-go-templating:v0.5.0
[x] schema validation error aws.platform.upbound.io/v1alpha1, Kind=XNetwork, configuration-aws-network : spec.parameters.deletionPolicy: Required value
[x] schema validation error aws.platform.upbound.io/v1alpha1, Kind=XNetwork, configuration-aws-network : spec.parameters.providerConfigName: Required value
[✓] ec2.aws.upbound.io/v1beta1, Kind=InternetGateway, configuration-aws-network-w7nb4 validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=MainRouteTableAssociation, configuration-aws-network-pg888 validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=Route, configuration-aws-network-9n2x8 validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=RouteTable, configuration-aws-network-vbcgd validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=RouteTableAssociation, configuration-aws-network-stjtq validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=RouteTableAssociation, configuration-aws-network-gfb42 validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=RouteTableAssociation, configuration-aws-network-jdqj2 validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=RouteTableAssociation, configuration-aws-network-pnvdg validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=SecurityGroup, configuration-aws-network-9m99r validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=SecurityGroupRule, configuration-aws-network-s4pzf validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=SecurityGroupRule, configuration-aws-network-kq5th validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=Subnet, configuration-aws-network-52h6d validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=Subnet, configuration-aws-network-phfgs validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=Subnet, configuration-aws-network-mxxs4 validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=Subnet, configuration-aws-network-gzzrb validated successfully
[✓] ec2.aws.upbound.io/v1beta1, Kind=VPC, configuration-aws-network-6jtl4 validated successfully
Total 17 resources: 0 missing schemas, 16 success cases, 1 failure cases
crossplane: error: cannot validate resources: could not validate all resources
```

You'll notice two errors. `validate` will compare the Composite
Resource (XR) to the XRD schema. We can either add these values
to the [`xr.yaml`](xr.yaml) file or update our upstream Configuration
to remove the `required` field and use default values instead.

```shell
[x] schema validation error aws.platform.upbound.io/v1alpha1, Kind=XNetwork, configuration-aws-network : spec.parameters.deletionPolicy: Required value
[x] schema validation error aws.platform.upbound.io/v1alpha1, Kind=XNetwork,configuration-aws-network : spec.parameters.providerConfigName: Required value

As can be seen in the example, dependencies were downloaded and each item in the render output was validated against the schema. Note that we have two errors, where in our request we don’t set a required field.

We’ve reached a good stopping point. Using the Crossplane CLI we’re able to locally render manifests and simulate real-world resources, and then validate the outputs against the Provider schemas and CompositeResourceDefinitions (XRDs).

What's Next?

Future blogs will cover End to End (e2e) testing, CI integration, and using native language tooling.

Yury Tsarev and I will be presenting Testing and Release Patterns for Crossplane at Kubecon 2024 Hong Kong Aug 22, 2024. Be sure to stop by if you're there or tune in online. Watch the recording here