Can AWS CodePipeline track multiple feature branches and run tests on each?

I was looking for a solution to exactly this problem. Eventually I settled on having a CodeBuild, which can be triggered off of a branch regex, begin the pipeline by pushing an archive to a specific S3 key. In my case, I also had that CodeBuild do my full build/test process, but you could also configure the CodeBuild to only pull the code and push it the the S3 key that triggers your CodePipeline.

Here's part of an example CodeBuild config matching 2 branches:

Then I set the CodeBuild artifact to go to a single key in a single bucket.

Then I setup a CodePipeline with an Amazon S3 source pointing to the same key/bucket.

CodePipeline is not the right tool for you. Create a separate, standalone CodeBuild project. It will work very much like other 3rd party CI services such as Travis.

Make sure to select the "Rebuild every time a code change is pushed to this repository" Source setting. This will trigger a build from push to any branch and also from other web hook events eg. created PRs. You can also create filters to manage them.

There are a number of ways you can connect this "CI stage" to a continuous delivery pipeline, which is what CodePipeline is for:

1. The simplest is just to use CodeBuild as a gate for pushing changes to the pipeline source branch, typically master.
2. You can also push artifacts created in CodeBuild to ECR or S3 and trigger a pipeline from those events.
3. If you want to get complicated, use some other jiggery pokery eg. SQS and Lambda.

I had the same confusion as CodeBuild and CodePipeline are tightly interlinked but also separate tools. CodePipeline does use CodeBuild but each has its own Git connector that works differently.

Further review shows that with Cloudformation, you can pick the branch that CodePipeline tracks,

AWS CodeCommit (CodeCommit)

• PollForSourceChanges¹ (Optional)
• RepositoryName (Required)
• BranchName (Required)

You can see an example of complete template but the CodePipeline stage looks like,

Name: CheckoutSourceTemplateActionTypeId:  Category: Source  Owner: AWS  Version: 1  Provider: CodeCommitConfiguration:  PollForSourceChanges: True  RepositoryName: !GetAtt [PipelineRepo, Name]  BranchName: masterOutputArtifacts:  - Name: TemplateSourceRunOrder: 1

With CodeCommit Repo's, you can create Triggers that can use these triggers to launch a Lambda function,

You can configure Lambda functions by creating the trigger in the Lambda console as part of the function. This is the simplest method, as triggers created in the Lambda console automatically include the permissions required for AWS CodeCommit to invoke the Lambda function. If you create the trigger in AWS CodeCommit, you must include a policy to allow AWS CodeCommit to invoke the function. For more information, see Create a Trigger for an Existing Lambda Function and Example 2: Create a Policy for AWS Lambda Integration.

So what could happen is to set up a CloudFormation template like above to track the master branch. Then have CodeCommit trigger on repository changes and call a Lambda function that uses Boto3 to get_pipeline to retrieve the master branch pipeline.

Then using either update_pipeline or create_pipeline to add a stage to the existing master branch pipeline or create an entirely new pipeline that tracks the additional branches desired.

This way, the CodePipeline can track the feature branches in a useful way.