Version control your CI/CD pipelines!

By 5th October 2018CI/CD, Coding

As a developer I dread having to log onto a CI/CD tools interface, fiddle around on settings pages (which are usually overloaded and lack UX principles) and finally, depending on how responsive the server is, have my new configurations applied. Let’s agree that this is not the most effective way to deal with our CI/CD pipelines.

This brings me to the topic that I want to discuss. I recently had a look at Team City’s Kotlin DSL functionality that can be used to version control all aspects of your applications build pipeline and store it alongside your application in source control.

Implementing Kotlin DSL in Team City Server 2018

I found the process of implementing this rather easy. I will take you through the process of creating a new sub project in Team City using the Kotlin DSL.

Before we begin

There are a couple of important details to note before we start.

I used Git (GitHub) as my version control. The Team City documentation notes that the two-way synchronisation necessary for this, is supported by: Git, Mecurial, Perforce, Subversion and TFS. I have however only tested it on Git. (https://confluence.jetbrains.com/display/TCD18/Storing+Project+Settings+in+Version+Control)

I also used Team City 2018.1 in this article.

Create read/write access between Team city and Repository

 

Setting up a read/write ssh key in GitHub

This would be the usual process of generating an ssh key, going to the settings page, accessing the SSH and GPG keys tab and adding a new SSH key. By default, the key should be assigned read/write access.

Add read/write permission to the user for the specified project

This was a step that initially tripped me up. It is very simple though. On GitHub you can create a team, add your user and any repositories that you want to use the Kotlin DSL onto this team’s repositories.

Setting up a read/write ssh key in TeamCity

This is also a relatively standard process. Go to your Root project. On the left set of tabs, you should see SSH Keys, select it and you will see an option to upload an SSH Key. Make sure that this is the same key (private key) that you uploaded on your GitHub account which would be associated with your user.

At this point you should now have your keys and permissions correctly set up in order for Team City to be able to read and write to a Git repository.

Setting up the VCS root

Now assuming that you have set up a VCS root (if not please refer to this page: https://confluence.jetbrains.com/display/TCD18/Configuring+VCS+Roots), the one important point to note here is that you will want to use the same SSH key that you previously added to Team City as your authentication method.

Project Configuration

I created a very simple maven project with a simple test goal. I then proceeded to add a .teamcity directory and created a pom.xml and settings.kts.

Add the following (shown below) in your pom.xml:

Please note that you must add your team city domain where you see team-city-server-url.

<?xml version="1.0"?>
<project>
  <modelVersion>4.0.0</modelVersion>
  <name>KotlinDslExample Config DSL Script</name>
  <groupId>KotlinDslExample</groupId>
  <artifactId>KotlinDslExample_dsl</artifactId>
  <version>1.0-SNAPSHOT</version>
  <parent>
    <groupId>org.jetbrains.teamcity</groupId>
    <artifactId>configs-dsl-kotlin-parent</artifactId>
    <version>1.0-SNAPSHOT</version>
  </parent>
  <repositories>
    <repository>
      <id>jetbrains-all</id>
      <url>http://download.jetbrains.com/teamcity-repository</url>
      <snapshots>
        <enabled>true</enabled>
      </snapshots>
    </repository>
    <repository>
      <id>teamcity-server</id>
      <url>https://team-city-server-url/app/dsl-plugins-repository</url>
      <snapshots>
        <enabled>true</enabled>
      </snapshots>
    </repository>
  </repositories>
  <pluginRepositories>
    <pluginRepository>
      <id>JetBrains</id>
      <url>http://download.jetbrains.com/teamcity-repository</url>
    </pluginRepository>
  </pluginRepositories>
  <build>
    <sourceDirectory>.</sourceDirectory>
    <plugins>
      <plugin>
        <artifactId>kotlin-maven-plugin</artifactId>
        <groupId>org.jetbrains.kotlin</groupId>
        <version>${kotlin.version}</version>
        <configuration/>
        <executions>
          <execution>
            <id>compile</id>
            <phase>process-sources</phase>
            <goals>
              <goal>compile</goal>
            </goals>
          </execution>
          <execution>
            <id>test-compile</id>
            <phase>process-test-sources</phase>
            <goals>
              <goal>test-compile</goal>
            </goals>
          </execution>
        </executions>
      </plugin>
      <plugin>
        <groupId>org.jetbrains.teamcity</groupId>
        <artifactId>teamcity-configs-maven-plugin</artifactId>
        <version>${teamcity.dsl.version}</version>
        <configuration>
          <format>kotlin</format>
          <dstDir>target/generated-configs</dstDir>
        </configuration>
      </plugin>
    </plugins>
  </build>
  <dependencies>
    <dependency>
      <groupId>org.jetbrains.teamcity</groupId>
      <artifactId>configs-dsl-kotlin</artifactId>
      <version>${teamcity.dsl.version}</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>org.jetbrains.teamcity</groupId>
      <artifactId>configs-dsl-kotlin-plugins</artifactId>
      <version>1.0-SNAPSHOT</version>
      <type>pom</type>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>org.jetbrains.kotlin</groupId>
      <artifactId>kotlin-stdlib-jdk8</artifactId>
      <version>${kotlin.version}</version>
      <scope>compile</scope>
    </dependency>
    <dependency>
      <groupId>org.jetbrains.kotlin</groupId>
      <artifactId>kotlin-script-runtime</artifactId>
      <version>${kotlin.version}</version>
      <scope>compile</scope>
    </dependency>
  </dependencies>
</project>

Add the following (shown below) in your settings.kts:

import jetbrains.buildServer.configs.kotlin.v2018_1.*
import jetbrains.buildServer.configs.kotlin.v2018_1.buildSteps.maven
import jetbrains.buildServer.configs.kotlin.v2018_1.triggers.vcs
/*
The settings script is an entry point for defining a TeamCity
project hierarchy. The script should contain a single call to the
project() function with a Project instance or an init function as
an argument.
VcsRoots, BuildTypes, Templates, and subprojects can be
registered inside the project using the vcsRoot(), buildType(),
template(), and subProject() methods respectively.
To debug settings scripts in command-line, run the
    mvnDebug org.jetbrains.teamcity:teamcity-configs-maven-plugin:generate
command and attach your debugger to the port 8000.
To debug in IntelliJ Idea, open the 'Maven Projects' tool window (View
-> Tool Windows -> Maven Projects), find the generate task node
(Plugins -> teamcity-configs -> teamcity-configs:generate), the
'Debug' option is available in the context menu for the task.
*/
version = "2018.1"
project {
    buildType {
        id = AbsoluteId("TestStepID")
        name = "Test Step"
        vcs { root(DslContext.settingsRoot)}
        steps {
            maven {
                mavenVersion = defaultProvidedVersion()
                goals = "clean test"
            }
        }
        artifactRules = "target/kotlin.dsl.project-*.jar"
        triggers {
            vcs {
                id = "vcsTrigger"
            }
        }
        cleanup {
            artifacts(days = 3)
            history(days = 10)
        }
    }
}

With this you will have added the necessary dependencies and file structure for Team City to detect

your configuration on import.

Team City Configuration

Now let’s create a new sub project off our root project.

Using the ‘From GitHub.com’ option, select the repository that you previously set up from the list of repositories

Now team city will load the list of repositories that are available on your GitHub account.

Team city will automatically detect the settings that you created in your .teamcity/settings.kts and import the configuration.

Once Team City has finished loading your configuration, you should now have a new project with a single build configuration.

The build should automatically execute and you will see that it has completed successfully.

You have now successfully created a project and your first build pipeline with Kotlin DSL!

This is by no means exhaustive in any way and only scratches the surface of what is possible, but will hopefully give you enough to get you started in the right direction.

by: Michael Stievenart

 

 

RMB FOUNDeRY

Author RMB FOUNDeRY

More posts by RMB FOUNDeRY