Using Docker with .Net Core in CI for OSS

I recently wrote a project for ASP.NET Core 2 and the time had come to get a CI system up and running. I develop on OSX and mainly test on OSX & Linux and so the defacto place to go is TravisCI. I’ve used it in the past and all has been great but I put out a tweet asking if Travis was still the place to go:

Adron Hall replied and said he’d been using Codeship as a Docker based CI system. Having experience in Docker I thought I’d take a look. My requirements were simple, I needed the CI to run dotnet restore and dotnet build and dotnet test. I also thought to myself how am I going to handle releasing a NuGet package? Normally I tend to run a build script locally check everything is ok before I push to NuGet so I can avoid an “oh-shit” release but it happens!

I looked at Codeship’s basic plan (free) and they supported most things on their systems out of the box but not .NET and so I moved to the pro plan (also free). At this point Codeship’s co-founder & CEO got in touch (Moritz Plassnig) and had said he had seen the conversation on Twitter with Adron and was there to help. I asked him about .NET Core etc and he confirmed that my choice to use the Pro account was the best decision and any other issues to give him a ping. Good stuff I thought!

I began to read Codeship’s documentation and quite comprehensive it is I must say. Essentially you have three files; a services file, a steps file and a docker file. The services file describes your service eg a name, the Dockerfile path and things like a path to encrypted envionment variables plus many other settings available. The steps file is a file where you describe each step in your CI system. For me the first step was obviously to run dotnet restore then another step for dotnet build then dotnet test. I pushed my files to my repo and watched on Codeship’s dashboard. The dotnet restore worked but the build failed. The thing I lost some time on was that each step is run in its own container and I couldn’t work out why the build was failing after I had successfully installed all the packages required for it to build. Ironically I was reading the documentation for golang projects where it mentioned this! During this process and me scratching my head I was tweeting Adron and Codeship to see if they knew why I was having issues and Kelly Andrews, Codeship’s Developer Advocate starting helping me out which was great. He suggested I could use my Dockerfile to do the dotnet restore and build and then have a step to do the dotnet test. That got me thinking and in the end I decided I would put the restore, build and test in the Dockerfile so when each push to the repo or PR is sent it would build the Dockerfile and although not part of the steps file Codeship would still report a failed build if it couldn’t build the Docker image. What I could use the steps file for was releasing to Nuget. This felt a bit scary as it increased the potential of releasing something I wasn’t happy with and I would end up releasing a “oh shit” patch release but I thought I’d just give it a go. The way I could control this is a feature of Codeship’s step files. (The files Codeship use is YAML). In my steps file I could filter when the step was executed. Here’s the resulting file:

- service: app
    tag: ^\d+.\d+.\d+(-.*|$)
    command: bash -c "dotnet pack -c Release -o /code/artifacts src/Botwin.csproj && dotnet nuget push -s https://www.nuget.org/api/v2/package -k $NUGETAPIKEY /code/artifacts/Botwin.$CI_BRANCH.nupkg"

This says the service it belongs to is app which is what I define in my services file. The tag is the way to filter the step so when it sees something in a commit message or git tag then it will execute. (You can also define the inverse using exclude, see the docs for more) Finally the command to execute if it passes the tag regex. In my file I have said execute this step if it sees number.number.number or number.number.number-something in a git tag. So if I’ve done a load of work and I’m happy that a new version is ready to be released I do a git tag 1.2.69 and then a git push and Codeship will see this tag and then build the Docker image then it see the tag and then execute dotnet pack and dotnet nuget push. Pretty good I thought and started to test it.

Codeship provides the same tooling that controls the CI process on their servers avaialble as a binary that can be installed via Homebrew so you can test the pipeline locally. This tooling is called Jet. So I followed the instructions and away I went. Again I was lucky as I had Kelly on hand to answer my questions but the documentation was very good. For example, as I wanted to publish to NuGet I needed to supply my API key and obviously didn’t want that sitting in my repo but Codeship’s docs described how you could pass in a file with the raw values, encrypt it using Jet, put the encrypted file in your repo and tell the services YAML file to look at the encrypted file to get environment variables out. So above you can see I use $NUGETAPIKEY and that comes from the encrypted file. You’ll also see that I use $CI_BRANCH. This is part of a number of environmental variables that Codeship provides that you have access too. Here I could use the git tag 5.6.7-rc79 which is found inside the $CI_BRANCH environmental variable, slightly badly named IMO but it means I can get access to the version I have just built in this scenario as just before I do my git tag and push I also change the csproj version number so they need to match, then I tag and push and Codeship builds and tests and releases to NuGet for me.

The other odd thing I did spot was the need to do bash -c "multiple statements go here" for multiple statements in a steps file because if I ran just dotnet restore all was fine but dotnet restore && dotnet build it didn’t like it so I needed to add the bash prefix. Thinking about it now I could move the dotnet restore, build and test to another step rather than make it part of the Dockerfile. I’m not sure there are any advantages/disadvantages to either approach really as I don’t think the layers in the Dockerfile when doing a restore/build are cached so it doesn’t speed up CI time.

When I got it all working I was pretty impressed and was thankful for the help I got from Kelly. The project I used it for (Botwin) has fairly small requirements and there was lots of documentation I didn’t even delve into so I think Codeship can probably provide a solution to much larger projects so please check them out. I’m hoping services like this expand as .NET Core gains more traction in the *nix worlds and the binding to Windows that .NET has always had truly disappears and .NET becomes a proper cross platform runtime. My next desire is to have a Linux .NET profiler, none exist currently although Jetbrains tell me they have some plans but it’s a gap in the market if you’re interested!

Link to getting started and defining services and steps YAML

Link to JET docs

Link to environment variables encryption