Ggthjy

Introduction

I'm writing a script in YAML for building ASP.NET Core 2.2 project using GitLab Continuous Integration. In all YAML samples I could find (and there are not many) for building .NET Core based applications using GitLab CI I could see something like this:

before_script:

 - 'dotnet restore'

before_script is running a dependency restore before every job. It makes sense because if you use free GitLab runners (like I do) every job is executed on a different machine. There is no possibility to preserve state from previous jobs (with exception on cache and artifacts, but I'll get to that later). What that means is that on the next job, all previously restored packages will be gone, and they need to be restored again. And on the next job once again. And again on every job in the pipeline which needs the packages. I noticed a redundancy there. A redundancy that takes precious time, because a huge project with a lot of third-party packages takes a while for a full restore.

Using job artifacts

I found a way to preserve those packages and then pass them to the next job via GitLab artifacts:

restore:

 stage: restore

 script:

  - 'dotnet restore --packages .nuget/'

 artifacts:

  paths:

   - 'src/**/obj/*'

   - '.nuget/'

Let's break it down. With dotnet restore --packages .nuget/ I explicitly specify a custom directory for packages to be restored. Then I specify two paths which GitLab CI will be interested in when creating a job artifacts. dotnet restore creates a few files with metadata about packages inside a obj/ directory, so these will be needed as well. I include them in src/**/obj/*. Finally, I include the .nuget/ directory which after dotnet restore should contain all restored dependencies.

Note: A dependency restore saves the path where the packages will be kept inside <PROJECT_NAME>/obj/project.assets.json file. After that, there is no need for explicitly specifying where the restored packages are e.g. when building the project.

Eventually, in the next job I use previously created job artifacts by specifying a job dependecy. In that way, GitLab CI knows that it should download job artifacts from the dependant job.

build:

 stage: build

 script:

  - 'dotnet build --no-restore'

 dependencies:

  - restore

Whole YAML script:

image: microsoft/dotnet:2.2-sdk



variables:

 SOURCE_CODE_DIRECTORY: 'src'

 BINARIES_DIRECTORY: 'bin'

 OBJECTS_DIRECTORY: 'obj'

 NUGET_PACKAGES_DIRECTORY: '.nuget'



stages:

 - restore

 - build



restore:

 stage: restore

 script:

  - 'dotnet restore --packages="$NUGET_PACKAGES_DIRECTORY"'

 artifacts:

  paths:

   - '$SOURCE_CODE_DIRECTORY/**/$OBJECTS_DIRECTORY/*'

   - '$NUGET_PACKAGES_DIRECTORY/'



build:

 stage: build

 script:

  - 'dotnet build --no-restore'

 dependencies:

  - restore

Feedback

Please tell me what you think, any weaknesses of my approach, code smells, or maybe a better solution. All kind of constructive feedback appreciated.

I had a wrong concept about GitLab artifacts. After a good read on GitLab docs, especially the section distinguishing artifacts and cache, I deduced that I should use cache instead of artifacts as it was designed precisely for storing restored dependencies. Artifacts are meant for passing build output and binaries.

I also removed the restore stage, placing the dotnet restore command in a global before_script. Cache can fail and in such scenario the script should gracefully fallback to default 'download-from-internet' behaviour. With --no-restore option enabled it would not happen. Thus, I removed that option from dotnet build command. It won't make a noticeable difference with successfully download cache as a dependency restore with already downloaded packages will execute in next-to-no-time.

Finally, I added cache key, which will keep cache bundles separate for branches and stages.

Updated script:

image: microsoft/dotnet:2.2-sdk



variables:

 SOURCE_CODE_DIRECTORY: 'src'

 BINARIES_DIRECTORY: 'bin'

 OBJECTS_DIRECTORY: 'obj'

 NUGET_PACKAGES_DIRECTORY: '.nuget'



stages:

  - build



cache:

 key: '$CI_JOB_STAGE-$CI_COMMIT_REF_SLUG'

 paths:

  - '$SOURCE_CODE_DIRECTORY/*/$OBJECTS_DIRECTORY/project.assets.json'

  - '$SOURCE_CODE_DIRECTORY/*/$OBJECTS_DIRECTORY/*.csproj.nuget.*'

  - '$NUGET_PACKAGES_DIRECTORY'



before_script:

  - 'dotnet restore --packages $NUGET_PACKAGES_DIRECTORY'



build:

 stage: build

 script:

  - 'dotnet build --no-restore'