The road to
devops

 

(A CICD post-mortem)

  • Dad of a 8 year old

  • Senior Developer @ Ops Management Team

  • ~10 years exp as a developer
2

Terrance Smith

outline

  • Where we started (the setup)
     
  • The Road (the conflict)
     
  • Where we stand (the payoff?)

Where we started

Our Back-end

  • Asp.Net Web API 2 (Service)
     
  • DynamoDb (DB)
     
  • Psake (Task Runner)
     
  • Gis.StratusBuild

 

Our Front-end

  • Angular 4*
     
  • Angular Cli
     
  • node\npm 

What could possibly go wrong?

the road

First we met jenkins

Road Bumps

  • Lack of Jenkins Knowledge
  • Lack of  groovy knowledge
  • VSTS Authentication Issues
  • Permissions\Access Issues for Remote Team (ongoing)

jenkins Lessons learned

  • Setup email notifications ASAP for remote folks
     
  • Navigation of the UI is Essential
     
  • Be picky about plugins
     
  • Test run locally where possible (setup a Jenkins server on your box)

jenkins Lessons learned 2

  • Filter your branches as much as possible for improved performance
     
  • Use the "Replay" feature in the Build to test changes to your jenkinsfile
     
  • Because you will live in the Console Output, you should log judiciously.

then we paid the aws toll

Cost of Crossing

  • Cloud Formation Template knowledge
     
  • Overwhelming number of tools
     
  • Acceptance that you will not remote into every box you work on

Aws Lessons learned

  • Reminder: You should treat your servers as cattle, not as pets.
     
  • Be prepared to spend a log of time tweaking Cloud Formation Templates
     
  • Leverage CodeDeploy and  CloudFormation Logs where possible to troubleshoot deployment issues 

Aws Lessons learned 2

  • If your data requires complex filtering\conditionals then consider a traditional db
     
  • If your data is lightweight and simply structured and required little configuration then consider DynamoDB
     
  • Use the AWS Cli for all configuration tokens\keys etc...

Then we hit .net build ROADBLOCKS

Road Blocks

  • Incorrect versions of msbuild
  • Invalid\Missing Web Targets
  • Binding Redirect Errors
  • Environment Inconsistencies 
  • Swagger Configuration Errors
  • Mixing Net Standard 1.4-2.0 code with Full Framework

.net build Lessons learned

  • Do not update nuget assemblies carelessly
     
  • Do not mix .net standard versions
     
  • YAGNI on refs; Remove what you don't need

.net build Lessons learned

  • Verify your build version is the same everywhere
     
  • Consider using Paket to lock down build versions
     
  • We are still trying to figure this one out.... :(
     

finally we went offroad front-end 

Off Road Action

  • Scripting integrations
  • IIS Configuration for static js 
  • npm version issues
  • Ng Tool versions
  • npm Global conflicts
  • Sass compiler issues

Front-end Lessons learned

  • Embrace angular cli tooling
     
  • Rely on local node_modules where possible
     
  • Avoid installing global npm packages on build server

  •  

Front-end Lessons learned 2

  • Plan for non-trivial effort when upgrading versions of angular and cli
  • Consider using Angular Cli Environment variables for use with deployment
     
  • Consider Static Hosting on IIS vs Serving via asp.net
     
  • Build configuration management tooling into application for fewer endpoint config headaches

Where we stand

Summary