node-devops-docker-demo

Practicing Node Devops with Docker

Devops with Docker and Node.js Youtube Playlist

Resource

• Nginx and Let’s Encrypt with Docker in Less Than 5 Minutes
• Setup Nginx and Let’s Encrypt with Docker
• Docker compose of nginx, express, letsencrypt SSL get 502 Bad gateway

Commands

• Build App Image

  docker build -t node-devops-docker-demo-image .

• Run a Container (port fowarding, bind mount(read-only), and anonymous volume)

  docker run -v $(pwd):/app:ro -v /app/node_modules -p 3000:4000 --env-file ./.env -d --name node-devops-docker-demo node-devops-docker-demo-image

• Access FileSystem

  docker exec -it node-devops-docker-demo bash

• Exit FileSystem

  exit

• Forcefully Remove a container with volume

  docker rm node-devops-docker-demo -fv

• Using Docker Compose

  • Build App

    docker-compose up -d

  • use this if there is a change in image (Dockerfile)

    docker-compose up -d --build

  • Delete App

    docker-compose down -v

  • Development docker-compose command

    • up

      docker-compose -f docker-compose.yml -f docker-compose.dev.yml up -d
      
      # force re-build image for changes
      docker-compose -f docker-compose.yml -f docker-compose.dev.yml up -d --build
      
      # re-build image without downing the container and re creating anonymous volumes
      docker-compose -f docker-compose.yml -f docker-compose.dev.yml up -d --build -V
      
      # scale the number of instances
      docker-compose -f docker-compose.yml -f docker-compose.dev.yml up -d --scale node-app=2

    • down

      docker-compose -f docker-compose.yml -f docker-compose.dev.yml down -v
      
      # don't remove volumes
      docker-compose -f docker-compose.yml -f docker-compose.dev.yml down

  • Production docker-compose command

    • up

      docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d
      
      # force re-build image for changes
      docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build

    • down

      docker-compose -f docker-compose.yml -f docker-compose.prod.yml down -v
      
      # don't remove volumes
      docker-compose -f docker-compose.yml -f docker-compose.prod.yml down

• Open MongoDB directly

  docker exec -it node-devops-docker-demo_mongodb_1 mongo -u "mugil" -p "mypassword"

  note: Don't pass in -v flag to remove the volumes when using databases as it would ultimately remove all volumes, Instead start up the containers and run docker volume prune to remove unused volumes.

• Get network details of container

  docker inspect <container_name>

  note: you can directly use the service name from docker-compose instead of using the ipAddress(works only for the services created/declared in docker-compose).

• Open Redis CLI

  docker exec -it node-devops-docker-demo_redis_1 redis-cli

  • View Session Keys

    KEYS *

  • Get Session Details

    GET <session_key>

Steps to Production

• Create Ubuntu Server on Cloud Platforms like Digital Ocean (I am using this one), AWS, etc..

• Create a Droplet

• Copy the IP Address of the Droplet provided

• Open a terminal in your computer and SSH into the droplet

  ssh root@<ip_address_of_droplet_here>

• Type in yes to conntinue connecting

• Type in your root password that you used while creating the droplet

• Install docker engine in the droplet

  curl -fsSL https://get.docker.com -o get-docker.sh
  
  sh get-docker.sh

• Install docker-compose by following instructions in the official documentation for linux

  sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
  
  sudo chmod +x /usr/local/bin/docker-compose

• Set Environment Variables

  export <key>=<value>

• Show all Env variables

  printenv

• Instead of individually adding env variables, setup .env file and add the .env variables into it

  vi .env

• Open .profile and add this at the bottom

  vi .profile

  <!-- Removed the lines for brevity -->
  
  set -o allexport; source /root/.env; set +o allexport 

• Create a directory and move into it

  mkdir app
  
  cd app

• Clone the Git repo in this folder and pull whenever there is a change

  git clone <git_repo_url> .

  Note: This works with public repo. If you want to work with private repos, check Deploy Keys section or How to clone your private repository from GitHub to server, Droplet, VDS, etc?

• Run the production docker-compose command

  docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d
  
  # do this to see any changes made
  docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build
  
  # rebuild only necessary service to avoid checking other containers unneccessarily
  docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build node-app
  
  # when you don't want to check for dependencies
  docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build --no-deps node-app
  
  # force rebuild containers even when there is no change
  docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d --force-recreate node-app
  
  # force rebuild containers even when there is no change without dependecies
  docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d --force-recreate --no-deps node-app

• Make requests to http://<ip_address_of_the_droplet>/ routes

• Push Images to docker to avoid building images on production server. Create a repo in dockerhub

  • Tag the local image with the name of that of the remote image

    docker tag <local_image> <username/remote_dockerhub_repo_name>    

  • Push the tagged image to dockerhub

    docker push <username/remote_dockerhub_repo_name>

• Build the image on local machine

  docker-compose -f docker-compose.yml -f docker-compose.prod.yml build
  
  # build only the services that is required (custom image)
  docker-compose -f docker-compose.yml -f docker-compose.prod.yml build node-app

• Push the custom build image to dockerhub

  docker-compose -f docker-compose.yml -f docker-compose.prod.yml push
  
  # push only the services that we want
  docker-compose -f docker-compose.yml -f docker-compose.prod.yml push node-app

• Pull the latest images in the production server

  docker-compose -f docker-compose.yml -f docker-compose.prod.yml pull
  
  # only specific image
  docker-compose -f docker-compose.yml -f docker-compose.prod.yml pull node-app

• Update the changes in producton server

  docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d
  
  # specific rebuild
  docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d --no-deps node-app

• Use watchtower to automatically look for changes in the remote custom build image (specify the images that needs to be watched at the end)

  docker run -d --name watchtower -e WATCHTOWER_TRACE=true -e WATCHTOWER_DEBUG=true -e WATCHTOWER_POLL_INTERVAL=50 -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower app_node-app_1

  NOTE: No need to pull and update changes in production server if watchtower is used

• Activate docker swarm for orchestration. Get IP of the droplet by ip add command

  docker swarm init --advertise-addr <public_ip>

  • Add more manager nodes, run this command and follow the instructions

    docker swarm join-token manager

  • Add worker node to swarm

    docker swarm join --token <token_provided> <ip>:<port>

• Deploy

  docker stack deploy -c docker-compose.yml -c docker-compose.prod.yml myapp

  • check how many nodes are running

    docker node ls

  • check how many stacks are there

    docker stack ls

  • list the services in the stack

    docker stack services myapp

  • list all the services across all stacks

    docker service ls

  • list the tasks in the stack

    docker stack ps myapp

TODO

• Add SSL Certificate (https)

• Add Domain Name

• Modify ./init-letsencrypt.sh to use docker swarm and production docker compose

• Run the modified ./init-letsencrypt.sh on the server for first time to generate fake certificates

• Everything must be automated

Docker Images

• node
• mongo
• redis
• nginx

DOCS

• Express behind proxies
• Docker Script for Ubuntu
• Install Docker Compose
• Watchtower
• Docker Compose Deploy Reference