When I installed modules with `npm install` or `bundle install` inside my Dockerfile they were not present in the steps on codeshipt-steps.yml

docker
codeship_pro
rails
nodejs

#1

I found a work around but I’d like if someone could explain to me this:

I have an application with Rails and React, so I have my Rspec specs and my Jest specs.

Inside my Dockerfile I have:

RUN bundle install
`RUN yarn install

The weird think it’s that when I run rspec or jest in my codeship-steps.yml the gems and modules loaded during the build are no longer installed, so I had to execute a pre step to install my dependencies.

Could it be related to volumes?


#2

Do you have any volumes in your Codeship services file? If you do, then during runtime they can over ride what was built.

If you add something like - path/to/node_modules as an additional volume with no host mapping, it should fix that.

The gems shouldn’t be affected, however, and there may be another issue there.


#3

Hey Walter, were you able to solve this? If not, would you mind sharing your Dockerfile, as well as the codeship-services.yml and codeship-steps.yml files with us?

If you don’t want to share them via the forum, feel free to open a ticket on https://helpdesk.codeship.com and we’ll see how we can fix this.

As Kelly already mentioned, volumes can definitely affect this, especially if you’re mounting your apps source via a volume.


#4

I did not solved it, but since I only need to run bundle install once for rspec I decided to do it before I set up the database.

npm install it’s a different story because as I understand when I run it in the Dockerfile it will get overwritten when I copy my code to the container.

codeship-services.yml

version: "2"

services:
  db:
    image: postgres:9.5.3
    environment:
      LC_ALL: C.UTF-8
      POSTGRES_PASSWORD: 12341234
    cached: true

  test:
    build:
      context: .
      dockerfile: Test.Dockerfile
    command: foreman start -f Procfile.test
    working_dir: /usr/src/app
    volumes:
      - .:/usr/src/app
    stdin_open: true
    tty: true
    links:
      - db
    depends_on:
      - db
    environment:
      DATABASE_URL: "postgres://postgres:12341234@db:5432/database_test"
      POSTGRESQL_USER: postgres
      POSTGRESQL_PASSWORD: 12341234
      RAILS_ENV: test
      RACK_ENV: test
      NODE_ENV: test
    env_file:
      - test.env
    cached: true

codeship-steps.yml

- type: parallel
  steps:
  - type: serial
    name: rails
    steps:
      - name: rspec
        service: test
        command: bin/ci rspec
  - type: serial
    name: webpack
    steps:
      - name: npm install
        service: test
        command: npm install --quiet
      - name: jest
        service: test
        command: npm test

bin/ci

#!/bin/sh
# Usage: bin/ci [setup]
set -e

bin/wait-for-postgres

bundle exec rake db:create
bundle exec rake db:migrate
bundle exec rake db:seed

bundle exec rspec $2

#5

@mlocher having the same experience here. Simple node based build step then deploy via aws-cli.

It feels like I should be able to just run yarn and yarn blendid build (that just generates a “public” folder) inside my Dockerfile, but if I ever attempt then the files seem to not persist.

The weird thing is that when running yarn from codeship-steps.yml the node_modules are cached somewhere, because the bundle process takes no time at all…

#codeship-services.yml
services:
  app:
    build:
      dockerfile: Dockerfile
    volumes_from:
      - data
  awsdeployment:
    image: codeship/aws-deployment
    encrypted_env_file: env.encrypted
    environment:
      - AWS_DEFAULT_REGION=ap-southeast-2
    volumes_from:
      - data
  data:
    image: busybox
    volumes:
      - .:/app
#codeship-steps.yml
- name: setup
  service: app
  command: yarn

- name: build
  service: app
  command: yarn blendid build

# - name: list build files
#   service: app
#   command: ls -la

# - name: list files
#   service: awsdeployment
#   command: ls -la /app

- name: deploy
  service: awsdeployment
  command: aws s3 sync /app/public s3://static-deploy-test
#Dockerfile
FROM mhart/alpine-node:9.3.0

# Create app directory
WORKDIR /app

# Install app dependencies
COPY package.json .
RUN yarn

# Bundle app src
COPY . .

# whether or not we run this here, we still have to run it in
# codeship-steps.yml, otherwise it will not persist?
# RUN yarn blendid build

#6

This is caused by a combination of copying the files into the image generated by building the Dockerfile, and then mounting the cloned repository when actually running your steps.

In your Dockerfile you switch COPY the package.json and run yarn (which implies the install subcommand). If you also specify cached: true via your codeship-services.yml file, the generated Docker image is cached by Codeship, which would explain the fast build times. If you also uncomment the RUN yarn blendid build step, the built artifacts would also be added to the Docker image and would be available during your build.

However, since you also specify a volume via your services file, that is mounted at the same /app path and mounted volumes take precedence over any files persisted via the Docker image, both the yarn [install] and currently commented out yarn blendid build steps are not taking effect.

Basically the contents of /app as modified by the commands in your Dockerfile are replaced by the pristine state as contained in your cloned repository.

To solve this, either rely on the volume to share those changes with future steps (and then run both yarn commands via the codeship-steps.yml file), or rely on the Dockerfile to run those commands, and then have a step copy the contents you want to deploy to a (separate) directory that is mounted as a volume.

For an example of the latter, take a look out our documentation repository. We rely on the Dockerfile to install required dependencies and work out of the /docs directory. For deployment we then copy the (in this case mostly HTML files) to the /site directory, which is mounted via a volume and shared with the actual deployment container.


#7

@mlocher I ended up moving the build steps to codeship-steps, thank you for the great reply as it makes perfect sense and is great to better understand the WHY behind the scenes!

giphy-downsized


#8

Awesome, that’s great to hear. Let me know if you have any further questions and happy shipping :slight_smile: