jest: Fresh jenkins installation crash when running jest.

Bug request

Bug

Current behavior

Using the official docker image of jenkins/ v2.46.2.

I have a webpack project that use jest-cli@19.0.2 that contain more than 500 hundreds of enzyme test mainly.

Reproduction

  1. Run jenkins (MUST have port 8080 free or change the host port)

     docker run -p 8080:8080 -v /var/jenkins_home jenkins
    
  2. Print the secret key to be inserted on install

     docker exec -i $(docker ps | grep jenkins | awk -F ' ' '{print $1}') cat /var/jenkins_home/secret.key
    
  3. Past the key on http://localhost:8080 and complete the jenkins installation

  4. Load from a CVS a project that use jest-cli 19.0.2 and contains around 500 enzyme (mount/shallow) unit test.

  5. Configure a new project on Jenkins and import project from CVS

  6. Create a build terminal step with npm install and npm test

With a fresh installation and one project, jenkins crash during the build project.

This is my hardware configuration, I don’t think it is the issue here

CPU
Intel(R) Xeon(R) CPU D-1540 @ 2.00GHz
Cores : 16
Cache : 12288KB
RAM
2x 16384MB
Disks
2 x 2000 GB
Motherboard
X10SDV-TLN4F
Kernel version
3.14.32-xxxx-grs-ipv6-64
node version 7.9.0

Expected behavior

jenkins should perform the jest command to the end.

About this issue

  • Original URL
  • State: closed
  • Created 7 years ago
  • Comments: 15 (3 by maintainers)

Most upvoted comments

Are you running the tests with the Jest --runInBand cli option? Else can try that.

I apologize for the tutorial, and would like to thanks in advance the person who will achieve it.

I am not confident that test will crash in your environment if you are not running your container on mesos/marathon.

In case you just wan’t to see the crash , I have recorded it with my smartphone, it’s not a professional footage but you can see it :

https://drive.google.com/file/d/0B6drRJ3_BeQSMVh2RWE4cnc2NUU/view

Prerequesite

In order to be exactly with the same configuration, you will need Host 32H offer at OVH.

CPU
Intel(R) Xeon(R) CPU D-1540 @ 2.00GHz
Cores : 16
Cache : 12288KB
RAM
2x 16384MB
Disks
2 x 2000 GB
Motherboard
X10SDV-TLN4F
Kernel version
3.14.32-xxxx-grs-ipv6-64

  1. Install Debian Jessie. (can be skipped but potentially you won’t get the error)
  2. Install Mesos/Marathon. (can be skipped but potentially you won’t get the error)
  3. Install docker-engine 1.13.1.

During my test, I use the following configuration with Debian Jessie 8.0:

Jenkins installation

  1. Download and run jenkins official image. From a separate bash: (you must have port 8080 available)

     docker run -p 8080:8080 jenkins
    
  2. You will see in your terminal password. Copy it.

     Jenkins initial setup is required. An admin user has been created and a password generated.
     Please use the following password to proceed to installation:
    
     39ae474fe5544c00881167cb8764cb14
    
     This may also be found at: /var/jenkins_home/secrets/initialAdminPassword
    

Or this will print the content of /var/jenkins_home/secrets/initialAdminPassword:

    docker exec -ti $(docker ps | grep jenkins | head -1 | awk -F ' ' '{print $1}') cat /var/jenkins_home/secrets/initialAdminPassword
  1. Now open chrome http://localhost:8080/ and past the code

image

  1. Click on Select plugin to install

image

  1. Go to manage jenkins > manage plugin and complete the installation by checking for installation the following plugins :
  • NodeJS

image

  • Git plugin

image

Validate the change

  1. Wait for the download and installation of the jenkins plugins :

image image

  1. Configure the administrator account as follow

image

  1. You will arrive on the jenkins dashboard

image

  1. Click on “Manage Jenkins”

image

  1. Now click on “Global Tool Configuration”

image

  1. Configure a NodeJS version 7.9.0

image

  1. Click on “New Item”

image

  1. Name it and choose “Freestyle project”

image

  1. Under “Source Code Management”, select the radio “git” and add the following repository url : https://github.com/kopax/jest-crashing-jenkins

image

  1. Under “Build Environment” check “provide Node & npm bin/ folder to path” and select the version “7.9.0” of NodeJS.

image

  1. Under “Build”, click on “Execute shell”

image

  1. Enter “npm install” and “npm test”, then click “Save”

image

  1. Now click on Build now

image

  1. There will be a new job added. You can click on it and then click on “Console Output”

image

  1. While running the build, you can use htop to view your memory usage (sudo apt-get update && sudo apt-get install htop -y)

image

  1. Also you should note the CONTAINER_ID so you will see when it crash

    docker ps | grep jenkins
    abf10c67f072        jenkins                 "/bin/tini -- /usr/lo"   About an hour ago   Up About an hour    0.0.0.0:8080->8080/tcp, 50000/tcp   reverent_saha
    

I wasn’t able to reproduce the crash it except in my production environment.

This is how my test fail in production :

image

Notice the Jobs id 48, because when Jenkins will come back, it will miss that ID and jenkins will do like nothing happened before.

Tips 1: If you wish to have a more verbose log level when doing logs of your container you can do the following withing the containers, I have already checked and there is nothing interesting into:

 docker exec -it $(docker ps | grep jenkins | head -1 | awk -F ' ' '{print $1}') cat > /var/jenkins_home/log.properties <<EOF
handlers=java.util.logging.ConsoleHandler
jenkins.level=FINEST
java.util.logging.ConsoleHandler.level=FINEST
EOF

And then restart your container with :

 docker stop $(docker ps | grep jenkins | head -1 | awk -F ' ' '{print $1}')
 docker start $(docker ps | grep jenkins | head -1 | awk -F ' ' '{print $1}') --env JAVA_OPTS="-Djava.util.logging.config.file=/var/jenkins_home/log.properties"

Tips 2: If you want to upgrade jenkins to the latest version (see the warning), you can activate a hidden button “Upgrade jenkins automatically button”. For that, you need to give the good permissions to /usr/share/jenkins within the container, this will make the button appear in the “Jenkins Configuration” page and you will then be able to upgrade Jenkins

    docker exec -it --user root $(docker ps | grep jenkins | head -1 | awk -F ' ' '{print $1}') chown jenkins:jenkins -R /usr/share/jenkins

Note that I am running my Jenkins container on Mesos/Marathon and I only have the issue here. As attachement this is the marathon deployement json I use for deploying my jenkins container:

{
  "id": "developers/jenkins",
  "cmd": null,
  "mem": 4096,
  "cpus": 2,
  "instances": 1,
  "container": {
    "type": "DOCKER",
    "docker": {
      "image": "jenkins:latest",
      "forcePullImage": true,
      "network": "BRIDGE",
      "portMappings": [
        {
          "containerPort": 8080,
          "hostPort": 0,
          "protocol": "tcp"
        }
      ]
    },
    "volumes": [
      {
        "containerPath": "/var/jenkins_home",
        "hostPath": "/srv/developers/jenkins/var/jenkins_home",
        "mode": "RW"
      },
      {
        "containerPath": "/var/run/docker.sock",
        "hostPath": "/var/run/docker.sock",
        "mode": "RW"
      },
      {
        "containerPath": "/usr/bin/docker",
        "hostPath": "/usr/bin/docker",
        "mode": "RW"
      },
      {
        "containerPath": "/lib/x86_64-linux-gnu/libapparmor.so.1",
        "hostPath": "/usr/lib/x86_64-linux-gnu/libapparmor.so.1",
        "mode": "RW"
      }
    ]
  },
  "healthChecks": [
    {
      "portIndex": 0,
      "protocol": "TCP",
      "gracePeriodSeconds": 30,
      "intervalSeconds": 10,
      "timeoutSeconds": 30,
      "maxConsecutiveFailures": 3
    }
  ],
  "upgradeStrategy": {
      "minimumHealthCapacity": 1,
      "maximumOverCapacity": 0.1
  }
}

It’s only in mesos where the container get killed with an exit code 137 :

processname:marathon groupname:marathon pid:19 channel:stdout
[2017-04-28 20:21:48,630] INFO Received status update for task kopaxgroup_developers_jenkins-2.dc2b0461-2c4d-11e7-9190-92077ff7c44b: TASK_FAILED (Docker container run error: Container exited on error: exited with status 137) (mesosphere.marathon.MarathonScheduler$$EnhancerByGuice$$a504fd7e:Thread-1442)