Uploaded image for project: 'FOLIO'
  1. FOLIO
  2. FOLIO-1804

Reference environment builds fail due to apparent timeout

    XMLWordPrintable

    Details

    • Template:
    • Development Team:
      Core: Platform

      Description

      Apologies for the rubbish title, please amend as we know more information.

      Symptoms

      Snapshot

      RUNNING HANDLER [folio-ansible/roles/okapi-deploy-config : Build tenant parameter list] ***
      ok: [10.36.1.64] => (item={'value': [{u'name': u'loadSample', u'value': u'true'}], 'key': u'mod-orders-storage'})
      ok: [10.36.1.64] => (item={'value': [{u'name': u'loadReference', u'value': u'true'}], 'key': u'mod-inventory-storage'})
      
      RUNNING HANDLER [folio-ansible/roles/okapi-deploy-config : Discard duplicates in tenant parameter list] ***
      ok: [10.36.1.64]
      
      RUNNING HANDLER [folio-ansible/roles/okapi-deploy-config : Convert tenant parameter list to query string] ***
      ok: [10.36.1.64] => (item={u'name': u'loadSample', u'value': u'true'})
      ok: [10.36.1.64] => (item={u'name': u'loadReference', u'value': u'true'})
      
      RUNNING HANDLER [folio-ansible/roles/okapi-deploy-config : Perform module installation] ***
      FATAL: command execution failed
      hudson.AbortException: Ansible playbook execution failed
      	at org.jenkinsci.plugins.ansible.AnsiblePlaybookBuilder.perform(AnsiblePlaybookBuilder.java:262)
      

      Testing

      TASK [folio-ansible/roles/okapi-tenant-deploy : Post install list for deployment and enabling] ***
      fatal: [10.36.1.212]: FAILED! => {"changed": false, "content": "", "msg": "Status code was -1 and not [200]: Connection failure: timed out", "redirected": false, "status": -1, "url": "http://10.36.1.212:9130/_/proxy/tenants/diku/install?deploy=true&tenantParameters=loadReference%3Dtrue%2CloadSample%3Dtrue"}
      	to retry, use: --limit @/home/jenkins/workspace/Automation/folio-testing-backend01/CI/ansible/folio-testing-backend.retry
      

      Environments Affected

      It did not seem to affect the folio-snapshot-core build

      This also affected the Vagrant builds for snapshot and testing backend. The snapshot core backend build failure seems unrelated (but I'm far from sure).

      Initial Investigation

      I was unable to discover much from the error messages in the builds except maybe that a long running operation timed out or was aborted.

      It might happen during the activation of modules that load reference or sample records during tenant activation, based upon this snippet:

      RUNNING HANDLER [folio-ansible/roles/okapi-deploy-config : Build tenant parameter list] ***
      ok: [10.36.1.64] => (item={'value': [{u'name': u'loadSample', u'value': u'true'}], 'key': u'mod-orders-storage'})
      ok: [10.36.1.64] => (item={'value': [{u'name': u'loadReference', u'value': u'true'}], 'key': u'mod-inventory-storage'})
      
      RUNNING HANDLER [folio-ansible/roles/okapi-deploy-config : Discard duplicates in tenant parameter list] ***
      ok: [10.36.1.64]
      
      RUNNING HANDLER [folio-ansible/roles/okapi-deploy-config : Convert tenant parameter list to query string] ***
      ok: [10.36.1.64] => (item={u'name': u'loadSample', u'value': u'true'})
      ok: [10.36.1.64] => (item={u'name': u'loadReference', u'value': u'true'})
      

      Folio-snapshot-core seems unaffected. Given that builds differently that might narrow down what could have caused that.

      I briefly investigated the recent changes to mod-inventory-storage, mod-orders-storage and folio-ansible but didn't spot anything obvious from yesterday that might have caused this.

      I've repeated the builds to check whether it is transient.

        TestRail: Results

          Attachments

            Issue Links

              Activity

                People

                Assignee:
                malc John Malconian
                Reporter:
                marcjohnson Marc Johnson
                Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                  Dates

                  Created:
                  Updated:
                  Resolved:

                    TestRail: Runs

                      TestRail: Cases