Uploaded image for project: 'edge-oai-pmh'
  1. edge-oai-pmh
  2. EDGOAIPMH-61

503 status returned after several initial harvests

    XMLWordPrintable

Details

    • Bug
    • Status: Closed (View Workflow)
    • P2
    • Resolution: Done
    • None
    • 2.3.1
    • None
    • Concorde - Sprint 112
    • 2
    • Concorde
    • R1 2021 Bug Fix

    Description

      Overview:
      After running several initial harvests, edge-oai-pmh returns 503 which is most likely related to the memory leak .

      Steps to Reproduce:

      1. Run several initial requests on a large data set (>6M)
        Expected Results:
        Harvesting continues
        Actual Results:
        503 is returned during consecutive run
        Additional Information:
        Excerpt from the harvester log:
      System.Net.WebException: The remote server returned an error: (503) Server Unavailable.
   at EBSCO.DDP.OAIPMHHarvester.WebClient.GetWebContent(String url, List`1 headers)
   at EBSCO.DDP.OAIPMHHarvester.Harvester.SaveXmlFromUrl(String url, String filename, List`1 headers, StringBuilder& failedXml, Int32 retries)
   at EBSCO.DDP.OAIPMHHarvester.Harvester.HarvestRecords(String repositoryUrl, String setSpec, String fromUTCDatetime, String untilUTCDatetime, List`1 headers)
2021-03-09 10:46:03,734 ERROR: Could not harvest from https://edge-concorde5c.int.aws.folio.org/oai/eyJzIjoiQmlzQ2pLZlJ3QyIsInQiOiJmczAwMDAxMDA2IiwidSI6ImZzMDAwMDEwMDYifQ==: Error while harvesting using setSpec 'all'. Will not continue
System.Exception: Error while harvesting using setSpec 'all'. Will not continue ---> System.Net.WebException: The remote server returned an error: (503) Server Unavailable.
   at EBSCO.DDP.OAIPMHHarvester.Harvester.HarvestRecords(String repositoryUrl, String setSpec, String fromUTCDatetime, String untilUTCDatetime, List`1 headers)
   at EBSCO.DDP.OAIPMHHarvester.Harvester.HarvestRecordBatch(String repositoryUrl, String setSpec, String fromUTCDatetime, String untilUTCDatetime, List`1 headers)
   at EBSCO.DDP.OAIPMHHarvester.Harvester.Harvest(String repositoryUrl, String fromUTCDatetime, String untilUTCDatetime, List`1 headers)
   --- End of inner exception stack trace ---
   at EBSCO.DDP.OAIPMHHarvester.Harvester.Harvest(String repositoryUrl, String fromUTCDatetime, String untilUTCDatetime, List`1 headers)
   at EBSCO.DDP.OAIPMHHarvester.Program.RunHarvestingDefinitions(String baseDownloadDir, DownloadMode mode, List`1 hDefs, Boolean resumeHarvest

      TestRail: Results

        Attachments

          Issue Links

            blocks

            Story - Created by Jira Software - do not edit or delete. Issue type for a user story. EDGOAIPMH-64 Release edge-oai-pmh v2.3.1

            • P3 - low priority level, item will be considered for inclusion in the next dev cycle
            • Closed
            defines

            New Feature - New feature requests UXPROD-2740 Stabilize OAI-PMH functionality

            • P2 - normal priority level, must be included in the current development cycle
            • Closed

            Activity

              People

                TotoroDev Illia Daliek
                magdaz Magda Zacharska
                Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                  Created:
                  Updated:
                  Resolved:

                  TestRail: Runs

                    TestRail: Cases