Details
-
Bug
-
Status: Closed (View Workflow)
-
P2
-
Resolution: Done
-
None
-
None
-
Concorde - Sprint 112
-
2
-
Concorde
-
R1 2021 Bug Fix
Description
Overview:
After running several initial harvests, edge-oai-pmh returns 503 which is most likely related to the memory leak .
Steps to Reproduce:
- Run several initial requests on a large data set (>6M)
Expected Results:
Harvesting continues
Actual Results:
503 is returned during consecutive run
Additional Information:
Excerpt from the harvester log:
System.Net.WebException: The remote server returned an error: (503) Server Unavailable. at EBSCO.DDP.OAIPMHHarvester.WebClient.GetWebContent(String url, List`1 headers) at EBSCO.DDP.OAIPMHHarvester.Harvester.SaveXmlFromUrl(String url, String filename, List`1 headers, StringBuilder& failedXml, Int32 retries) at EBSCO.DDP.OAIPMHHarvester.Harvester.HarvestRecords(String repositoryUrl, String setSpec, String fromUTCDatetime, String untilUTCDatetime, List`1 headers) 2021-03-09 10:46:03,734 ERROR: Could not harvest from https://edge-concorde5c.int.aws.folio.org/oai/eyJzIjoiQmlzQ2pLZlJ3QyIsInQiOiJmczAwMDAxMDA2IiwidSI6ImZzMDAwMDEwMDYifQ==: Error while harvesting using setSpec 'all'. Will not continue System.Exception: Error while harvesting using setSpec 'all'. Will not continue ---> System.Net.WebException: The remote server returned an error: (503) Server Unavailable. at EBSCO.DDP.OAIPMHHarvester.Harvester.HarvestRecords(String repositoryUrl, String setSpec, String fromUTCDatetime, String untilUTCDatetime, List`1 headers) at EBSCO.DDP.OAIPMHHarvester.Harvester.HarvestRecordBatch(String repositoryUrl, String setSpec, String fromUTCDatetime, String untilUTCDatetime, List`1 headers) at EBSCO.DDP.OAIPMHHarvester.Harvester.Harvest(String repositoryUrl, String fromUTCDatetime, String untilUTCDatetime, List`1 headers) --- End of inner exception stack trace --- at EBSCO.DDP.OAIPMHHarvester.Harvester.Harvest(String repositoryUrl, String fromUTCDatetime, String untilUTCDatetime, List`1 headers) at EBSCO.DDP.OAIPMHHarvester.Program.RunHarvestingDefinitions(String baseDownloadDir, DownloadMode mode, List`1 hDefs, Boolean resumeHarvest
TestRail: Results
Attachments
Issue Links
- blocks
-
EDGOAIPMH-64
Release edge-oai-pmh v2.3.1
-
- Closed
-
- defines
-
UXPROD-2740 Stabilize OAI-PMH functionality
-
- Closed
-