Uploaded image for project: 'mod-source-record-manager'
  1. mod-source-record-manager
  2. MODSOURMAN-98

Memory leak in mod-source-record-manager

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: P2
    • Resolution: Done
    • Affects Version/s: 1.2.0, 1.2.1
    • Fix Version/s: 1.3.0
    • Labels:
    • Template:
    • Sprint:
      EPAM BatchLoader Sprint 14, EPAM BatchLoader Sprint 15
    • Story Points:
      8
    • Development Team:
      Folijet

      Description

      When file with 30,000 record is uploaded local env breaks down:

      16:18:38.815 [vert.x-eventloop-thread-0] ERROR org.folio.dataimport.util.ExceptionHelper - Java heap space
      java.lang.OutOfMemoryError: Java heap space
              at java.util.Arrays.copyOfRange(Arrays.java:3664) ~[?:1.8.0_181]
              at java.lang.String.<init>(String.java:207) ~[?:1.8.0_181]
              at java.lang.StringBuilder.toString(StringBuilder.java:407) ~[?:1.8.0_181]
              at com.fasterxml.jackson.core.util.TextBuffer.contentsAsString(TextBuffer.java:404) ~[mod-source-record-manager-server-fat.jar:?]
              at com.fasterxml.jackson.core.io.SegmentedStringWriter.getAndClear(SegmentedStringWriter.java:83) ~[mod-source-record-manager-server-fat.jar:?]
              at com.fasterxml.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:3225) ~[mod-source-record-manager-server-fat.jar:?]
              at org.folio.rest.tools.ClientHelpers.pojo2json(ClientHelpers.java:25) ~[mod-source-record-manager-server-fat.jar:?]
              at org.folio.rest.client.SourceStorageClient.postSourceStorageRecords(SourceStorageClient.java:361) ~[mod-source-record-manager-server-fat.jar:?]
              at org.folio.services.ChangeEngineServiceImpl.postRecord(ChangeEngineServiceImpl.java:152) ~[mod-source-record-manager-server-fat.jar:?]
              at org.folio.services.ChangeEngineServiceImpl.lambda$parseRawRecordsChunkForJobExecution$0(ChangeEngineServiceImpl.java:54) ~[mod-source-record-manager-server-fat.jar:?]
              at org.folio.services.ChangeEngineServiceImpl$$Lambda$323/560974523.accept(Unknown Source) ~[?:?]
              at java.util.ArrayList.forEach(ArrayList.java:1257) ~[?:1.8.0_181]
              at org.folio.services.ChangeEngineServiceImpl.parseRawRecordsChunkForJobExecution(ChangeEngineServiceImpl.java:54) ~[mod-source-record-manager-server-fat.jar:?]
              at org.folio.services.ChunkProcessingServiceImpl.lambda$processChunk$2(ChunkProcessingServiceImpl.java:43) ~[mod-source-record-manager-server-fat.jar:?]
              at org.folio.services.ChunkProcessingServiceImpl$$Lambda$301/534015602.apply(Unknown Source) ~[?:?]
              at io.vertx.core.Future.lambda$compose$1(Future.java:265) ~[mod-source-record-manager-server-fat.jar:?]
              at io.vertx.core.Future$$Lambda$253/171066894.handle(Unknown Source) ~[?:?]
              at io.vertx.core.impl.FutureImpl.tryComplete(FutureImpl.java:125) ~[mod-source-record-manager-server-fat.jar:?]
              at io.vertx.core.impl.FutureImpl.complete(FutureImpl.java:86) ~[mod-source-record-manager-server-fat.jar:?]
              at io.vertx.core.impl.FutureImpl.handle(FutureImpl.java:151) ~[mod-source-record-manager-server-fat.jar:?]
              at io.vertx.core.impl.FutureImpl.handle(FutureImpl.java:18) ~[mod-source-record-manager-server-fat.jar:?]
              at io.vertx.core.impl.FutureImpl.tryComplete(FutureImpl.java:125) ~[mod-source-record-manager-server-fat.jar:?]
              at io.vertx.core.impl.FutureImpl.complete(FutureImpl.java:86) ~[mod-source-record-manager-server-fat.jar:?]
              at io.vertx.core.impl.FutureImpl.handle(FutureImpl.java:151) ~[mod-source-record-manager-server-fat.jar:?]
              at io.vertx.core.impl.FutureImpl.handle(FutureImpl.java:18) ~[mod-source-record-manager-server-fat.jar:?]
              at io.vertx.core.impl.SucceededFuture.setHandler(SucceededFuture.java:40) ~[mod-source-record-manager-server-fat.jar:?]
              at io.vertx.core.Future.lambda$compose$1(Future.java:270) ~[mod-source-record-manager-server-fat.jar:?]
              at io.vertx.core.Future$$Lambda$253/171066894.handle(Unknown Source) ~[?:?]
              at io.vertx.core.impl.FutureImpl.tryComplete(FutureImpl.java:125) ~[mod-source-record-manager-server-fat.jar:?]
              at io.vertx.core.impl.FutureImpl.complete(FutureImpl.java:86) ~[mod-source-record-manager-server-fat.jar:?]
              at io.vertx.core.Future.lambda$map$2(Future.java:306) ~[mod-source-record-manager-server-fat.jar:?]
              at io.vertx.core.Future$$Lambda$210/383799480.handle(Unknown Source) ~[?:?]
      

      folio-testing.aws.indexdata.com breaks down with

      15:13:47.900 [vert.x-eventloop-thread-0] ERROR org.folio.dataimport.util.ExceptionHelper - GC overhead limit exceeded
      java.lang.OutOfMemoryError: GC overhead limit exceeded
      	at java.lang.StringCoding$StringEncoder.encode(StringCoding.java:300) ~[?:1.8.0_181]
      	at java.lang.StringCoding.encode(StringCoding.java:344) ~[?:1.8.0_181]
      	at java.lang.String.getBytes(String.java:918) ~[?:1.8.0_181]
      	at org.marc4j.MarcJsonWriter.write(MarcJsonWriter.java:479) ~[mod-source-record-manager-server-fat.jar:?]
      	at org.folio.services.parsers.MarcRecordParser.parseRecord(MarcRecordParser.java:37) ~[mod-source-record-manager-server-fat.jar:?]
      	at org.folio.services.ChangeEngineServiceImpl.lambda$parseRecords$6(ChangeEngineServiceImpl.java:102) ~[mod-source-record-manager-server-fat.jar:?]
      	at org.folio.services.ChangeEngineServiceImpl$$Lambda$331/1588658038.apply(Unknown Source) ~[?:?]
      	at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_181]
      	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) ~[?:1.8.0_181]
      	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_181]
      	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_181]
      	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_181]
      	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_181]
      	at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:1.8.0_181]
      	at org.folio.services.ChangeEngineServiceImpl.parseRecords(ChangeEngineServiceImpl.java:128) ~[mod-source-record-manager-server-fat.jar:?]
      	at org.folio.services.ChangeEngineServiceImpl.parseRawRecordsChunkForJobExecution(ChangeEngineServiceImpl.java:53) ~[mod-source-record-manager-server-fat.jar:?]
      	at org.folio.services.ChunkProcessingServiceImpl.lambda$processChunk$2(ChunkProcessingServiceImpl.java:43) ~[mod-source-record-manager-server-fat.jar:?]
      	at org.folio.services.ChunkProcessingServiceImpl$$Lambda$313/1586559089.apply(Unknown Source) ~[?:?]
      	at io.vertx.core.Future.lambda$compose$1(Future.java:265) ~[mod-source-record-manager-server-fat.jar:?]
      	at io.vertx.core.Future$$Lambda$272/1555584168.handle(Unknown Source) ~[?:?]
      	at io.vertx.core.impl.FutureImpl.tryComplete(FutureImpl.java:125) ~[mod-source-record-manager-server-fat.jar:?]
      	at io.vertx.core.impl.FutureImpl.complete(FutureImpl.java:86) ~[mod-source-record-manager-server-fat.jar:?]
      	at io.vertx.core.impl.FutureImpl.handle(FutureImpl.java:151) ~[mod-source-record-manager-server-fat.jar:?]
      	at io.vertx.core.impl.FutureImpl.handle(FutureImpl.java:18) ~[mod-source-record-manager-server-fat.jar:?]
      	at io.vertx.core.impl.FutureImpl.tryComplete(FutureImpl.java:125) ~[mod-source-record-manager-server-fat.jar:?]
      	at io.vertx.core.impl.FutureImpl.complete(FutureImpl.java:86) ~[mod-source-record-manager-server-fat.jar:?]
      	at io.vertx.core.impl.FutureImpl.handle(FutureImpl.java:151) ~[mod-source-record-manager-server-fat.jar:?]
      	at io.vertx.core.impl.FutureImpl.handle(FutureImpl.java:18) ~[mod-source-record-manager-server-fat.jar:?]
      	at io.vertx.core.impl.SucceededFuture.setHandler(SucceededFuture.java:40) ~[mod-source-record-manager-server-fat.jar:?]
      	at io.vertx.core.Future.lambda$compose$1(Future.java:270) ~[mod-source-record-manager-server-fat.jar:?]
      	at io.vertx.core.Future$$Lambda$272/1555584168.handle(Unknown Source) ~[?:?]
      	at io.vertx.core.impl.FutureImpl.tryComplete(FutureImpl.java:125) ~[mod-source-record-manager-server-fat.jar:?]
      

        TestRail: Results

          Attachments

            Issue Links

              Activity

                People

                Assignee:
                Kateryna Senchenko Kateryna Senchenko
                Reporter:
                Kateryna Senchenko Kateryna Senchenko
                Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                  Dates

                  Created:
                  Updated:
                  Resolved:

                    TestRail: Runs

                      TestRail: Cases