SPIKE: identify additional blocks of code for instrumentation

In order to get an idea of how Okapi performs on a detailed level, the following custom metrics will be useful to have added:

Blocks of code that are suspicious of performing slowly should be measured as well to get concrete data. Currently these include serialization and deserialization time and rate, logging time and rate.

Identify what blocks of code should be decorated with instrumentation.

In the scope of work:

• hazelcast
• serialization
• logging
• CPU utilization (if possible)
  * etc. To be checked with PTF

And contact PTF team to discuss a list of expected data.

[Update]

Below are the top slow methods provided by mtraneis from PTF team. Here is the report. In case the link will expire, I also attached screenshots to show stacktraces for each slow method.

• org.folio.okapi.managers.ProxyService.getModulesForRequest; CPU time - 213161 ms
• org.folio.okapi.managers.ProxyService.match; CPU time - 119636 ms
• com.fasterxml.jackson.databind.ObjectMapper.readValue; CPU time - 104819 ms
• org.apache.logging.log4j.spi.AbstractLogger.info; CPU time - 96760 ms
• org.apache.logging.log4j.spi.AbstractLogger.debug; CPU time - 29073 ms
• com.fasterxml.jackson.core.JsonFactory.createParser; CPU time - 10692 ms
• org.apache.logging.log4j.LogManager.getLogger; CPU time - 4040 ms