Skip to content

Java analysis optimization tuning on OpenPOWER

Ashish Kumar edited this page Dec 19, 2016 · 1 revision
  • System Performance Analysis

        1) **Using SAR Utility to collect performance input of system** 
                  
           -> sar collects a lot of data with little cpu utilization (usually less than 1%).  The information sar collects includes cpu, memory, network, disk io, run queue, ....   The following is a brief description for the usage.
           -> To collect sar output file.
           -> sar -P -ALL -o sar.out interval_in_seconds iteration > /dev/null&
              example:
              sar -P -ALL -o sar.out 1 120 > /dev/null&
              This will collect 120 data with 1 second interval with a total of 120 seconds.
           -> Some tips:
              1. If you might exit the terminal before the end of the collection, you would need to use "nohup".
                 nohup sar -P -ALL -o sar.out interval_in_seconds iteration > /dev/null&
              2. Make sure your filesystem has enough space to store the data.
              3. If the space is not a concern, it is recommended to use the smallest interval, i.e. 1 second, to collect data.   When reporting, we can increase the interval, but can not decrease the interval.
              4. sar can collect data across midnight.  However, data collected after midnight will not be available to read with "-s" and "-e" options.
                  -> To extract data: 
                  -> To get system cpu unitlization:   sar -f sar.out     or sar -f sar.out -u
                  -> To get per cpu unitlization:   sar -f sar.out -P ALL
                  -> To get memory utilization:   sar -f sar.out -r
                  -> To see run queue:  sar -f sar.out -q
                  -> To see disk io:  sar -f sar.out -d
               5. You can add -i interval to the command to replace the interval used in collection of sar data.   However, if the report interval is smaller than the collection interval, the collection interval will be used.
               6. You can also add -s and -e options to specify the beginning and ending time of the report.   The time format is hh:mm[:ss] in 24 hour format.  Only the part of data before the first midnight in the data (if any) is recognized.   
                  -> Example:
                     sar -f sar.out -s 21:00 -e 23:59:59
    
  • Analyze GC logs

        1) Below java options would need to be appended to jvm settings and jobs would need to be rerun . 
           -Xcompressedrefs -Xdisableexplicitgc -Xtrace:none -Xshareclasses:none -XtlhPrefetch --Xverbosegclog:/home/ashish/jvm_gc_logs/<FILENAME> 
    
        2) I would like to clarify that verbosegclogs parameter would need to be run against each process . We have created a directory where the gc logs would get dumped "/home/ashish/jvm_gc_logs/" . 
    
        3)  Some sample settings
            "-Xms8g -Xmx8g -Xgcthreads4 -Xlp" . or "-Xmn512m -Xms1280m -Xmx1280m -Xlp"
    
  • Heap dump analysis

        1) Creating Heap Dumps
              a) Using jmap tool get the heap details
                    jmap -heap <process-id>
    
              b) Find the right process-id using jvmtop and generate the heap dump
                    jmap -dump:file=DumpFile.txt <pid>
    
              c) Take heap dump of live objects only
                    jmap -dump:live,format=b,file=DumpFile.txt <pid>
    
              d) Heap Summary: 
                    jmap -heap PID
                    This command dumps the current memory usage with the division of heap space (Young Gen/Old Gen/Survivor Space) depending on the Garbage Collector used
    
        1) Analysing Heap Dump
    
              a) using jhat tool 
                     Use jhat command to analyse the Heap Dump 
                     jhat -J-d64 -J-mx16g myheap.hprof
                     jhat -port 8080 -J-mx1024m myheap.hprof
    
              b) Using Eclipse Memory Analysis Tool
                     -Install the Eclipse plugin for Memory Analysis
    
              c) Using IBM Heap Analyzer
                      java -Xmx4g -jar ha456.jar
    
  • References

       1) IBM DeveloperWorks Article
             https://www.ibm.com/developerworks/community/groups/service/html/communityview?communityUuid=22d56091-3a7b-4497-b36e-634b51838e11
    
       2) GCViewer Tool
              https://github.com/chewiebug/GCViewer/wiki/Changelog
    
       3) IBM Heap Analyzer Tool
              https://www.ibm.com/developerworks/community/groups/service/html/communityview?communityUuid=4544bafe-c7a2-455f-9d43-eb866ea60091