- 浏览: 282058 次
- 性别:
- 来自: 天津
文章分类
最新评论
-
狼子六:
已经开始用了
Java 枚举7常见种用法 -
相约的旋律:
方法一下面的"public static fianl ...
Java 枚举7常见种用法 -
kiddy2012:
...
[转载]程序员装B指南 -
fancy105:
写得很好,简单明了双全面!
Java 枚举7常见种用法 -
he_wen:
请问一下 G1算法是否在线上使用了?
Garbage First(G1) 日志分析工具
G1介绍:http://softbeta.iteye.com/blog/1562185
用python写的一个分析G1日志工具,现在还不是很完善,有时间继续改进:
目前只支持以下配置输出的GC日志:
-XX:+UseG1GC -XX:+PrintGCDateStamps -XX:+PrintGCDetails -Xloggc:/Users/lichengwu/tmp/mtct/mtct_g1.log
下面是代码:
# coding=utf-8 __author__ = 'lichengwu' import datetime import re ''' Garbage First Log Analyse Util ''' class G1LogUtil: __path = '' # some regular expression pattern # like this '2012-12-19T10:25:19' __START_LINE_PATTERN = re.compile('^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:') # like this '112M(112M)->0B(112M)' __MEMORY_INFO_PATTERN = re.compile('(\d+)([M|B|K|G])\((\d+)([M|B|K|G])\)->(\d+)([M|B|K|G])\((\d+)([M|B|K|G])\)') # like this '16M->16M' __SIMPLE_MEMORY_INFO_PATTERN = re.compile('(\d+)([M|B|K|G])->(\d+)([M|B|K|G])') # constructor def __init__(self, path): self.__path = path """ analyse G1 log for java """ def analyse(self): # get log reader reader = self.__reader(self.__path) # number of GC times gc_count = 0 gc_count_young = 0 gc_count_mixed = 0 # total stop the world time total_stop_time = 0.000000 # max heap size max_heap_size = 0 # min heap size min_heap_size = 0xEFFFFFFFFFFFFFFFF # max eden size max_eden_size = 0 # min eden size min_eden_size = 0xEFFFFFFFFFFFFFFFF # survivor size survivor_size = None # total eden size for statistics total_eden_size = 0 # map store memory info memory_usage_map = {'young_garbage_percent': [], 'heap_garbage_percent': [], 'young_usage_percent': [], 'heap_usage_percent': []} #log start time start_time = None # log end time finish_time = None # gc work thread count gc_work_thread_number = None each_line = reader.next() while each_line: if self.__is_start_line(each_line): token = each_line.split(' ') total_stop_time += float(token[5]) if start_time is None: start_time = self.__get_datetime(token[0]) finish_time = token[0] gc_count += 1 gc_type = token[4][1:-2] if gc_type == 'young': gc_count_young += 1 elif gc_type == 'mixed': gc_count_mixed += 1 elif each_line.find(' [Eden:') == 0: ''' parse memory info ''' memory_info = each_line.split(' ') eden_info = self.__parse_memory_info(memory_info[4]) survivor_info = self.__parse_memory_info(memory_info[6]) if survivor_size is None: survivor_size = survivor_info[1] heap_info = self.__parse_memory_info(memory_info[8]) max_heap_size = max(max_heap_size, heap_info[1]) min_heap_size = min(heap_info[1], min_heap_size) # garbage (heap) / before gc (heap) memory_usage_map['heap_garbage_percent'].append(float(heap_info[0] - heap_info[2]) / heap_info[0]) # before gc (heap) / heap size memory_usage_map['heap_usage_percent'].append(float(heap_info[0]) / heap_info[1]) max_eden_size = max(max_eden_size, eden_info[1]) min_eden_size = min(eden_info[1], min_eden_size) # garbage (eden+survivor) / before gc (eden+survivor) memory_usage_map['young_garbage_percent'].append( float(eden_info[0] + survivor_info[0] - eden_info[2] - survivor_info[1]) / ( eden_info[0] + survivor_info[0])) # before gc(eden+survivor) / eden+survivor*2 memory_usage_map['young_usage_percent'].append( float(eden_info[0] + survivor_info[0]) / (eden_info[1] + survivor_info[1] * 2)) total_eden_size += eden_info[1] elif gc_work_thread_number is None and each_line.find(' [GC Worker Start') == 0: gc_work_thread_number = len(each_line.strip().split(' ')) - 1 each_line = reader.next() finish_time = self.__get_datetime(finish_time) reader.close() print '''G1 log Time: [%s] - [%s]''' % ( start_time.strftime('%Y-%m-%d %H:%M:%S'), finish_time.strftime('%Y-%m-%d %H:%M:%S')) summary = '''Memory Info: Min Heap Size\t= %sM Max Heap Size\t= %sM Min Eden Size\t= %sM Max Eden Size\t= %sM Avg Eden Size\t= %sM Survivor Size\t= %sM''' % ( (max_heap_size / 1024), (min_heap_size / 1024), (max_eden_size / 1024), (min_eden_size / 1023), (total_eden_size / gc_count / 1024), survivor_size / 1024) print summary gc_info = '''GC Info: GC Work Threads\t= %s Avg Stop Time\t= %.2fms GC Throughput\t= %.2f%% ''' % (gc_work_thread_number, (total_stop_time * 1000 / gc_count), total_stop_time * 100 / (finish_time.second - start_time.second)) gc_info += '''GC(yong) Times\t= %s GC(mixed) Times\t= %s Total GC Times\t= %s ''' % (gc_count_young, gc_count_mixed, gc_count) gc_info += '''Avg Yong Generation Garbage Rate\t= %.2f%% Avg Heap Garbage rate\t= %.2f%% ''' % (sum(memory_usage_map['young_garbage_percent']) * 100 / len(memory_usage_map['young_garbage_percent']), sum(memory_usage_map['heap_garbage_percent']) * 100 / len(memory_usage_map['heap_garbage_percent'])) gc_info += '''Avg Max Young Generation Usage Rate\t=%.2f%% Avg Max Heap Usage Rate\t=%.2f%% ''' % (sum(memory_usage_map['young_usage_percent']) * 100 / len(memory_usage_map['young_garbage_percent']), sum(memory_usage_map['heap_usage_percent']) * 100 / len(memory_usage_map['heap_usage_percent'])) print gc_info # get datetime from header line def __get_datetime(self, str): # time like this '2012-12-12T19:01:28.610' datetime_string = str if len(str) > 23: datetime_string = str[0:23] return datetime.datetime.strptime(datetime_string, '%Y-%m-%dT%H:%M:%S.%f') # test if the line is g1 log start line def __is_start_line(self, line): #pattern = re.compile('^[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:') return self.__START_LINE_PATTERN.match(line) is not None # private reader for read each line def __reader(self, path): log_file = open(path, 'r') line = log_file.readline() while line: yield line line = log_file.readline() log_file.close() yield None ''' parse memory info to a tuple in kilobyte eg: 1M->1M parse to (1024,1024) 2M(2M)->0B(1M) parse to (2028,2048,0,1024) ''' def __parse_memory_info(self, info): match = self.__MEMORY_INFO_PATTERN.match(info) if match: cell = match.groups() return int(cell[0]) * self.__unit2kb(cell[1]), int(cell[2]) * self.__unit2kb(cell[3]), int( cell[4]) * self.__unit2kb(cell[5]), int(cell[6]) * self.__unit2kb(cell[7]) match = self.__SIMPLE_MEMORY_INFO_PATTERN.match(info) if match: cell = match.groups() return int(cell[0]) * self.__unit2kb(cell[1]), int(cell[2]) * self.__unit2kb(cell[3]) return None # covert unit to KB # M = 1024K # G = 1024K = 1024K*1024 def __unit2kb(self, unit): if unit == 'M': return 1024 elif unit == 'K': return 1 elif unit == "G": return 1048576 else: return 1 if __name__ == '__main__': analyseG1Log = G1LogUtil('/Users/lichengwu/tmp/mtcms-debug.gc.log.2012-12-20') analyseG1Log.analyse()
输出日志:
G1 log Time: [2012-12-19 10:25:12] - [2012-12-20 17:22:16] Memory Info: Min Heap Size = 1024M Max Heap Size = 1024M Min Eden Size = 128M Max Eden Size = 112M Avg Eden Size = 117M Survivor Size = 16M GC Info: GC Work Threads = 6 Avg Stop Time = 61.31ms GC Throughput = 91.97% GC(yong) Times = 60 GC(mixed) Times = 0 Total GC Times = 60 Avg Yong Generation Garbage Rate = 91.38% Avg Heap Garbage rate = 39.49% Avg Max Young Generation Usage Rate =92.06% Avg Max Heap Usage Rate =29.38%
几个参数解释:
- GC Work Threads GC工作线程数
- Avg Stop Time 平均停顿时间
- GC Throughput 吞吐量
- GC(yong) Times 新生代GC次数(严格来说是新生代region 下同)
- GC(mixed) Times 混合GC次数(新生代region+老年代region)
- Total GC Times 总gc次数
- Avg Yong Generation Garbage Rate 新生代垃圾比例
- Avg Heap Garbage rate 老年代垃圾比例
- Avg Max Young Generation Usage Rate 新生代内粗平均最大利用率
- Avg Max Heap Usage Rate 老年代内存平均利用率
稍后可能会加入一些图表和其他分析结果:
持续更新地址:https://gist.github.com/4353790
发表评论
-
Java压缩/解压缩二进制文件
2013-02-08 13:56 5675在Java中提供Deflater和Inflater工具类来 ... -
java常用并发工具介绍
2013-02-06 23:22 1566本文主要介绍的工具包括: CountDownLa ... -
Spring源码阅读——Ioc初始化过程
2012-11-20 21:52 1214以web项目启动为例,介绍一下Ioc容器的初始化。 下面这个 ... -
Spring源码阅读——获得bean
2012-10-20 20:57 3124已XmlWebApplicationContext为例,从ge ... -
Spring源码阅读——BeanFactory体系结构
2012-10-10 18:10 2164以下是看代的码随笔,想到哪写到哪,没有组织逻辑,见谅见谅。 ... -
生成随机数组
2012-09-11 20:23 10161.要求生成一组n位的数字,0-9...(n个)之间的稠密集合 ... -
O(1)复杂度获得栈中的极值
2012-09-09 23:47 1196设计一个栈,O(1)复杂度实现入栈,出栈,栈中最大值,栈中最小 ... -
Garbage First(G1)介绍
2012-07-08 13:34 2192介绍: Garbage First(G1)致 ... -
ubuntu12.04上编译openjdk7
2012-06-13 15:31 4823获取源码 从openjdk代码仓库获取(比较慢) ... -
jinfo 查看、设置JVM参数
2012-04-25 15:36 5097用法: # jinfo -h Usage: ... -
一些参数,弥补CMS(Concurrent Mark-Sweep)收集器的缺点
2012-03-28 21:11 1636CMS缺点:http://softbeta.iteye ... -
[转载]PrintCompilation 参数解释
2012-03-27 13:59 1503英文不好,所以就不翻译了: b Blocking co ... -
《Head First 设计模式》读书笔记
2012-03-01 11:30 15081.策略模式(Strategy) ... -
解决jrobin图像中文乱码
2012-02-14 23:31 1117目前发现一种方法可以解决,做个标记。 利用字体: RrdG ... -
Java内存管理——垃圾收集概念及特点
2012-02-03 12:37 1013碎碎念,介绍Java内存管 ... -
[转载]hotspot源码(JDK7)
2012-01-19 19:25 1494源码结构图,方便理解: ├─agent ... -
跨平台获取java进程id(Process ID in Java)
2012-01-18 19:20 9086原创地址:http://blog.lichengwu.cn/ ... -
了解CMS(Concurrent Mark-Sweep)垃圾回收器
2011-12-27 20:13 120141.总体介绍: CMS(Concurrent Mark- ... -
fail-fast
2011-12-13 19:34 1014fail-fast:快速失败 一般情况下,在对集合进行迭代( ... -
[转载]理解 CMS GC 日志
2011-11-22 14:52 1526CMS GC with -XX:+PrintGCDetai ...
相关推荐
G1垃圾回收器论文。 Garbage-First Garbage Collection October 24-25, 2004
Garbage-First is a server-style garbage collector, targeted for multi-processors with large memories, that meets a soft real-time goal with high probability, while achieving high throughput. Whole-...
NULL 博文链接:https://softbeta.iteye.com/blog/1562185
G1垃圾收集器的总体流程与重要算法
GarbageFirst Garbage Collection (java G1垃圾回收)
G1 是一个 服务器风格(Server-Style)的垃圾收集器。其适用目标是具有大内存的多处理器计算机。它试图在实现高吞吐量的同时,高概率地满足 GC暂停时间 目标。整个堆操作,如 全局标记,与业务线程同时运行。这可以...
Picking up where Charlie Hunt and Binu John’s classic Java Performance left off, this book provides unprecedented detail on two powerful Java platform innovations: the Garbage First (G1) garbage ...
IBM Pattern Modelin and Analysis Tool for Java Garbage Collector IBM Weblogic Application Service(WAS)GC(垃圾回收)日志分析工具
IBM的java 内存回收分析工具,用于分析gc日志
NULL 博文链接:https://seanzhou.iteye.com/blog/2003197
[GA458]IBM Pattern Modeling and Analysis Tool for Java Garbage Collector
WP-Understanding Java Garbage Collection
The Garbage Collection Handbook 内存管理,垃圾收集,GC
The Garbage Collection Handbook:The Art of Automatic Memory Manage
The Garbage Collection Handbook The Art of Automatic Memory Management, Jones, Hosking, Moss, 2012
系统级编程的课程实验,实现内存的的自动回收管理。 这里需要声明的是,资源不是来自本人,资源来自网络。本着造福广大学生的目的。但是本人实在太菜,不知道如何设置资源分数为0,所以就设置为1. ...
IBM Pattern Modeling and Analysis Tool for Java Garbage Collector gc分析,没有时间格式错误的bug
CMU的垃圾回收实验(IN C) 1.这个实验由我主讲,我制作了PPT,包含三种思路, 2.并提供其中最重要也是最简单的扫描Stack Frame的源码。 3.此外,有整体思考的思维导图,供全局预览。
The Garbage Collection Handbook 关于垃圾回收器算法的书籍,非常不错