- 浏览: 414722 次
- 性别:
- 来自: 深圳
文章分类
最新评论
-
Glogo:
楼主您好,我在试验您的代码的时候发现新开的三个子线程并没有一直 ...
java 高并发 ReentrantLock -- 可重入的锁 -
univasity:
最近发觉也被限速了,投诉一下就好一会~~ 看来明天又要和电信M ...
ADSL上网速度慢 都是帐号限速惹的祸 -
liuyuanhui0301:
java 高并发 ReentrantLock -- 可重入的锁 -
dang_java:
呵.很好的说明文档.
JXTA技术与应用发展 -
helloqidi:
谢谢,学习了
SQL中exists和in的区别
Erlang memory architecture vs Java memory architecture
- 博客分类:
- JVM/内存模型/调优/排错
I read a really, really interesting article on memory management strategies for the Erlang VM. It was written as a thesis by Jesper Wilhelmsson: http://www.it.uu.se/research/publications/lic/2005-001/ I thought it might be nice to discuss the differences between Erlang's memory setup and Sun's Java VM.
As a real short introduction for those who have never heard of Erlang; it is a functional language that uses asynchronous message passing as its basis for concurrency. The message passing uses copy semantics, making distribution over more than one Erlang VM, running on more than one machine essentially transparent to the programmer.
Erlang and Java are similar in the sense that both use a virtual machine to abstract the hardware into a portable layer. Both languages employ machine independent byte code. Both run-time systems rely on garbage collection to free the programmer of doing memory management.
Threading overhead in Erlang is very low. I believe that the memory requirements for a thread in Erlang is about 512 bytes. In Java threads typically need about 512 kilobytes, about 1000 times more. For a programmer, the upshot is that creating threads to do some work asynchronously is not something you have to sit down and think about. Typical Erlang systems have thousands or tens of thousands of threads. There is no futzing with threadpools and executors like we do in Java.
From what little I have dabbled with it, I found Erlang to be a pleasant compromise between a functional language, and a language that allows you to write real-world applications. (I know I'll get flak for this) Robust distributed error handling is a pleasant surprise and writing a network server of any kind is actually easy. The state-machine approach to web servers makes rolling back on errors completely natural.
But this post is not about the programming model of Erlang. It is about the way the Erlang VM deals with memory.
The current Java virtual machine uses what an Erlang programmer would call a shared heap topology. There is one big heap that is used by all threads. Most memory is allocated on that heap. In addition to the heap, the JVM uses some specialised data areas like the code cache and the permanent generation. These too are shared between all threads.
By contrast, Erlang uses a private heap topology. Each thread has its own tiny heap that contains all data the thread uses and the thread's stack as well. All data for a thread is on that local heap. It is reserved when the thread is created. When the thread dies, the entire heap is simply returned to the pool of free memory.
Aside from the private heaps, all threads share access to a so-called binary heap and a message heap. These are specialised heaps. The binary heap is for allocating large chunks of arbitrary data that are likely to be shared between threads. This is where for example file input or network buffers live.
The message heap is a heap for data that is used in messages. Messages too are shared between processes. Messages are passed between threads by copying a pointer over from the sending thread to the receiving thread. The data for the message is stored on the message heap.
I was impressed by the Erlang memory model. It just strikes me as a lot more scalable than Java's single heap model. The language semantics and the memory model match beautifully.
For instance; the simple fact that heaps are private to a thread relieves the threads of all forms of lock checking on their own data. Add to that the fact that there are no destructive writes and suddenly there is no need for lock checking for shared data either.
The latest version of the Erlang VM take this yet another step further by having more than one scheduler. Once scheduler per physical processor to be precise. This eliminates another entire class of locks to check. Only when a scheduler is bored it needs to go out, gather a lock and get some processes off of another scheduler.
In Java, we have a lot to learn still. That said, we have a few nice things in Java that I miss working with large Erlang systems.
The Erlang VM will reallocate and grow heaps when thread accumulate a lot of data. However, the reallocation algorithm causes heap sizes to grow rapidly. Under high load, we have seen Erlang VM's eat up 16GB of RAM in a matter of minutes. Each release has to be carefully load tested to see if its memory requirements are still sane.
There are no mechanisms in the Erlang VM to curb the growth of the memory. The VM will happily allocate so much memory that the system shoots into swap, or that the virtual memory is exhausted. These may cause the machine to become unresponsive even to KVM console access. In the past we have had to power cycle machines to get access to them again.
The queue-based programming model that makes Erlang so much fun to write code for, is also it Achilles heel in production. Every queue in Erlang is unbounded. The VM will not throw exceptions or limit the number of messages in a queue. Sometimes a process stops processing due to a bug, or a process fails to keep up with the flow of messages being sent to it. In that case, Erlang will simply allow the queue for that process to grow until either the VM is killed or the machine locks up, whichever comes first.
This means that when you run large Erlang VM's in a production environment you need to have OS-level checks that will kill the process if memory use skyrockets. Remote hands for the machine, or remote access cards is a must-have for machines that run large Erlang VM's.
In summary, for every day performance I believe the private heap memory model to be a very powerful tool in Erlang's box. It cuts whole classes of locking mechanisms out of the run-time system and that means it will scale better than Java will for the same purpose. Java's hard limits on memory will save your bacon when your system is being flooded or DDoSed.
Kees Jan
PS. There is a command line switch for Erlang's VM to switch it from using the private heap topology to using the share heap topology.
PPS. I like Erlang and Java. They are hard to compare because the have so little in common for a developer. In general, I would use Java for most systems, though. Tool support is better and the number of libraries available is staggering. I would opt for Erlang in the case where I have a stream oriented messaging system. That's when the Erlang programming model really shines.
<!-- / message -->
发表评论
-
JVM 深入笔记
2012-04-12 20:36 909JVM 深入笔记(1)内存区域是如何划分的? 一个超短的前言 ... -
JVM启动参数
2011-06-10 16:27 933java [jvmargs] class [arguments ... -
JVM指令集 和 一个异常例子
2011-05-19 16:14 983指令码 助记符 说明 0x00 nop ... -
The Top Java Memory Problems – Part 1
2011-05-18 11:01 867转载:http://blog.dynatrace.com/20 ... -
BTrace使用简介
2011-04-19 13:51 752该文转载自:http://rdc.taobao.com/tea ... -
大方法的执行性能与调优过程小记
2011-04-18 16:20 763该文章转载自:http://rdc.taobao.com/te ... -
十个最好的Java性能故障排除工具
2011-04-18 16:02 888推荐十个最好的Java性能 ... -
两个OOM Cases排查过程的分享
2011-04-18 15:39 897分享一下两个OOM Cases的查找过程,一个应用是Nativ ... -
Monitoring Java garbage collection with jstat
2011-04-18 15:28 832The original article is at : ht ... -
理解Heap Profling名词-Shallow和Retained Sizes
2011-04-18 14:58 939所有包含Heap Profling功能的工具(MAT, You ... -
Java深度历险(四)——Java垃圾回收机制与引用类型
2011-04-01 08:18 928Java语言的一个重要特性 ... -
Thread Dump 和Java应用诊断
2011-03-30 08:08 1098Thread Dump 和Java应用诊断 ... -
Memory Analysis Part 1 – Obtaining a Java Heapdump
2011-03-14 16:06 1087For troubleshooting Java memory ... -
Java Memory Leaks et al. (2. Act)
2011-03-14 15:37 897The first act of this blog-seri ... -
The Java Memory Architecture (1. Act)
2011-03-14 15:15 1387One of the biggest strength of ... -
Permanent generation
2011-03-14 14:53 1199版权声明:转载时请以 ... -
JVM内存模型以及垃圾回收
2011-03-13 09:38 912内存由 Perm 和 Heap 组成. 其中 Heap ... -
Java虚拟机垃圾回收机制
2011-03-13 09:31 769一、相关概念 基本 ... -
JVM结构
2011-03-10 14:43 861类文件格式 JVM使用一 ... -
OOM与JVM(转)
2009-03-24 15:49 896OOM与JVM(转) 2008年08月08日 星期五 15: ...
相关推荐
初级的erlang和Java交互的聊天代码
erlang调用java,在yaws应用里调用J2EE方法
java与Erlang 之间的通讯,这个列子我已经测试过了,简单易懂。
Erlang核心开发者Lukas Larsson在2014年3月份Erlang Factory上的一个演讲详细介绍了Erlang内存体系的原理以及调优案例 根据siyao zheng博客上听写的资源进行的翻译,大致只翻译了80%但核心部分已经完整,希望对大家...
java php python erlang 千万级内存数据性能比较,
Erlang架构行为 该项目包含Erlang中许多体系结构模式的示例实现。 到目前为止已实现的行为/架构: 管道和过滤器 待实施的行为/架构: 主从 点对点
本系统是基于erlang开发的后台,java swing开发的前端的qq聊天系统,希望能给初学erlang的人带来一点小小的帮助,具体操作步骤见redme.txt文件 功能如下: 功能: 1.用户登录功能,账号和密码从服务端的mysql数据库...
erlang入门电子书 erlang编程 Introducing Erlang,作者Simon.St.Laurent
6,安装失败,你骂我好了,并把你的ubunt版本、是否是在虚拟机和其下的虚拟机软件名称和版本与再其下的os名称和版本、硬件(cpu、memory、Mainboard、disk)等信息email给我,我帮你找能用的版本。 软件相关信息: ...
erlang 安装包
Erlang及其应用Erlang及其应用Erlang及其应用
erlang25.0 windows版本
Erlang 中的Module级别热部署,可以看看
erlang otp25 win安装包
erlang22最新下载包 erlang22.1.tar.gz erlang22最新下载包 erlang22最新下载包
erlang 中文基础教程erlang 中文基础教程
ErlangB和ErlangC计算工具(exe可执行文件+excel两个) ErlangB和ErlangC计算工具(exe可执行文件+excel两个)
c# 版ErlangOtp跨平台通信框架(Java版的转译)
erlang安装包
erlang port driver test vs project. can run on linux