java.io和java.nio性能简单对比

我从java1.3开始学习java，后来主要用1.4，再后来1.5和1.6中的很多新特性，都停留在“知道”的状态，比如nio，虽然据说可以提升性能，但并没有真正深入使用和测试过，工作操作文件的情况不多，所以关注也不多，即便用到，也还是习惯性的用java.io。今天看到的这篇文章，虽然测试手段非常简单，所得结论也难免有些片面 ，但依然说明，在顺序访问的时候，NIO的性能相对java.io有很大的提升。

也许应该update一下自己的知识了，否则就要OUT，或者早已经OUT了。
下次操作文件或者写socket要用NIO了。

---
转自：http://links.techwebnewsletters.com/ctt?kn=28&m=34038811&r=MzI1Mjc3MDAzOAS2&b=0&j=NTc5NjM4MTAS1&mt=1&rt=0
以下为翻译的内容：

最近我在工作中用到了java i/o相关功能。因为对java.io的了解更多（毕竟面世较早），所以一开始我使用的是java.io包下的类，后来为了测试一下是不是能够通过NIO提高文件操作性能，于是转向了java.nio。我得到的结论让我感到有些震惊，下面是对比测试的一些细节：

   1、在java.io的测试代码中，我使用RandomAccessFile直接向文件写数据，并搜索到特定的位置执行记录的插入、读取和删除。
   2、在java.nio的初步测试代码中，使用FileChannel对象。NIO之所以比java.io更加高效，是因为NIO面向的是data chunks，而java.io基本上是面向byte的。
   3、为了进一步挖掘NIO的能力，我又改用MappedByteBuffer执行测试，这个类是构建在操作系统的虚拟内存机制上的。根据java文档所说，这个类在性能方面是最好的。

 

为了进行测试，我写了一个模拟员工数据库的小程序，员工数据的结构如下：

    view plaincopy to clipboardprint?
class Employee {  
        String last; // the key  
        String first;  
        int id;  
        int zip;  
        boolean employed;  
        String comments;  
    } 
class Employee {
        String last; // the key
        String first;
        int id;
        int zip;
        boolean employed;
        String comments;
    }

员工数据写入文件，并将last name作为索引key，日后可以通过这个key从文件中加载该员工对应的数据。无论使用IO、NIO还是MappedByteBuffers，首先都需要打开一个RandomAccessFile。以下代码在用户的home目录下创建一个名为employee.ejb的文件，设置为可读可写，并初始化对应的Channel和MappedByteBuffer：
    view plaincopy to clipboardprint?
String userHome = System.getProperty("user.home");  
    StringBuffer pathname = new StringBuffer(userHome);  
    pathname.append(File.separator);  
    pathname.append("employees.ejb");  
    java.io.RandomAccessFile journal =  
        new RandomAccessFile(pathname.toString(), "rw");  
   
    //下面这一句是为了NIO  
    java.nio.channels.FileChannel channel = journal.getChannel();  
     
    //下面这两句是为了使用MappedByteBuffer  
    journal.setLength(PAGE_SIZE);  
    MappedByteBuffer mbb =  
        channel.map(FileChannel.MapMode.READ_WRITE, 0, journal.length() ); 
String userHome = System.getProperty("user.home");
    StringBuffer pathname = new StringBuffer(userHome);
    pathname.append(File.separator);
    pathname.append("employees.ejb");
    java.io.RandomAccessFile journal =
        new RandomAccessFile(pathname.toString(), "rw");
 
    //下面这一句是为了NIO
    java.nio.channels.FileChannel channel = journal.getChannel();
  
    //下面这两句是为了使用MappedByteBuffer
    journal.setLength(PAGE_SIZE);
    MappedByteBuffer mbb =
        channel.map(FileChannel.MapMode.READ_WRITE, 0, journal.length() );

 
使用channel.map进行映射后，当该文件被追加了新的数据时，之前的MappedByteBuffer是看不到这些数据的。因为我们想测试读和写，所以当文件中追加写入新的记录后，需要重新做映射才能使得MappedByteBuffer读取新数据。为了提高效率，降低重新映射的次数，每次空间不够的时候，我们将文件扩张特定的大小（比如说1K）以防止每次追加新记录都要重新映射。
 

下面是写入员工记录的对比测试：

使用java.io的代码：
    view plaincopy to clipboardprint?
public boolean addRecord_IO(Employee emp) {  
        try {  
            byte[] last = emp.last.getBytes();  
            byte[] first = emp.first.getBytes();  
            byte[] comments = emp.comments.getBytes();  
             
            // Just hard-code the sizes for perfomance  
            int size = 0;  
            size += emp.last.length();  
            size += 4; // strlen - Integer  
            size += emp.first.length();  
            size += 4; // strlen - Integer  
            size += 4; // emp.id - Integer  
            size += 4; // emp.zip - Integer  
            size += 1; // emp.employed - byte  
            size += emp.comments.length();  
            size += 4; // strlen - Integer  
            long offset = getStorageLocation(size);  
            //  
            // Store the record by key and save the offset  
            //  
            if ( offset == -1 ) {  
                // We need to add to the end of the journal. Seek there  
                // now only if we're not already there  
                long currentPos = journal.getFilePointer();  
                long jounralLen = journal.length();  
                if ( jounralLen != currentPos )  
                    journal.seek(jounralLen);  
                     
                offset = jounralLen;  
            }else {  
                // Seek to the returned insertion point  
                journal.seek(offset);  
            }  
            // Fist write the header  
            journal.writeByte(1);  
            journal.writeInt(size);  
            // Next write the data  
            journal.writeInt(last.length);  
            journal.write(last);  
            journal.writeInt(first.length);  
            journal.write(first);  
            journal.writeInt(emp.id);  
            journal.writeInt(emp.zip);  
            if ( emp.employed )  
                journal.writeByte(1);  
            else 
                journal.writeByte(0);  
            journal.writeInt(comments.length);  
            journal.write(comments);  
            // Next, see if we need to append an empty record if we inserted  
            // this new record at an empty location  
            if ( newEmptyRecordSize != -1 ) {  
                // Simply write a header  
                journal.writeByte(0); //inactive record  
                journal.writeLong(newEmptyRecordSize);  
            }  
            employeeIdx.put(emp.last, offset);  
            return true;  
        }  
        catch ( Exception e ) {  
            e.printStackTrace();  
        }  
        return false;  
    } 
public boolean addRecord_IO(Employee emp) {
        try {
            byte[] last = emp.last.getBytes();
            byte[] first = emp.first.getBytes();
            byte[] comments = emp.comments.getBytes();
          
            // Just hard-code the sizes for perfomance
            int size = 0;
            size += emp.last.length();
            size += 4; // strlen - Integer
            size += emp.first.length();
            size += 4; // strlen - Integer
            size += 4; // emp.id - Integer
            size += 4; // emp.zip - Integer
            size += 1; // emp.employed - byte
            size += emp.comments.length();
            size += 4; // strlen - Integer
            long offset = getStorageLocation(size);
            //
            // Store the record by key and save the offset
            //
            if ( offset == -1 ) {
                // We need to add to the end of the journal. Seek there
                // now only if we're not already there
                long currentPos = journal.getFilePointer();
                long jounralLen = journal.length();
                if ( jounralLen != currentPos )
                    journal.seek(jounralLen);
                  
                offset = jounralLen;
            }else {
                // Seek to the returned insertion point
                journal.seek(offset);
            }
            // Fist write the header
            journal.writeByte(1);
            journal.writeInt(size);
            // Next write the data
            journal.writeInt(last.length);
            journal.write(last);
            journal.writeInt(first.length);
            journal.write(first);
            journal.writeInt(emp.id);
            journal.writeInt(emp.zip);
            if ( emp.employed )
                journal.writeByte(1);
            else
                journal.writeByte(0);
            journal.writeInt(comments.length);
            journal.write(comments);
            // Next, see if we need to append an empty record if we inserted
            // this new record at an empty location
            if ( newEmptyRecordSize != -1 ) {
                // Simply write a header
                journal.writeByte(0); //inactive record
                journal.writeLong(newEmptyRecordSize);
            }
            employeeIdx.put(emp.last, offset);
            return true;
        }
        catch ( Exception e ) {
            e.printStackTrace();
        }
        return false;
    }
 

使用java.nio的代码：

    view plaincopy to clipboardprint?
public boolean addRecord_NIO(Employee emp) {  
        try {  
            data.clear();  
            byte[] last = emp.last.getBytes();  
            byte[] first = emp.first.getBytes();  
            byte[] comments = emp.comments.getBytes();  
            data.putInt(last.length);  
            data.put(last);  
            data.putInt(first.length);  
            data.put(first);  
            data.putInt(emp.id);  
            data.putInt(emp.zip);  
            byte employed = 0;  
            if ( emp.employed )  
                employed = 1;  
            data.put(employed);  
            data.putInt(comments.length);  
            data.put(comments);  
            data.flip();  
            int dataLen = data.limit();  
            header.clear();  
            header.put((byte)1); // 1=active record  
            header.putInt(dataLen);  
            header.flip();  
            long headerLen = header.limit();  
            int length = (int)(headerLen + dataLen);  
            long offset = getStorageLocation((int)dataLen);  
            //  
            // Store the record by key and save the offset  
            //  
            if ( offset == -1 ) {  
                // We need to add to the end of the journal. Seek there  
                // now only if we're not already there  
                long currentPos = channel.position();  
                long jounralLen = channel.size();  
                if ( jounralLen != currentPos )  
                    channel.position(jounralLen);  
                offset = jounralLen;  
            }  
            else {  
                // Seek to the returned insertion point  
                channel.position(offset);  
            }  
            // Fist write the header  
            long written = channel.write(srcs);  
            // Next, see if we need to append an empty record if we inserted  
            // this new record at an empty location  
            if ( newEmptyRecordSize != -1 ) {  
                // Simply write a header  
                data.clear();  
                data.put((byte)0);  
                data.putInt(newEmptyRecordSize);  
                data.flip();  
                channel.write(data);  
            }  
            employeeIdx.put(emp.last, offset);  
            return true;  
        }  
        catch ( Exception e ) {  
            e.printStackTrace();  
        }  
        return false;  
    } 
public boolean addRecord_NIO(Employee emp) {
        try {
            data.clear();
            byte[] last = emp.last.getBytes();
            byte[] first = emp.first.getBytes();
            byte[] comments = emp.comments.getBytes();
            data.putInt(last.length);
            data.put(last);
            data.putInt(first.length);
            data.put(first);
            data.putInt(emp.id);
            data.putInt(emp.zip);
            byte employed = 0;
            if ( emp.employed )
                employed = 1;
            data.put(employed);
            data.putInt(comments.length);
            data.put(comments);
            data.flip();
            int dataLen = data.limit();
            header.clear();
            header.put((byte)1); // 1=active record
            header.putInt(dataLen);
            header.flip();
            long headerLen = header.limit();
            int length = (int)(headerLen + dataLen);
            long offset = getStorageLocation((int)dataLen);
            //
            // Store the record by key and save the offset
            //
            if ( offset == -1 ) {
                // We need to add to the end of the journal. Seek there
                // now only if we're not already there
                long currentPos = channel.position();
                long jounralLen = channel.size();
                if ( jounralLen != currentPos )
                    channel.position(jounralLen);
                offset = jounralLen;
            }
            else {
                // Seek to the returned insertion point
                channel.position(offset);
            }
            // Fist write the header
            long written = channel.write(srcs);
            // Next, see if we need to append an empty record if we inserted
            // this new record at an empty location
            if ( newEmptyRecordSize != -1 ) {
                // Simply write a header
                data.clear();
                data.put((byte)0);
                data.putInt(newEmptyRecordSize);
                data.flip();
                channel.write(data);
            }
            employeeIdx.put(emp.last, offset);
            return true;
        }
        catch ( Exception e ) {
            e.printStackTrace();
        }
        return false;
    }
 
使用MappedByteBuffer的代码如下:
 
   view plaincopy to clipboardprint?
public boolean addRecord_MBB(Employee emp) {  
        try {  
            byte[] last = emp.last.getBytes();  
            byte[] first = emp.first.getBytes();  
            byte[] comments = emp.comments.getBytes();  
            int datalen = last.length + first.length + comments.length + 12 + 9;  
            int headerlen = 5;  
            int length = headerlen + datalen;  
            //  
            // Store the record by key and save the offset  
            //  
            long offset = getStorageLocation(datalen);  
            if ( offset == -1 ) {  
                // We need to add to the end of the journal. Seek there  
                // now only if we're not already there  
                long currentPos = mbb.position();  
                long journalLen = channel.size();  
                if ( (currentPos+length) >= journalLen ) {  
                    //log("GROWING FILE BY ANOTHER PAGE");  
                    mbb.force();  
                    journal.setLength(journalLen + PAGE_SIZE);  
                    channel = journal.getChannel();  
                    journalLen = channel.size();  
                    mbb = channel.map(FileChannel.MapMode.READ_WRITE, 0, journalLen);  
                    currentPos = mbb.position();  
                }  
                if ( currentEnd != currentPos )  
                    mbb.position(currentEnd);  
                offset = currentEnd;//journalLen;  
            }  
            else {  
                // Seek to the returned insertion point  
                mbb.position((int)offset);  
            }  
            // write header  
            mbb.put((byte)1); // 1=active record  
            mbb.putInt(datalen);  
            // write data  
            mbb.putInt(last.length);  
            mbb.put(last);  
            mbb.putInt(first.length);  
            mbb.put(first);  
            mbb.putInt(emp.id);  
            mbb.putInt(emp.zip);  
            byte employed = 0;  
            if ( emp.employed )  
                employed = 1;  
            mbb.put(employed);  
            mbb.putInt(comments.length);  
            mbb.put(comments);  
            currentEnd += length;  
            // Next, see if we need to append an empty record if we inserted  
            // this new record at an empty location  
            if ( newEmptyRecordSize != -1 ) {  
                // Simply write a header  
                mbb.put((byte)0);  
                mbb.putInt(newEmptyRecordSize);  
                currentEnd += 5;  
            }  
            employeeIdx.put(emp.last, offset);  
            return true;  
        }  
        catch ( Exception e ) {  
            e.printStackTrace();  
        }  
        return false;  
    } 
public boolean addRecord_MBB(Employee emp) {
        try {
            byte[] last = emp.last.getBytes();
            byte[] first = emp.first.getBytes();
            byte[] comments = emp.comments.getBytes();
            int datalen = last.length + first.length + comments.length + 12 + 9;
            int headerlen = 5;
            int length = headerlen + datalen;
            //
            // Store the record by key and save the offset
            //
            long offset = getStorageLocation(datalen);
            if ( offset == -1 ) {
                // We need to add to the end of the journal. Seek there
                // now only if we're not already there
                long currentPos = mbb.position();
                long journalLen = channel.size();
                if ( (currentPos+length) >= journalLen ) {
                    //log("GROWING FILE BY ANOTHER PAGE");
                    mbb.force();
                    journal.setLength(journalLen + PAGE_SIZE);
                    channel = journal.getChannel();
                    journalLen = channel.size();
                    mbb = channel.map(FileChannel.MapMode.READ_WRITE, 0, journalLen);
                    currentPos = mbb.position();
                }
                if ( currentEnd != currentPos )
                    mbb.position(currentEnd);
                offset = currentEnd;//journalLen;
            }
            else {
                // Seek to the returned insertion point
                mbb.position((int)offset);
            }
            // write header
            mbb.put((byte)1); // 1=active record
            mbb.putInt(datalen);
            // write data
            mbb.putInt(last.length);
            mbb.put(last);
            mbb.putInt(first.length);
            mbb.put(first);
            mbb.putInt(emp.id);
            mbb.putInt(emp.zip);
            byte employed = 0;
            if ( emp.employed )
                employed = 1;
            mbb.put(employed);
            mbb.putInt(comments.length);
            mbb.put(comments);
            currentEnd += length;
            // Next, see if we need to append an empty record if we inserted
            // this new record at an empty location
            if ( newEmptyRecordSize != -1 ) {
                // Simply write a header
                mbb.put((byte)0);
                mbb.putInt(newEmptyRecordSize);
                currentEnd += 5;
            }
            employeeIdx.put(emp.last, offset);
            return true;
        }
        catch ( Exception e ) {
            e.printStackTrace();
        }
        return false;
    }
 

接下来，调用每种方法插入100,000条记录, 耗时对比如下:
    * With java.io: ~10,000 milliseconds
    * With java.nio: ~2,000 milliseconds
    * With MappedByteBuffer: ~970 milliseconds

 
使用NIO的性能改善效果非常明显，使用MappedByteBuffer的性能，更是让人吃惊。

使用三种方式读取数据的性能对比如下：
    * With java.io: ~6,900 milliseconds
    * With java.nio: ~1,400 milliseconds
    * With MappedByteBuffer: ~355 milliseconds

和写入的时候情况差不多，NIO有很明显的性能提升，而MappedByteBuffer则有惊人的高效率。从java.io迁移到nio并使用MappedByteBuffer，通常可以获得10倍以上的性能提升。