Performance of various methods of binary serialization in Java

by Mikhail Vorontsov

We are going to find out what is the performance of binary serialization in Java. Following classes will be compared:

  • DataInputStream(ByteArrayInputStream) and its counterpart DataOutputStream(ByteArrayOutputStream)
  • See how synchronization affects ByteArrayInput/OutputStream and check performance of BAInputStream – copy of ByteArrayInputStream w/o synchronization
  • ByteBuffer in its 4 flavours – heap/direct, big/little endian
  • sun.misc.Unsafe – based memory operations on heap byte arrays

My experience has shown me that all these serialization methods depend on on data item size as well as on buffer/stream type. So, two sets of tests were written. First test works on an object having a single field – byte[500], while second test is using another object with another single field – long[500]. In case of ByteBuffer and Unsafe we will test both bulk operations and serialization of every array element as a separate method call.

First of all, here are results for the “second” league of this test: streams. We will write results for heap little-endian ByteBuffer as well (for the same number of iterations) just for comparison purposes. Please, note that we use lower number of iterations for “second” league test cases.

Two times in each cell are Java 6/Java 7 times in seconds.

Second league

Structure: DataOutputStream (ByteArrayOutputStream) DataInputStream (ByteArrayInputStream) DataInputStream (BAInputStream – not synchronized) ByteBuffer
Type       Heap
Endian       Little
Write 1M byte buffers (500 bytes) 16.178/15.74 0.362/0.379
Read 1M byte buffers (500 bytes) 16.106/16.424 2.432/2.361 0.471/0.508
Write 1M long buffers (500 longs) 11.735/11.632 3.771/3.691
Read 1M long buffers (500 longs) 11.747/11.622 9.966/9.532 6.062/4.069

The only thing to note here: synchronization on ByteArray streams affects smaller data types more severely rather than larger data types. 8 times difference between synchronized and not synchronized versions on byte operations are actually a problem.

Premier league

In our “premier” league we are going to compare various ByteBuffers with memory access via sun.misc.Unsafe. Unfortunately, Unsafe class works only with native byte order. This can be fixed by using Short/Integer/Long.reverseBytes method (to cover 2/4/8 bytes data types), but bulk operations on non-native byte order will not be possible without a temporary buffer (and with a buffer it doesn’t make any sense – it means 2 data copies instead of one).

Structure: ByteBuffer ByteBuffer ByteBuffer ByteBuffer Unsafe
Type Heap Heap Direct Direct  
Endian Little Big Little Big Little (native)
Write 80M byte buffers (as bytes) 26.21/26.26 27.457/27.365 101.967/43.338 102.912/43.396 31.578/31.846
Write 80M byte buffers (as 1 byte[]) 2.966/2.888 3.489/3.321 4.036/4.016 4.27/4.038 2.456/2.035
Read 80M byte buffers (as bytes) 56.163/36.089 56.322/36.084 40.1/41.411 40.519/41.195 36.68/41.711
Read 80M byte buffers (as 1 byte[]) 6.909/6.842 6.926/7.036 6.795/6.808 7.377/7.176 6.307/6.317
Write 10M long buffers (as longs) 36.179/37.316 51.063/51.221 14.702/7.366 64.305/10.127 7.301/6.699
Write 10M long buffers (as 1 long[]) 32.301/29.651 53.477/54.115 2.014/1.912 27.265/28.319 1.703/1.701
Read 10M long buffers (as longs) 59.625/39.923 53.978/52.097 26.715/10.941 67.355/15.932 8.492/10.026
Read 10M long buffers (as 1 long[]) 47.19/36.373 60.107/35.186 6.668/6.754 32.925/35.071 6.075/6.143

What have we found?

  1. It is extremely slow to write single bytes to direct byte buffers. You should avoid using direct byte buffers for writing records with mostly single byte fields.
  2. If you have primitive array fields – always use bulk methods to process them. ByteBuffer bulk methods performance is close to those of Unsafe (but ByteBuffer are always a little slower). If you need to store/load any other primitive array except byte – use ByteBuffer.as[YourType]Buffer.put(array) method call followed by your byte buffer position update.
  3. The higher your average field length – the slower is a heap buffer and the faster is a direct byte buffer. Unsafe access even to separate fields is still faster.
  4. In Java 7 many types of ByteBuffer accesses were seriously optimized compared to Java 6.
  5. Always try to serialize primitive arrays using direct byte buffer with your platform native byte order – its performance is very close to Unsafe performance, but it is portable.

Source code

This article required rather large source code to be written in order to test all these various memory access methods. Half of this code was mostly copy-pasted from byte access mode to long access mode. Selected sources are following. ByteObject – a class with byte[] field and all serialization methods. LongObject used for long tests is absolutely similar.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
private static class ByteObject
{
    public byte[] data;
 
    public ByteObject() {
    }
 
    private void init() {
        data = new byte[ ARRAY_LENGTH ];
        for ( int i = 0; i < ARRAY_LENGTH; ++i )
            data[ i ] = (byte) i;
    }
 
    public void writeToStream( final DataOutputStream os ) throws Exception {
        os.writeInt( data.length );
        for (final byte b : data) os.writeByte(b);
    }
 
    public static ByteObject readFromStream( final DataInputStream is ) throws Exception {
        final ByteObject obj = new ByteObject();
        obj.data = new byte[ is.readInt() ];
        for ( int i = 0; i < obj.data.length; ++i )
            obj.data[ i ] = is.readByte();
        return obj;
    }
 
    public void writeToByteBuffer( final ByteBuffer buf )
    {
        buf.putInt( data.length );
        for (final byte b: data ) buf.put( b );
    }
 
    public void writeToByteBufferArray( final ByteBuffer buf )
    {
        buf.putInt( data.length );
        buf.put( data );
    }
 
    public static ByteObject readFromByteBuffer( final ByteBuffer buf ) {
        final ByteObject obj = new ByteObject();
        obj.data = new byte[ buf.getInt() ];
        for ( int i = 0; i < obj.data.length; ++i )
            obj.data[ i ] = buf.get();
        return obj;
    }
 
    public static ByteObject readFromByteBufferArray( final ByteBuffer buf ) {
        final ByteObject obj = new ByteObject();
        obj.data = new byte[ buf.getInt() ];
        buf.get( obj.data );
        return obj;
    }
 
    public void writeToUnsafe( final UnsafeMemory buf )
    {
        buf.putByteArray( data );
    }
 
    public void writeToUnsafeSingle( final UnsafeMemory buf )
    {
        buf.putInt( data.length );
        for ( final byte b : data ) buf.putByte( b );
    }
 
    public static ByteObject readFromUnsafe( final UnsafeMemory buf )
    {
        final ByteObject obj = new ByteObject();
        obj.data = buf.getByteArray();
        return obj;
    }
 
    public static ByteObject readFromUnsafeSingle( final UnsafeMemory buf )
    {
        final ByteObject obj = new ByteObject();
        obj.data = new byte[ buf.getInt() ];
        for ( int i = 0; i < obj.data.length; ++i )
            obj.data[ i ] = buf.getByte();
        return obj;
    }
}
private static class ByteObject
{
    public byte[] data;

    public ByteObject() {
    }

    private void init() {
        data = new byte[ ARRAY_LENGTH ];
        for ( int i = 0; i < ARRAY_LENGTH; ++i )
            data[ i ] = (byte) i;
    }

    public void writeToStream( final DataOutputStream os ) throws Exception {
        os.writeInt( data.length );
        for (final byte b : data) os.writeByte(b);
    }

    public static ByteObject readFromStream( final DataInputStream is ) throws Exception {
        final ByteObject obj = new ByteObject();
        obj.data = new byte[ is.readInt() ];
        for ( int i = 0; i < obj.data.length; ++i )
            obj.data[ i ] = is.readByte();
        return obj;
    }

    public void writeToByteBuffer( final ByteBuffer buf )
    {
        buf.putInt( data.length );
        for (final byte b: data ) buf.put( b );
    }

    public void writeToByteBufferArray( final ByteBuffer buf )
    {
        buf.putInt( data.length );
        buf.put( data );
    }

    public static ByteObject readFromByteBuffer( final ByteBuffer buf ) {
        final ByteObject obj = new ByteObject();
        obj.data = new byte[ buf.getInt() ];
        for ( int i = 0; i < obj.data.length; ++i )
            obj.data[ i ] = buf.get();
        return obj;
    }

    public static ByteObject readFromByteBufferArray( final ByteBuffer buf ) {
        final ByteObject obj = new ByteObject();
        obj.data = new byte[ buf.getInt() ];
        buf.get( obj.data );
        return obj;
    }

    public void writeToUnsafe( final UnsafeMemory buf )
    {
        buf.putByteArray( data );
    }

    public void writeToUnsafeSingle( final UnsafeMemory buf )
    {
        buf.putInt( data.length );
        for ( final byte b : data ) buf.putByte( b );
    }

    public static ByteObject readFromUnsafe( final UnsafeMemory buf )
    {
        final ByteObject obj = new ByteObject();
        obj.data = buf.getByteArray();
        return obj;
    }

    public static ByteObject readFromUnsafeSingle( final UnsafeMemory buf )
    {
        final ByteObject obj = new ByteObject();
        obj.data = new byte[ buf.getInt() ];
        for ( int i = 0; i < obj.data.length; ++i )
            obj.data[ i ] = buf.getByte();
        return obj;
    }
}

I don't want to quote the full LongObject source code here - it is too similar to ByteObject. Though, a couple of methods worth mentioning: bulk reading from/writing to ByteBuffer:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
public void writeToByteBufferArray( final ByteBuffer buf )
{
    buf.putInt( data.length );
    buf.asLongBuffer().put( data );
    buf.position( buf.position() + data.length * 8 );
}
 
public static LongObject readFromByteBufferArray( final ByteBuffer buf ) {
    final LongObject obj = new LongObject();
    obj.data = new long[ buf.getInt() ];
    buf.asLongBuffer().get( obj.data );
    buf.position( buf.position() + obj.data.length * 8 );
    return obj;
}
public void writeToByteBufferArray( final ByteBuffer buf )
{
    buf.putInt( data.length );
    buf.asLongBuffer().put( data );
    buf.position( buf.position() + data.length * 8 );
}

public static LongObject readFromByteBufferArray( final ByteBuffer buf ) {
    final LongObject obj = new LongObject();
    obj.data = new long[ buf.getInt() ];
    buf.asLongBuffer().get( obj.data );
    buf.position( buf.position() + obj.data.length * 8 );
    return obj;
}

Test cases for byte data. All checks after reading/writing are required in order to forbid optimizer to throw away test code. For ByteBuffer tests, only little-endian heap buffer tests were left. All other ByteBuffer tests could be produced by changing ByteBuffer declaration.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
private static void testByte() throws Exception {
    final ByteObject obj = new ByteObject();
    obj.init();
 
    final PerformanceByteTest[] slow = new PerformanceByteTest[]{
            new PerformanceByteTest( "BufferReadHeapLittle", obj ) {
                final ByteBuffer buf = ByteBuffer.allocate( ARRAY_LENGTH + 20 ).order( ByteOrder.LITTLE_ENDIAN );
                {
                    obj.writeToByteBuffer( buf );
                }
                public void test() throws Exception {
                    buf.flip();
                    final ByteObject obj = ByteObject.readFromByteBuffer( buf );
                    if ( obj.data[ 7 ] != 7 )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },
 
            new PerformanceByteTest( "StreamWrite", obj ) {
                final PeekableBAOutputStream bos = new PeekableBAOutputStream( ARRAY_LENGTH + 20 );
                public void test() throws Exception {
                    bos.reset();
                    final DataOutputStream os = new DataOutputStream( bos );
                    obj.writeToStream( os );
                    os.close();
                    if ( bos.getArray()[ 3 ] != LAST_LENGTH_BYTE )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },
 
            new PerformanceByteTest( "StreamRead", obj ) {
                byte[] data;
                final ByteArrayInputStream bis;
 
                {
                    try
                    {
                        ByteArrayOutputStream bos = new ByteArrayOutputStream(ARRAY_LENGTH + 20);
                        final DataOutputStream dos = new DataOutputStream(bos);
                        obj.writeToStream( dos );
                        data = bos.toByteArray();
                    }
                    catch (Exception ignored){ data = null; }
                    bis = new ByteArrayInputStream( data );
                }
 
                public void test() throws Exception {
                    bis.reset();
                    final DataInputStream is = new DataInputStream( bis );
                    final ByteObject obj = ByteObject.readFromStream( is );
                    if ( obj.data[ 7 ] != 7 )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },
 
            new PerformanceByteTest( "StreamReadUnsync", obj ) {
                byte[] data;
                final BAInputStream bis;
 
                {
                    try
                    {
                        ByteArrayOutputStream bos = new ByteArrayOutputStream(ARRAY_LENGTH + 20);
                        final DataOutputStream dos = new DataOutputStream(bos);
                        obj.writeToStream( dos );
                        data = bos.toByteArray();
                    }
                    catch (Exception ignored){ data = null; }
                    bis = new BAInputStream( data );
                }
 
                public void test() throws Exception {
                    bis.reset();
                    final DataInputStream is = new DataInputStream( bis );
                    final ByteObject obj = ByteObject.readFromStream( is );
                    if ( obj.data[ 7 ] != 7 )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },
            new PerformanceByteTest( "BufferWriteHeapLittle", obj ) {
                final ByteBuffer buf = ByteBuffer.allocate( ARRAY_LENGTH + 20 ).order( ByteOrder.LITTLE_ENDIAN );
                public void test() throws Exception {
                    buf.clear();
                    obj.writeToByteBuffer( buf );
                    if ( buf.get( 0 ) != LAST_LENGTH_BYTE )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },
 
    };
 
    final PerformanceByteTest[] tests = new PerformanceByteTest[]{
            new PerformanceLongTest( "BufferWriteHeapArrayLittle", obj ) {
                final ByteBuffer buf = ByteBuffer.allocate( ARRAY_LENGTH * 8 + 20 ).order( ByteOrder.LITTLE_ENDIAN );
                public void test() throws Exception {
                    buf.clear();
                    obj.writeToByteBufferArray( buf );
                    if ( buf.get( 0 ) != LAST_LENGTH_BYTE )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },
            new PerformanceLongTest( "BufferReadArrayHeapLittle", obj ) {
                final ByteBuffer buf = ByteBuffer.allocate( ARRAY_LENGTH * 8 + 20 ).order( ByteOrder.LITTLE_ENDIAN );
                {
                    obj.writeToByteBuffer( buf );
                }
                public void test() throws Exception {
                    buf.flip();
                    final LongObject obj = LongObject.readFromByteBufferArray( buf );
                    if ( obj.data[ 7 ] != 7 )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },
 
 
            new PerformanceByteTest( "UnsafeWriteSingle", obj ) {
                final byte[] buffer = new byte[ ARRAY_LENGTH + 20 ];
                final UnsafeMemory buf = new UnsafeMemory( buffer );
                public void test() throws Exception {
                    buf.reset();
                    obj.writeToUnsafeSingle( buf );
                    if ( buffer[ 0 ] != LAST_LENGTH_BYTE )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },
 
            new PerformanceByteTest( "UnsafeReadSingle", obj ) {
                final UnsafeMemory buf = new UnsafeMemory( new byte[ ARRAY_LENGTH + 20 ] );
                {
                    obj.writeToUnsafeSingle( buf );
                }
                public void test() throws Exception {
                    buf.reset();
                    final ByteObject obj = ByteObject.readFromUnsafeSingle( buf );
                    if ( obj.data[ 7 ] != 7 )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },
 
 
 
            new PerformanceByteTest( "BufferWriteHeapLittleArray", obj ) {
                final ByteBuffer buf = ByteBuffer.allocate(ARRAY_LENGTH + 20).order( ByteOrder.LITTLE_ENDIAN );
                public void test() throws Exception {
                    buf.clear();
                    obj.writeToByteBufferArray( buf );
                    if ( buf.get( 0 ) != LAST_LENGTH_BYTE )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },
 
            new PerformanceByteTest( "BufferWriteHeapLittle", obj ) {
                final ByteBuffer buf = ByteBuffer.allocate( ARRAY_LENGTH + 20 ).order( ByteOrder.LITTLE_ENDIAN );
                public void test() throws Exception {
                    buf.clear();
                    obj.writeToByteBuffer( buf );
                    if ( buf.get( 0 ) != LAST_LENGTH_BYTE )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },
 
 
            new PerformanceByteTest( "BufferReadHeapLittle", obj ) {
                final ByteBuffer buf = ByteBuffer.allocate( ARRAY_LENGTH + 20 ).order( ByteOrder.LITTLE_ENDIAN );
                {
                    obj.writeToByteBuffer( buf );
                }
                public void test() throws Exception {
                    buf.flip();
                    final ByteObject obj = ByteObject.readFromByteBuffer( buf );
                    if ( obj.data[ 7 ] != 7 )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },
 
 
            new PerformanceByteTest( "BufferReadArrayHeapLittle", obj ) {
                final ByteBuffer buf = ByteBuffer.allocate( ARRAY_LENGTH + 20 ).order( ByteOrder.LITTLE_ENDIAN );
                {
                    obj.writeToByteBuffer( buf );
                }
                public void test() throws Exception {
                    buf.flip();
                    final ByteObject obj = ByteObject.readFromByteBufferArray(buf);
                    if ( obj.data[ 7 ] != 7 )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },
 
            new PerformanceByteTest( "UnsafeWrite", obj ) {
                final byte[] buffer = new byte[ ARRAY_LENGTH + 20 ];
                final UnsafeMemory buf = new UnsafeMemory( buffer );
                public void test() throws Exception {
                    buf.reset();
                    obj.writeToUnsafe( buf );
                    if ( buffer[ 0 ] != LAST_LENGTH_BYTE )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },
 
            new PerformanceByteTest( "UnsafeRead", obj ) {
                final UnsafeMemory buf = new UnsafeMemory( new byte[ ARRAY_LENGTH + 20 ] );
                {
                    obj.writeToUnsafe( buf );
                }
                public void test() throws Exception {
                    buf.reset();
                    final ByteObject obj = ByteObject.readFromUnsafe( buf );
                    if ( obj.data[ 7 ] != 7 )
                    {
                        System.out.println( "Failed" );
                    }
                }
            }
    };
 
    //warmup
    for ( final PerformanceByteTest test : slow )
    {
        for ( int i = 0; i < WARMUP_ITERS; ++i )
            test.test();
    }
 
    for ( final PerformanceByteTest test : tests )
    {
        for ( int i = 0; i < WARMUP_ITERS; ++i )
            test.test();
    }
    System.gc();
    Thread.sleep(2000);
 
    //actual tests
    final int slowIters = ITERS / 10;
    for ( final PerformanceByteTest test : slow )
    {
        final long start = System.currentTimeMillis();
        for ( int i = 0; i < slowIters; ++i )
        {
            test.test();
        }
        final long time = System.currentTimeMillis() - start;
        System.out.println( test.name + ", cnt = " + slowIters + ", time = " + time / 1000.0 );
        System.gc();
        Thread.sleep(2000);
    }
 
    final int byteIters = ITERS * 8;
    for ( final PerformanceByteTest test : tests )
    {
        final long start = System.currentTimeMillis();
        for ( int i = 0; i < byteIters; ++i )
        {
            test.test();
        }
        final long time = System.currentTimeMillis() - start;
        System.out.println( test.name + ", cnt = " + byteIters + ", time = " + time / 1000.0 );
        System.gc();
        Thread.sleep(2000);
    }
}
private static void testByte() throws Exception {
    final ByteObject obj = new ByteObject();
    obj.init();

    final PerformanceByteTest[] slow = new PerformanceByteTest[]{
            new PerformanceByteTest( "BufferReadHeapLittle", obj ) {
                final ByteBuffer buf = ByteBuffer.allocate( ARRAY_LENGTH + 20 ).order( ByteOrder.LITTLE_ENDIAN );
                {
                    obj.writeToByteBuffer( buf );
                }
                public void test() throws Exception {
                    buf.flip();
                    final ByteObject obj = ByteObject.readFromByteBuffer( buf );
                    if ( obj.data[ 7 ] != 7 )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },

            new PerformanceByteTest( "StreamWrite", obj ) {
                final PeekableBAOutputStream bos = new PeekableBAOutputStream( ARRAY_LENGTH + 20 );
                public void test() throws Exception {
                    bos.reset();
                    final DataOutputStream os = new DataOutputStream( bos );
                    obj.writeToStream( os );
                    os.close();
                    if ( bos.getArray()[ 3 ] != LAST_LENGTH_BYTE )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },

            new PerformanceByteTest( "StreamRead", obj ) {
                byte[] data;
                final ByteArrayInputStream bis;

                {
                    try
                    {
                        ByteArrayOutputStream bos = new ByteArrayOutputStream(ARRAY_LENGTH + 20);
                        final DataOutputStream dos = new DataOutputStream(bos);
                        obj.writeToStream( dos );
                        data = bos.toByteArray();
                    }
                    catch (Exception ignored){ data = null; }
                    bis = new ByteArrayInputStream( data );
                }

                public void test() throws Exception {
                    bis.reset();
                    final DataInputStream is = new DataInputStream( bis );
                    final ByteObject obj = ByteObject.readFromStream( is );
                    if ( obj.data[ 7 ] != 7 )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },

            new PerformanceByteTest( "StreamReadUnsync", obj ) {
                byte[] data;
                final BAInputStream bis;

                {
                    try
                    {
                        ByteArrayOutputStream bos = new ByteArrayOutputStream(ARRAY_LENGTH + 20);
                        final DataOutputStream dos = new DataOutputStream(bos);
                        obj.writeToStream( dos );
                        data = bos.toByteArray();
                    }
                    catch (Exception ignored){ data = null; }
                    bis = new BAInputStream( data );
                }

                public void test() throws Exception {
                    bis.reset();
                    final DataInputStream is = new DataInputStream( bis );
                    final ByteObject obj = ByteObject.readFromStream( is );
                    if ( obj.data[ 7 ] != 7 )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },
            new PerformanceByteTest( "BufferWriteHeapLittle", obj ) {
                final ByteBuffer buf = ByteBuffer.allocate( ARRAY_LENGTH + 20 ).order( ByteOrder.LITTLE_ENDIAN );
                public void test() throws Exception {
                    buf.clear();
                    obj.writeToByteBuffer( buf );
                    if ( buf.get( 0 ) != LAST_LENGTH_BYTE )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },

    };

    final PerformanceByteTest[] tests = new PerformanceByteTest[]{
            new PerformanceLongTest( "BufferWriteHeapArrayLittle", obj ) {
                final ByteBuffer buf = ByteBuffer.allocate( ARRAY_LENGTH * 8 + 20 ).order( ByteOrder.LITTLE_ENDIAN );
                public void test() throws Exception {
                    buf.clear();
                    obj.writeToByteBufferArray( buf );
                    if ( buf.get( 0 ) != LAST_LENGTH_BYTE )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },
            new PerformanceLongTest( "BufferReadArrayHeapLittle", obj ) {
                final ByteBuffer buf = ByteBuffer.allocate( ARRAY_LENGTH * 8 + 20 ).order( ByteOrder.LITTLE_ENDIAN );
                {
                    obj.writeToByteBuffer( buf );
                }
                public void test() throws Exception {
                    buf.flip();
                    final LongObject obj = LongObject.readFromByteBufferArray( buf );
                    if ( obj.data[ 7 ] != 7 )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },


            new PerformanceByteTest( "UnsafeWriteSingle", obj ) {
                final byte[] buffer = new byte[ ARRAY_LENGTH + 20 ];
                final UnsafeMemory buf = new UnsafeMemory( buffer );
                public void test() throws Exception {
                    buf.reset();
                    obj.writeToUnsafeSingle( buf );
                    if ( buffer[ 0 ] != LAST_LENGTH_BYTE )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },

            new PerformanceByteTest( "UnsafeReadSingle", obj ) {
                final UnsafeMemory buf = new UnsafeMemory( new byte[ ARRAY_LENGTH + 20 ] );
                {
                    obj.writeToUnsafeSingle( buf );
                }
                public void test() throws Exception {
                    buf.reset();
                    final ByteObject obj = ByteObject.readFromUnsafeSingle( buf );
                    if ( obj.data[ 7 ] != 7 )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },



            new PerformanceByteTest( "BufferWriteHeapLittleArray", obj ) {
                final ByteBuffer buf = ByteBuffer.allocate(ARRAY_LENGTH + 20).order( ByteOrder.LITTLE_ENDIAN );
                public void test() throws Exception {
                    buf.clear();
                    obj.writeToByteBufferArray( buf );
                    if ( buf.get( 0 ) != LAST_LENGTH_BYTE )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },

            new PerformanceByteTest( "BufferWriteHeapLittle", obj ) {
                final ByteBuffer buf = ByteBuffer.allocate( ARRAY_LENGTH + 20 ).order( ByteOrder.LITTLE_ENDIAN );
                public void test() throws Exception {
                    buf.clear();
                    obj.writeToByteBuffer( buf );
                    if ( buf.get( 0 ) != LAST_LENGTH_BYTE )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },


            new PerformanceByteTest( "BufferReadHeapLittle", obj ) {
                final ByteBuffer buf = ByteBuffer.allocate( ARRAY_LENGTH + 20 ).order( ByteOrder.LITTLE_ENDIAN );
                {
                    obj.writeToByteBuffer( buf );
                }
                public void test() throws Exception {
                    buf.flip();
                    final ByteObject obj = ByteObject.readFromByteBuffer( buf );
                    if ( obj.data[ 7 ] != 7 )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },


            new PerformanceByteTest( "BufferReadArrayHeapLittle", obj ) {
                final ByteBuffer buf = ByteBuffer.allocate( ARRAY_LENGTH + 20 ).order( ByteOrder.LITTLE_ENDIAN );
                {
                    obj.writeToByteBuffer( buf );
                }
                public void test() throws Exception {
                    buf.flip();
                    final ByteObject obj = ByteObject.readFromByteBufferArray(buf);
                    if ( obj.data[ 7 ] != 7 )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },

            new PerformanceByteTest( "UnsafeWrite", obj ) {
                final byte[] buffer = new byte[ ARRAY_LENGTH + 20 ];
                final UnsafeMemory buf = new UnsafeMemory( buffer );
                public void test() throws Exception {
                    buf.reset();
                    obj.writeToUnsafe( buf );
                    if ( buffer[ 0 ] != LAST_LENGTH_BYTE )
                    {
                        System.out.println( "Failed" );
                    }
                }
            },

            new PerformanceByteTest( "UnsafeRead", obj ) {
                final UnsafeMemory buf = new UnsafeMemory( new byte[ ARRAY_LENGTH + 20 ] );
                {
                    obj.writeToUnsafe( buf );
                }
                public void test() throws Exception {
                    buf.reset();
                    final ByteObject obj = ByteObject.readFromUnsafe( buf );
                    if ( obj.data[ 7 ] != 7 )
                    {
                        System.out.println( "Failed" );
                    }
                }
            }
    };

    //warmup
    for ( final PerformanceByteTest test : slow )
    {
        for ( int i = 0; i < WARMUP_ITERS; ++i )
            test.test();
    }

    for ( final PerformanceByteTest test : tests )
    {
        for ( int i = 0; i < WARMUP_ITERS; ++i )
            test.test();
    }
    System.gc();
    Thread.sleep(2000);

    //actual tests
    final int slowIters = ITERS / 10;
    for ( final PerformanceByteTest test : slow )
    {
        final long start = System.currentTimeMillis();
        for ( int i = 0; i < slowIters; ++i )
        {
            test.test();
        }
        final long time = System.currentTimeMillis() - start;
        System.out.println( test.name + ", cnt = " + slowIters + ", time = " + time / 1000.0 );
        System.gc();
        Thread.sleep(2000);
    }

    final int byteIters = ITERS * 8;
    for ( final PerformanceByteTest test : tests )
    {
        final long start = System.currentTimeMillis();
        for ( int i = 0; i < byteIters; ++i )
        {
            test.test();
        }
        final long time = System.currentTimeMillis() - start;
        System.out.println( test.name + ", cnt = " + byteIters + ", time = " + time / 1000.0 );
        System.gc();
        Thread.sleep(2000);
    }
}

PeekableBAOutputStream - small extension to ByteArrayOutputStream, allowing direct access to a byte buffer inside a stream. This is required in order to avoid expensive array duplication for checking a byte out of stream.

1
2
3
4
5
6
7
8
9
10
11
private static class PeekableBAOutputStream extends ByteArrayOutputStream
{
    private PeekableBAOutputStream(int size) {
        super(size);
    }
 
    public byte[] getArray()
    {
        return this.buf;
    }
}
private static class PeekableBAOutputStream extends ByteArrayOutputStream
{
    private PeekableBAOutputStream(int size) {
        super(size);
    }

    public byte[] getArray()
    {
        return this.buf;
    }
}

UnsafeMemory class, borrowed from a brilliant Martin Thompson article and slightly changed. Simple wrapper for sun.misc.Unsafe memory methods.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
import sun.misc.Unsafe;
import java.lang.reflect.Field;
 
public class UnsafeMemory
{
    private static final Unsafe unsafe;
    static
    {
        try
        {
            Field field = Unsafe.class.getDeclaredField("theUnsafe");
            field.setAccessible(true);
            unsafe = (Unsafe)field.get(null);
        }
        catch (Exception e)
        {
            throw new RuntimeException(e);
        }
    }
 
    private static final long byteArrayOffset = unsafe.arrayBaseOffset(byte[].class);
    private static final long longArrayOffset = unsafe.arrayBaseOffset(long[].class);
 
    private static final int SIZE_OF_BYTE = 1;
    private static final int SIZE_OF_INT = 4;
    private static final int SIZE_OF_LONG = 8;
 
    private int pos = 0;
    private final byte[] buffer;
 
    public UnsafeMemory(final byte[] buffer)
    {
        if (null == buffer)
        {
            throw new NullPointerException("buffer cannot be null");
        }
        this.buffer = buffer;
    }
 
    public void reset()
    {
        this.pos = 0;
    }
 
    ////////////////////////// Byte ////////////////////////////////
 
    public void putByte(final byte value)
    {
        unsafe.putByte(buffer, byteArrayOffset + pos, value);
        pos += SIZE_OF_BYTE;
    }
 
    public byte getByte()
    {
        byte value = unsafe.getByte(buffer, byteArrayOffset + pos);
        pos += SIZE_OF_BYTE;
        return value;
    }
 
    public void putByteArray(final byte[] values)
    {
        putInt(values.length);
 
        long bytesToCopy = values.length;
        unsafe.copyMemory(values, byteArrayOffset,
                buffer, byteArrayOffset + pos,
                bytesToCopy);
        pos += bytesToCopy;
    }
 
    public byte[] getByteArray()
    {
        int arraySize = getInt();
        byte[] values = new byte[arraySize];
 
        long bytesToCopy = values.length;
        unsafe.copyMemory(buffer, byteArrayOffset + pos,
                values, byteArrayOffset,
                bytesToCopy);
        pos += bytesToCopy;
 
        return values;
    }
 
    /////////////////////////// Int ////////////////////////////////
 
 
    public void putInt(final int value)
    {
        unsafe.putInt(buffer, byteArrayOffset + pos, value);
        pos += SIZE_OF_INT;
    }
 
    public int getInt()
    {
        int value = unsafe.getInt(buffer, byteArrayOffset + pos);
        pos += SIZE_OF_INT;
        return value;
    }
 
    //////////////// Long ///////////////////////////////////////////
 
    public void putLong(final long value)
    {
        unsafe.putLong(buffer, byteArrayOffset + pos, value);
        pos += SIZE_OF_LONG;
    }
 
    public long getLong()
    {
        long value = unsafe.getLong(buffer, byteArrayOffset + pos);
        pos += SIZE_OF_LONG;
        return value;
    }
 
    public void putLongArray(final long[] values)
    {
        putInt(values.length);
 
        long bytesToCopy = values.length << 3;
        unsafe.copyMemory(values, longArrayOffset,
                buffer, byteArrayOffset + pos,
                bytesToCopy);
        pos += bytesToCopy;
    }
 
    public long[] getLongArray()
    {
        int arraySize = getInt();
        long[] values = new long[arraySize];
 
        long bytesToCopy = values.length << 3;
        unsafe.copyMemory(buffer, byteArrayOffset + pos,
                values, longArrayOffset,
                bytesToCopy);
        pos += bytesToCopy;
 
        return values;
    }
}
import sun.misc.Unsafe;
import java.lang.reflect.Field;

public class UnsafeMemory
{
    private static final Unsafe unsafe;
    static
    {
        try
        {
            Field field = Unsafe.class.getDeclaredField("theUnsafe");
            field.setAccessible(true);
            unsafe = (Unsafe)field.get(null);
        }
        catch (Exception e)
        {
            throw new RuntimeException(e);
        }
    }

    private static final long byteArrayOffset = unsafe.arrayBaseOffset(byte[].class);
    private static final long longArrayOffset = unsafe.arrayBaseOffset(long[].class);

    private static final int SIZE_OF_BYTE = 1;
    private static final int SIZE_OF_INT = 4;
    private static final int SIZE_OF_LONG = 8;

    private int pos = 0;
    private final byte[] buffer;

    public UnsafeMemory(final byte[] buffer)
    {
        if (null == buffer)
        {
            throw new NullPointerException("buffer cannot be null");
        }
        this.buffer = buffer;
    }

    public void reset()
    {
        this.pos = 0;
    }

    ////////////////////////// Byte ////////////////////////////////

    public void putByte(final byte value)
    {
        unsafe.putByte(buffer, byteArrayOffset + pos, value);
        pos += SIZE_OF_BYTE;
    }

    public byte getByte()
    {
        byte value = unsafe.getByte(buffer, byteArrayOffset + pos);
        pos += SIZE_OF_BYTE;
        return value;
    }

    public void putByteArray(final byte[] values)
    {
        putInt(values.length);

        long bytesToCopy = values.length;
        unsafe.copyMemory(values, byteArrayOffset,
                buffer, byteArrayOffset + pos,
                bytesToCopy);
        pos += bytesToCopy;
    }

    public byte[] getByteArray()
    {
        int arraySize = getInt();
        byte[] values = new byte[arraySize];

        long bytesToCopy = values.length;
        unsafe.copyMemory(buffer, byteArrayOffset + pos,
                values, byteArrayOffset,
                bytesToCopy);
        pos += bytesToCopy;

        return values;
    }

    /////////////////////////// Int ////////////////////////////////


    public void putInt(final int value)
    {
        unsafe.putInt(buffer, byteArrayOffset + pos, value);
        pos += SIZE_OF_INT;
    }

    public int getInt()
    {
        int value = unsafe.getInt(buffer, byteArrayOffset + pos);
        pos += SIZE_OF_INT;
        return value;
    }

    //////////////// Long ///////////////////////////////////////////

    public void putLong(final long value)
    {
        unsafe.putLong(buffer, byteArrayOffset + pos, value);
        pos += SIZE_OF_LONG;
    }

    public long getLong()
    {
        long value = unsafe.getLong(buffer, byteArrayOffset + pos);
        pos += SIZE_OF_LONG;
        return value;
    }

    public void putLongArray(final long[] values)
    {
        putInt(values.length);

        long bytesToCopy = values.length << 3;
        unsafe.copyMemory(values, longArrayOffset,
                buffer, byteArrayOffset + pos,
                bytesToCopy);
        pos += bytesToCopy;
    }

    public long[] getLongArray()
    {
        int arraySize = getInt();
        long[] values = new long[arraySize];

        long bytesToCopy = values.length << 3;
        unsafe.copyMemory(buffer, byteArrayOffset + pos,
                values, longArrayOffset,
                bytesToCopy);
        pos += bytesToCopy;

        return values;
    }
}

Full test source code


Leave a Reply

Your email address will not be published. Required fields are marked *