Tag Archives: memory allocation

Various types of memory allocation in Java

by Mikhail Vorontsov

This article discusses various types of a memory buffer allocation in Java. We will see how to treat any sort of Java buffer uniformly using sun.misc.Unsafe memory access methods. This article may be especially interesting for ex-C programmers willing to work with the memory on the lowest possible level in Java.

If you are more interested in general Java memory optimization, take a look at An overview of memory saving techniques in Java article in this blog as well as its following parts: one, two.

Array allocation limitations

Array size in Java is limited by the fact of using int as an array index. This means that you can not allocate an array with more than Integer.MAX_VALUE ( = 2^31 - 1 ) elements. This doesn’t mean that the longest chunk of memory you can allocate in Java is 2 Gb. You can allocate an array of bigger type instead. For example,

1
final long[] ar = new long[ Integer.MAX_VALUE ];
final long[] ar = new long[ Integer.MAX_VALUE ];

will allocate 16Gb - 8 bytes, if you have sufficiently high -Xmx Java setting (usually you should have about 50% more memory in heap – so in order to allocate 16Gb buffer, you will have to specify -Xmx24G (this is a general rule, actual required heap size may vary).

Unfortunately, you will be limited by your array element type in pure Java. The only useful class for working with arrays is a ByteBuffer, which offers methods for getting/writing various Java data types in the buffer (see Various methods of binary serialization in Java for more details). The disadvantage of a ByteBuffer – you are limited with byte[] as a source array type, which means a limitation of 2Gb for your buffer.

Treating any arrays as a byte buffer

For a while let’s assume that 2Gb buffers were not sufficient for our needs, but a 16Gb buffer will make us happy. We have allocated a long[], but want to treat this buffer as a byte array. We need to use a best C programmer friend in Java – sun.misc.Unsafe. This class has 2 sets of methods: getN( Object, offset ), where N is a result type for reading a value of given type from the given offset in the object and putN( Object, offset, value ) for writing a value at a given offset.

Unfortunately, these methods set or get only an individual value. If you want to copy data to/from an array, you will need one more Unsafe method: copyMemory(srcObject, srcOffset, destObject, destOffset, count). It works similar to System.arraycopy, but copies bytes instead of array elements.

In order to access array data using sun.misc.Unsafe, you will need 2 components:

Continue reading