Exception in thread main java lang outofmemoryerror java heap space ошибка

If you keep on allocating & keeping references to object, you will fill up any amount of memory you have.

One option is to do a transparent file close & open when they switch tabs (you only keep a pointer to the file, and when the user switches tab, you close & clean all the objects… it’ll make the file change slower… but…), and maybe keep only 3 or 4 files on memory.

Other thing you should do is, when the user opens a file, load it, and intercept any OutOfMemoryError, then (as it is not possible to open the file) close that file, clean its objects and warn the user that he should close unused files.

Your idea of dynamically extending virtual memory doesn’t solve the issue, for the machine is limited on resources, so you should be carefull & handle memory issues (or at least, be carefull with them).

A couple of hints i’ve seen with memory leaks is:

—> Keep on mind that if you put something into a collection and afterwards forget about it, you still have a strong reference to it, so nullify the collection, clean it or do something with it… if not you will find a memory leak difficult to find.

—> Maybe, using collections with weak references (weakhashmap…) can help with memory issues, but you must be carefull with it, for you might find that the object you look for has been collected.

—> Another idea i’ve found is to develope a persistent collection that stored on database objects least used and transparently loaded. This would probably be the best approach…

How to solve java.lang.outofmemoryerror: java heap space or exception in thread “main”? Know reasons caused by and how to solve it guide.

Introduction

In Java JVM allocates a defined memory size for storing objects created during program execution known as Java Heap Space. Along with it, JVM allocates another memory called PermGen space: permanent generation space.

However, we can change the default size with the JVM options.

Most importantly, Oracle completely removed this memory space in the JDK 8 release.

Understanding OutOfMemoryError in Java

There most common reason for this error is simple –

If we try to fit a large application into a smaller space. In other words, the application just requires more Java heap space than available to it to operate normally.

Other causes:

1. Spikes in usage/data volume- The application was designed to handle a certain amount of users or a certain amount of data. When the number of users or the volume of data suddenly spikes and crosses the expected limit. The operation which functioned normally before the spike ceases to operate and triggers the OutOfMemoryError.
2. Memory leaks- A particular type of programming error will lead your application to constantly consume more memory. Every time the leaking functionality of the application is used it leaves some objects behind in the Java heap space. Over time the leaked objects consume all of the available heap space and trigger the already familiar OutOfMemoryError.

Exception in thread “main” java.lang.outofmemoryerror: java heap space

Exception in thread "main" java.lang.OutOfMemoryError

What is causing it?

Generally, bad programming results in OutOfMemoryError. OutOfMemoryError usually means that we’re doing something wrong, either holding onto objects too long or trying to process too much data at a time. Sometimes, it indicates a problem that’s out of our control, such as a third-party library that caches strings, or an application server that doesn’t clean up after deploys.

GC Overhead limit exceeded- This error indicates that the garbage collector is running all the time and the Java program is making very slow progress. If such an event occurs then an OutOfMemoryError is thrown.

Stack vs Java Heap Space

Heap Space
Whenever we create an object, it’s always created in the Heap space.

Stack Memory
Java Stack memory is used for the execution of a thread. It also contains method references.

Note: String Pool is also a part of Java Heap Memory.

Solving  java.lang.outofmemoryerror: java heap space error

Increase Java Heap size in Java

The default size of Heap space in Java is 128MB on most of 32 bit Sun’s JVM but it highly varies from JVM to JVM.

For instance, the default maximum and start heap size for the 32-bit Solaris Operating System (SPARC Platform Edition) is -Xms=3670K and -Xmx=64M. And default values of heap size parameters on 64-bit systems have been increased up by approximately 30%.

Also, if we are using a throughput garbage collector in Java 1.5 default maximum heap size of JVM would be Physical Memory/4, and the default initial heap size would be Physical Memory/16.

Another way to find the default heap size of JVM is to start an application with default heap parameters and monitor using JConsole. It is available on JDK 1.5 onwards, on the VMSummary tab, you will be able to see the maximum heap size.

Moreover, we can increase the size of java heap space based on our application need and it is always recommended to avoid using default JVM heap values. Therefore, if our application is large and lots of objects are created. We can change the size of heap space by using JVM options -Xms and -Xmx. Here, Xms denotes the starting size of Heap while -Xmx denotes the maximum size of Heap in Java.

There is another parameter called -Xmn. It denotes the size of the new generation of Java Heap Space. The only thing is we can’t change the size of Heap in Java dynamically. We can only provide the Java Heap Size parameter while starting JVM.

Conclusion

We hope you got your error resolved. Let us know by commenting if you need any help or have any questions.

If this article helped you feel free to share. Keep reading and sharing. Kudos!!

In Java, all objects are stored in a heap. They are allocated using a new operator. The OutOfMemoryError Exception in Java looks like this: 

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

Usually, this error is thrown when the Java Virtual Machine cannot allocate an object because it is out of memory. No more memory could be made available by the garbage collector.

OutOfMemoryError usually means that you’re doing something wrong, either holding onto objects too long or trying to process too much data at a time. Sometimes, it indicates a problem that’s out of your control, such as a third-party library that caches strings or an application server that doesn’t clean up after deploys. And sometimes, it has nothing to do with objects on the heap.

The java.lang.OutOfMemoryError exception can also be thrown by native library code when a native allocation cannot be satisfied (for example, if swap space is low). Let us understand various cases when the OutOfMemory error might occur.

Symptom or Root cause?

To find the cause, the text of the exception includes a detailed message at the end. Let us examine all the errors. 

Error 1 – Java heap space: 

This error arises due to the applications that make excessive use of finalizers. If a class has a finalize method, objects of that type do not have their space reclaimed at garbage collection time. Instead, after garbage collection, the objects are queued for finalization, which occurs later. 

Implementation: 

• finalizers are executed by a daemon thread that services the finalization queue.
• If the finalizer thread cannot keep up with the finalization queue, the Java heap could fill up, and this type of OutOfMemoryError exception would be thrown.
• The problem can also be as simple as a configuration issue, where the specified heap size (or the default size, if it is not specified) is insufficient for the application.

Output:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at Heap.main(Heap.java:11)

When you execute the above code above you might expect it to run forever without any problems. As a result, over time, with the leaking code constantly used, the “cached” results end up consuming a lot of Java heap space, and when the leaked memory fills all of the available memory in the heap region and Garbage Collection is not able to clean it, the java.lang.OutOfMemoryError:Java heap space is thrown.

Prevention: Check how to monitor objects for which finalization is pending in Monitor the Objects Pending Finalization.

Error 2 – GC Overhead limit exceeded: 

This error indicates that the garbage collector is running all the time and Java program is making very slow progress. After a garbage collection, if the Java process is spending more than approximately 98% of its time doing garbage collection and if it is recovering less than 2% of the heap and has been doing so far the last 5 (compile-time constant) consecutive garbage collections, then a java.lang.OutOfMemoryError is thrown. 

This exception is typically thrown because the amount of live data barely fits into the Java heap having little free space for new allocations. 

If you run this program with java -Xmx100m -XX:+UseParallelGC Wrapper, then the output will be something like this : 

Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
    at java.lang.Integer.valueOf(Integer.java:832)
    at Wrapper.main(error.java:9)

Prevention: Increase the heap size and turn off it with the command line flag -XX:-UseGCOverheadLimit. 

Error 3 – Permgen space is thrown: 

Java memory is separated into different regions. The size of all those regions, including the permgen area, is set during the JVM launch. If you do not set the sizes yourself, platform-specific defaults will be used. 

The java.lang.OutOfMemoryError: PermGen space error indicates that the Permanent Generation’s area in memory is exhausted. 

In the above sample code, code iterates over a loop and generates classes at run time. Class generation complexity is being taken care of by the Javassist library. 

Running the above code will keep generating new classes and loading their definitions into Permgen space until the space is fully utilized and the java.lang.OutOfMemoryError: Permgen space is thrown. 

Prevention : When the OutOfMemoryError due to PermGen exhaustion is caused during the application launch, the solution is simple. The application just needs more room to load all the classes to the PermGen area, so we need to increase its size. To do so, alter your application launch configuration and add (or increase if present) the -XX:MaxPermSize parameter similar to the following example: 

java -XX:MaxPermSize=512m com.saket.demo.Permgen

Error 4 – Metaspace: 

Java class metadata is allocated in native memory. Suppose metaspace for class metadata is exhausted, a java.lang.OutOfMemoryError exception with a detail MetaSpace is thrown. 

The amount of metaspace used for class metadata is limited by the parameter MaxMetaSpaceSize, which is specified on the command line. When the amount of native memory needed for a class metadata exceeds MaxMetaSpaceSize, a java.lang.OutOfMemoryError exception with a detail MetaSpace is thrown.

This code will keep generating new classes and loading their definitions to Metaspace until the space is fully utilized and the java.lang.OutOfMemoryError: Metaspace is thrown. When launched with -XX:MaxMetaspaceSize=64m then on Mac OS X my Java 1.8.0_05 dies at around 70, 000 classes loaded.

Prevention: If MaxMetaSpaceSize, has been set on the command line, increase its value. MetaSpace is allocated from the same address spaces as the Java heap. Reducing the size of the Java heap will make more space available for MetaSpace. This is only a correct trade-off if there is an excess of free space in the Java heap. 

Error 5 – Requested array size exceeds VM limit: 

This error indicates that the application attempted to allocate an array that is larger than the heap size. For example, if an application attempts to allocate an array of 1024 MB but the maximum heap size is 512 MB then OutOfMemoryError will be thrown with “Requested array size exceeds VM limit”. 

Output:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at GFG.main(GFG.java:12)

The java.lang.OutOfMemoryError: Requested array size exceeds VM limit can appear as a result of either of the following situations: 

• Your arrays grow too big and end up having a size between the platform limit and the Integer.MAX_INT
• You deliberately try to allocate arrays larger than 2^31-1 elements to experiment with the limits.

Error 6 – Request size bytes for a reason. Out of swap space?: 

This apparent exception occurred when an allocation from the native heap failed and the native heap might be close to exhaustion. The error indicates the size (in bytes) of the request that failed and the reason for the memory request. Usually, the reason is the name of the source module reporting the allocation failure, although sometimes it is the actual reason. 

The java.lang.OutOfMemoryError: Out of swap space error is often caused by operating-system-level issues, such as: 

• The operating system is configured with insufficient swap space.
• Another process on the system is consuming all memory resources.

Prevention: When this error message is thrown, the VM invokes the fatal error handling mechanism (that is, it generates a deadly error log file, which contains helpful information about the thread, process, and system at the time of the crash). In the case of native heap exhaustion, the heap memory and memory map information in the log can be useful.

Error 7 – reason stack_trace_with_native_method: 

Whenever this error message(reason stack_trace_with_native_method) is thrown then a stack trace is printed in which the top frame is a native method, then this is an indication that a native method has encountered an allocation failure. The difference between this and the previous message is that the allocation failure was detected in a Java Native Interface (JNI) or native method rather than the JVM code. 

The exact native thread limit is platform-dependent. For example, tests Mac OS X reveals that: 64-bit Mac OS X 10.9, Java 1.7.0_45 – JVM dies after #2031 threads have been created

Prevention: Use native utilities of the OS to diagnose the issue further. For more information about tools available for various operating systems, see Native Operating System tools. 

This article is contributed by Saket Kumar. If you like GeeksforGeeks and would like to contribute, you can also write an article using write.geeksforgeeks.org or mail your article to review-team@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks. Please write comments if you find anything incorrect, or you want to share more information about the topic discussed above.

The OutOfMemoryError is a common issue in Java applications, occurring when the JVM cannot allocate enough memory to fulfill a request. The OutOfMemoryError can lead to application crashes and affect overall performance. Proper memory management is critical in preventing it.

This article provides insights and strategies to handle and prevent this error in Java applications, ensuring efficient memory usage and optimal application performance.

The OutOfMemoryError class is present in ‘java.lang‘ package and has been there since Java version 1.0. It extends VirtualMachineError class; hence, it is an unchecked exception, and we don’t have to declare it in a method or a constructor throws clause.

This issue is more due to the system’s limitation (heap memory space) rather than due to the programming mistakes made by the developer.

Constructors defined in this class:

• OutOfMemoryError(): Creates an instance of the OutOfMemoryError class, setting null as its message.
• OutOfMemoryError(String message): Creates an instance of the OutOfMemoryError class, using the specified message.

2. Various Causes and Fixtures For OutOfMemoryError

OutOfMemoryError occurs when something is wrong in the application either the application code references large-size objects for a longer time than necessary or the user tries to process larger amounts of data in memory at a certain point of time for which JVM won’t allocate sufficient memory.

Let us now dive into some of the main causes for this error and the corresponding fixes for the same:

2.1. Java Heap Space

Problem

In Java, all the new objects are stored in the heap area of JVM, and the heap memory has a limited storage space. This error occurs when heap memory doesn’t have enough space to store new objects.

This can happen for various reasons:

• When we try to process a large amount of data or files which is too large into the application memory to store and process.
• When we keep references to the old objects (ones that we are not using anywhere in our application) and because of this, heap memory piles up with these old objects.
• When we have some programmatic loopholes, and there occurs a memory leak within our application code which fills up the heap memory unnecessarily.

Let us now look at an example to produce a Java heap space problem:

public class JavaHeapSpace {

  public static void main(String[] args) throws Exception {
    String[] array = new String[100000 * 100000];
  }
}

Here, we are trying to create a string array of quite a larger size, and with the default memory settings of JVM, this code on execution will raise OutOfMemoryError. This error indicates that there is not enough memory on the heap to assign this string array.

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
  at memory.JavaHeapSpace.main(JavaHeapSpace.java:5)

Solution

In most cases, increasing the maximum heap size will resolve this problem so that the JVM can allocate the required amount of memory to the program.

Two main JVM attributes that determine the Java heap size, and we can set them at JVM application startup settings:

• -Xms to set the initial heap size
• -Xmx to set the maximum heap size

For example, to increase the maximum heap size to 8 GB, we must add the -Xmx8g parameter to our JVM start parameters.

-Xms2G -Xmx8G

This fix will work only till this new memory space wouldn’t gets filled up again. Suppose we try to process some data or files larger than 8 GB then again our application will raise this error or in case our application has memory leaks or useless references then also we will run into the same problem for quite some time again.

For this, we have to go through our application code and find any potential memory leaks and remove useless references from our code to prevent the heap memory from filling up with unnecessary data.

 2.2. GC Overhead Limit

Problem

This error occurs when Garbage Collector won’t able to reclaim memory despite running most of the time, meaning, JVM is spending approx. 98%  of its time doing the garbage collection for 5 consecutive garbage collection cycles and still reclaiming less than 2% of the heap space.

We will usually get this error with older java versions with lesser heap memory and older implementations of Garbage collectors like Parallel GC or Serial GC.

Let’s now look at an example to produce GC overhead Limit error:

public class GCOverhead {

  public static void main(String[] args) throws Exception {
    Map<Long, Long> map = new HashMap<>();
    for (long i = 0l; i < Long.MAX_VALUE; i++) {
      map.put(i, i);
    }
  }
}

When we run the above code with a heap size of 50 MB and using Java version 8 that uses Parallel GC then we will get the below error,

Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
        at java.base/java.lang.Long.valueOf(Long.java:1211)
        at memory.GCOverhead.main(GCOverhead.java:10)

Solution

We can resolve this type of error by increasing the heap size using -Xmx attribute so that JVM will have enough heap memory to meet our programming needs and it won’t run GC most of the time to reclaim the memory. We have to also use new implementations of GC that come up with new versions of Java to have better algorithms implemented for garbage collection.

-Xmx4G

2.3. Array Size Limits

Problem

In a Java application, if we try to create an array of size greater than Integer.MAX_INT or if our array grows too big and ends up with a size greater than the heap memory size then we will get java.lang.OutOfMemoryError: Requested array size exceeds VM limit error.

For example, if an application attempts to allocate an array of size 1024 MB but the maximum heap size is 512 MB then this error occurs.

Solution

If we want to use an array of larger size then we have to increase the heap memory size using -Xmx attribute of JVM. We need to revisit our code as well to remove such situations where our array grows upto heap space, and we should come up with better memory management and data allocation measures in our code.

-Xmx4G

2.4. Perm Gen Issues

JVM memory area is divided into 2 parts:

• Heap Space
• PermGen (Permanent Generation)

At the time JVM launches, the size of the Heap space and permgen sets up. Heap space stores all the newly created objects in Java whereas PermGen space keeps track of all the loaded classes metadata, static methods and references to static objects.

PermGen space size is limited in Java, we have a PermGen of size approx. 85 MB in a 64-bit version of JVM and because of that if we have lot many classes, static methods or references to static objects in the application then this limited PermGen space fills up and our application will raise java.lang.OutOfMemoryError: PermGen space error.

Since Java version 8, PermGen space removes from the JVM, and hence we won’t get this error anymore with java versions after JDK7.

Solution

We can resolve this error by increasing the size of the PermGen space by including -XX:PermSize and -XX:MaxPermSize JVM parameters.

-XX:PermSize128m
-XX:MaxPermSize512m

2.5. Metaspace Issues

As the PermGen space has been removed from JVM as part of the Java-8 release, all the class metadata is now been stored in native space also called metaspace which is part of the heap memory of JVM.

This metaspace region is still limited and can be exhausted if we have a lot of classes, resulting in raising java.lang.OutOfMemoryError: Metaspace error.

Solution

We can resolve this error by increasing the size of the Metaspace by adding -XX:MaxMetaspaceSize flag to VM arguments or parameters of our Java application. For example, to set the Metaspace region size to 128M, we would add the following parameter:

-XX:MaxMetaspaceSize128m

2.6. Out of Swap

Problem

Operating-system level issues cause java.lang.OutOfMemoryError: Out of swap space error, such as:

• When the operating system configures with insufficient swap space.
• When another process on the system consumes all memory resources.

Solution

For resolving this issue, we have to check our operating system swap settings and increase if that is too low. We also need to verify if there are other heavy memory consumers running on the same machine as our application.

2.7. Number of Threads

Problem

There is a limit to creating and running threads in an application and java.lang.OutOfMemoryError: unable to create native thread error occurs when we hit the thread creation threshold and the operating system runs out of resources to create a new thread for our application.

Let us now look at an example to understand this error more clearly,

public class ThreadsLimits {

  public static void main(String[] args) throws Exception {

    while (true) {
      new Thread(
          new Runnable() {
            @Override
            public void run() {
              try {
                Thread.sleep(1000 * 60 * 60 * 24);
              } catch (Exception ex) {}
            }
          }
      ).start();
    }
  }
}

In this program, we continuously create new threads and put them to sleep. After some time, our code exhausts the operating system limits, and it can’t create more threads, and we will end up with the error,

Exception in thread "main" java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
  at java.base/java.lang.Thread.start0(Native Method)
  at java.base/java.lang.Thread.start(Thread.java:802)
  at memory.ThreadsLimits.main(ThreadsLimits.java:15

Solution

To get rid of these kinds of issues, we should be careful while creating new threads in our application and we should create a new thread only when it’s been required and try to implement Thread Pools so that we can make use of same threads for multiple jobs instead of creating new threads for each and every operation.

3. How to Catch OutOfMemoryError

If a program raises OutOfMemoryError, then being a programmer we can’t do much regarding the lack of memory in our application. It is the system or server admin who can increase the heap memory of the application to resolve this error. But it is highly recommended to handle exceptions wherever it is possible like in cases where the lines of code raising this exception are known then it will be a good idea to handle this error.

To catch this error, we just need to surround the code that may cause memory issues within the try-catch block,

public class OutOfMemoryErrorExample {

    public void initializeArray (int size) {
        try {
            Integer[] arr = new Integer[size];
        } catch (OutOfMemoryError e) {
            //Log the error msg
            System.err.println("Array size too large :: "+ e);
          }
    }

    public static void main(String[] args) {
        OutOfMemoryErrorExample obj = new OutOfMemoryErrorExample();
        obj.initializeArray(100000 * 100000);
    }
}

Output:
Array size too large :: java.lang.OutOfMemoryError: Java heap space

Here, as we know the lines of code that may cause OutOfMemoryError we have kept those in try-catch block so that in case this error occurs then it will be handled gracefully, and the program is terminated/continued normally.

4. Various Tools for Detecting OutOfMemoryError

Detecting OutOfMemoryError is not easy, and one has to investigate a lot to find out the root cause of this error. We can use various tools available in the market that helps us to analyze the objects that are taking a lot of memory in our application, or find any kind of memory leakages within the application. We can also set up alerts or alarming mechanisms as well in our logs which will trigger a notification in case OutOfMemoryError gets logged-in in our application logs.

Some of the popular tools for analyzing OutOfMemoryError are:

4.1. VisualVM

VisualVM is another command line JDK tool that monitors and troubleshoots applications running on Java 1.4+. It can monitor the application’s CPU usage, GC activity, heap and metaspace / permanent generation memory, number of loaded classes and running threads.

VisualVM can take and display the thread dumps for an immediate insight of what is going on in the target process to help with uncovering inefficient heap usage and debugging memory leaks.

4.2. Eclipse Memory Analyzer (MAT)

Eclipse Memory Analyzer is an Eclipse foundation tool that is used for analyzing the Java heap dump. It is used for finding the memory leaks and the classloader leaks in our application code.

We can take the heap dump of our application that may contain millions of objects and analyze the same using this tool to detect any memory leakages and identify the places where can reduce the object creations.

4.3. Visualgc

Visualgc (Visual Garbage Collection Monitoring Tool) is another popular tool that can be attached to the instrumented hotspot JVM.

It is mainly used to represent all the key data in a graphical manner, including class loader, JVM compiler performance data, and Garbage Collection which we can analyze and figure out the culprits for high memory consumption or any kind of memory leaks.

4.4. JProfiler

JProfiler is a popular Java profiler tool in the market. It provides an intuitive UI for viewing system performance, memory usage, potential memory leaks, and thread profiling.

Using this tool, we can easily track performance issues and optimize our Java application. It is a cross-platform tool and various operating systems like Windows, Mac OS, Linux, Solaris, etc. support it. It also supports different IDEs such as NetBeans, Eclipse, IntelliJ, etc.

This tool provides many features such as Memory Profiling, Heap Walker, CPU profiling, Thread Profiling, Databases, etc. to track the performance issue in different sections. It also provides support for both the SQL and NoSQL databases.

5. Conclusion

In this tutorial, we have learned about Java OutOfMemoryError and the answers to the following questions:

• What is OutOfMemoryError class in Java?
• What are the various causes and corresponding fixes for OutOfMemoryError?
• How to catch OutOfMemoryError in our application code?
• Various tools by which we can analyzeOutOfMemoryError and remove memory leaks and improve the application’s performance.

Happy Learning !!

Document Information

Preface

1.  Diagnostic Tools and Options

2.  Detailed Tool Descriptions

3.  Troubleshooting Memory Leaks

3.1 Meaning of OutOfMemoryError

3.1.1 Detail Message: Java heap space

3.1.2 Detail Message: PermGen space

3.1.3 Detail Message: Requested array size exceeds VM limit

3.1.4 Detail Message: request <size> bytes for <reason>. Out of swap space?

3.1.5 Detail Message: <reason> <stack trace> (Native method)

3.2 Crash Instead of OutOfMemoryError

3.3 Diagnosing Leaks in Java Language Code

3.3.1 NetBeans Profiler

3.3.2 Using the jhat Utility

3.3.3 Creating a Heap Dump

3.3.3.1 HPROF Profiler

3.3.3.2 jmap Utility

3.3.3.3 JConsole Utility

3.3.3.4 -XX:+HeapDumpOnOutOfMemoryError Command-line Option

3.3.4 Obtaining a Heap Histogram on a Running Process

3.3.5 Obtaining a Heap Histogram at OutOfMemoryError

3.3.6 Monitoring the Number of Objects Pending Finalization

3.3.7 Third Party Memory Debuggers

3.4 Diagnosing Leaks in Native Code

3.4.1 Tracking All Memory Allocation and Free Calls

3.4.2 Tracking Memory Allocation in a JNI Library

3.4.3 Tracking Memory Allocation With OS Support

3.4.4 Using dbx to Find Leaks

3.4.5 Using libumem to Find Leaks

4.  Troubleshooting System Crashes

5.  Troubleshooting Hanging or Looping Processes

6.  Integrating Signal and Exception Handling

7.  Submitting Bug Reports

A.  Environment Variables and System Properties

B.  Command-Line Options

C.  Fatal Error Log

D.  Summary of Tools in This Release

Chapter 3

Troubleshooting Memory Leaks

If your application’s execution time becomes longer and longer, or if the operating system seems to be performing slower and slower, this could be an indication of a memory leak. In other words, virtual memory is being allocated but is not being returned when it is no longer needed. Eventually the application or the system runs out of memory, and the application terminates abnormally.

This chapter provides some suggestions on diagnosing problems involving possible memory leaks.

3.1 Meaning of OutOfMemoryError

One common indication of a memory leak is the java.lang.OutOfMemoryError error. This error is thrown when there is insufficient space to allocate an object in the Java heap or in a particular area of the heap. The garbage collector cannot make any further space available to accommodate a new object, and the heap cannot be expanded further.

When the java.lang.OutOfMemoryError error is thrown, a stack trace is printed also.

A java.lang.OutOfMemoryError can also be thrown by native library code when a native allocation cannot be satisfied, for example, if swap space is low.

An early step to diagnose an OutOfMemoryError is to determine what the error means. Does it mean that the Java heap is full, or does it mean that the native heap is full? To help you answer this question, the following subsections explain some of the possible error messages, with reference to the detail part of the message:

• Exception in thread "main": java.lang.OutOfMemoryError: Java heap space

See 3.1.1 Detail Message: Java heap space .

• Exception in thread "main": java.lang.OutOfMemoryError: PermGen space

See 3.1.2 Detail Message: PermGen space .

• Exception in thread "main": java.lang.OutOfMemoryError: Requested array size exceeds VM limit

See 3.1.3 Detail Message: Requested array size exceeds VM limit .

• Exception in thread "main": java.lang.OutOfMemoryError: request <size> bytes for <reason>. Out of swap space?

See 3.1.4 Detail Message: request <size> bytes for <reason>. Out of swap space? .

• Exception in thread "main": java.lang.OutOfMemoryError: <reason> <stack trace> (Native method)

  See 3.1.5 Detail Message: <reason> <stack trace> (Native method) .

3.1.1 Detail Message: Java heap space

The detail message Java heap space indicates that an object could not be allocated in the Java heap. This error does not necessarily imply a memory leak. The problem can be as simple as a configuration issue, where the specified heap size (or the default size, if not specified) is insufficient for the application.

In other cases, and in particular for a long-lived application, the message might be an indication that the application is unintentionally holding references to objects, and this prevents the objects from being garbage collected. This is the Java language equivalent of a memory leak. Note that APIs that are called by an application could also be unintentionally holding object references.

One other potential source of OutOfMemoryError arises with applications that make excessive use of finalizers. If a class has a finalize method, then objects of that type do not have their space reclaimed at garbage collection time. Instead, after garbage collection the objects are queued for finalization, which occurs at a later time. In the Sun implementation, finalizers are executed by a daemon thread that services the finalization queue. If the finalizer thread cannot keep up with the finalization queue, then the Java heap could fill up and OutOfMemoryError would be thrown. One scenario that can cause this situation is when an application creates high-priority threads that cause the finalization queue to increase at a rate that is faster than the rate at which the finalizer thread is servicing that queue. Section 3.3.6 Monitoring the Number of Objects Pending Finalization discusses how to monitor objects for which finalization is pending.

3.1.2 Detail Message: PermGen space

The detail message PermGen space indicates that the permanent generation is full. The permanent generation is the area of the heap where class and method objects are stored. If an application loads a very large number of classes, then the size of the permanent generation might need to be increased using the -XX:MaxPermSize option.

Interned java.lang.String objects are also stored in the permanent generation. The java.lang.String class maintains a pool of strings. When the intern method is invoked, the method checks the pool to see if an equal string is already in the pool. If there is, then the intern method returns it; otherwise it adds the string to the pool. In more precise terms, the java.lang.String.intern method is used to obtain the canonical representation of the string; the result is a reference to the same class instance that would be returned if that string appeared as a literal. If an application interns a huge number of strings, the permanent generation might need to be increased from its default setting.

When this kind of error occurs, the text String.intern or ClassLoader.defineClass might appear near the top of the stack trace that is printed.

The jmap -permgen command prints statistics for the objects in the permanent generation, including information about internalized String instances. See 2.7.4 Getting Information on the Permanent Generation.

3.1.3 Detail Message: Requested array size exceeds VM limit

The detail message Requested array size exceeds VM limit indicates that the application (or APIs used by that application) attempted to allocate an array that is larger than the heap size. For example, if an application attempts to allocate an array of 512MB but the maximum heap size is 256MB then OutOfMemoryError will be thrown with the reason Requested array size exceeds VM limit. In most cases the problem is either a configuration issue (heap size too small), or a bug that results in an application attempting to create a huge array, for example, when the number of elements in the array are computed using an algorithm that computes an incorrect size.

3.1.4 Detail Message: request <size> bytes for <reason>. Out of swap space?

The detail message request <size> bytes for <reason>. Out of swap space? appears to be an OutOfMemoryError. However, the HotSpot VM code reports this apparent exception when an allocation from the native heap failed and the native heap might be close to exhaustion. The message indicates the size (in bytes) of the request that failed and the reason for the memory request. In most cases the <reason> part of the message is the name of a source module reporting the allocation failure, although in some cases it indicates a reason.

When this error message is thrown, the VM invokes the fatal error handling mechanism, that is, it generates a fatal error log file, which contains useful information about the thread, process, and system at the time of the crash. In the case of native heap exhaustion, the heap memory and memory map information in the log can be useful. See AppendixC, Fatal Error Log for detailed information about this file.

If this type of OutOfMemoryError is thrown, you might need to use troubleshooting utilities on the operating system to diagnose the issue further. See 2.16 Operating-System-Specific Tools.

The problem might not be related to the application, for example:

• The operating system is configured with insufficient swap space.
• Another process on the system is consuming all memory resources.

If neither of the above issues is the cause, then it is possible that the application failed due to a native leak, for example, if application or library code is continuously allocating memory but is not releasing it to the operating system.

3.1.5 Detail Message: <reason> <stack trace> (Native method)

If the detail part of the error message is <reason> <stack trace> (Native method) and a stack trace is printed in which the top frame is a native method, then this is an indication that a native method has encountered an allocation failure. The difference between this and the previous message is that the allocation failure was detected in a JNI or native method rather than in Java VM code.

If this type of OutOfMemoryError is thrown, you might need to use utilities on the operating system to further diagnose the issue. See 2.16 Operating-System-Specific Tools.

3.2 Crash Instead of OutOfMemoryError

Sometimes an application crashes soon after an allocation from the native heap fails. This occurs with native code that does not check for errors returned by memory allocation functions.

For example, the malloc system call returns NULL if there is no memory available. If the return from malloc is not checked, then the application might crash when it attempts to access an invalid memory location. Depending on the circumstances, this type of issue can be difficult to locate.

However, in some cases the information from the fatal error log or the crash dump might be sufficient to diagnose this issue. The fatal error log is covered in detail in AppendixC, Fatal Error Log. If the cause of a crash is determined to be the failure to check an allocation failure, then the reason for the allocation failure must be examined. As with any other native heap issue, the system might be configured with insufficient swap space, another process on the system might be consuming all memory resources, or there might be a leak in the application (or in the APIs that it calls) that causes the system to run out of memory.

3.3 Diagnosing Leaks in Java Language Code

Diagnosing leaks in Java language code can be a difficult task. In most cases it requires very detailed knowledge of the application. In addition the process is often iterative and lengthy. This section provides the following subsections:

• 3.3.1 NetBeans Profiler
• 3.3.2 Using the jhat Utility
• 3.3.3 Creating a Heap Dump
• 3.3.4 Obtaining a Heap Histogram on a Running Process
• 3.3.5 Obtaining a Heap Histogram at OutOfMemoryError
• 3.3.6 Monitoring the Number of Objects Pending Finalization
• 3.3.7 Third Party Memory Debuggers

3.3.1 NetBeans Profiler

The NetBeans Profiler (previously known as JFluid) is an excellent profiler, which can locate memory leaks very quickly. Most commercial memory leak debugging tools can often take a long time to locate a leak in a large application. The NetBeans Profiler, however, uses the pattern of memory allocations and reclamations that such objects typically demonstrate. This process includes also the lack of memory reclamations. The profiler can check where these objects were allocated, which in many cases is sufficient to identify the root cause of the leak.

More details can be found at http://profiler.netbeans.org.

3.3.2 Using the jhat Utility

The jhat utility (see 2.5 jhat Utility ) is useful when debugging unintentional object retention (or memory leaks). It provides a way to browse an object dump, view all reachable objects in the heap, and understand which references are keeping an object alive.

To use jhat you must obtain one or more heap dumps of the running application, and the dumps must be in binary format. Once the dump file is created, it can be used as input to jhat, as described in 2.5 jhat Utility .

3.3.3 Creating a Heap Dump

A heap dump provides detailed information on the allocation of heap memory. The following sections describe several ways to produce a heap dump:

• 3.3.3.1 HPROF Profiler
• 3.3.3.2 jmap Utility
• 3.3.3.3 JConsole Utility
• 3.3.3.4 -XX:+HeapDumpOnOutOfMemoryError Command-line Option

3.3.3.1 HPROF Profiler

The HPROF profiler agent can create a heap dump while the application is executing. The following is an example of the command line:

  $  
       java -agentlib:hprof=file=snapshot.hprof,format=b 
       application
      

If the VM is embedded or is not started using a command line launcher that allows additional options to be provided, then it might be possible to use the JAVA_TOOLS_OPTIONS environment variable so that the -agentlib option is automatically added to the command line. See A.2 JAVA_TOOL_OPTIONS Environment Variable for further information on this environment variable.

Once the application is running with HPROF, a heap dump is created by pressing Ctrl- or Ctrl-Break (depending on the platform) on the application console. An alternative approach on Solaris OS and Linux is to send a QUIT signal with the kill -QUIT pid command. When the signal is received, a heap dump is created; in the above example the file snapshot.hprof is created.

The heap dump file contains all the primitive data and stack traces.

A dump file can contain multiple heap dumps. If Ctrl- or Ctrl-Break is pressed a number of times then the subsequent dumps are appended to the file. The jhat utility uses the # n syntax to distinguish the dumps, where n is the dump number.

3.3.3.2 jmap Utility

A heap dump can also be obtained using the jmap utility (see 2.7 jmap Utility ). The following is an example of the command line:

  $  
        jmap -dump:format=b,file=snapshot.jmap 
      process-pid

      

Regardless of how the Java VM was started, the jmap tool will produce a head dump snapshot, in the above example in a file called snapshot.jmap. The jmap output files should contain all the primitive data, but will not include any stack traces showing where the objects have been created.

3.3.3.3 JConsole Utility

Another way to obtain a heap dump is with the JConsole utility. In the MBeans tab, select the HotSpotDiagnostic MBean, then the Operations display, and choose the dumpHeap operation.

3.3.3.4 -XX:+HeapDumpOnOutOfMemoryError Command-line Option

If you specify the -XX:+HeapDumpOnOutOfMemoryError command-line option, and if an OutOfMemoryError is thrown, the VM generates a heap dump.

3.3.4 Obtaining a Heap Histogram on a Running Process

You can try to quickly narrow down a memory leak by examining a heap histogram. This information can be obtained in several ways:

• A heap histogram can be obtained from a running process using the command jmap -histo pid. The output shows the total size and instance count for each class type in the heap. If a sequence of histograms is obtained (for example, every 2 minutes), then you might be able to observe a trend that can lead to further analysis.
• On Solaris OS and Linux, the jmap utility can also provide a histogram from a core file.
• If the Java process is started with the -XX:+PrintClassHistogram command-line option, then the Ctrl-Break handler will produce a heap histogram.

3.3.5 Obtaining a Heap Histogram at OutOfMemoryError

If you specify the -XX:+HeapDumpOnOutOfMemoryError command-line option, and if an OutOfMemoryError is thrown, the VM generates a heap dump. You can then use the jmap utility to obtain a histogram from the heap dump.

If a core file is produced when the OutOfMemoryError is thrown, you can execute jmap on the core file to get a histogram, as in the following example.

$

       jmap -histo  /java/re/javase/6/latest/binaries/solaris-sparc/bin/java core.27421

Attaching to core core.27421 from executable 
/java/re/javase/6/latest/binaries/solaris-sparc/bin/java, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 1.6.0-beta-b63
Iterating over heap. This may take a while...
Heap traversal took 8.902 seconds.

Object Histogram:
 
Size      Count   Class description
-------------------------------------------------------
86683872  3611828 java.lang.String
20979136  204     java.lang.Object[]
403728    4225    * ConstMethodKlass
306608    4225    * MethodKlass
220032    6094    * SymbolKlass
152960    294     * ConstantPoolKlass
108512    277     * ConstantPoolCacheKlass
104928    294     * InstanceKlassKlass
68024     362     byte[]
65600     559     char[]
31592     359     java.lang.Class
27176     462     java.lang.Object[]
25384     423     short[]
17192     307     int[]
:
      

The example shows that the OutOfMemoryError is caused by the number of java.lang.String objects (3611828 instances in the heap). Without further analysis it is not clear where the strings are allocated. However, the information is still useful and the investigation can continue with tools such as HPROF or jhat to find out where the strings are allocated, as well as what references are keeping them alive and preventing them from being garbage collected.

3.3.6 Monitoring the Number of Objects Pending Finalization

As noted in 3.1.1 Detail Message: Java heap space , excessive use of finalizers can be the cause of OutOfMemoryError. You have several options for monitoring the number of objects that are pending finalization.

• The JConsole management tool (see 2.3 JConsole Utility) can be used to monitor the number of objects that are pending finalization. This tool reports the pending finalization count in the memory statistics on the “Summary” tab pane. The count is approximate but it can be used to characterize an application and understand if it relies a lot on finalization.
• On Solaris OS and Linux, the jmap -finalizerinfo option prints information on objects awaiting finalization.
• An application can report the approximate number of objects pending finalization using the getObjectPendingFinalizationCount method in the java.lang.management.MemoryMXBean class. Links to the API documentation and example code can be found in 2.17 Developing Diagnostic Tools. The example code can easily be extended to include the reporting of the pending finalization count.

3.3.7 Third Party Memory Debuggers

In addition to the tools mentioned in the previous chapters, there are a large number of third-party memory debuggers available. JProbe from Quest Software, and OptimizeIt from Borland are two examples of commercial tools with memory debugging capability. There are many others and no specific product is recommended.

3.4 Diagnosing Leaks in Native Code

Several techniques can be used to find and isolate native code memory leaks. In general there is no single ideal solution for all platforms.

3.4.1 Tracking All Memory Allocation and Free Calls

A very common practice is to track all allocation and free calls of the native allocations. This can be a fairly simple process or a very sophisticated one. Many products over the years have been built up around the tracking of native heap allocations and the use of that memory.

Tools like Purify and Sun’s dbx Run Time Checking (see 3.4.4 Using dbx to Find Leaks ) functionality can be used to find these leaks in normal native code situations and also find any access to native heap memory that represents assignments to uninitialized memory or accesses to freed memory.

Not all these types of tools will work with Java applications that use native code, and usually these tools are platform-specific. Since the virtual machine dynamically creates code at runtime, these tools can wrongly interpret the code and fail to run at all, or give false information. Check with your tool vendor to make sure the version of the tool works with the version of the virtual machine you are using.

Many simple and portable native memory leak detecting examples can be found at http://sourceforge.net/. Most of these libraries and tools assume that you can recompile or edit the source of the application and place wrapper functions over the allocation functions. The more powerful of these tools allow you to run your application unchanged by interposing over these allocation functions dynamically. This is the case with the library libumem.so, starting with Solaris 9 OS update 3; see 3.4.5 Using libumem to Find Leaks .

3.4.2 Tracking Memory Allocation in a JNI Library

If you write a JNI library, it would probably be wise to create some kind of localized way to make sure your library does not leak memory, using a simple wrapper approach.

The following procedure is an easy localized allocation tracking approach for a JNI library. First, define the following lines in all source files:

#include <stdlib.h>

#define malloc(n) debug_malloc(n, __FILE__, __LINE__)
#define free(p) debug_free(p, __FILE__, __LINE__)

Then you can use the following functions to watch for leaks.

  /* Total bytes allocated */
static int total_allocated;
/* Memory alignment is important */
typedef union { double d; struct {size_t n; char *file; int line;} s; } Site;
void *
debug_malloc(size_t n, char *file, int line) 
{ 
    char *rp;
    rp = (char*)malloc(sizeof(Site)+n); 
    total_allocated += n; 
    ((Site*)rp)->s.n = n;
    ((Site*)rp)->s.file = file;
    ((Site*)rp)->s.line = line;
    return (void*)(rp + sizeof(Site));
}
void 
debug_free(void *p, char *file, int line)
{
    char *rp;
    rp = ((char*)p) - sizeof(Site);
    total_allocated -= ((Site*)rp)->s.n;
    free(rp);
}

The JNI library would then need to periodically (or at shutdown) check the value of the total_allocated variable to make sure that it made sense. The above code could also be expanded to save in a linked list the allocations that remained and report where the leaked memory was allocated. This is a localized and portable way to track memory allocations in a single set of sources. You would need to make sure that debug_free() was called only with a pointer that came from debug_malloc(), and you would also need to create similar functions for realloc(), calloc(), strdup(), and so forth, if they were used.

A more global way to look for native heap memory leaks would involve interposition of the library calls for the entire process.

3.4.3 Tracking Memory Allocation With OS Support

Most operating systems include some form of global allocation tracking support.

• On Windows, go to http://msdn.microsoft.com/library/default.asp and search for debug support. The Microsoft C++ compiler has the /Md and /Mdd compiler options that will automatically include extra support for tracking memory allocations.
• Linux systems have tools such as mtrace and libnjamd to help in dealing with allocation tracking.
• Solaris Operating Systems provide the watchmalloc tool. Solaris 9 OS update 3 started providing the libumem tool (see 3.4.5 Using libumem to Find Leaks ).

3.4.4 Using dbx to Find Leaks

The Sun debugger dbx includes the Run Time Checking (RTC) functionality, which can find leaks. The dbx debugger is also available on Linux.

Below is a sample dbx session.

  $  
        dbx ${java_home}/bin/java
Reading java
Reading ld.so.1
Reading libthread.so.1
Reading libdl.so.1
Reading libc.so.1
(dbx)  
       
       dbxenv rtc_inherit on
(dbx)  
        check -leaks
leaks checking - ON
(dbx)  
         run HelloWorld
Running: java HelloWorld 
(process id 15426)
Reading rtcapihook.so
Reading rtcaudit.so
Reading libmapmalloc.so.1
Reading libgen.so.1
Reading libm.so.2
Reading rtcboot.so
Reading librtc.so
RTC: Enabling Error Checking...
RTC: Running program...
dbx: process 15426 about to exec("/net/bonsai.sfbay/export/home2/user/ws/j2se/build/solaris-i586/bin/java")
dbx: program "/net/bonsai.sfbay/export/home2/user/ws/j2se/build/solaris-i586/bin/java"
just exec'ed
dbx: to go back to the original program use "debug $oprog"
RTC: Enabling Error Checking...
RTC: Running program...
t@1 (l@1) stopped in main at 0x0805136d
0x0805136d: main       :        pushl    %ebp
(dbx)  
         when dlopen libjvm { suppress all in libjvm.so; }
(2) when dlopen libjvm { suppress all in libjvm.so; }  
(dbx)  
         when dlopen libjava { suppress all in libjava.so; }
(3) when dlopen libjava { suppress all in libjava.so; }  
(dbx) cont                                             
Reading libjvm.so
Reading libsocket.so.1
Reading libsched.so.1
Reading libCrun.so.1
Reading libm.so.1
Reading libnsl.so.1
Reading libmd5.so.1
Reading libmp.so.2
Reading libhpi.so
Reading libverify.so
Reading libjava.so
Reading libzip.so
Reading en_US.ISO8859-1.so.3
hello world
hello world
Checking for memory leaks...

Actual leaks report    (actual leaks:           27  total size:      46851 bytes)

  Total     Num of  Leaked     Allocation call stack
  Size      Blocks  Block
                    Address
==========  ====== =========== =======================================
     44376       4      -      calloc < zcalloc 
      1072       1  0x8151c70  _nss_XbyY_buf_alloc < get_pwbuf < _getpwuid <
                               GetJavaProperties < Java_java_lang_System_initProperties <
                               0xa740a89a< 0xa7402a14< 0xa74001fc
       814       1  0x8072518  MemAlloc < CreateExecutionEnvironment < main 
       280      10      -      operator new < Thread::Thread 
       102       1  0x8072498  _strdup < CreateExecutionEnvironment < main 
        56       1  0x81697f0  calloc < Java_java_util_zip_Inflater_init < 0xa740a89a<
                               0xa7402a6a< 0xa7402aeb< 0xa7402a14< 0xa7402a14< 0xa7402a14
        41       1  0x8072bd8  main 
        30       1  0x8072c58  SetJavaCommandLineProp < main 
        16       1  0x806f180  _setlocale < GetJavaProperties <
                               Java_java_lang_System_initProperties < 0xa740a89a< 0xa7402a14<
                               0xa74001fc< JavaCalls::call_helper < os::os_exception_wrapper 
        12       1  0x806f2e8  operator new < instanceKlass::add_dependent_nmethod <
                               nmethod::new_nmethod < ciEnv::register_method <
                               Compile::Compile #Nvariant 1 < C2Compiler::compile_method <
                               CompileBroker::invoke_compiler_on_method <
                               CompileBroker::compiler_thread_loop 
        12       1  0x806ee60  CheckJvmType < CreateExecutionEnvironment < main 
        12       1  0x806ede8  MemAlloc < CreateExecutionEnvironment < main 
        12       1  0x806edc0  main 
         8       1  0x8071cb8  _strdup < ReadKnownVMs < CreateExecutionEnvironment < main 
         8       1  0x8071cf8  _strdup < ReadKnownVMs < CreateExecutionEnvironment < main
      

The output shows that the dbx debugger reports memory leaks if memory is not freed at the time the process is about to exit. However, memory that is allocated at initialization time and needed for the life of the process is often never freed in native code. Therefore, in such cases the dbx debugger can report memory leaks that are not leaks in reality.

Note that the example used two suppress commands to suppress the leaks reported in the virtual machine ( libjvm.so) and the Java support library ( libjava.so).

3.4.5 Using libumem to Find Leaks

Starting with Solaris 9 OS update 3, the libumem.so library and the modular debugger ( mdb) can be used to debug memory leaks. Before using libumem, you must preload the libumem library and set an environment variable as follows:

$

         LD_PRELOAD=libumem.so
$  
        export LD_PRELOAD

$  
        UMEM_DEBUG=default
$  
        export UMEM_DEBUG
      

Now, run the Java application but stop it before it exits. The following example uses truss to stop the process when it calls the _exit system call:

  $  
        truss -f -T _exit java MainClass 
         arguments

      

At this point you can attach the mdb debugger, as follows:

  $  
         mdb -p 
       pid
>::findleaks
      

The ::findleaks command is the mdb command to find memory leaks. If a leak is found, the findleaks command prints the address of the allocation call, buffer address, and nearest symbol.

It is also possible to get the stack trace for the allocation which resulted in the memory leak by dumping the bufctl structure. The address of this structure can be obtained from the output of the ::findleaks command. The description of the commands to perform these functions, as well as more information on using libumem to identify memory managements bugs.