NOTICE! This is a static HTML version of a legacy Fiji BugZilla bug.

The Fiji project now uses GitHub Issues for issue tracking.

Please file all new issues there.

Bug 990 - Mac OS out of memory
Mac OS out of memory
Status: RESOLVED FIXED
Product: Fiji
Classification: Unclassified
Component: Plugins
unspecified
Macintosh Mac OS
: P4 normal
Assigned To: ImageJ Bugs Mailing List
Depends on:
Blocks:
 
Reported: 2015-01-21 20:09 CST by Glen MacDonald
Modified: 2015-02-18 12:20 CST
3 users (show)

See Also:

Description Glen MacDonald 2015-01-21 20:09:59 CST
Montages are created by making a blank stack and using the Stack Inserter to add stacks. Each stack is closed before opening the next component stack. I can observe tht the stacks are closing.  the montages range from 2.5 to 9 Gb in size.  Component stacks are 200-300 Mb in size.  With 12 Gb allocated to Fiji, I keep running out of memory.  If the next blank montage exceeds remaining memory I get a formal out of memory error.  If the blank montage can be created, then Fiji will stitch until one of the stacks exceeds the available memory then stop with a message to the log window.  the only solution is to quit Fiji.  calls to the garbage collector do not help.   SciFio is not being used. IJ2 Data structure has no effect.  Is this a bug or is there a workaround?
thanks, Glen

Information about your version of Java:

  os.arch => x86_64
  os.name => Mac OS X
  os.version => 10.9.5
  java.version => 1.6.0_65
  java.vendor => Apple Inc.
  java.runtime.name => Java(TM) SE Runtime Environment
  java.runtime.version => 1.6.0_65-b14-462-11M4609
  java.vm.name => Java HotSpot(TM) 64-Bit Server VM
  java.vm.version => 20.65-b04-462
  java.vm.vendor => Apple Inc.
  java.vm.info => mixed mode
  java.awt.graphicsenv => apple.awt.CGraphicsEnvironment
  java.specification.name => Java Platform API Specification
  java.specification.version => 1.6
  sun.cpu.endian => little
  sun.desktop => null
  file.separator => /

The up-to-date check says: CHECK_TURNED_OFF

Information relevant to JAVA_HOME related problems:

  JAVA_HOME is set to: null
  imagej.dir => /Applications/Fiji.app

Information about the version of each plugin:

Activated update sites:
ImageJ: http://update.imagej.net/ (last check:20150116081452)
Fiji: http://fiji.sc/update/ (last check:20150116084028)
3D ImageJ Suite: http://sites.imagej.net/Tboudier/ (last check:20141207235245)
Bio-Formats: http://sites.imagej.net/Bio-Formats/ (last check:20150114182640)
Biomedgroup: http://sites.imagej.net/Biomedgroup/ (last check:20141216080640)
BioVoxxel: http://sites.imagej.net/BioVoxxel/ (last check:20150112120702)
CMCI-EMBL: http://sites.imagej.net/Miura/ (last check:20140718070940)
FFMPEG: http://fiji.sc/~schindelin/ffmpeg-plugins/ (last check:20130606082449)
IMCF Uni Basel: http://sites.imagej.net/UniBas-IMCF/ (last check:20141116104718)
Morphology: http://sites.imagej.net/Landini/ (last check:20140723075558)
MOSAIC ToolSuite: http://mosaic.mpi-cbg.de/Downloads/update/Fiji/MosaicToolsuite/ (last check:20150120074915)
NucleusJ: http://sites.imagej.net/PouletAxel/ (last check:20141101023602)
Stowers: http://research.stowers.org/imagejplugins/updates/ (last check:20150106143220)
WormSizer: http://sites.imagej.net/Bradtmoore/ (last check:20140504092939)

Files not up-to-date:
  2418366b (MODIFIED) 20150121161955 Contents/Info.plist
  1ad3be0d (LOCAL_ONLY) 20141119213514 jars/jpedalSTD.jar
  07f4044a (LOCAL_ONLY) 20140612110153 macros/CLAHE stack.ijm
  dc3f77db (LOCAL_ONLY) 20140612103947 macros/Float stack.ijm
  37aecc8a (LOCAL_ONLY) 20141007123738 macros/Old versions/FVstack inserter_.ijm
  a6b787e7 (LOCAL_ONLY) 20140625172146 macros/Old versions/Stereology_grid2_4.ijm
  65fe498d (LOCAL_ONLY) 20120730154750 macros/Old versions/drift3b.ijm
  07eaea1d (LOCAL_ONLY) 20120821154843 macros/Old versions/waterjet2.ijm
  9d962521 (LOCAL_ONLY) 20120618120603 macros/PIMP_2f-3ivariant.ijm
  b69e6838 (LOCAL_ONLY) 20120622105927 macros/PIMP_2g.ijm
  eb509ed0 (LOCAL_ONLY) 20120717095350 macros/PIMP_2gbgsubtraction.ijm
  e56187b3 (LOCAL_ONLY) 20120710174431 macros/PIMP_2gtests.ijm
  41baa729 (LOCAL_ONLY) 20120717105146 macros/PIMP_2h.ijm
  c46ef207 (LOCAL_ONLY) 20120717110725 macros/PIMP_2i.ijm
  22596c68 (LOCAL_ONLY) 20120615161654 macros/PIMP_finalGHMarrays2e3DguassThold.ijm
  7fb3cc04 (LOCAL_ONLY) 20120615161647 macros/PIMP_finalGHMarrays2e3Dguassian.ijm
  87255328 (LOCAL_ONLY) 20140206150200 macros/delete2slices.ijm
  fd6a1c44 (LOCAL_ONLY) 20141203103226 macros/print properties_.ijm
  9aafb25c (LOCAL_ONLY) 20141203114918 macros/random numbers_.ijm
  9f59bd3b (LOCAL_ONLY) 20141105091214 macros/tools/Calibration_Menu.ijm
  bf41d3c0 (LOCAL_ONLY) 20141105091215 macros/tools/List_Folder_Menu.ijm
  8364abb4 (LOCAL_ONLY) 20141105091214 macros/tools/Segment_Profile.ijm
  f3cfc0b6 (LOCAL_ONLY) 20141105091214 macros/tools/Shortcuts_Menu.ijm
  74f75823 (LOCAL_ONLY) 20120719100435 macros/toolsets/3DClusterAnalysis.ijm
  5e7a3919 (LOCAL_ONLY) 20140815160318 macros/toolsets/Analyze Utricles-BioFormats.ijm
  0fe955a3 (LOCAL_ONLY) 20120305144333 macros/toolsets/Andrew Macro4b.ijm
  6cc0aae2 (LOCAL_ONLY) 20110901171448 macros/toolsets/Arminda timelapse macro 3.8b.ijm
  f4074bbb (LOCAL_ONLY) 20120702084321 macros/toolsets/BPCounting3g.ijm
  3f9a11ae (LOCAL_ONLY) 20150116170843 macros/toolsets/Convert to Unsigned_.ijm
  8bf6c467 (LOCAL_ONLY) 20141007164319 macros/toolsets/FV Optical Disector.ijm
  48c586d9 (LOCAL_ONLY) 20141212165342 macros/toolsets/FV-OME_Montage_1.ijm
  6ec9c1fd (LOCAL_ONLY) 20150114153429 macros/toolsets/FV_Montage_ACDC3.ijm
  8e86396d (LOCAL_ONLY) 20131018145317 macros/toolsets/FWHM.ijm
  2fe75fec (LOCAL_ONLY) 20131107171559 macros/toolsets/FWHM3.ijm
  9e64fe4d (LOCAL_ONLY) 20141104120029 macros/toolsets/FWHMJayb.ijm
  750b79cc (LOCAL_ONLY) 20141204150658 macros/toolsets/FWHMJayc.ijm
  c68a09eb (LOCAL_ONLY) 20120911102147 macros/toolsets/Fill ROI Macro.ijm
  edbc43b2 (LOCAL_ONLY) 20141205145444 macros/toolsets/Find and List Files by Name_.ijm
  ead280e3 (LOCAL_ONLY) 20120808153016 macros/toolsets/GTTR Segmentation3.ijm
  77158612 (LOCAL_ONLY) 20150102131424 macros/toolsets/Import Channel Order3.98c.ijm
  17d1aba0 (LOCAL_ONLY) 20140404113537 macros/toolsets/Merge HuPro Quadrants_3b.ijm
  9f68e1dd (LOCAL_ONLY) 20150109143800 macros/toolsets/Multi-Line Intensity Plot.ijm
  01179b26 (LOCAL_ONLY) 20120913130241 macros/toolsets/Multiply.ijm
  6fabf894 (LOCAL_ONLY) 20141114145758 macros/toolsets/NewSlidebook2_.ijm
  680ee77a (LOCAL_ONLY) 20141111130926 macros/toolsets/NewSlidebook_.ijm
  4651471a (LOCAL_ONLY) 20121116153429 macros/toolsets/PIMP_3_Batch.ijm
  0c5c4a4c (LOCAL_ONLY) 20121116153510 macros/toolsets/PIMP_3_Single file.ijm
  4b5104ee (LOCAL_ONLY) 20120627100227 macros/toolsets/Placenta macro3.ijm
  1e56302a (LOCAL_ONLY) 20141118152407 macros/toolsets/RAMSlidebook2_.ijm
  0c256129 (LOCAL_ONLY) 20141105091214 macros/toolsets/ROI Manager Tools.ijm
  ca90b65b (LOCAL_ONLY) 20141002163033 macros/toolsets/Scaled sampling grid.ijm
  24c3ebb4 (LOCAL_ONLY) 20131218123203 macros/toolsets/Scan macro.ijm
  7b436426 (LOCAL_ONLY) 20141107172453 macros/toolsets/Slidebook montage_.ijm
  cb9c4438 (LOCAL_ONLY) 20141104113030 macros/toolsets/Stereology_grid2_6.ijm
  232bca5e (LOCAL_ONLY) 20120910121056 macros/toolsets/Temporal Time Encoder.ijm
  3af0fe66 (LOCAL_ONLY) 20141105091214 macros/toolsets/Toolset Creator.ijm
  3d17a094 (LOCAL_ONLY) 20140115144252 macros/toolsets/Utricle_RGB_merge2_2.ijm
  618286ae (LOCAL_ONLY) 20121115152953 macros/toolsets/batchStackreg.ijm
  e8677e33 (LOCAL_ONLY) 20141028141133 macros/toolsets/ctbp2a macro.ijm
  6cedb193 (LOCAL_ONLY) 20120730162148 macros/toolsets/drift3c.ijm
  fc424882 (LOCAL_ONLY) 20120305133512 macros/toolsets/macro development/Andrew Macro4a.ijm
  0b91930b (LOCAL_ONLY) 20111007164331 macros/toolsets/macro development/Arminda 3.98test.ijm
  07f111f8 (LOCAL_ONLY) 20120620161839 macros/toolsets/macro development/FWHM.ijm
  607b8de2 (LOCAL_ONLY) 20110928161804 macros/toolsets/macro development/Import Channel Order3.98.ijm
  891da2e3 (LOCAL_ONLY) 20110831161129 macros/toolsets/macro development/Import_Channel_Order3.8atlapse.txt
  c1e7d84a (LOCAL_ONLY) 20110831163521 macros/toolsets/macro development/Import_Channel_Order3.8btlapse.txt
  3fedd17e (LOCAL_ONLY) 20110901165057 macros/toolsets/macro development/Import_Channel_Order3.8btlapse.txt.ijm
  6ddde9e7 (LOCAL_ONLY) 20110124183506 macros/toolsets/macro development/Import_Channel_Order3.8f.txt
  76c91a2c (LOCAL_ONLY) 20120529122545 macros/toolsets/macro development/PIMP_final.ijm
  45be256a (LOCAL_ONLY) 20120601113233 macros/toolsets/macro development/PIMP_finalGHMarrays2b.ijm
  de11be1e (LOCAL_ONLY) 20120403145549 macros/toolsets/macro development/Scale CellVizio Tiff stack.ijm
  ee6d3193 (LOCAL_ONLY) 20120221135146 macros/toolsets/macro development/testscale.ijm
  83d10173 (LOCAL_ONLY) 20120712161724 macros/toolsets/plot coordinates.ijm
  d1a3a016 (LOCAL_ONLY) 20120712161713 macros/toolsets/plotresults.ijm
  833e093f (LOCAL_ONLY) 20140926165030 macros/toolsets/testbatchopen.ijm
  c1dcbd92 (LOCAL_ONLY) 20121008113151 macros/toolsets/transition stacks.ijm
  301245dd (LOCAL_ONLY) 20120822090138 macros/toolsets/waterjet2deltaf.ijm
  81fecfd4 (LOCAL_ONLY) 20120822172438 macros/toolsets/waterjet3.ijm
  d054d71f (LOCAL_ONLY) 20120717163446 macros/use copyand restore ROI.ijm
  d5294aaa (LOCAL_ONLY) 20141210131239 plugins/Hill_Shade.class
  fb824711 (MODIFIED) 20141210130824 plugins/Ridge_Detect.jar
  b2afde18 (LOCAL_ONLY) 20141216103055 plugins/cell_counter.jar
Comment 1 Wayne Rasband 2015-01-21 20:54:40 CST
Could you provide a small test macro that we can use to reproduce this problem? The following macro works as expected. The memory used by the four 512x512x100 temporary stacks it opens (25MB each) is reclaimed.

  newImage("Montage", "8-bit ramp", 1024, 1024, 100);
  x=0; y=0
  for (i=0; i<4; i++) {
    newImage("Stack", "8-bit ramp", 512, 512, 100);
    run("Insert...", "source=Stack destination=Montage x=&x y=&y");
    close();
    x += 512;
    if (x==1024) {
       x = 0;
       y += 512;
    }
  }
  setBatchMode(false);
Comment 2 Glen MacDonald 2015-02-06 12:21:07 CST
Hi Wayne,
First time replying on BugZilla, so hoping this is correct.
Here are macros that [1] generates 4 stacks then [2] opens files, creates a montage and also prints out memory usage at each step.  I’ve tried with various stack sizes. Behavior is not entirely consistent on any computer, sometimes all memory is released when the macro finishes with small stacks. But never with large stacks.  
When the finished montage does release memory,I was surprised that the memory for each component stack is not released when that stack is closed, only when the macro exits.  This means the maximum size of a montage is 1/2 the available memory.   I ran the debugger bug but saw nothing intelligible to me.  


macro "make stacks [1]"{
	target=getDirectory("Source");
	for (s=0;s<4;s++){
		newImage("HyperStack", "16-bit ramp", 1024, 1024, 3,100,1);
		saveAs("Tiff",target+"Stack000"+s);
	}
}

 macro "make montages [2]"{
 	dir=getDirectory("Source");
 	dirList=getFileList(dir);
 	fList=newArray(0);
 	print("Start Macro- Free memory:",IJ.freeMemory(), "Currently in use:",IJ.currentMemory());
	newImage("Hyperstack", "16-bit composite-mode", 2048,2048,3,100,1);	
	print("Create Stack- Free memory:",IJ.freeMemory(), "Currently in use:",IJ.currentMemory());
	xx=0;
	yy=0;
	strname="montage";
	grayopt="color_mode=Grayscale view=Hyperstack stack_order=XYCZT";
	for(i=0;i<dirList.length;i++){
		if(endsWith(dirList[i],".tif")||(endsWith(dirList[i],".tiff")))
			fList=Array.concat(fList,dirList[i]);
	}	
		for(f=0;f<fList.length;f++){
			fpath=dir+fList[f];
		//showStatus("Opening stack");
		run("Bio-Formats Windowless Importer", "open=[fpath] [grayopt]");
		rename("Source");
		run("Insert...", "source=Source destination=Hyperstack x=xx y=yy");
		run("Close");
			print("Add Stack "+f+":",IJ.freeMemory(), "Currently in use:",IJ.currentMemory());
		xx+=1024;
  		if (xx==2048) {
     		xx = 0;
      		yy += 1024;
   	}
	}
	run("Close");
	//print("Free memory:",IJ.freeMemory(), "Currently in use:",IJ.currentMemory());
}



regards

Glen MacDonald
	Core for Communication Research
Virginia Merrill Bloedel Hearing Research Center
	Cellular Morphology Core
Center on Human Development and Disability
Box 357923
University of Washington
Seattle, WA 98195-7923  USA
(206) 616-4156
glenmac@uw.edu







On Jan 21, 2015, at 6:54 PM, bugzilla@fiji.sc wrote:

Wayne Rasband changed bug 990 
What	Removed	Added
CC	  	wsr@nih.gov

Comment # 1 on bug 990 from Wayne Rasband
Could you provide a small test macro that we can use to reproduce this problem?
The following macro works as expected. The memory used by the four 512x512x100
temporary stacks it opens (25MB each) is reclaimed.

 newImage("Montage", "8-bit ramp", 1024, 1024, 100);
 x=0; y=0
 for (i=0; i<4; i++) {
   newImage("Stack", "8-bit ramp", 512, 512, 100);
   run("Insert...", "source=Stack destination=Montage x=&x y=&y");
   close();
   x += 512;
   if (x==1024) {
      x = 0;
      y += 512;
   }
 }
 setBatchMode(false);


You are receiving this mail because:
	• You are on the CC list for the bug.
	• You reported the bug.
Comment 3 Wayne Rasband 2015-02-08 20:39:47 CST
Hi Glen,

I am not able to consistently reproduce this problem. Yesterday, when testing your macro, I noticed that sometimes not all memory was reclaimed after the macro finished and I ran the garbage collector by clicking on the status bar or "Memory" window. Today, however, the memory is always reclaimed after running the macro and clicking in the "Memory" window.

The following version of your macro displays the "Memory" window and creates 10 montages. The "Make Montage" macro is first so you can run it by clicking on "Run" in Fiji's script editor. Watch the "Memory" window as the macro runs and you will see that memory is reclaimed as needed or you click in the "Memory" window. This version of the macro also fixes a few problems with passing variables and it removes the 'grayopt' string, which was not seen by the Insert command because it was not passed correctly.


  macro "Make Montage [1]" {
     dir=getDirectory("Source");
     n = 10; 
     list=getFileList(dir);
      for (i=0; i<n; i++) {
        montage = "Montage-"+(i+1)+"/"+n;
        newImage(montage, "16-bit composite-mode", 2048,2048,3,100,1);    
        doCommand("Monitor Memory...");
        x=0; y=0;
        for (f=0; f<list.length; f++) {
           fpath=dir+list[f];
           //open(fpath);
           run("Bio-Formats Windowless Importer", "open=&fpath");
           rename("Source");
           run("Insert...", "source=Source destination=&montage x=&x y=&y");
           close;
           x += 1024;
           if (x==2048) {
              x = 0;
              y += 1024;
           }
        }
        close;
     }
  }

  macro "Make Stacks [2]" {
      target=getDirectory("Choose Directory");
      newImage("HyperStack", "16-bit ramp", 1024, 1024, 3,100,1);
      for (s=0;s<4;s++)
          saveAs("Tiff",target+"Stack000"+s);
  }
Comment 4 Glen MacDonald 2015-02-13 20:19:51 CST
Hi Wayne,
thanks for the cleanup on the macro. I still have much to learn (or remember). I ran it on 5 Macs, all OS10.9.5, latest Fiji 1.49m RC24, Java 1.60_65. oldest was Early 2009 MacPro, newest was Late 2013 iMac. All gave variable results: sometimes releasing all memory at each interation, sometimes retaining 1,2 or 3 stacks or a montage until the next montage loop, other times releasing nothing on every montage.  I'm not seeing that the garbage collector is doing anything when clicking on status bar or calling from the macro. Behavior is different every time Fiji starts up.  I've read that too many GC calls can result in OOM errors. Could there be Java settings I should investigate?  I've never before had an iterative operation behave like this.  Interestingly, the older 2 macs had more OOM, they are Xeons instead of i5 or i7.  

Regards,
glen
Comment 5 Curtis Rueden 2015-02-13 21:53:39 CST
Glen, might I suggest trying with Java 7 or Java 8 instead of Java 6? Java 7 fixes many of Java 6's problems.

http://imagej.net/FAQ#How_do_I_launch_ImageJ_with_a_different_version_of_Java.3F

(As an aside: there will soon be a new version of the ImageJ launcher that makes it easier to use Java 7 or 8 on OS X without needing to install Java 6 anymore.)
Comment 6 Wayne Rasband 2015-02-14 19:16:24 CST
Hi Glen,

You should not expect memory to be released at each iteration or when the macro finishes. The garbage collector will run as needed to reclaim memory and prevent OOM errors. If you are still getting OOM errors, try replacing

   run("Bio-Formats Windowless Importer", "open=&fpath");

with

   open(fpath);
Comment 7 Glen MacDonald 2015-02-18 11:07:13 CST
Hi Curtis,
Java 1.7.0_75 seems to have resolved the issue.  Good to hear there is a Java selection option in the works.  I was going to inquire on the devel list whether Fiji could check for installed Java versions and provide a checkbox for choosing which ver. to run on restart.  

Its interesting that memory use reported by the memory monitor and IJ.getcurrent memory drops as the size of the test montage is increased.  

Now that Huygens runs under Yosemite, I'm going to start upgrading systems.  

thanks,
Glen
Comment 8 Curtis Rueden 2015-02-18 12:20:38 CST
Glad to hear that Java 7 fixes the issue!