Bug 118 – OOM during regenerating mipmaps

Bug 118 - OOM during regenerating mipmaps


Summary:	OOM during regenerating mipmaps

Status:	RESOLVED FIXED

Product:	Fiji
Classification:	Unclassified
Component:	TrakEM2
Version:	unspecified
Hardware:	PC Linux

Importance:	P2 normal
Assigned To:	Albert Cardona

URL:

Depends on:
Blocks:

Reported:	2009-12-07 14:27 CST by Stephan Saalfeld
Modified:	2010-08-21 13:29 CDT
CC List:	0 users

See Also:

Description Stephan Saalfeld 2009-12-07 14:27:06 CST

During regenerating mipmaps by browsing and the menu call `Project->regenerate mipmaps (selected images)', I get:

==================
ERROR:
java.lang.OutOfMemoryError: Java heap space
	at ij.process.TypeConverter.convertFloatToShort(TypeConverter.java:152)
	at ij.process.TypeConverter.convertToShort(TypeConverter.java:125)
	at ij.process.ImageProcessor.convertToShort(ImageProcessor.java:1884)
	at ini.trakem2.utils.Utils.convertTo(Utils.java:909)
	at ini.trakem2.persistence.FSLoader.generateMipMaps(FSLoader.java:1926)
	at ini.trakem2.persistence.FSLoader$7.call(FSLoader.java:2540)
	at ini.trakem2.persistence.FSLoader$7.call(FSLoader.java:2535)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:619)
==================

In addition, I get several:

*** ERROR: Can't generate mipmaps for patch 08apr15a_gb27932_A1_c1024_00009gr_00731ex.mrc z=1.0 #216

and

Cannot regenerate mipmaps for patch 08apr15a_gb27932_A1_c1024_00009gr_00853ex.mrc z=2.0 #340

without further explanations.

I am experiencing this on a large project whose images are located on a fileserver mounted in an in-house network through samba.  That is, files can be accessed but sometimes the delay seems large.  To prevent any missleading discussion in prior (;-)):  Not being able to read a file immediately shouldn't result in exceptional behaviour because the aim of TrakEM2 is to handle a large number of files, and to store them on a fileserver instead of a local harddrive is absolutely appropriate.

Comment 1 Stephan Saalfeld 2009-12-09 12:15:32 CST

This happens also while navigating through a project with not pre-generated mipmaps such that the mipmaps to be re-generated are accumulating in the queue.

The result is that random tiles are not re-generated.

Comment 2 Stephan Saalfeld 2009-12-09 13:51:43 CST

When navigating afterwards through the layers, the not-regenerated tiles are displayed at wrong contrast and aliased quality---so probably from the original files (but why then with wrong contrast?).

Going into such layer logs:

Cannot regenerate mipmaps for patch 08apr15a_gb27932_C5_c1024_00008gr_01362ex.mrc z=148.0 #19020

for each of these images.  But selecting them and doing `Project->Regenerate mipmaps (selected images)' works just fine.

Comment 3 Albert Cardona 2009-12-09 15:09:30 CST

So there are at least 2 issues here:
- insufficient memory clearance
- inconsistent test for "un-regenerability" of an image

As for the memory, could you try out the new "memory" branch I just pushed?
Among other things, it simplifies image preloading and removes waiting from the canvas offscreen thread, and it requires 150 Mb always free for each core, if there is that much physical memory (otherwise it warns you).

Comment 4 Albert Cardona 2009-12-09 15:49:09 CST

> When navigating afterwards through the layers, the not-regenerated tiles are
> displayed at wrong contrast and aliased quality---so probably from the original
> files (but why then with wrong contrast?).


This part is fixed now in master.

The data returned by Patch.createTransformedImage use to include the min and max but not anymore (since quite a while) and thus the Loader.fetchImage has to explicitly set the min and max.

Comment 5 Albert Cardona 2010-08-21 13:29:08 CDT

This error was fixed long ago.