NOTICE! This is a static HTML version of a legacy Fiji BugZilla bug.

The Fiji project now uses GitHub Issues for issue tracking.

Please file all new issues there.

Bug 835 - Fiji hangs when installing macro tool
Fiji hangs when installing macro tool
Status: ASSIGNED
Product: Fiji
Classification: Unclassified
Component: Other
unspecified
PC Linux
: P5 enhancement
Assigned To: Curtis Rueden
Depends on:
Blocks:
 
Reported: 2014-07-04 00:29 CDT by Ilan Tal
Modified: 2014-08-31 00:45 CDT
4 users (show)

See Also:


Attachments
install from more (23.28 KB, image/png)
2014-07-23 00:29 CDT, Ilan Tal
install from plugin tools (22.93 KB, image/png)
2014-07-23 00:30 CDT, Ilan Tal
startup (24.44 KB, image/png)
2014-07-23 00:30 CDT, Ilan Tal

Description Ilan Tal 2014-07-04 00:29:59 CDT
1) Make a macro tool like
  macro "Postage Action Tool - R0063R0663R0c63Ra063Ra663Rac63" {
     print("Tile");
  }

2) Call the file "Postage_Action_Tool.ijm".
3) Place this file in Fiji.app/plugins/Tools
4) Start Fiji and press ">>" on the tool bar and choose "Postage Action Tool".

Fiji will hang and not accept any input from the menu or from the keyboard.
I wanted to make a thread dump by using ctrl+\ but the keyboard had no response.
I am using a Linux 32 bit machine but I don't believe it is OS dependent.

I had a discussion with Wayne Rasband on the problem in Bug 821. The same macro in Fiji.app/macros/toolsets will install properly and run. The problem with this directory is the installation is forgotten on the next startup.

In any case this bug is about the system hanging.
Thanks,
Ilan
Comment 1 Ilan Tal 2014-07-17 04:50:11 CDT
This is what Wayne wrote in bug 821:

Do you have a "Postage_Action_Tool.ijm" file in Fiji.app/plugins/tools? Do you have a "Postage Action Tool" command in the Plugins>Tool menu?

In ImageJ, when I select Plugins>Tools>Postage Action Tool, it install the Postage Action Tool in the toolbar, and that tool remains in the toolbar when I restart ImageJ, as long as there are seven or fewer tools. In Fiji, when I select Plugins>Tools>Postage Action Tool, the tool is not installed. Instead, "Tile" is displayed in the Log window. This is a bug.
Comment 2 Ilan Tal 2014-07-23 00:29:00 CDT
Created attachment 181
install from more
Comment 3 Ilan Tal 2014-07-23 00:30:03 CDT
Created attachment 182
install from plugin tools
Comment 4 Ilan Tal 2014-07-23 00:30:49 CDT
Created attachment 183
startup
Comment 5 Ilan Tal 2014-07-23 01:23:20 CDT
My debugging environment is NetBeans and I use ImageJ, not Fiji for memory considerations. For example there have been cases where a large whole body scan would not fit into memory for debugging.

With this restriction, I was able to find out what goes on inside ImageJ when I install a macro tool. I can verify that ImageJ works properly but Fiji fails. I would like to be able to work with someone who can check the Fiji environment.

I will describe what happens in 3 different cases:
1) installing using the toolbar "More" command and choosing the macro.
2) installing using Plugins -> Tools and choosing the macro.
3) restarting ImageJ where the tools are reinstalled from the registry values.

In Fiji case 1 causes a crash which freezes the keyboard. Fiji case 2 causes the macro to run but it doesn't get installed to the toolbar. Case 3 cannot be checked in Fiji since installation of the tool always fails.

In the case of both a plugin jar file as well as a macro, both receive the run command. The plugin jar is simpler since its run does the install. The macro runs through some 6 different run commands. The stack dumps are shown in the 3 attachments.

The crucial point is the macro name. Here is the code in macro runner which does the job
		else if (name.endsWith("Tool.ijm") || name.endsWith("Tool.txt")
		|| name.endsWith("Menu.ijm") || name.endsWith("Menu.txt"))
			(new MacroInstaller()).installTool(Menus.getPlugInsPath()+name);

My tool is called Postage_Action_Tool.ijm and the Tool.ijm is the key. In ImageJ all works correctly so that it reaches this point. In case 2 of Fiji, it won't reach this point - instead of installing the tool, it runs it. In case 1 Fiji just crashes.

Now to consider the different cases. In case 1 the interesting things start at

			String label = item.getLabel();
			String cmd = item.getActionCommand();
			boolean isTool = cmd.equals("Tool") || cmd.equals("Plugin Tool");
			if (!(label.equals("Help...")||label.equals("Remove Custom Tools")) && !isTool)
				currentSet = label;
			if (isTool) {
				if (cmd.equals("Tool")) // built in tool
					installBuiltinTool(label);
				else  // plugin or macro tool in ImageJ/plugins/Tools
					IJ.run(label);
				return;

in itemStateChanged. The cmd is "Plugin Tool" and not "Tool" so that it skips the installBuiltinTool and does the IJ.run(label). In Fiji is it Tool, or Plugin Tool?

Case 2 does a run directly which in ImageJ ends up checking the file name and causing the install, whereas in Fiji it just runs with no install. Why does Fiji act differently?

Case 3 is the startup which works in ImageJ and can't be reached in Fiji. The code is

	public void installStartupTools() {
		if (IJ.debugMode) IJ.log("Toolbar.installStartupTools");
		for (int i=0; i<=6; i++) {
			String name = Prefs.get(TOOL_KEY + (i/10)%10 + i%10, "");
			if (IJ.debugMode) IJ.log("  "+i+" "+name);
			if (name.equals("")) continue;
			installingStartupTool = true;
			boolean ok = installBuiltinTool(name);
			if (!ok) {
				if (name.endsWith("Menu Tool"))
					name = name.substring(0, name.length()-5);
				Hashtable commands = Menus.getCommands();
				if (commands!=null && commands.get(name)!=null)
					IJ.run(name);
			}
			installingStartupTool = false;
		}
	}

The 6 additional keys are checked from the registry. Clearly this is not a built in tool so installBuiltinTool will fail. It then sees that commands are not null and commands.get(name) is in the hashtable, so the IJ.run(name) takes over. This reaches the same point as cases 1 and 2 causing the tool to be installed.

Again, my question is: who has a Fiji environment and can check what the difference is in Fiji?

Hoping to hear from someone,
Ilan
Comment 6 Ilan Tal 2014-07-23 02:45:24 CDT
Hi Wayne,
I would like to make an additional request for my external jar file plugin. You have these really great hints for most of your built in buttons, but they are hard coded. I would really like to have a hint for my external button tool as well.

For example I have in my code, which I have just changed for experimental purposes:

	public String getToolName() {
		return "Window Level Tool (right click for Reset, Auto)";
	}

The problem is that the hint will kill the possibility to save the tool to the registry because it won't fit the command. What I would like is to save to the registry the original name, without the hint. That way I get the hint on the status bar and the registry value fits the command.

I added a "second chance" to the setPrefs and hope you agree to the change:

	private void setPrefs(int id) {
		if (doNotSavePrefs)
			return;
		boolean ok = isBuiltInTool(names[id]);
		String registryValue = instance.names[id];
		if (!ok) {
			Hashtable commands = Menus.getCommands();
			String name = names[id];
			if (name.endsWith("Menu Tool"))
				name = name.substring(0, name.length()-5);
			ok = commands!=null && commands.get(name)!=null;
			// give a second chance for hints, i.e. (hint 123)
			if( !ok) {
				int i = name.indexOf(" (");
				if( i > 0) {
					name = name.substring(0, i);
					ok = commands!=null && commands.get(name)!=null;
					if(ok) registryValue = name;
				}
			}
		}
		if (!ok)
			return;
		int index = id - CUSTOM1;
		String key = TOOL_KEY + (index/10)%10 + index%10;
		Prefs.set(key, registryValue);
	}

Please let me know.
Thanks,
Ilan
Comment 7 Mark Hiner 2014-07-28 08:30:13 CDT
Confirmed that this is a bug in the compatibility layer between ImageJ 1.x and ImageJ 2.

See this issue: https://github.com/imagej/imagej-legacy/issues/69
Comment 8 Ilan Tal 2014-07-29 07:35:56 CDT
Hi Mark,
If you will be able to gain control in the Fiji environment, there are actually 2 sides to the problem. You can see the stack dump attachments I included.

Case 1 is activating the plugin from >> "More tools". This is the one which causes the EDT crash which you caught. The attachment "Install from more" shows the stack dump which runs correctly under ImageJ.

Case 2 doesn't result in any crash, but it doesn't do the correct thing in Fiji. Instead of installing the tool, Fiji just runs it. Again the attachment "Install from plugin tools" shows the stack dump which runs correctly under ImageJ. Apparently Fiji follows a different route in its execution of the command and the question is: what causes it to deviate from the correct path?

I still haven't managed to set up a Fiji debug environment, so I can't check it. If you can somehow check it, it would be a great help.

Thanks,
Ilan
Comment 9 Ilan Tal 2014-08-24 06:33:57 CDT
Hi Mark,
This is the case "Install from more" which is attachment 181. The process starts at Toolbar.itemStateChanged:1270 and the critical difference is at Executer.runCommand:131. The interesting code is

	void runCommand(String cmd) {
		Hashtable table = Menus.getCommands();
		String className = (String)table.get(cmd);
		if (className!=null) {
			String arg = "";
			if (className.endsWith("\")")) {
				// extract string argument (e.g. className("arg"))
				int argStart = className.lastIndexOf("(\"");
				if (argStart>0) {
					arg = className.substring(argStart+2, className.length()-2);
					className = className.substring(0, argStart);
				}
			}
			if (IJ.shiftKeyDown() && className.startsWith("ij.plugin.Macro_Runner") && !Menus.getShortcuts().contains("*"+cmd))
    			IJ.open(IJ.getDirectory("plugins")+arg);
    		else
				IJ.runPlugIn(cmd, className, arg);

There is a hash table of commands and the question is: what is the command which gets picked up? In the ImageJ environment the className comes back as

ij.plugin.Macro_Runner("Tools/Postage_Action_Tool.ijm")
This ends with ") so it is parsed into 2 strings:
className = ij.plugin.Macro_Runner
and arg = Tools/Postage_Action_Tool.ijm

In Fiji something else comes out. The className comes back as
script:/home/ilan/Fiji.app/plugins/Tools/Postage_Action_Tool.ijm

Of course no parsing is performed and arg=""
Not surpringly the IJ.runPlugIn(cmd, className, arg)
works totally differently. Later on down the line, this results in the crash.
So the hash table has been changed. Where it has been changed and why it has been changed, I don't know, but the change results in a crash.
If you have any idea why it isn't picking up "ij.plugin.Macro_Runner" but instead is using script:/...., I would be interested in hearing about it.

Now I'll go look at the second case.
Thanks,
Ilan
Comment 10 Ilan Tal 2014-08-24 07:06:33 CDT
This is the case Install from plugin tools, attachment 182.
The ImageJ environment returns the same
ij.plugin.Macro_Runner("Tools/Postage_Action_Tool.ijm")

Likewise Fiji returns its same
script:/home/ilan/Fiji.app/plugins/Tools/Postage_Action_Tool.ijm

In this case the macro is executed instead of being installed. The execution involves a log file print, but there is no crash. Why the first case crashes must be some unwinding of a different stack sequence, after it has attempted to write to the log file.
In this case the bug is that it executes the macro instead of installing it. At least it is more friendly in that no crash occurs.

Still the basic problem is the change in the hash table of menu commands.
I would like to help, but I don't know where to go from here.

Ilan
Comment 11 Ilan Tal 2014-08-25 07:12:53 CDT
I managed to track down the problem to Menus.java. Here is the relevant code

	/** Returns a list of the plugins in the plugins menu. */
	public static synchronized String[] getPlugins() {
		File f = pluginsPath!=null?new File(pluginsPath):null;
		if (f==null || (f!=null && !f.isDirectory()))
			return null;
		String[] list = f.list();
		if (list==null)
			return null;
		Vector v = new Vector();
		jarFiles = null;
		macroFiles = null;
		for (int i=0; i<list.length; i++) {
			String name = list[i];
			boolean isClassFile = name.endsWith(".class");
			boolean hasUnderscore = name.indexOf('_')>=0;
			if (hasUnderscore && isClassFile && name.indexOf('$')<0 ) {
				name = name.substring(0, name.length()-6); // remove ".class"
				v.addElement(name);
			} else if (hasUnderscore && (name.endsWith(".jar") || name.endsWith(".zip"))) {
				if (jarFiles==null) jarFiles = new Vector();
				jarFiles.addElement(pluginsPath + name);
			} else if (validMacroName(name,hasUnderscore)) {
				if (macroFiles==null) macroFiles = new Vector();
				macroFiles.addElement(name);
			} else {
				if (!isClassFile)
					checkSubdirectory(pluginsPath, name, v);
			}
		}
		list = new String[v.size()];
		v.copyInto((String[])list);
		StringSorter.sort(list);
		return list;
	}
	
	/** Looks for plugins and jar files in a subdirectory of the plugins directory. */
	private static void checkSubdirectory(String path, String dir, Vector v) {
		if (dir.endsWith(".java"))
			return;
		File f = new File(path, dir);
		if (!f.isDirectory())
			return;
		String[] list = f.list();
		if (list==null)
			return;
		dir += "/";
		int classCount=0, otherCount=0;
		String className = null;
		for (int i=0; i<list.length; i++) {
			String name = list[i];
			boolean hasUnderscore = name.indexOf('_')>=0;
			if (hasUnderscore && name.endsWith(".class") && name.indexOf('$')<0) {
				name = name.substring(0, name.length()-6); // remove ".class"
				v.addElement(dir+name);
				classCount++;
				className = name;
				//IJ.write("File: "+f+"/"+name);
			} else if (hasUnderscore && (name.endsWith(".jar") || name.endsWith(".zip"))) {
				if (jarFiles==null) jarFiles = new Vector();
				jarFiles.addElement(f.getPath() + File.separator + name);
				otherCount++;
			} else if (validMacroName(name,hasUnderscore)) {
				if (macroFiles==null) macroFiles = new Vector();
				macroFiles.addElement(dir + name);
				otherCount++;
			} else {
				File f2 = new File(f, name);
				if (f2.isDirectory()) installSubdirectorMacros(f2, dir+name);
			}
		}
		if (Prefs.moveToMisc && classCount==1 && otherCount==0 && dir.indexOf("_")==-1)
			v.setElementAt("Miscellaneous/" + className,
				v.size() - 1);
	}

The problem is macroFiles never gets a non null value in Fiji. In ImageJ, it works with no problems. In the end the problem comes down to

	private static boolean validMacroName(String name, boolean hasUnderscore) {
		return (hasUnderscore&&name.endsWith(".txt"))||name.endsWith(".ijm")||name.endsWith(".js")
			||(name.endsWith(".bsh")&&!isFiji)||(name.endsWith(".py")&&!isFiji);
	}
 which returns a value of FALSE for a valid macro name. I verified that this routine hasn't changed even in the latest 1.49f. It should return TRUE if the name end in ".txt" and has an underscore, or if name ends in ".ijm" without any connection to an underscore. Again, in ImageJ it works fine, but not in Fiji.

The process begins in getPlugins where macroFiles gets initialized to null. From there it looks for valid macros to store. In my case I've got Postage_Action_Tool.ijm which is a perfectly valid macro file name. This file name is in the Tools directory inside the plugins directory.

The getPlugins gets a file list where the interesting entry is Tools. Of course Tools isn't a macro so it gets down to checkSubdirectory(...). Inside checkSubdirectory there is directory list which includes my file Postage_Action_Tool.ijm. Even though this is a perfectly valid macro name, validMacroName insists it isn't valid. Thus macroFiles never gets a non null value and no elements are added to it.

I thought maybe there were some invisible characters which might cause the endsWith(".ijm") to fail, so I copied over the exact same file from ImageJ into Fiji. I was going to suggest to add a trim to the list line

String name = list[i].trim();

but now I have strong doubts that it would help. This is especially after I looked at some other macros which should have been added. They are

About_Plugins_Macros.txt
batch_convert_any_to_tif.txt
Polygon_.txt
RGB_Histogram.txt

All of them should have passed the test and none of them passed in fact. These are all .txt files, but they all have underscores.

So the problem is wider than the bug I reported, but it also affects me, which is the part which is most annoying to me. Someone may be familiar with one or more of the above macros. If I knew what to expect from them, I might verify that they work differently in ImageJ.

Thanks,
Ilan
Comment 12 Ilan Tal 2014-08-26 00:26:45 CDT
I think it is time for a minor overhaul on Menus.java. I took Mark's idea of running the actual, up-to-date Fiji program and attaching Netbeans to it. This gives me the ability to look at all the variables and to compare the run of ImageJ to the run of Fiji. It is a very powerful idea.

Inside validMacroName there is a isFiji static variable which I noticed was always false. This morning I decided to have a look. This code is in the Menus.java constructor

		isFiji = title!=null && title.contains("Fiji");

and it clearly is no longer working. It used to be that Fiji was started by fiji-linux, but now ImageJ-linux32 is used. A breakpoint on the constructor shows that Fiji returns a title of ImageJ. So if the 2 cases in validMacroName still need to depend upon Fiji or not Fiji, this needs to be fixed (how is a very good question).

Now back to my problem with macros, which has been extended to many macros I wasn't even aware of. I have a Window_Level_Tool.jar and a Postage_Action_Tool.ijm inside the Tools folder. Both work perfectly fine under ImageJ. The Window_Level_Tool.jar works fine under Fiji as well but Postage_Action_Tool.ijm fails under Fiji. Why?

Yesterday I saw that name.endsWith(".jar") picks up my jar file while name.endsWith(".ijm") works in ImageJ and fails in Fiji. Why? One possibility is there is some unseen character in the file name. The trim() command would remove this problem. Another possibility is name.endsWith(".jar") is operated in the current instance of Menus.java, whereas name.endsWith(".ijm") is operated in a static program which may have some subtle difference.

I did a search for validMacroName and found references to it only inside Menus.java. From this there is a possibility that it need not be static. In any case it is broken because isFiji is rubbish (always false).

As a minimum I would suggest

		String name = list[i].trim();

as something which can't hurt and might possibly help. The real test is to make an experimental version with the trim() and test it. This is no problem for me inside ImageJ, but I have no control over Fiji and its contents.

If anyone can give some help and suggestions maybe we can move this bug to a solution.

Thanks,
Ilan
Comment 13 Curtis Rueden 2014-08-26 09:14:47 CDT
It is wrong for ImageJ 1.x core to have _any_ case logic regarding whether the current installation is a Fiji one. I personally hope that the hack you describe gets removed, and any relevant problems fixed on the Fiji side instead.
Comment 14 Ilan Tal 2014-08-26 11:45:33 CDT
Hi Curtis,
I didn't describe any hack for this: I didn't have a clue what would work. All I said was "as is" it is rubbish in that it always returns false. If any program is building on it, the program will fail.

There are multiple things which don't work here, and in my opinion at least, they need to be dealt with.

Ilan
Comment 15 Curtis Rueden 2014-08-26 15:03:10 CDT
> I didn't describe any hack for this

I was referring to the existence of the "isFiji" variable, which is a hack inside ImageJ 1.x core. My preference would be for ImageJ 1.x not to care whatsoever whether it is running from a Fiji installation or a vanilla one. Having ImageJ 1.x detect a Fiji installation creates circular reasoning between ImageJ1 and Fiji, with each part of the program trying to "work around" the behavior of the other.

> One possibility is there is some unseen character in the file name.

This seems highly unlikely. However, you can easily verify that hypothesis for yourself by trying something like "ls Postage_Action_Tool.ijm" (typing it all out without tab completion) from the command line. If it works, then you got the file name right.

> I think it is time for a minor overhaul on Menus.java

Unfortunately, it is probably far too late to make structural changes to Menus.java without far-reaching consequences on backward compatibility.

> It used to be that Fiji was started by fiji-linux, but now ImageJ-linux32 is used.

That is not why the "isFiji" hack no longer works, though. The "title" in this case refers to the title of the main window Frame.

> Fiji will hang and not accept any input from the menu or from the keyboard.
> I wanted to make a thread dump by using ctrl+\ but the keyboard had no response.

I ran Fiji from the command line on my OS X system and was able to generate a stack trace after the program hang using Ctrl+\ on the console. Here are the relevant parts:

"SciJava-4a0ece36-Thread-2" prio=5 tid=115aa6000 nid=0x12232a000 in Object.wait() [122329000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <6c62f46c0> (a java.awt.EventQueue$1AWTInvocationLock)
	at java.lang.Object.wait(Object.java:485)
	at java.awt.EventQueue.invokeAndWait(EventQueue.java:1121)
	- locked <6c62f46c0> (a java.awt.EventQueue$1AWTInvocationLock)
	at java.awt.EventQueue.invokeAndWait(EventQueue.java:1103)
	at org.scijava.thread.DefaultThreadService.invoke(DefaultThreadService.java:98)
	at org.scijava.event.DefaultEventBus.publishNow(DefaultEventBus.java:191)
	at org.scijava.event.DefaultEventBus.publishNow(DefaultEventBus.java:82)
	at org.scijava.event.DefaultEventService.publish(DefaultEventService.java:94)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:153)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:126)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:65)
	at org.scijava.thread.DefaultThreadService$2.call(DefaultThreadService.java:164)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
	at java.lang.Thread.run(Thread.java:695)

"Run$_AWT-EventQueue-0" prio=6 tid=106c61800 nid=0x11d66a000 waiting on condition [11d667000]
   java.lang.Thread.State: WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <6c634e290> (a java.util.concurrent.FutureTask$Sync)
	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:969)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1281)
	at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
	at java.util.concurrent.FutureTask.get(FutureTask.java:83)
	at net.imagej.legacy.DefaultLegacyService.runLegacyCompatibleCommand(DefaultLegacyService.java:267)
	at net.imagej.legacy.DefaultLegacyHooks.interceptRunPlugIn(DefaultLegacyHooks.java:160)
	at ij.IJ.runPlugIn(IJ.java)
	at ij.Executer.runCommand(Executer.java:131)
	at ij.Executer.run(Executer.java:64)
	at ij.IJ.run(IJ.java:272)
	at ij.IJ.run(IJ.java:250)
	at ij.gui.Toolbar.itemStateChanged(Toolbar.java:1270)

In English:
- ImageJ1 invokes the macro using IJ.runPlugIn.
- ImageJ2's legacy service intercepts the execution, and executes the macro using the SciJava scripting framework.
- This launches a new thread ("SciJava-4a0ece36-Thread-2" in this case), and blocks until execution is complete.
- On the new thread, the SciJava module framework tries to broadcast a ModuleStartedEvent, blocking till event handling is complete.
- Deadlock occurs as each thread is blocking the other.

> In Fiji, when I select Plugins>Tools>Postage Action Tool, the tool is not
> installed. Instead, "Tile" is displayed in the Log window. This is a bug.

Yes, that is a separate bug, caused by the fact that the macro is executed using the SciJava scripting framework, which works differently.

I am investigating how to fix the deadlock right now, and will keep you posted.
Comment 16 Curtis Rueden 2014-08-26 16:38:11 CDT
It seems that IJ.run(String) does not spawn a different thread, but _does_ still prepend "Run$_" to the current thread, resulting in the EDT being renamed, which seems like a bug.

The threading issues here are complex, due to many different possible code paths within ImageJ 1.x: IJ.run, IJ.runPlugIn, IJ.runMacro, multiple method signatures, called from many places in different ways. Many of these code paths end up running on the EDT, which has major ramifications.

I'll discuss further with Johannes, when I get a chance.
Comment 17 Wayne Rasband 2014-08-26 18:31:36 CDT
I noticed that Fiji runs the "Postage Action Tool" command when what it should do is install the tool in the toolbar. In ImageJ, type "L", find the "Postage Action Tool", run it, and it is installed in the toolbar. Do the same thing in Fiji and it is not installed in the toolbar. Instead, "Tile" is displayed in the Log window, which indicates Fiji is trying to run the tool as if it were an ordinary macro. ImageJ installs the tool using the MacroInstaller#installTool() function, which is called from Macro_Runner when the name of the macro file ends in "Tools.ijm".
Comment 18 Ilan Tal 2014-08-27 00:33:28 CDT
Hi Curtis,
First and foremost I am delighted that you are thinking about the problem. Until I used Mark's idea of attaching a debugger to the actual running Fiji program, I could do absolutely nothing. With the debugger I can set break points, investigate object values and the world opens up to investigation.

To clear up a minor point, isFiji is probably a variable which someone added long ago for some long forgotten purpose. With the debugger I demonstrated that it no longer does anything useful. You say it shouldn't be there and I said it should either be fixed or be removed. To have something which might work if you start Fiji under the old fiji command, but won't work if you start it with the current imageJ command is just another bug waiting to appear.

What I did was spread around break points until I caught the path which was being followed and then I could compare values and see what the differences were between Fiji and ImageJ. In looking at the differences I could see that in ImageJ macroFiles had valid information including my file, whereas in Fiji it was null.

The next logical step was to find out why macroFiles was null. That led me to Menus.java and see what the differences were in that program. In Menus.java I found that validMacroName wasn't working in Fiji, but worked fine in ImageJ. In comment 11 I found 5 additional files which should have passed the test but failed. After 5 files, I stopped counting but macroFiles never reached a non null value.

What is clear is that validMacroName is broken inside Fiji. The question is: why? The 2 possibilities which I could see is either there is some invisible character which trim() could eliminate, or something subtle in the fact that it is a static procedure. If name.endsWith(".jar") works correctly, common sense would say the name.endsWith(".ijm") should work as well. The problem is that it fails. Why?


I see that you looked at the fallout after the explosion. Here is what I think is happening. The macro, instead of being installed, is being executed. The macro calls to log to print a message. That apparently starts a new thread and something goes hay wire in unwinding the stack.

You can solve that problem, but that isn't the real problem. The real problem is why is going down the wrong path in the first place? I traced it back to validMacroName which isn't doing its job correctly. Solve that and the other problem will disappear.

Thanks again for all your help,
Ilan
Comment 19 Ilan Tal 2014-08-28 02:50:10 CDT
I wanted to try something different. I have the Window_Level_Tool compiled as a jar file which both Fiji and ImageJ pick up with no problems. Wayne prefers a class file, so I thought I would erase my jar file and put a class file in instead. How would that work?

It turns out that both the jar and class files work perfectly well in both ImageJ and Fiji. Still I decided to see the results under control of the debugger. In addition to the .txt files I mentioned earlier, I picked up a whole load of other files:

Dynamic_ROI_Profiler.clj
random_noise_example.clj
chess_.py
downsample_.js
compose_rgb-stacks.bsh
Delayed_Snapshot.py
list_all_threads.py
Plasma_Cloud.rb
Fiji_Cube.ijm
Window_Level_Tool.class
Postage_Action_Tool.ijm

I got bored at a certain stage and entered only interesting files. .txt files we already know don't get picked up by validMacroName, but it seems like nothing else gets picked up either. clj is not supposed to be picked up, but py, js, bsh are supposed to be picked up (isFiji = false).

I saw there is another Fiji_Cube.ijm which has a similar name which also failed. Window_Level_Tool.class gets properly classified which is what I wanted to check, compared to the jar file which also works.

Of course my Postage_Action_Tool.ijm also fails - no big surprise here. In sort validMacroName always fails in Fiji and it works in ImageJ. If I look at the code it looks plain vanilla to me, so I have no explanation of why it should fail. Adding trim() would do no damage, but it would probably do no good as well. Aside from classifying this as an unsolvable bug, I have no idea what to think.

Ilan
Comment 20 Wayne Rasband 2014-08-28 17:12:57 CDT
Upgrade to the latest ImageJ daily build (1.49h3) and you can work around this bug by putting the macro tool in the macros/toolsets folder. Single macro tools in the toolsets folder added to the toolbar using the toolbar's ">>" menu are now restored to the toolbar when ImageJ restarts as long as the file name matches the tool name and there is at least one free tool slot.
Comment 21 Ilan Tal 2014-08-31 00:45:12 CDT
Thanks Wayne,
This morning I tried the daily build and the work around does the job. There is one user who needs it and I told him about your work around.

Ilan