edit:
  >> File->Open.  After <enter> there is no CPU activity (ie 1%) for 22 seconds, then all cores run high for ~10 seconds, and then the image appears.

Example of macro lethargy.  With IJ1 my image processing macro executed in a very respectable 6-7 seconds.  

In IJ2 with SCIFIO disabled, the same macro takes ~15 seconds to execute, high CPU activity for the full 15 seconds.  

With SCIFIO enabled [with setBatchMode(true), images suppressed], that jumps to 50 seconds.  There is the 22-second CPU idle period but it occurs after ~8 seconds into the macro.  It's as though IJ2 waits for ~22 seconds before resuming high CPU activity for the remainder 20 seconds of the macro (total 50 seconds).

The timings of this strange activity seem related to manual Open, but are "inverted."

In short, there are two issues IJ2:
  1 - 10X longer image file open times (10s vs 1s of IJ1)
  2 - an additional 22-second CPU idle time when SCIFIO is enabled during File Open and during a macro that opens an image file.

The delay behavior is worst, ie most obvious, when opening regular JPG files.  Mine are 10 mpx image files, typically 2-4 MB JPGs.

Thanks for the excellent bug report, Stick. This is a very serious issue which we will address as soon as possible.

Sure Curtis, my pleasure.  

Also useful (give 'im an inch and he'll take a mile) .... if you or a colleague are dabbling in that area, it would be useful to give us a programmatic get/set of SCIFIO flag, eg. flag = getSCIFIOflag().  The reason macro knowledge is needed is that SCIFIO seems to treat some files differently with SCIFIO disabled (like IJ1).  

For example, with SCIFIO off, a 16-bit PNG gets opened in IJ1 style as an 8-bit RGB (like an 8-bit JPG), with I presume resolution loss due to mapping to lo-res.  With SCIFIO on, the same PNG file is opened as a 3-plane hyperstack (with the 1/3 etc color slider on bottom) preserving its original 16-bit per plane resolution.  

However, that results in a different window naming convention for SCIFIO ON|OFF after I do "Split Channels" for either state, which I then have to handle with different code paths (that's OK).  While true 16-bittiness offered by SCIFIO is very welcome, at present, without knowing SCIFIO state programmatically, I can't automate subsequent image processing.

Thanks.

Please file an issue at https://github.com/imagej/imagej-legacy/issues for the feature request regarding macro interrogation of the SCIFIO status.

As for this bug, I think I found the problem, and I fear it will take some work to resolve properly. A stack trace during the initial display of a largish image shows:

"Open..." prio=4 tid=11c85d000 nid=0x128101000 runnable [1280ff000]
   java.lang.Thread.State: RUNNABLE
	at net.imglib2.display.projector.composite.CompositeXYProjector.map(CompositeXYProjector.java:193)
	at net.imagej.display.DefaultDatasetView.rebuild(DefaultDatasetView.java:401)
	- locked <6d1c538f8> (a net.imagej.display.DefaultDatasetView)
	at net.imagej.display.DefaultImageDisplay.rebuild(DefaultImageDisplay.java:136)
	at net.imagej.display.DefaultImageDisplay.display(DefaultImageDisplay.java:274)

This is in the ImageJ2 display layer: it is trying to build a projection of the image for use in the ImageJ2 UI, even though that UI is inactive. In other words: ImageJ is doing twice as much work as necessary, rendering both the IJ1 and IJ2 image planes. Furthermore, the IJ2/ImgLib2 rendering layer is very much unoptimized at the moment, so the (unnecessary!) IJ2 part of that work tanks performance rather severely, especially with images over 1 MPx.

The proper fix is to render image data on the IJ2 side only as needed to actually paint pixels. But we'll need another design iteration of the display architecture (which has numerous problems) in order to accommodate that.

In the meantime, I'll see about putting a hack somewhere to suppress this undesirable behavior.

To avoid this sort of issue for most users most of the time, I added a toggle to the ImageJ2 options. Work (unfinished) is at:

   https://github.com/imagej/imagej-legacy/compare/ij2-sync

I'll file a PR when it's ready.

There is now a tentative fix:
  https://github.com/imagej/imagej-legacy/pull/82

Some testing would be helpful. If you check out the branch, you can build using "mvn clean package" and then copy "target/imagej-legacy-0.8.2-SNAPSHOT.jar" into your ImageJ.app/jars, deleting the old "target/imagej-legacy-0.8.1.jar".

If you then launch ImageJ and run Edit > Options > ImageJ2 you'll see a new option, "Enable ImageJ2 data structures", off by default. As long as that option stays off, performance _should_ return to what it was in the pure IJ1 days.

I tested this only lightly, since I am actually out of the office today. We will do more thorough testing on Monday, and if all is well, merge and release.

Curtis, I'm not sure if your previous comment was intended for me, but I expect I don't have js build capability, unless it's included by default in the java package that runs fiji in Vista and there's a way of getting at it that I don't know.  

Nonetheless, this morning I am noting these differences:

  a) There is no new toggle (yet, expected I suppose) in Options->ImageJ that you describe.

With SCIFIO enabled .....
    
  b) If I manually open a 10 mpx 3 MB JPG there is still that 22s of CPU idle time before the image appears.  It takes a total of ~35s for the image to show up.

  c) But, if I open a 16-bit 40 MB PNG version of the same image, it shows up in ~12s with no idle time -- much better.  So there must be an improvement here.

  d) The macro that executed in ~50s a few days ago now runs in ~30s.  I see cores run high with no ~22s CPU idle time, so the idle time appears to be eliminated.  The interesting thing here is that there is no idle time either for PNGs or JPGs, which is curious because with PNGs performance agrees with (c) but for JPGs contradicts (b), to the better of course!

Moral of the story ... if you can make (b) work like (c) then I expect (not tested yet) that SetBatchMode(false) for debugging my macros will give me better performance too.

After some more thought, the "curiosity" in (d) vs (b) seems to be related to setBatchMode().  I am wondering if what I'm reporting above makes any sense.

I uploaded the fix. So now you can test simply by running Help > Update and installing the updates.

Chris, I am not sure why you saw a discrepancy in your second round of testing, since this fix was not yet uploaded to the ImageJ update site. Perhaps there is an intermittent timing issue. But in any case, the fix should completely sidestep the issue, returning the behavior to pre-ImageJ2 days unless you explicitly toggle ImageJ2 syncing and/or SCIFIO in the Edit > Options > ImageJ2... dialog (which I would not recommend doing at this time).

RESULTS
-------

IJ2 data structs OFF
SCIFIO ON

Manually open 3 MB 10 mpx JPG:  12s, no CPU idle
Manually open 40 MB 10 mpx 16-bit PNG: 12s, no CPU idle
Macro with 16-bit PNG: 28s, no CPU idle
Macro with JPG: not supported format (expected - due to SCIFIO not being detected yet in my macro)

IJ2 data structs OFF
SCIFIO OFF

Manually open 3 MB 10 mpx JPG:  less than 1s - crazy good
Manually open 40 MB 10 mpx 16-bit PNG: less than 2s -- crazy good
Macro with 16-bit PNG:  not supported format (expected - due to SCIFIO not being detected yet in my macro)
Macro with JPG: 18s, OK, passes

IJ2 data structs ON
SCIFIO ON

Manually open 3 MB 10 mpx JPG:  11s, no CPU idle
Manually open 40 MB 10 mpx 16-bit PNG: 11s, no CPU idle
Macro with 16-bit PNG: 28s, no CPU idle
Macro with JPG: 28s, no CPU idle

COMMENTS
--------
It looks like the new switch works.  Despite the switch, the nice thing now is that the 22s CPU idle is gone everywhere, even though I understand you didn't find it.  So it may come back to haunt us.  The issue remains also with the long file loads with IJ2 structs ON, basically ~10 sec vs ~1 sec (potential CPU idle lurking gremlin notwithstanding).  Right now thus, I would not be able to make decent use of SCIFIO on the macros where I read upwards of 36 of these image files.  Please give a heads up if you fix the load delay.

If you do [test #3 JPG macro] minus [test #2 JPG macro], you get 28s-18s = 10s, the file load overhead caused by SCIFIO ON during manual load.   If you can get rid of that overhead, you'll be back to the performance of IJ1 but with the nice features of IJ2 SCIFIO.  Looking forward to a solution.
All the best,
Chris.

> Looking forward to a solution.

Note that this is an Open Source project. That means that you can make happen what you desire to see. Or you can wait and lean on volunteers to do the work for you, at a time when their goals match yours. It is fundamentally *your* choice which route you choose.

Johannes, while I know this is open source and for features specific to my projects that could also help a small group of others, I would be responsible in crafting out.  If I am not mistaken the move to IJ2 and its features was funded by the NIH, and that I presume must have been funneled to a core group of developers.  Whether Curtis is one or he could be working on IJ2 for other reasons, I don't know because I did not ask.  But who ever implemented the new functionality that impacts everyone using IJ2 at the *front* end, should be the one interested in my bug report and improve his code.  There is a good reason why there is a bug report facility in Fiji and that is what I understand it to be.

To interrogate whether SCIFIO is active in your ImageJ, you can use the OptionsService API. Here is an example Jython script:

  # @OptionsService optionsService
  from net.imagej.legacy import ImageJ2Options
  scifio = optionsService.getOptions(ImageJ2Options).isSciJavaIO()
  print("SCIFIO? " + str(scifio))

The same approach should work with any scripting language _except_ for ImageJ 1.x macros, since they do not support access to the complete Java API of ImageJ etc.

Regarding performance improvements to SCIFIO and ImageJ-Legacy: yes, they are certainly needed and planned, and will remain on the radar. But it requires a substantial retooling of the ImageJ-Common library, among other involved efforts. Thus, due to limited developer resources, I cannot promise a timeline.

Since improving SCIFIO performance is more of a feature request than a bug, and since there is a simple workaround to any performance issues (i.e.: turn off SCIFIO and ImageJ2 data structures), I am closing this bug as resolved.

In the meantime, for your 16-bit PNGs, you can force use SCIFIO via File > Import > Image... (way at the bottom of the menu! or use the Command Finder by pressing L). That way you can at least write a macro that will support all your images via case logic.

I'm not yet at the level of coding Jython and the like, but I appreciate your solution offer and will return to this thread when I get up to speed.  So far macros have been working very well for me, are fast to develop and are easy to debug because of their tight relationship with menu commands. 

WRT to 16-bit PNGs ..."you can force use SCIFIO via File > Import > Image... (way at the bottom of the menu!"

Yes nice, I tried this from the menu and it worked ... I turned OFF both SCIFIO and IJ2 structs and Fiji presented the image in full 16-bit, as though both options were ON.  The same 10s delay takes place but it worked.  I got excited because this looked like a good solution for both the many 1-second JPGs and the hi-res solitary PNG as needed, but alas, macro record emitted only this line:
   run("Image... ")
without file information or other parameters in the window.  It's as though the macro script is incomplete for the command???

> alas, macro record emitted only this line: run("Image... ") without
> file information or other parameters in the window.  It's as though
> the macro script is incomplete for the command???

Right, I forgot that ImageJ2 commands do not record properly yet. The issue is that ImageJ2 commands are _not_ ImageJ 1.x plugins, and need separate handling for the recorder to emit something sensible.

It's been a known issue for quite some time, but AFAIK there was no existing issue for it. So I filed one:
   https://github.com/imagej/imagej-legacy/issues/87

And unfortunately, no workaround for this problem occurs to me at the moment.

Just a quick note that ImageJ2 commands such as File > Import > Image... were made fully macro recordable a few months ago.

See the GitHub issue (https://github.com/imagej/imagej-legacy/issues/87; now closed) for details.