"Could not apply Classifier!" while running example "apply classifier to all images in folder" script on TWS 3.3.2 as well as 3.2.34

[mcmaxl]

Hi everyone,

while trying to apply a previously trained model to a number of files in a folder using the standard script provided here https://imagej.net/plugins/tws/scripting#example-apply-classifier-to-all-images-in-folder I get the follow error in a pop up window: "Could not apply Classifier!"

The script window gives me the following output:

Started Pentachrom_Weka_Script.bsh at Tue Apr 04 19:17:49 CEST 2023
java.lang.NullPointerException
	at trainableSegmentation.WekaSegmentation.applyClassifier(WekaSegmentation.java:5692)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at bsh.Reflect.invokeMethod(Reflect.java:160)
	at bsh.Reflect.invokeObjectMethod(Reflect.java:93)
	at bsh.Name.invokeMethod(Name.java:854)
	at bsh.BSHMethodInvocation.eval(BSHMethodInvocation.java:67)
	at bsh.BSHPrimaryExpression.eval(BSHPrimaryExpression.java:94)
	at bsh.BSHPrimaryExpression.eval(BSHPrimaryExpression.java:39)
	at bsh.BSHAssignment.eval(BSHAssignment.java:69)
	at bsh.BSHBlock.evalBlock(BSHBlock.java:123)
	at bsh.BSHBlock.eval(BSHBlock.java:73)
	at bsh.BSHBlock.eval(BSHBlock.java:39)
	at bsh.BSHIfStatement.eval(BSHIfStatement.java:40)
	at bsh.BSHBlock.evalBlock(BSHBlock.java:123)
	at bsh.BSHBlock.eval(BSHBlock.java:73)
	at bsh.BSHBlock.eval(BSHBlock.java:39)
	at bsh.BSHIfStatement.eval(BSHIfStatement.java:40)
	at bsh.BSHBlock.evalBlock(BSHBlock.java:123)
	at bsh.BSHBlock.eval(BSHBlock.java:73)
	at bsh.BSHBlock.eval(BSHBlock.java:39)
	at bsh.BSHForStatement.eval(BSHForStatement.java:103)
	at bsh.Interpreter.eval(Interpreter.java:673)
	at bsh.Interpreter.eval(Interpreter.java:764)
	at bsh.Interpreter.eval(Interpreter.java:753)
	at bsh.BshScriptEngine.evalSource(BshScriptEngine.java:89)
	at bsh.BshScriptEngine.eval(BshScriptEngine.java:61)
	at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264)
	at org.scijava.script.ScriptModule.run(ScriptModule.java:164)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:163)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)

while the Fiji console gives me this:

WARNING: core mtj jar files are not available as resources to this classloader (sun.misc.Launcher$AppClassLoader@764c12b6)
java.lang.ArrayIndexOutOfBoundsException: 78
	weka.core.DenseInstance.value(DenseInstance.java:347)
	weka.core.AbstractInstance.isMissing(AbstractInstance.java:319)
	hr.irb.fastRandomForest.FastRandomTree.distributionForInstance(FastRandomTree.java:269)
	hr.irb.fastRandomForest.FastRandomTree.distributionForInstance(FastRandomTree.java:299)
	hr.irb.fastRandomForest.FastRandomTree.distributionForInstance(FastRandomTree.java:301)
	hr.irb.fastRandomForest.FastRfBagging.distributionForInstance(FastRfBagging.java:666)
	hr.irb.fastRandomForest.FastRandomForest.distributionForInstance(FastRandomForest.java:667)
	weka.classifiers.AbstractClassifier.classifyInstance(AbstractClassifier.java:173)
	trainableSegmentation.WekaSegmentation$9.call(WekaSegmentation.java:7231)
	trainableSegmentation.WekaSegmentation$9.call(WekaSegmentation.java:7197)
	java.util.concurrent.FutureTask.run(FutureTask.java:266)
	java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	java.lang.Thread.run(Thread.java:750)
Exception in thread "Thread-9" java.lang.NullPointerException
	trainableSegmentation.WekaSegmentation.applyClassifier(WekaSegmentation.java:6897)
	trainableSegmentation.WekaSegmentation$1ApplyClassifierThread.run(WekaSegmentation.java:5634)

	at trainableSegmentation.WekaSegmentation.applyClassifier(WekaSegmentation.java:6897)
	at trainableSegmentation.WekaSegmentation$1ApplyClassifierThread.run(WekaSegmentation.java:5634)

I am running Fiji/ImageJ2 Version 2.9.0/1.53t Build: a33148d777 and have tried this with both the inbuilt Trainable_Segmentation-3.3.2 as well as Trainable_Segmentation-3.2.34.

I also tried using an older Fiji Version (ImageJ 2.1.0 Build 5f23140693) with inbuilt Trainable_Segmentation-3.2.34. Same error. Script log output shows:

Started Pentachrom_Weka_Script.bsh at Tue Apr 04 19:27:51 CEST 2023
Sourced file: inline evaluation of: ``     import trainableSegmentation.WekaSegmentation; import trainableSegmentation . . . '' : Method Invocation segmentator.applyClassifier : at Line: 35 : in file: inline evaluation of: ``     import trainableSegmentation.WekaSegmentation; import trainableSegmentation . . . '' : segmentator .applyClassifier ( image , 0 , getProbs ) 

Target exception: java.lang.NullPointerException

	at bsh.BSHMethodInvocation.eval(BSHMethodInvocation.java:91)
	at bsh.BSHPrimaryExpression.eval(BSHPrimaryExpression.java:96)
	at bsh.BSHPrimaryExpression.eval(BSHPrimaryExpression.java:41)
	at bsh.BSHAssignment.eval(BSHAssignment.java:71)
	at bsh.BSHBlock.evalBlock(BSHBlock.java:125)
	at bsh.BSHBlock.eval(BSHBlock.java:75)
	at bsh.BSHBlock.eval(BSHBlock.java:41)
	at bsh.BSHIfStatement.eval(BSHIfStatement.java:42)
	at bsh.BSHBlock.evalBlock(BSHBlock.java:125)
	at bsh.BSHBlock.eval(BSHBlock.java:75)
	at bsh.BSHBlock.eval(BSHBlock.java:41)
	at bsh.BSHIfStatement.eval(BSHIfStatement.java:42)
	at bsh.BSHBlock.evalBlock(BSHBlock.java:125)
	at bsh.BSHBlock.eval(BSHBlock.java:75)
	at bsh.BSHBlock.eval(BSHBlock.java:41)
	at bsh.BSHForStatement.eval(BSHForStatement.java:105)
	at bsh.Interpreter.eval(Interpreter.java:659)
	at bsh.Interpreter.eval(Interpreter.java:750)
	at bsh.Interpreter.eval(Interpreter.java:739)
	at org.scijava.plugins.scripting.beanshell.BeanshellScriptEngine.eval(BeanshellScriptEngine.java:68)
	at org.scijava.script.ScriptModule.run(ScriptModule.java:157)
	at org.scijava.module.ModuleRunner.run(ModuleRunner.java:165)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:124)
	at org.scijava.module.ModuleRunner.call(ModuleRunner.java:63)
	at org.scijava.thread.DefaultThreadService.lambda$wrap$2(DefaultThreadService.java:225)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.NullPointerException
	at trainableSegmentation.WekaSegmentation.applyClassifier(WekaSegmentation.java:5566)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at bsh.Reflect.invokeMethod(Reflect.java:131)
	at bsh.Reflect.invokeObjectMethod(Reflect.java:77)
	at bsh.Name.invokeMethod(Name.java:852)
	at bsh.BSHMethodInvocation.eval(BSHMethodInvocation.java:69)
	... 28 more

while Fiji console shows:

java.lang.ArrayIndexOutOfBoundsException: 78
	weka.core.DenseInstance.value(DenseInstance.java:347)
	weka.core.AbstractInstance.isMissing(AbstractInstance.java:319)
	hr.irb.fastRandomForest.FastRandomTree.distributionForInstance(FastRandomTree.java:248)
	hr.irb.fastRandomForest.FastRandomTree.distributionForInstance(FastRandomTree.java:280)
	hr.irb.fastRandomForest.FastRandomTree.distributionForInstance(FastRandomTree.java:280)
	hr.irb.fastRandomForest.FastRfBagging.distributionForInstance(FastRfBagging.java:645)
	hr.irb.fastRandomForest.FastRandomForest.distributionForInstance(FastRandomForest.java:646)
	weka.classifiers.AbstractClassifier.classifyInstance(AbstractClassifier.java:173)
	trainableSegmentation.WekaSegmentation$9.call(WekaSegmentation.java:7096)
	trainableSegmentation.WekaSegmentation$9.call(WekaSegmentation.java:7062)
	java.util.concurrent.FutureTask.run(FutureTask.java:266)
	java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	java.lang.Thread.run(Thread.java:750)

	at weka.core.DenseInstance.value(DenseInstance.java:347)
	at weka.core.AbstractInstance.isMissing(AbstractInstance.java:319)
	at hr.irb.fastRandomForest.FastRandomTree.distributionForInstance(FastRandomTree.java:248)
	at hr.irb.fastRandomForest.FastRandomTree.distributionForInstance(FastRandomTree.java:280)
	at hr.irb.fastRandomForest.FastRandomTree.distributionForInstance(FastRandomTree.java:280)
	at hr.irb.fastRandomForest.FastRfBagging.distributionForInstance(FastRfBagging.java:645)
	at hr.irb.fastRandomForest.FastRandomForest.distributionForInstance(FastRandomForest.java:646)
	at weka.classifiers.AbstractClassifier.classifyInstance(AbstractClassifier.java:173)
	at trainableSegmentation.WekaSegmentation$9.call(WekaSegmentation.java:7096)
	at trainableSegmentation.WekaSegmentation$9.call(WekaSegmentation.java:7062)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:750)
Exception in thread "Thread-8" java.lang.NullPointerException
	trainableSegmentation.WekaSegmentation.applyClassifier(WekaSegmentation.java:6763)
	trainableSegmentation.WekaSegmentation$1ApplyClassifierThread.run(WekaSegmentation.java:5508)

	at trainableSegmentation.WekaSegmentation.applyClassifier(WekaSegmentation.java:6763)
	at trainableSegmentation.WekaSegmentation$1ApplyClassifierThread.run(WekaSegmentation.java:5508)

I am running Mac OS 13.2.1

Any ideas what the problem might be? Thanks very much for any help!

[mlt]

This may happen when you change enabled features after you load your classifier.
Here is how to reproduce with 3.2.34

diff --git a/src/test/java/trainableSegmentation/BasicTest.java b/src/test/java/trainableSegmentation/BasicTest.java
index 5ad8639..bc0ca82 100644
--- a/src/test/java/trainableSegmentation/BasicTest.java
+++ b/src/test/java/trainableSegmentation/BasicTest.java
@@ -47,6 +47,16 @@ public class BasicTest
 		{
 			assertTrue("Misclassified training sample", pix[i] == pixTrue[i]);
 		}
+
+		// TODO: avoid testing the file system
+		boolean[] ff = segmentator.getEnabledFeatures();
+		segmentator.saveClassifier("test1.classifier");
+		segmentator = new WekaSegmentation();
+		segmentator.loadClassifier("test1.classifier"); // <- load
+		ff[0] = false;                                  // <- turn off gaussian blur
+		segmentator.setEnabledFeatures(ff);
+		result = segmentator.applyClassifier(image);
+		assertTrue("Failed to apply trained classifier", null != result);
 	}
 
 	@Test

mvn test (after adjusting pom.xml to some recent scijava version say 37.0.0) would produce

Could not apply Classifier!
java.lang.ArrayIndexOutOfBoundsException: Index 74 out of bounds for length 72
	weka.core.DenseInstance.value(DenseInstance.java:347)
	weka.core.AbstractInstance.isMissing(AbstractInstance.java:319)
	hr.irb.fastRandomForest.FastRandomTree.distributionForInstance(FastRandomTree.java:248)
	hr.irb.fastRandomForest.FastRfBagging.distributionForInstance(FastRfBagging.java:645)
	hr.irb.fastRandomForest.FastRandomForest.distributionForInstance(FastRandomForest.java:646)
	weka.classifiers.AbstractClassifier.classifyInstance(AbstractClassifier.java:173)
	trainableSegmentation.WekaSegmentation$9.call(WekaSegmentation.java:7096)
	trainableSegmentation.WekaSegmentation$9.call(WekaSegmentation.java:1)
	java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	java.base/java.lang.Thread.run(Thread.java:829)

	at weka.core.DenseInstance.value(DenseInstance.java:347)
	at weka.core.AbstractInstance.isMissing(AbstractInstance.java:319)
	at hr.irb.fastRandomForest.FastRandomTree.distributionForInstance(FastRandomTree.java:248)
	at hr.irb.fastRandomForest.FastRfBagging.distributionForInstance(FastRfBagging.java:645)
	at hr.irb.fastRandomForest.FastRandomForest.distributionForInstance(FastRandomForest.java:646)
	at weka.classifiers.AbstractClassifier.classifyInstance(AbstractClassifier.java:173)
	at trainableSegmentation.WekaSegmentation$9.call(WekaSegmentation.java:7096)
	at trainableSegmentation.WekaSegmentation$9.call(WekaSegmentation.java:1)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Exception in thread "Thread-2" java.lang.NullPointerException
	trainableSegmentation.WekaSegmentation.applyClassifier(WekaSegmentation.java:6763)
	trainableSegmentation.WekaSegmentation$1ApplyClassifierThread.run(WekaSegmentation.java:5508)

	at trainableSegmentation.WekaSegmentation.applyClassifier(WekaSegmentation.java:6763)
	at trainableSegmentation.WekaSegmentation$1ApplyClassifierThread.run(WekaSegmentation.java:5508)

This perfectly matches your output.

This might be related to #74 as I saw the stack trace from this issue while trying to incorrectly fix 74.