Can produce output yet will have some other errors

[alvin9964]

2020-09-09 14:01:37 WARN BatchNormalization:382 - CuDNN BatchNormalization forward pass execution failed - falling back on built-in implementation
java.lang.RuntimeException: cuDNN status = 3: CUDNN_STATUS_BAD_PARAM
at org.deeplearning4j.cuda.BaseCudnnHelper.checkCudnn(BaseCudnnHelper.java:48)
at org.deeplearning4j.cuda.normalization.CudnnBatchNormalizationHelper.preOutput(CudnnBatchNormalizationHelper.java:320)
at org.deeplearning4j.nn.layers.normalization.BatchNormalization.preOutput(BatchNormalization.java:462)
at org.deeplearning4j.nn.layers.normalization.BatchNormalization.activate(BatchNormalization.java:404)
at org.deeplearning4j.nn.graph.vertex.impl.LayerVertex.doForward(LayerVertex.java:111)
at org.deeplearning4j.nn.graph.ComputationGraph.outputOfLayersDetached(ComputationGraph.java:2380)
at org.deeplearning4j.nn.graph.ComputationGraph.output(ComputationGraph.java:1741)
at org.deeplearning4j.nn.graph.ComputationGraph.output(ComputationGraph.java:1697)
at org.deeplearning4j.nn.graph.ComputationGraph.output(ComputationGraph.java:1627)
at SocialDistanceCheckerVideo.getPredictedObjects(SocialDistanceCheckerVideo.java:150)
at SocialDistanceCheckerVideo.main(SocialDistanceCheckerVideo.java:76)

Pom file as below :

4.0.0

<groupId>Certifai</groupId>
<artifactId>SocialDistancingChecker</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <configuration>
                <source>8</source>
                <target>8</target>
            </configuration>
        </plugin>
    </plugins>
</build>
<properties>
    <nd4j.backend>nd4j-native-platform</nd4j.backend>
    <nd4j.backend>nd4j-cuda-10.2-platform</nd4j.backend>
    <nd4j.version>1.0.0-beta7</nd4j.version>
    <dl4j.version>1.0.0-beta7</dl4j.version>
    <datavec.version>1.0.0-beta7</datavec.version>
</properties>

<repositories>
    <repository>
        <id>snapshots-repo</id>
        <url>https://oss.sonatype.org/content/repositories/snapshots</url>
        <releases>
            <enabled>false</enabled>
        </releases>
        <snapshots>
            <enabled>true</enabled>
        </snapshots>
    </repository>
</repositories>
<dependencies>
    <dependency>
        <groupId>org.nd4j</groupId>
        <artifactId>${nd4j.backend}</artifactId>
        <version>${nd4j.version}</version>
    </dependency>
    <dependency>
        <groupId>org.deeplearning4j</groupId>
        <artifactId>deeplearning4j-cuda-10.2</artifactId>
        <version>${dl4j.version}</version>
    </dependency>
    <dependency>
        <groupId>org.bytedeco</groupId>
        <artifactId>javacv-platform</artifactId>
        <version>1.5.3</version>
    </dependency>
    <dependency>
        <groupId>org.deeplearning4j</groupId>
        <artifactId>deeplearning4j-ui</artifactId>
        <version>${dl4j.version}</version>
    </dependency>
    <!--        Logging libraries-->
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-api</artifactId>
        <version>2.0.0-alpha1</version>
    </dependency>
    <dependency>
        <groupId>org.slf4j</groupId>
        <artifactId>slf4j-log4j12</artifactId>
        <version>2.0.0-alpha1</version>
    </dependency>
</dependencies>

Cuda version 10.2
Cudnn 7.6.5

[bewithme]

can you reproduce this problem ?

[alvin9964]

So apparently I tired several combination of CUDA version with CUDNN version, here is the result

10.1 - 7.6.1 same error
10.1 - 7.6.2 OK!
10.1 - 7.6.3 OK!
10.1 - 7.6.4 OK!
10.1 - 7.6.5 OK!

10.2 - 7.6.5 same error

I think the workaround solution for me is to prevent those CUDNN versions that cause error and stick with CUDA 10.1 first.