Demos Applications Components Optimizers Experiments Datasets

Image_Float

Typical 3x3 image convolution (32-bit)


Project maintained by SimiaCryptus Java, CuDNN, and CUDA are others' trademarks. No endorsement is implied.
  1. Serialization
    1. Raw Json
  2. Example Input/Output Pair
  3. Batch Execution
  4. Differential Validation
    1. Feedback Validation
    2. Learning Validation

Target Description: This convolution layer only supports an equal number of input and output bands. It is used as the foundational component for ConvolutionLayer, since the CuDNN api has this restriction (in recent versions).

Report Description: Typical 3x3 image convolution (32-bit)

Serialization

This run will demonstrate the layer’s JSON serialization, and verify deserialization integrity.

Raw Json

Code from SerializationTest.java:84 executed in 0.00 seconds:

    final JsonObject json = layer.getJson();
    final NNLayer echo = NNLayer.fromJson(json);
    if (echo == null) throw new AssertionError("Failed to deserialize");
    if (layer == echo) throw new AssertionError("Serialization did not copy");
    if (!layer.equals(echo)) throw new AssertionError("Serialization not equal");
    return new GsonBuilder().setPrettyPrinting().create().toJson(json);

Returns:

    {
      "class": "com.simiacryptus.mindseye.layers.cudnn.SimpleConvolutionLayer",
      "id": "501bf533-7681-4f31-a32f-a9b361a17c6a",
      "isFrozen": false,
      "name": "SimpleConvolutionLayer/501bf533-7681-4f31-a32f-a9b361a17c6a",
      "filter": [
        [
          [
            -1.66,
            -0.728,
            -0.232
          ],
          [
            0.972,
            0.62,
            1.228
          ],
          [
            -0.78,
            -1.776,
            1.896
          ]
        ],
        [
          [
            1.888,
            -0.284,
            -1.148
          ],
          [
            -0.652,
            1.9,
            -1.728
          ],
          [
            1.036,
            1.892,
            1.568
          ]
        ],
        [
          [
            -0.892,
            -1.052,
            0.012
          ],
          [
            -1.144,
            0.976,
            -0.224
          ],
          [
            1.924,
            -1.756,
            0.216
          ]
        ],
        [
          [
            0.868,
            0.196,
            0.636
          ],
          [
            0.04,
            1.444,
            -0.236
          ],
          [
            0.08,
            0.936,
            1.14
          ]
        ],
        [
          [
            -1.436,
            1.096,
            0.456
          ],
          [
            0.144,
            -1.252,
            1.652
          ],
          [
            -1.052,
            -0.276,
            -1.116
          ]
        ],
        [
          [
            -0.052,
            0.972,
            -1.556
          ],
          [
            -0.164,
            -1.02,
            1.148
          ],
          [
            0.448,
            0.388,
            -0.44
          ]
        ],
        [
          [
            0.592,
            -1.464,
            -1.628
          ],
          [
            0.072,
            1.344,
            -1.26
          ],
          [
            0.812,
            -0.476,
            -0.084
          ]
        ],
        [
          [
            0.624,
            -0.168,
            0.204
          ],
          [
            -1.052,
            0.176,
            0.312
          ],
          [
            0.188,
            0.036,
            -1.432
          ]
        ],
        [
          [
            1.268,
            0.432,
            0.436
          ],
          [
            1.632,
            0.744,
            -1.148
          ],
          [
            -0.584,
            -0.416,
            0.768
          ]
        ]
      ],
      "strideX": 1,
      "strideY": 1,
      "paddingX": 1,
      "paddingY": 1,
      "simple": false,
      "precision": "Float"
    }

Wrote Model to SimpleConvolutionLayer_Image_Float.json; 2127 characters

Example Input/Output Pair

Display input/output pairs from random executions:

Code from ReferenceIO.java:69 executed in 0.00 seconds:

    final SimpleEval eval = SimpleEval.run(layer, inputPrototype);
    return String.format("--------------------\nInput: \n[%s]\n--------------------\nOutput: \n%s\n--------------------\nDerivative: \n%s",
                         Arrays.stream(inputPrototype).map(t -> t.prettyPrint()).reduce((a, b) -> a + ",\n" + b).get(),
                         eval.getOutput().prettyPrint(),
                         Arrays.stream(eval.getDerivative()).map(t -> t.prettyPrint()).reduce((a, b) -> a + ",\n" + b).get());

Returns:

    --------------------
    Input: 
    [[
    	[ [ -0.152, 1.848, 0.8 ], [ 0.56, -0.164, 1.936 ], [ 0.076, 1.456, 0.148 ] ],
    	[ [ -0.416, 0.232, -0.96 ], [ 1.16, 0.196, -0.276 ], [ -1.12, 0.096, 1.528 ] ],
    	[ [ 0.112, 1.78, -0.704 ], [ -0.388, -0.528, 0.584 ], [ -1.964, 0.404, -1.54 ] ]
    ]]
    --------------------
    Output: 
    [
    	[ [ 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0 ] ],
    	[ [ 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0 ] ],
    	[ [ 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0 ] ]
    ]
    --------------------
    Derivative: 
    [
    	[ [ 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0 ] ],
    	[ [ 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0 ] ],
    	[ [ 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0 ] ]
    ]

GPU Log

Batch Execution

Most layers, including this one, should behave the same no matter how the items are split between batches. We verify this:

Code from BatchingTester.java:113 executed in 0.04 seconds:

    return test(reference, inputPrototype);

Returns:

    ToleranceStatistics{absoluteTol=0.0000e+00 +- 0.0000e+00 [0.0000e+00 - 0.0000e+00] (540#), relativeTol=0.0000e+00 +- 0.0000e+00 [Infinity - -Infinity] (0#)}

Differential Validation

Code from SingleDerivativeTester.java:292 executed in 0.00 seconds:

    log.info(String.format("Inputs: %s", Arrays.stream(inputPrototype).map(t -> t.prettyPrint()).reduce((a, b) -> a + ",\n" + b).get()));
    log.info(String.format("Inputs Statistics: %s", Arrays.stream(inputPrototype).map(x -> new ScalarStatistics().add(x.getData()).toString()).reduce((a, b) -> a + ",\n" + b).get()));
    log.info(String.format("Output: %s", outputPrototype.prettyPrint()));
    log.info(String.format("Outputs Statistics: %s", new ScalarStatistics().add(outputPrototype.getData())));

Logging:

    Inputs: [
    	[ [ -0.344, -1.056, 0.592 ], [ -1.66, 1.3, 1.976 ], [ -0.564, 1.968, -1.06 ] ],
    	[ [ 0.556, -1.46, -1.3 ], [ 0.06, 1.204, -0.324 ], [ 0.792, -0.332, -1.176 ] ],
    	[ [ 1.076, -0.32, -0.432 ], [ 0.268, -1.144, 1.864 ], [ 1.84, -0.056, -0.852 ] ]
    ]
    Inputs Statistics: {meanExponent=-0.15596833283686654, negative=15, min=-0.852, max=-0.852, mean=0.052444444444444474, count=27, positive=12, stdDev=1.1122293040067583, zeros=0}
    Output: [
    	[ [ 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0 ] ],
    	[ [ 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0 ] ],
    	[ [ 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0 ], [ 0.0, 0.0, 0.0 ] ]
    ]
    Outputs Statistics: {meanExponent=NaN, negative=0, min=0.0, max=0.0, mean=0.0, count=27, positive=0, stdDev=0.0, zeros=27}
    

Feedback Validation

We validate the agreement between the implemented derivative of the inputs with finite difference estimations:

Code from SingleDerivativeTester.java:303 executed in 0.13 seconds:

    return testFeedback(statistics, component, inputPrototype, outputPrototype);

Logging:

    Feedback for input 0
    Inputs Values: [
    	[ [ -0.344, -1.056, 0.592 ], [ -1.66, 1.3, 1.976 ], [ -0.564, 1.968, -1.06 ] ],
    	[ [ 0.556, -1.46, -1.3 ], [ 0.06, 1.204, -0.324 ], [ 0.792, -0.332, -1.176 ] ],
    	[ [ 1.076, -0.32, -0.432 ], [ 0.268, -1.144, 1.864 ], [ 1.84, -0.056, -0.852 ] ]
    ]
    Value Statistics: {meanExponent=-0.15596833283686654, negative=15, min=-0.852, max=-0.852, mean=0.052444444444444474, count=27, positive=12, stdDev=1.1122293040067583, zeros=0}
    Implemented Feedback: [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], ... ]
    Implemented Statistics: {meanExponent=NaN, negative=0, min=0.0, max=0.0, mean=0.0, count=729, positive=0, stdDev=0.0, zeros=729}
    Measured Feedback: [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], ... ]
    Measured Statistics: {meanExponent=NaN, negative=0, min=0.0, max=0.0, mean=0.0, count=729, positive=0, stdDev=0.0, zeros=729}
    Feedback Error: [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], ... ]
    Error Statistics: {meanExponent=NaN, negative=0, min=0.0, max=0.0, mean=0.0, count=729, positive=0, stdDev=0.0, zeros=729}
    

Returns:

    ToleranceStatistics{absoluteTol=0.0000e+00 +- 0.0000e+00 [0.0000e+00 - 0.0000e+00] (729#), relativeTol=0.0000e+00 +- 0.0000e+00 [Infinity - -Infinity] (0#)}

Learning Validation

We validate the agreement between the implemented derivative of the internal weights with finite difference estimations:

Code from SingleDerivativeTester.java:311 executed in 0.00 seconds:

    return testLearning(statistics, component, inputPrototype, outputPrototype);

Logging:

    Learning Gradient for weight setByCoord 0
    Implemented Gradient: [ [ 0.05999999865889549, 0.2680000066757202, 0.0, 0.7919999957084656, 1.840000033378601, 0.0, 0.0, 0.0, ... ], [ -1.659999966621399, 0.05999999865889549, 0.2680000066757202, -0.5640000104904175, 0.7919999957084656, 1.840000033378601, 0.0, 0.0, ... ], [ 0.0, -1.659999966621399, 0.05999999865889549, 0.0, -0.5640000104904175, 0.7919999957084656, 0.0, 0.0, ... ], [ 0.5559999942779541, 1.0759999752044678, 0.0, 0.05999999865889549, 0.2680000066757202, 0.0, 0.7919999957084656, 1.840000033378601, ... ], [ -0.3440000116825104, 0.5559999942779541, 1.0759999752044678, -1.659999966621399, 0.05999999865889549, 0.2680000066757202, -0.5640000104904175, 0.7919999957084656, ... ], [ 0.0, -0.3440000116825104, 0.5559999942779541, 0.0, -1.659999966621399, 0.05999999865889549, 0.0, -0.5640000104904175, ... ], [ 0.0, 0.0, 0.0, 0.5559999942779541, 1.0759999752044678, 0.0, 0.05999999865889549, 0.2680000066757202, ... ], [ 0.0, 0.0, 0.0, -0.3440000116825104, 0.5559999942779541, 1.0759999752044678, -1.659999966621399, 0.05999999865889549, ... ], ... ]
    Implemented Statistics: {meanExponent=-0.17140811599175057, negative=231, min=-0.3240000009536743, max=-0.3240000009536743, mean=0.013349793652080215, count=2187, positive=210, stdDev=0.4975561422391111, zeros=1746}
    Measured Gradient: [ [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], [ 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ... ], ... ]
    Measured Statistics: {meanExponent=NaN, negative=0, min=0.0, max=0.0, mean=0.0, count=2187, positive=0, stdDev=0.0, zeros=2187}
    Gradient Error: [ [ -0.05999999865889549, -0.2680000066757202, 0.0, -0.7919999957084656, -1.840000033378601, 0.0, 0.0, 0.0, ... ], [ 1.659999966621399, -0.05999999865889549, -0.2680000066757202, 0.5640000104904175, -0.7919999957084656, -1.840000033378601, 0.0, 0.0, ... ], [ 0.0, 1.659999966621399, -0.05999999865889549, 0.0, 0.5640000104904175, -0.7919999957084656, 0.0, 0.0, ... ], [ -0.5559999942779541, -1.0759999752044678, 0.0, -0.05999999865889549, -0.2680000066757202, 0.0, -0.7919999957084656, -1.840000033378601, ... ], [ 0.3440000116825104, -0.5559999942779541, -1.0759999752044678, 1.659999966621399, -0.05999999865889549, -0.2680000066757202, 0.5640000104904175, -0.7919999957084656, ... ], [ 0.0, 0.3440000116825104, -0.5559999942779541, 0.0, 1.659999966621399, -0.05999999865889549, 0.0, 0.5640000104904175, ... ], [ 0.0, 0.0, 0.0, -0.5559999942779541, -1.0759999752044678, 0.0, -0.05999999865889549, -0.2680000066757202, ... ], [ 0.0, 0.0, 0.0, 0.3440000116825104, -0.5559999942779541, -1.0759999752044678, 1.659999966621399, -0.05999999865889549, ... ], ... ]
    Error Statistics: {meanExponent=-0.17140811599175057, negative=210, min=0.3240000009536743, max=0.3240000009536743, mean=-0.013349793652080215, count=2187, positive=231, stdDev=0.4975561422391111, zeros=1746}
    

Returns:

    java.lang.AssertionError: ToleranceStatistics{absoluteTol=1.8916e-01 +- 4.6039e-01 [0.0000e+00 - 1.9760e+00] (2187#), relativeTol=1.0000e+00 +- 0.0000e+00 [1.0000e+00 - 1.0000e+00] (441#)}
    	at com.simiacryptus.mindseye.test.unit.SingleDerivativeTester.lambda$testLearning$23(SingleDerivativeTester.java:353)
    	at java.util.stream.IntPipeline$4$1.accept(IntPipeline.java:250)
    	at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
    	at java.util.Spliterator$OfInt.forEachRemaining(Spliterator.java:693)
    	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
    	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
    	at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
    	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    	at java.util.stream.ReferencePipeline.reduce(ReferencePipeline.java:479)
    	at com.simiacryptus.mindseye.test.unit.SingleDerivativeTester.testLearning(SingleDerivativeTester.java:386)
    	at com.simiacryptus.mindseye.test.unit.SingleDerivativeTester.lambda$test$18(SingleDerivativeTester.java:312)
    	at com.simiacryptus.util.io.MarkdownNotebookOutput.lambda$null$1(MarkdownNotebookOutput.java:205)
    	at com.simiacryptus.util.lang.TimedResult.time(TimedResult.java:59)
    	at com.simiacryptus.util.io.MarkdownNotebookOutput.lambda$code$2(MarkdownNotebookOutput.java:205)
    	at com.simiacryptus.util.test.SysOutInterceptor.withOutput(SysOutInterceptor.java:107)
    	at com.simiacryptus.util.io.MarkdownNotebookOutput.code(MarkdownNotebookOutput.java:203)
    	at com.simiacryptus.util.io.NotebookOutput.code(NotebookOutput.java:82)
    	at com.simiacryptus.mindseye.test.unit.SingleDerivativeTester.test(SingleDerivativeTester.java:311)
    	at com.simiacryptus.mindseye.test.unit.SingleDerivativeTester.test(SingleDerivativeTester.java:42)
    	at com.simiacryptus.mindseye.test.unit.StandardLayerTests.lambda$run$5(StandardLayerTests.java:257)
    	at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
    	at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
    	at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1374)
    	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
    	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
    	at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
    	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
    	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
    	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
    	at com.simiacryptus.mindseye.test.unit.StandardLayerTests.run(StandardLayerTests.java:256)
    	at com.simiacryptus.mindseye.test.NotebookReportBase.lambda$run$0(NotebookReportBase.java:105)
    	at com.simiacryptus.util.lang.TimedResult.time(TimedResult.java:76)
    	at com.simiacryptus.mindseye.test.NotebookReportBase.run(NotebookReportBase.java:103)
    	at com.simiacryptus.mindseye.layers.LayerTestBase.test(LayerTestBase.java:37)
    	at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:498)
    	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    	at org.junit.runners.Suite.runChild(Suite.java:128)
    	at org.junit.runners.Suite.runChild(Suite.java:27)
    	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
    	at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
    	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
    	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)