Demos Applications Components Optimizers Experiments Datasets

Basic

Basic Test


Project maintained by SimiaCryptus Java, CuDNN, and CUDA are others' trademarks. No endorsement is implied.
  1. Network Diagram
  2. Serialization
    1. Raw Json
  3. Example Input/Output Pair
  4. Differential Validation
    1. Feedback Validation
    2. Learning Validation
    3. Total Accuracy
    4. Frozen and Alive Status
  5. Performance
  6. Training Characteristics
    1. Input Learning

Target Description: The type Std dev meta layer.

Report Description: Basic Test

Network Diagram

This is a network with the following layout:

Code from StandardLayerTests.java:251 executed in 0.22 seconds:

    return Graphviz.fromGraph(TestUtil.toGraph((DAGNetwork) layer))
                   .height(400).width(600).render(Format.PNG).toImage();

Returns:

Result

Serialization

This run will demonstrate the layer’s JSON serialization, and verify deserialization integrity.

Raw Json

Code from SerializationTest.java:84 executed in 0.00 seconds:

    final JsonObject json = layer.getJson();
    final NNLayer echo = NNLayer.fromJson(json);
    if (echo == null) throw new AssertionError("Failed to deserialize");
    if (layer == echo) throw new AssertionError("Serialization did not copy");
    if (!layer.equals(echo)) throw new AssertionError("Serialization not equal");
    return new GsonBuilder().setPrettyPrinting().create().toJson(json);

Returns:

    {
      "class": "com.simiacryptus.mindseye.layers.java.StdDevMetaLayer",
      "id": "bb20a895-440e-43e5-82c3-883eb6752a76",
      "isFrozen": false,
      "name": "StdDevMetaLayer/bb20a895-440e-43e5-82c3-883eb6752a76",
      "inputs": [
        "f0eb8e32-62d5-4552-a9d6-05aa02e10913"
      ],
      "nodes": {
        "4e1e0617-2f1f-44ad-b400-783baa8255cd": "30496926-faa3-4d0c-85b4-bf54c3a1b1ce",
        "e94e9fd4-4f09-411c-8aa5-9a2598263148": "86544111-8b27-47e6-b537-d1111a665457",
        "aa2d9b67-b2b3-4045-8545-7f9c42066540": "a3f691fe-4e48-455a-87fe-b55ee12610f4",
        "be6eea00-ea0a-4c91-b6e9-47acfb12012b": "81c1a6cc-de32-482a-ad3b-79c4f77f11b6",
        "bd6921f3-1a6b-4efa-98f2-3ddf223e41bb": "01214095-73f8-4376-9f25-36be2c674bd6",
        "1899b1a7-10ca-4acc-bd3c-b44c791816a8": "92c533d5-09e5-4e1f-b62d-11dffaffbe43",
        "5f253905-879c-4578-8e5c-0f0c327addd2": "15c8970a-4ed8-423c-9f3e-611eede02b9b",
        "a36841a9-570b-4203-a833-7a8340daeca9": "b5280fba-d6b6-4e13-b9a1-c53d7488ef14",
        "87f1ef37-30d2-49b3-b897-16e710fdaa5d": "eacb164f-46f9-4431-8da9-e3904168ffa3"
      },
      "layers": {
        "30496926-faa3-4d0c-85b4-bf54c3a1b1ce": {
          "class": "com.simiacryptus.mindseye.layers.java.SqActivationLayer",
          "id": "30496926-faa3-4d0c-85b4-bf54c3a1b1ce",
          "isFrozen": true,
          "name": "SqActivationLayer/30496926-faa3-4d0c-85b4-bf54c3a1b1ce"
        },
        "86544111-8b27-47e6-b537-d1111a665457": {
          "class": "com.simiacryptus.mindseye.layers.java.AvgMetaLayer",
          "id": "86544111-8b27-47e6-b537-d1111a665457",
          "isFrozen": false,
          "name": "AvgMetaLayer/86544111-8b27-47e6-b537-d1111a665457",
          "minBatchCount": 0
        },
        "a3f691fe-4e48-455a-87fe-b55ee12610f4": {
          "class": "com.simiacryptus.mindseye.layers.java.AvgReducerLayer",
          "id": "a3f691fe-4e48-455a-87fe-b55ee12610f4",
          "isFrozen": false,
          "name": "AvgReducerLayer/a3f691fe-4e48-455a-87fe-b55ee12610f4"
        },
        "81c1a6cc-de32-482a-ad3b-79c4f77f11b6": {
          "class": "com.simiacryptus.mindseye.layers.java.AvgMetaLayer",
          "id": "81c1a6cc-de32-482a-ad3b-79c4f77f11b6",
          "isFrozen": false,
          "name": "AvgMetaLayer/81c1a6cc-de32-482a-ad3b-79c4f77f11b6",
          "minBatchCount": 0
        },
        "01214095-73f8-4376-9f25-36be2c674bd6": {
          "class": "com.simiacryptus.mindseye.layers.java.AvgReducerLayer",
          "id": "01214095-73f8-4376-9f25-36be2c674bd6",
          "isFrozen": false,
          "name": "AvgReducerLayer/01214095-73f8-4376-9f25-36be2c674bd6"
        },
        "92c533d5-09e5-4e1f-b62d-11dffaffbe43": {
          "class": "com.simiacryptus.mindseye.layers.java.SqActivationLayer",
          "id": "92c533d5-09e5-4e1f-b62d-11dffaffbe43",
          "isFrozen": true,
          "name": "SqActivationLayer/92c533d5-09e5-4e1f-b62d-11dffaffbe43"
        },
        "15c8970a-4ed8-423c-9f3e-611eede02b9b": {
          "class": "com.simiacryptus.mindseye.layers.java.LinearActivationLayer",
          "id": "15c8970a-4ed8-423c-9f3e-611eede02b9b",
          "isFrozen": true,
          "name": "LinearActivationLayer/15c8970a-4ed8-423c-9f3e-611eede02b9b",
          "weights": [
            -1.0,
            0.0
          ]
        },
        "b5280fba-d6b6-4e13-b9a1-c53d7488ef14": {
          "class": "com.simiacryptus.mindseye.layers.java.SumInputsLayer",
          "id": "b5280fba-d6b6-4e13-b9a1-c53d7488ef14",
          "isFrozen": false,
          "name": "SumInputsLayer/b5280fba-d6b6-4e13-b9a1-c53d7488ef14"
        },
        "eacb164f-46f9-4431-8da9-e3904168ffa3": {
          "class": "com.simiacryptus.mindseye.layers.java.NthPowerActivationLayer",
          "id": "eacb164f-46f9-4431-8da9-e3904168ffa3",
          "isFrozen": false,
          "name": "NthPowerActivationLayer/eacb164f-46f9-4431-8da9-e3904168ffa3",
          "power": 0.5
        }
      },
      "links": {
        "4e1e0617-2f1f-44ad-b400-783baa8255cd": [
          "f0eb8e32-62d5-4552-a9d6-05aa02e10913"
        ],
        "e94e9fd4-4f09-411c-8aa5-9a2598263148": [
          "4e1e0617-2f1f-44ad-b400-783baa8255cd"
        ],
        "aa2d9b67-b2b3-4045-8545-7f9c42066540": [
          "e94e9fd4-4f09-411c-8aa5-9a2598263148"
        ],
        "be6eea00-ea0a-4c91-b6e9-47acfb12012b": [
          "f0eb8e32-62d5-4552-a9d6-05aa02e10913"
        ],
        "bd6921f3-1a6b-4efa-98f2-3ddf223e41bb": [
          "be6eea00-ea0a-4c91-b6e9-47acfb12012b"
        ],
        "1899b1a7-10ca-4acc-bd3c-b44c791816a8": [
          "bd6921f3-1a6b-4efa-98f2-3ddf223e41bb"
        ],
        "5f253905-879c-4578-8e5c-0f0c327addd2": [
          "1899b1a7-10ca-4acc-bd3c-b44c791816a8"
        ],
        "a36841a9-570b-4203-a833-7a8340daeca9": [
          "aa2d9b67-b2b3-4045-8545-7f9c42066540",
          "5f253905-879c-4578-8e5c-0f0c327addd2"
        ],
        "87f1ef37-30d2-49b3-b897-16e710fdaa5d": [
          "a36841a9-570b-4203-a833-7a8340daeca9"
        ]
      },
      "labels": {},
      "head": "87f1ef37-30d2-49b3-b897-16e710fdaa5d"
    }

Wrote Model to StdDevMetaLayer_Basic.json; 4653 characters

Example Input/Output Pair

Display input/output pairs from random executions:

Code from ReferenceIO.java:69 executed in 0.00 seconds:

    final SimpleEval eval = SimpleEval.run(layer, inputPrototype);
    return String.format("--------------------\nInput: \n[%s]\n--------------------\nOutput: \n%s\n--------------------\nDerivative: \n%s",
                         Arrays.stream(inputPrototype).map(t -> t.prettyPrint()).reduce((a, b) -> a + ",\n" + b).get(),
                         eval.getOutput().prettyPrint(),
                         Arrays.stream(eval.getDerivative()).map(t -> t.prettyPrint()).reduce((a, b) -> a + ",\n" + b).get());

Returns:

    --------------------
    Input: 
    [[ 0.088, 0.864, -0.112 ]]
    --------------------
    Output: 
    [ 0.42094496869147474 ]
    --------------------
    Derivative: 
    [ -0.15203887624300796, 0.4624515819058159, -0.31041270566280793 ]

Differential Validation

Code from BatchDerivativeTester.java:388 executed in 0.00 seconds:

    BatchDerivativeTester.log.info(String.format("Inputs: %s", Arrays.stream(inputPrototype).map(t -> t.prettyPrint()).reduce((a, b) -> a + ",\n" + b).get()));
    BatchDerivativeTester.log.info(String.format("Inputs Statistics: %s", Arrays.stream(inputPrototype).map(x -> new ScalarStatistics().add(x.getData()).toString()).reduce((a, b) -> a + ",\n" + b).get()));
    BatchDerivativeTester.log.info(String.format("Output: %s", ioPair.getOutputPrototype().prettyPrint()));
    BatchDerivativeTester.log.info(String.format("Outputs Statistics: %s", new ScalarStatistics().add(ioPair.getOutputPrototype().getData())));

Logging:

    Inputs: [ 1.396, -0.584, -0.216 ]
    Inputs Statistics: {meanExponent=-0.2514159944831758, negative=2, min=-0.216, max=-0.216, mean=0.19866666666666666, count=3, positive=1, stdDev=0.8598687238306916, zeros=0}
    Output: [ 0.8598687238306916 ]
    Outputs Statistics: {meanExponent=-0.06556784743988038, negative=0, min=0.8598687238306916, max=0.8598687238306916, mean=0.8598687238306916, count=1, positive=1, stdDev=0.0, zeros=0}
    

Feedback Validation

We validate the agreement between the implemented derivative of the inputs with finite difference estimations:

Code from BatchDerivativeTester.java:402 executed in 0.01 seconds:

    return testFeedback(component, ioPair, statistics);

Logging:

    Feedback for input 0
    Inputs Values: [ 1.396, -0.584, -0.216 ]
    Value Statistics: {meanExponent=-0.2514159944831758, negative=2, min=-0.216, max=-0.216, mean=0.19866666666666666, count=3, positive=1, stdDev=0.8598687238306916, zeros=0}
    Implemented Feedback: [ [ 0.046415353884844426 ], [ -0.030340548697554207 ], [ -0.01607480518729022 ] ]
    Implemented Statistics: {meanExponent=-1.5483897283575636, negative=2, min=-0.01607480518729022, max=-0.01607480518729022, mean=0.0, count=3, positive=1, stdDev=0.03333333333333333, zeros=0}
    Measured Feedback: [ [ 0.04641710227182294 ], [ -0.03033872854829589 ], [ -0.016072946538159272 ] ]
    Measured Statistics: {meanExponent=-1.5484096996125096, negative=2, min=-0.016072946538159272, max=-0.016072946538159272, mean=1.8090617892596583E-6, count=3, positive=1, stdDev=0.0333332938358456, zeros=0}
    Feedback Error: [ [ 1.7483869785145378E-6 ], [ 1.8201492583171108E-6 ], [ 1.8586491309473263E-6 ] ]
    Error Statistics: {meanExponent=-5.742686006886181, negative=0, min=1.8586491309473263E-6, max=1.8586491309473263E-6, mean=1.8090617892596583E-6, count=3, positive=3, stdDev=4.569197261629849E-8, zeros=0}
    Feedback for input 1
    Inputs Values: [ 1.396, -0.584, -0.216 ]
    Value Statistics: {meanExponent=-0.2514159944831758, negative=2, min=-0.216, max=-0.216, mean=0.19866666666666666, count=3, positive=1, stdDev=0.8598687238306916, zeros=0}
    Implemented Feedback: [ [ 0.046415353884844426 ], [ -0.030340548697554207 ], [ -0.01607480518729022 ] ]
    Implemented Statistics: {meanExponent=-1.5483897283575636, negative=2, min=-0.01607480518729022, max=-0.01607480518729022, mean=0.0, count=3, positive=1, stdDev=0.03333333333333333, zeros=0}
    Measured Feedback: [ [ 0.04641710227182294 ], [ -0.03033872854829589 ], [ -0.016072946538159272 ] ]
    Measured Statistics: {meanExponent=-1.5484096996125096, negative=2, min=-0.016072946538159272, max=-0.016072946538159272, mean=1.8090617892596583E-6, count=3, positive=1, stdDev=0.0333332938358456, zeros=0}
    Feedback Error: [ [ 1.7483869785145378E-6 ], [ 1.8201492583171108E-6 ], [ 1.8586491309473263E-6 ] ]
    Error Statistics: {meanExponent=-5.742686006886181, negative=0, min=1.8586491309473263E-6, max=1.8586491309473263E-6, mean=1.8090617892596583E-6, count=3, positive=3, stdDev=4.569197261629849E-8, zeros=0}
    Feedback for input 2
    Inputs Values: [ 1.396, -0.584, -0.216 ]
    Value Statistics: {meanExponent=-0.2514159944831758, negative=2, min=-0.216, max=-0.216, mean=0.19866666666666666, count=3, positive=1, stdDev=0.8598687238306916, zeros=0}
    Implemented Feedback: [ [ 0.046415353884844426 ], [ -0.030340548697554207 ], [ -0.01607480518729022 ] ]
    Implemented Statistics: {meanExponent=-1.5483897283575636, negative=2, min=-0.01607480518729022, max=-0.01607480518729022, mean=0.0, count=3, positive=1, stdDev=0.03333333333333333, zeros=0}
    Measured Feedback: [ [ 0.04641710227182294 ], [ -0.03033872854829589 ], [ -0.016072946538159272 ] ]
    Measured Statistics: {meanExponent=-1.5484096996125096, negative=2, min=-0.016072946538159272, max=-0.016072946538159272, mean=1.8090617892596583E-6, count=3, positive=1, stdDev=0.0333332938358456, zeros=0}
    Feedback Error: [ [ 1.7483869785145378E-6 ], [ 1.8201492583171108E-6 ], [ 1.8586491309473263E-6 ] ]
    Error Statistics: {meanExponent=-5.742686006886181, negative=0, min=1.8586491309473263E-6, max=1.8586491309473263E-6, mean=1.8090617892596583E-6, count=3, positive=3, stdDev=4.569197261629849E-8, zeros=0}
    Feedback for input 3
    Inputs Values: [ 1.396, -0.584, -0.216 ]
    Value Statistics: {meanExponent=-0.2514159944831758, negative=2, min=-0.216, max=-0.216, mean=0.19866666666666666, count=3, positive=1, stdDev=0.8598687238306916, zeros=0}
    Implemented Feedback: [ [ 0.046415353884844426 ], [ -0.030340548697554207 ], [ -0.01607480518729022 ] ]
    Implemented Statistics: {meanExponent=-1.5483897283575636, negative=2, min=-0.01607480518729022, max=-0.01607480518729022, mean=0.0, count=3, positive=1, stdDev=0.03333333333333333, zeros=0}
    Measured Feedback: [ [ 0.04641710227182294 ], [ -0.03033872854829589 ], [ -0.016072946538159272 ] ]
    Measured Statistics: {meanExponent=-1.5484096996125096, negative=2, min=-0.016072946538159272, max=-0.016072946538159272, mean=1.8090617892596583E-6, count=3, positive=1, stdDev=0.0333332938358456, zeros=0}
    Feedback Error: [ [ 1.7483869785145378E-6 ], [ 1.8201492583171108E-6 ], [ 1.8586491309473263E-6 ] ]
    Error Statistics: {meanExponent=-5.742686006886181, negative=0, min=1.8586491309473263E-6, max=1.8586491309473263E-6, mean=1.8090617892596583E-6, count=3, positive=3, stdDev=4.569197261629849E-8, zeros=0}
    Feedback for input 4
    Inputs Values: [ 1.396, -0.584, -0.216 ]
    Value Statistics: {meanExponent=-0.2514159944831758, negative=2, min=-0.216, max=-0.216, mean=0.19866666666666666, count=3, positive=1, stdDev=0.8598687238306916, zeros=0}
    Implemented Feedback: [ [ 0.046415353884844426 ], [ -0.030340548697554207 ], [ -0.01607480518729022 ] ]
    Implemented Statistics: {meanExponent=-1.5483897283575636, negative=2, min=-0.01607480518729022, max=-0.01607480518729022, mean=0.0, count=3, positive=1, stdDev=0.03333333333333333, zeros=0}
    Measured Feedback: [ [ 0.04641710227182294 ], [ -0.03033872854829589 ], [ -0.016072946538159272 ] ]
    Measured Statistics: {meanExponent=-1.5484096996125096, negative=2, min=-0.016072946538159272, max=-0.016072946538159272, mean=1.8090617892596583E-6, count=3, positive=1, stdDev=0.0333332938358456, zeros=0}
    Feedback Error: [ [ 1.7483869785145378E-6 ], [ 1.8201492583171108E-6 ], [ 1.8586491309473263E-6 ] ]
    Error Statistics: {meanExponent=-5.742686006886181, negative=0, min=1.8586491309473263E-6, max=1.8586491309473263E-6, mean=1.8090617892596583E-6, count=3, positive=3, stdDev=4.569197261629849E-8, zeros=0}
    Feedback for input 5
    Inputs Values: [ 1.396, -0.584, -0.216 ]
    Value Statistics: {meanExponent=-0.2514159944831758, negative=2, min=-0.216, max=-0.216, mean=0.19866666666666666, count=3, positive=1, stdDev=0.8598687238306916, zeros=0}
    Implemented Feedback: [ [ 0.046415353884844426 ], [ -0.030340548697554207 ], [ -0.01607480518729022 ] ]
    Implemented Statistics: {meanExponent=-1.5483897283575636, negative=2, min=-0.01607480518729022, max=-0.01607480518729022, mean=0.0, count=3, positive=1, stdDev=0.03333333333333333, zeros=0}
    Measured Feedback: [ [ 0.04641710227182294 ], [ -0.03033872854829589 ], [ -0.016072946538159272 ] ]
    Measured Statistics: {meanExponent=-1.5484096996125096, negative=2, min=-0.016072946538159272, max=-0.016072946538159272, mean=1.8090617892596583E-6, count=3, positive=1, stdDev=0.0333332938358456, zeros=0}
    Feedback Error: [ [ 1.7483869785145378E-6 ], [ 1.8201492583171108E-6 ], [ 1.8586491309473263E-6 ] ]
    Error Statistics: {meanExponent=-5.742686006886181, negative=0, min=1.8586491309473263E-6, max=1.8586491309473263E-6, mean=1.8090617892596583E-6, count=3, positive=3, stdDev=4.569197261629849E-8, zeros=0}
    Feedback for input 6
    Inputs Values: [ 1.396, -0.584, -0.216 ]
    Value Statistics: {meanExponent=-0.2514159944831758, negative=2, min=-0.216, max=-0.216, mean=0.19866666666666666, count=3, positive=1, stdDev=0.8598687238306916, zeros=0}
    Implemented Feedback: [ [ 0.046415353884844426 ], [ -0.030340548697554207 ], [ -0.01607480518729022 ] ]
    Implemented Statistics: {meanExponent=-1.5483897283575636, negative=2, min=-0.01607480518729022, max=-0.01607480518729022, mean=0.0, count=3, positive=1, stdDev=0.03333333333333333, zeros=0}
    Measured Feedback: [ [ 0.04641710227182294 ], [ -0.03033872854829589 ], [ -0.016072946538159272 ] ]
    Measured Statistics: {meanExponent=-1.5484096996125096, negative=2, min=-0.016072946538159272, max=-0.016072946538159272, mean=1.8090617892596583E-6, count=3, positive=1, stdDev=0.0333332938358456, zeros=0}
    Feedback Error: [ [ 1.7483869785145378E-6 ], [ 1.8201492583171108E-6 ], [ 1.8586491309473263E-6 ] ]
    Error Statistics: {meanExponent=-5.742686006886181, negative=0, min=1.8586491309473263E-6, max=1.8586491309473263E-6, mean=1.8090617892596583E-6, count=3, positive=3, stdDev=4.569197261629849E-8, zeros=0}
    Feedback for input 7
    Inputs Values: [ 1.396, -0.584, -0.216 ]
    Value Statistics: {meanExponent=-0.2514159944831758, negative=2, min=-0.216, max=-0.216, mean=0.19866666666666666, count=3, positive=1, stdDev=0.8598687238306916, zeros=0}
    Implemented Feedback: [ [ 0.046415353884844426 ], [ -0.030340548697554207 ], [ -0.01607480518729022 ] ]
    Implemented Statistics: {meanExponent=-1.5483897283575636, negative=2, min=-0.01607480518729022, max=-0.01607480518729022, mean=0.0, count=3, positive=1, stdDev=0.03333333333333333, zeros=0}
    Measured Feedback: [ [ 0.04641710227182294 ], [ -0.03033872854829589 ], [ -0.016072946538159272 ] ]
    Measured Statistics: {meanExponent=-1.5484096996125096, negative=2, min=-0.016072946538159272, max=-0.016072946538159272, mean=1.8090617892596583E-6, count=3, positive=1, stdDev=0.0333332938358456, zeros=0}
    Feedback Error: [ [ 1.7483869785145378E-6 ], [ 1.8201492583171108E-6 ], [ 1.8586491309473263E-6 ] ]
    Error Statistics: {meanExponent=-5.742686006886181, negative=0, min=1.8586491309473263E-6, max=1.8586491309473263E-6, mean=1.8090617892596583E-6, count=3, positive=3, stdDev=4.569197261629849E-8, zeros=0}
    Feedback for input 8
    Inputs Values: [ 1.396, -0.584, -0.216 ]
    Value Statistics: {meanExponent=-0.2514159944831758, negative=2, min=-0.216, max=-0.216, mean=0.19866666666666666, count=3, positive=1, stdDev=0.8598687238306916, zeros=0}
    Implemented Feedback: [ [ 0.046415353884844426 ], [ -0.030340548697554207 ], [ -0.01607480518729022 ] ]
    Implemented Statistics: {meanExponent=-1.5483897283575636, negative=2, min=-0.01607480518729022, max=-0.01607480518729022, mean=0.0, count=3, positive=1, stdDev=0.03333333333333333, zeros=0}
    Measured Feedback: [ [ 0.04641710227182294 ], [ -0.03033872854829589 ], [ -0.016072946538159272 ] ]
    Measured Statistics: {meanExponent=-1.5484096996125096, negative=2, min=-0.016072946538159272, max=-0.016072946538159272, mean=1.8090617892596583E-6, count=3, positive=1, stdDev=0.0333332938358456, zeros=0}
    Feedback Error: [ [ 1.7483869785145378E-6 ], [ 1.8201492583171108E-6 ], [ 1.8586491309473263E-6 ] ]
    Error Statistics: {meanExponent=-5.742686006886181, negative=0, min=1.8586491309473263E-6, max=1.8586491309473263E-6, mean=1.8090617892596583E-6, count=3, positive=3, stdDev=4.569197261629849E-8, zeros=0}
    Feedback for input 9
    Inputs Values: [ 1.396, -0.584, -0.216 ]
    Value Statistics: {meanExponent=-0.2514159944831758, negative=2, min=-0.216, max=-0.216, mean=0.19866666666666666, count=3, positive=1, stdDev=0.8598687238306916, zeros=0}
    Implemented Feedback: [ [ 0.046415353884844426 ], [ -0.030340548697554207 ], [ -0.01607480518729022 ] ]
    Implemented Statistics: {meanExponent=-1.5483897283575636, negative=2, min=-0.01607480518729022, max=-0.01607480518729022, mean=0.0, count=3, positive=1, stdDev=0.03333333333333333, zeros=0}
    Measured Feedback: [ [ 0.04641710227182294 ], [ -0.03033872854829589 ], [ -0.016072946538159272 ] ]
    Measured Statistics: {meanExponent=-1.5484096996125096, negative=2, min=-0.016072946538159272, max=-0.016072946538159272, mean=1.8090617892596583E-6, count=3, positive=1, stdDev=0.0333332938358456, zeros=0}
    Feedback Error: [ [ 1.7483869785145378E-6 ], [ 1.8201492583171108E-6 ], [ 1.8586491309473263E-6 ] ]
    Error Statistics: {meanExponent=-5.742686006886181, negative=0, min=1.8586491309473263E-6, max=1.8586491309473263E-6, mean=1.8090617892596583E-6, count=3, positive=3, stdDev=4.569197261629849E-8, zeros=0}
    

Returns:

    ToleranceStatistics{absoluteTol=1.8091e-06 +- 4.5692e-08 [1.7484e-06 - 1.8586e-06] (30#), relativeTol=3.5549e-05 +- 1.6391e-05 [1.8834e-05 - 5.7816e-05] (30#)}

Learning Validation

We validate the agreement between the implemented derivative of the internal weights with finite difference estimations:

Code from BatchDerivativeTester.java:410 executed in 0.01 seconds:

    return testLearning(component, ioPair, statistics);

Logging:

    Learning Gradient for weight setByCoord 0
    Weights: [ -1.0, 0.0 ]
    Implemented Gradient: [ [ 0.022950273309519622 ], [ 0.581484110472717 ] ]
    Implemented Statistics: {meanExponent=-0.9373371432035269, negative=0, min=0.581484110472717, max=0.581484110472717, mean=0.3022171918911183, count=2, positive=2, stdDev=0.2792669185815987, zeros=0}
    Measured Gradient: [ [ 0.02295024268184065 ], [ 0.5814644504420485 ] ]
    Measured Statistics: {meanExponent=-0.9373447748839, negative=0, min=0.5814644504420485, max=0.5814644504420485, mean=0.3022073465619446, count=2, positive=2, stdDev=0.27925710388010394, zeros=0}
    Gradient Error: [ [ -3.0627678972855454E-8 ], [ -1.9660030668444683E-5 ] ]
    Error Statistics: {meanExponent=-6.110150861343371, negative=2, min=-1.9660030668444683E-5, max=-1.9660030668444683E-5, mean=-9.84532917370877E-6, count=2, positive=0, stdDev=9.814701494735914E-6, zeros=0}
    

Returns:

    ToleranceStatistics{absoluteTol=2.3113e-06 +- 3.1315e-06 [3.0628e-08 - 1.9660e-05] (32#), relativeTol=3.3876e-05 +- 1.7262e-05 [6.6726e-07 - 5.7816e-05] (32#)}

Total Accuracy

The overall agreement accuracy between the implemented derivative and the finite difference estimations:

Code from BatchDerivativeTester.java:418 executed in 0.00 seconds:

    //log.info(String.format("Component: %s\nInputs: %s\noutput=%s", component, Arrays.toString(inputPrototype), outputPrototype));
    BatchDerivativeTester.log.info(String.format("Finite-Difference Derivative Accuracy:"));
    BatchDerivativeTester.log.info(String.format("absoluteTol: %s", statistics.absoluteTol));
    BatchDerivativeTester.log.info(String.format("relativeTol: %s", statistics.relativeTol));

Logging:

    Finite-Difference Derivative Accuracy:
    absoluteTol: 2.3113e-06 +- 3.1315e-06 [3.0628e-08 - 1.9660e-05] (32#)
    relativeTol: 3.3876e-05 +- 1.7262e-05 [6.6726e-07 - 5.7816e-05] (32#)
    

Frozen and Alive Status

Code from BatchDerivativeTester.java:426 executed in 0.01 seconds:

    testFrozen(component, ioPair.getInputPrototype());
    testUnFrozen(component, ioPair.getInputPrototype());

Performance

Now we execute larger-scale runs to benchmark performance:

Code from PerformanceTester.java:183 executed in 0.01 seconds:

    test(component, inputPrototype);

Logging:

    100 batches
    Input Dimensions:
    	[100]
    Performance:
    	Evaluation performance: 0.001124s +- 0.000100s [0.001023s - 0.001267s]
    	Learning performance: 0.000032s +- 0.000021s [0.000005s - 0.000065s]
    

Per-layer Performance Metrics:

Code from TestUtil.java:216 executed in 0.00 seconds:

    final Map<NNLayer, MonitoringWrapperLayer> metrics = new HashMap<>();
    network.visitNodes(node -> {
      if (node.getLayer() instanceof MonitoringWrapperLayer) {
        final MonitoringWrapperLayer layer = node.getLayer();
        metrics.put(layer.getInner(), layer);
      }
    });
    TestUtil.log.info("Performance: \n\t" + metrics.entrySet().stream().map(e -> {
      final PercentileStatistics performanceF = e.getValue().getForwardPerformance();
      final PercentileStatistics performanceB = e.getValue().getBackwardPerformance();
      return String.format("%s -> %.6fs +- %.6fs (%d)", e.getKey(), performanceF.getMean(), performanceF.getStdDev(), performanceF.getCount()) +
        (performanceB.getCount() == 0 ? "" : String.format("%n\tBack: %.6fs +- %.6fs (%s)", performanceB.getMean(), performanceB.getStdDev(), performanceB.getCount()));
    }).reduce((a, b) -> a + "\n\t" + b).get());

Logging:

    Performance: 
    	AvgReducerLayer/1b634500-b8e2-457e-b65b-67f045475740 -> 0.000004s +- 0.000001s (11)
    	Back: 0.000015s +- 0.000000s (1)
    	SqActivationLayer/46d5cc7c-7a20-4eba-a1e8-35fd440a2c29 -> 0.000002s +- 0.000001s (11)
    	Back: 0.000002s +- 0.000000s (1)
    	AvgReducerLayer/4cce6d01-b147-4aba-ae33-c543c7f5d7a1 -> 0.000003s +- 0.000002s (11)
    	Back: 0.000009s +- 0.000000s (1)
    	LinearActivationLayer/2ca5d56e-0b10-4e30-83ca-02bd804d6086 -> 0.000003s +- 0.000001s (11)
    	Back: 0.000025s +- 0.000000s (1)
    	AvgMetaLayer/335a8864-a393-4ece-bfe9-8614e737ea50 -> 0.000339s +- 0.000053s (11)
    	Back: 0.000213s +- 0.000000s (1)
    	SumInputsLayer/2d6a793e-e837-4a38-b1b6-b87e516ec9d4 -> 0.000020s +- 0.000019s (11)
    	Back: 0.000059s +- 0.000000s (1)
    	SqActivationLayer/fac7c257-88dc-459e-9f21-a472b52b5f71 -> 0.000155s +- 0.000081s (11)
    	Back: 0.000005s +- 0.000000s (1)
    	NthPowerActivationLayer/5bef21f2-39ee-44b1-a660-dd02d223a79f -> 0.000003s +- 0.000001s (11)
    	Back: 0.000034s +- 0.000022s (6)
    	AvgMetaLayer/e6420b2b-6d15-410a-8659-c8504d2f535d -> 0.000430s +- 0.000050s (11)
    	Back: 0.000279s +- 0.000000s (1)
    

Training Characteristics

Input Learning

In this run, we use a network to learn this target input, given it’s pre-evaluated output:

Code from TrainingTester.java:423 executed in 0.00 seconds:

    return Arrays.stream(input_target)
                 .flatMap(x -> Arrays.stream(x))
                 .map(x -> x.prettyPrint())
                 .reduce((a, b) -> a + "\n" + b)
                 .orElse("");

Returns:

    [ 1.28, 1.008, 1.56, -1.528, 0.488, 1.6, -1.352, 0.944, ... ]
    [ 0.488, -0.956, 1.88, 0.924, -1.972, 0.388, 1.292, -0.096, ... ]
    [ -0.552, 0.212, -1.98, -0.424, -1.74, 0.696, -1.26, -1.524, ... ]