onnx-go: Gorgonia's evaluation of the MNIST model does not give expected result

This is related to the issue #2 I have with gorgonnx (the previous test implementation of onnx-to-gorgonia).

The problem is exactly the same with the new version of the unmarsaler (from the directed-graph branch)

To investigate, I will check every operator to see where the bug is hidden.

To do so, I have created a test file here. This file contains the evaluated input and output of all the node that compose the MNIST model (from the ONNX model zoo).

The next task is to evaluate all the tests to see if the results are ok.

To help me, each test function generates a “numpy” compatible tensor for input and output. For simple operators, that should be enough to run them within python and to compute the expected result.

Any help welcome.

HOWTO:

go-get this repository
checkout the directed-graph branch
cd into examples/gorgonia
go run mnist.go run the (unsuccessful) test (gorgonia has been vendorer in this directory)
go test generate a numpy subdirectory with the tests files.
find which operator is not ok

Remark: I did not export the attributes of the Convolution operator yet, but you can find their values in the internal/examples/mnist directory

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 16 (6 by maintainers)

Most upvoted comments

I extracted the inputs and outputs from 2nd conv node, and run some tests on gorgonia conv node, it didn’t give out the correct result, I suspect it’s a bug in gorgonia’s conv layer. @owulveryck @chewxy To reproduce the tests, follow this README. https://github.com/lynic/gorgonnx/tree/test_conv/eltest

Submit a related issue to gorgonia https://github.com/gorgonia/gorgonia/issues/268

lynic on Feb 22, 2019

First of all: many thanks for your help.

I have just tested it with a fresh and empty GOPATH; this seems to work:

GOPATH=/tmp go get -v github.com/owulveryck/onnx-go
cd /tmp/src/github.com/owulveryck/onnx-go
git checkout b7af8bd
cd example/gorgonia
GOPATH=/tmp go get -v ./...
GOPATH=/tmp go run mnist.go

owulveryck on Feb 6, 2019