You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@mxnet.apache.org by GitBox <gi...@apache.org> on 2021/01/22 18:50:23 UTC
[GitHub] [incubator-mxnet] Johnny-dai-git opened a new issue #19781: MNISTiterator miss channel information while flatten causing conv1d layer differed initialization failed.
Johnny-dai-git opened a new issue #19781:
URL: https://github.com/apache/incubator-mxnet/issues/19781
## Description
(A clear and concise description of what the bug is.)
I am using Mxnet 1.6.0.
The problem shows up when I am trying to flatten the Mnist dataset from 2D(1,28,28) to (1,784). And feed the data to a Conv1d layer.
Here is the iterator I am using:
""" Flatten the dataset from (1,28,28) to (1,784)
train = mx.io.MNISTIter(
image = "data/train-images-idx3-ubyte",
label = "data/train-labels-idx1-ubyte",
batch_size = 20,
flat=True,
data_shape = (784, ))
val = mx.io.MNISTIter(
image="data/train-images-idx3-ubyte",
label="data/train-labels-idx1-ubyte",
batch_size=20,
flat=True,
data_shape=(784, ))
"After that, I try to feed it a model with a Conv1d as the first layer"
"Also I run the forward pass"
net = nn.Sequential()
net.add(
nn.Conv1D(channels=1,kernel_size=1,in_channels=1),
nn.Dense(units=10)
)
net.initialize()
for i, batch in enumerate(train):
print(batch)
print(i)
data = gluon.utils.split_and_load(batch.data[0].astype("float16"), ctx_list=ctx, batch_axis=0)
label = gluon.utils.split_and_load(batch.label[0].astype("float16"), ctx_list=ctx, batch_axis=0)
with ag.record():
for x, y in zip(data, label):
z = net(x)
### Error Message
[13:35:18] ../src/io/iter_mnist.cc:110: MNISTIter: load 60000 images, shuffle=1, shape=(20,784)
[13:35:20] ../src/io/iter_mnist.cc:110: MNISTIter: load 60000 images, shuffle=1, shape=(20,784)
DataBatch: data shapes: [(20, 784)] label shapes: [(20,)]
0
Traceback (most recent call last):
File "/Users/xiangrenbaibaoxiang/Desktop/johnny_source_code/souce_code/mxnet/example/gluon/imdb.py", line 54, in <module>
z = net(x)
File "/Users/xiangrenbaibaoxiang/Desktop/johnny_source_code/pcap/conv2d/tmp/venv/lib/python3.9/site-packages/mxnet/gluon/block.py", line 682, in __call__
out = self.forward(*args)
File "/Users/xiangrenbaibaoxiang/Desktop/johnny_source_code/pcap/conv2d/tmp/venv/lib/python3.9/site-packages/mxnet/gluon/nn/basic_layers.py", line 55, in forward
x = block(x)
File "/Users/xiangrenbaibaoxiang/Desktop/johnny_source_code/pcap/conv2d/tmp/venv/lib/python3.9/site-packages/mxnet/gluon/block.py", line 682, in __call__
out = self.forward(*args)
File "/Users/xiangrenbaibaoxiang/Desktop/johnny_source_code/pcap/conv2d/tmp/venv/lib/python3.9/site-packages/mxnet/gluon/block.py", line 1258, in forward
return self.hybrid_forward(ndarray, x, *args, **params)
File "/Users/xiangrenbaibaoxiang/Desktop/johnny_source_code/pcap/conv2d/tmp/venv/lib/python3.9/site-packages/mxnet/gluon/nn/conv_layers.py", line 147, in hybrid_forward
act = getattr(F, self._op_name)(x, weight, bias, name='fwd', **self._kwargs)
File "<string>", line 169, in Convolution
File "/Users/xiangrenbaibaoxiang/Desktop/johnny_source_code/pcap/conv2d/tmp/venv/lib/python3.9/site-packages/mxnet/_ctypes/ndarray.py", line 82, in _imperative_invoke
check_call(_LIB.MXImperativeInvokeEx(
File "/Users/xiangrenbaibaoxiang/Desktop/johnny_source_code/pcap/conv2d/tmp/venv/lib/python3.9/site-packages/mxnet/base.py", line 246, in check_call
raise get_last_ffi_error()
mxnet.base.MXNetError: Traceback (most recent call last):
File "../src/operator/nn/convolution.cc", line 103
MXNetError: Check failed: dshp.ndim() == 3U (2 vs. 3) : Input data should be 3D in batch-num_filter-x
## To Reproduce
Here is the iterator I am using:
""" Flatten the dataset from (1,28,28) to (1,784)
train = mx.io.MNISTIter(
image = "data/train-images-idx3-ubyte",
label = "data/train-labels-idx1-ubyte",
batch_size = 20,
flat=True,
data_shape = (784, ))
val = mx.io.MNISTIter(
image="data/train-images-idx3-ubyte",
label="data/train-labels-idx1-ubyte",
batch_size=20,
flat=True,
data_shape=(784, ))
"After that, I try to feed it a model with a Conv1d as the first layer"
"Also I run the forward pass"
net = nn.Sequential()
net.add(
nn.Conv1D(channels=1,kernel_size=1,in_channels=1),
nn.Dense(units=10)
)
net.initialize()
for i, batch in enumerate(train):
print(batch)
print(i)
data = gluon.utils.split_and_load(batch.data[0].astype("float16"), ctx_list=ctx, batch_axis=0)
label = gluon.utils.split_and_load(batch.label[0].astype("float16"), ctx_list=ctx, batch_axis=0)
with ag.record():
for x, y in zip(data, label):
z = net(x)
### Steps to reproduce
(Paste the commands you ran that produced the error.)
Just run it.
## What have you tried to solve it?
## Environment
Mnxet 1.6.0, Maxos, Ubuntu16,04 LTS
***We recommend using our script for collecting the diagnostic information with the following command***
`curl --retry 10 -s https://raw.githubusercontent.com/apache/incubator-mxnet/master/tools/diagnose.py | python3.`
<details>
<summary>Environment Information</summary>
```
# Paste the diagnose.py command output here
```
</details>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] github-actions[bot] commented on issue #19781: MNISTiterator miss channel information while flatten causing conv1d layer differed initialization failed.
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on issue #19781:
URL: https://github.com/apache/incubator-mxnet/issues/19781#issuecomment-765617066
Welcome to Apache MXNet (incubating)! We are on a mission to democratize AI, and we are glad that you are contributing to it by opening this issue.
Please make sure to include all the relevant context, and one of the @apache/mxnet-committers will be here shortly.
If you are interested in contributing to our project, let us know! Also, be sure to check out our guide on [contributing to MXNet](https://mxnet.apache.org/community/contribute) and our [development guides wiki](https://cwiki.apache.org/confluence/display/MXNET/Developments).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org