You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mxnet.apache.org by GitBox <gi...@apache.org> on 2020/07/08 13:58:52 UTC
[GitHub] [incubator-mxnet] cosmincatalin opened a new issue #18671: Scala 2.11 binding for version 1.5.1 throws memory leak warnings
cosmincatalin opened a new issue #18671:
URL: https://github.com/apache/incubator-mxnet/issues/18671
I am using the Scala 2.11 binding in my project (🤞 for #18655) and keep getting memory leak warnings. This happens for both the MacOS and Linux builds.
It is not clear to me if these warnings are valid or not, since I would assume the issue was fixed a long time ago (#10436 and some others).
```
//libraryDependencies += "org.apache.mxnet" % "mxnet-full_2.11-linux-x86_64-cpu" % "1.5.1"
libraryDependencies += "org.apache.mxnet" % "mxnet-full_2.11-osx-x86_64-cpu" % "1.5.1"
```
```
020-07-08 15:46:48 WARN WarnIfNotDisposed:70 - LEAK: An instance of class org.apache.mxnet.NDArray was not disposed. Creation point of this resource was:
java.lang.Thread.getStackTrace(Thread.java:1559)
org.apache.mxnet.WarnIfNotDisposed$class.$init$(WarnIfNotDisposed.scala:52)
org.apache.mxnet.NDArray.<init>(NDArray.scala:744)
org.apache.mxnet.NDArray.at(NDArray.scala:927)
org.apache.mxnet.NDArray$$anonfun$org$apache$mxnet$NDArray$$buildStringHelper$1.apply$mcVI$sp(NDArray.scala:873)
scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)
org.apache.mxnet.NDArray.org$apache$mxnet$NDArray$$buildStringHelper(NDArray.scala:872)
org.apache.mxnet.NDArray$$anonfun$org$apache$mxnet$NDArray$$buildStringHelper$1.apply$mcVI$sp(NDArray.scala:873)
scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)
org.apache.mxnet.NDArray.org$apache$mxnet$NDArray$$buildStringHelper(NDArray.scala:872)
org.apache.mxnet.NDArray$$anonfun$org$apache$mxnet$NDArray$$buildStringHelper$1.apply$mcVI$sp(NDArray.scala:873)
scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)
org.apache.mxnet.NDArray.org$apache$mxnet$NDArray$$buildStringHelper(NDArray.scala:872)
org.apache.mxnet.NDArray.toString(NDArray.scala:846)
java.lang.String.valueOf(String.java:2994)
scala.collection.mutable.StringBuilder.append(StringBuilder.scala:200)
scala.collection.TraversableOnce$$anonfun$addString$1.apply(TraversableOnce.scala:364)
scala.collection.Iterator$class.foreach(Iterator.scala:891)
scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
scala.collection.TraversableOnce$class.addString(TraversableOnce.scala:357)
scala.collection.AbstractIterator.addString(Iterator.scala:1334)
scala.collection.TraversableOnce$class.mkString(TraversableOnce.scala:323)
scala.collection.AbstractIterator.mkString(Iterator.scala:1334)
scala.runtime.ScalaRunTime$._toString(ScalaRunTime.scala:166)
com.audienceproject.User.toString(User.scala:6)
....
```
I run my code with `-Dmxnet.traceLeakedObjects=true` so that I can get the verbose trace. The warnings are coming up from just about anywhere I use `NDArray`, I'm not even doing any inference or training at this point.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-mxnet] cosmincatalin commented on issue #18671: Scala 2.11 binding for version 1.5.1 throws memory leak warnings
Posted by GitBox <gi...@apache.org>.
cosmincatalin commented on issue #18671:
URL: https://github.com/apache/incubator-mxnet/issues/18671#issuecomment-752213400
Great stuff @johsbk
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] johsbk edited a comment on issue #18671: Scala 2.11 binding for version 1.5.1 throws memory leak warnings
Posted by GitBox <gi...@apache.org>.
johsbk edited a comment on issue #18671:
URL: https://github.com/apache/incubator-mxnet/issues/18671#issuecomment-751772048
@cosmincatalin this is because your NDarrays are not deallocated automatically, you can use ResourceScope, like this:
`ResourceScope.using(new ResourceScope()) {
val bareArray = row.getAs[Seq[Double]](fieldName).map(_.toFloat).toArray
val shape = schema.getShape(fieldName)
val transposedShape = shape.reverse
val expectedSize = shape.product
val paddedArray = bareArray.padTo(expectedSize, 0f)
NDArray.array(paddedArray, Shape(transposedShape), ctx).T.reshape(Shape(List(1, 1) ++ shape: _*)).toArray
}`
to automatically close NDarrays
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] johsbk commented on issue #18671: Scala 2.11 binding for version 1.5.1 throws memory leak warnings
Posted by GitBox <gi...@apache.org>.
johsbk commented on issue #18671:
URL: https://github.com/apache/incubator-mxnet/issues/18671#issuecomment-751772048
@cosmincatalin this is because you NDarrays are not deallocated automatically, you can do use ResourceScope, like this:
`ResourceScope.using(new ResourceScope()) {
val bareArray = row.getAs[Seq[Double]](fieldName).map(_.toFloat).toArray
val shape = schema.getShape(fieldName)
val transposedShape = shape.reverse
val expectedSize = shape.product
val paddedArray = bareArray.padTo(expectedSize, 0f)
NDArray.array(paddedArray, Shape(transposedShape), ctx).T.reshape(Shape(List(1, 1) ++ shape: _*)).toArray
}`
to automatically close NDarrays
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org
[GitHub] [incubator-mxnet] cosmincatalin closed issue #18671: Scala 2.11 binding for version 1.5.1 throws memory leak warnings
Posted by GitBox <gi...@apache.org>.
cosmincatalin closed issue #18671:
URL: https://github.com/apache/incubator-mxnet/issues/18671
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@mxnet.apache.org
For additional commands, e-mail: issues-help@mxnet.apache.org