[GitHub] [tvm] tkonolige commented on a change in pull request #8056: [Relay, TOPI] Add negative log likelihood loss (nll_loss) op

[GitBox <gi...@apache.org>]

tkonolige commented on a change in pull request #8056:
URL: https://github.com/apache/tvm/pull/8056#discussion_r633772853



##########
File path: include/tvm/topi/nn.h
##########
@@ -642,6 +643,54 @@ inline tvm::te::Tensor batch_to_space_nd(const tvm::te::Tensor& data,
   out = strided_slice(out, begin_idx, end_idx, strides);
   return out;
 }
+
+/*!
+ * \brief Negative log likelihood loss.
+ *
+ * \param input The input tensor.
+ * \param target The target tensor.
+ * \param weight A manual rescaling weight given to each class.
+ * \param reduction The reduction method to apply to the output.
+ * \param ignore_index The target value to ignore.
+ * \param name The name of the operation.
+ * \param tag The tag to mark the operation.
+ *
+ * \return A Tensor whose op member is the batch_to_space_nd operation
+ */
+inline Tensor nll_loss(const Tensor& input, const Tensor& target, const Tensor& weight,
+                       std::string reduction = "mean", int ignore_index = -100,

Review comment:
       What is the significance of the -100 for `ignore_index`?

##########
File path: python/tvm/relay/op/nn/nn.py
##########
@@ -2973,6 +2973,34 @@ def cross_entropy_with_logits(predictions, targets):
     return _make.cross_entropy_with_logits(predictions, targets)
 
 
+def nll_loss(input, target, weight, reduction="mean", ignore_index=-100):
+    """Negative log likelihood loss.

Review comment:
       Can you document how `nll_loss` is computed (i.e. a mathematical expression).

##########
File path: src/relay/op/nn/nn.cc
##########
@@ -1091,6 +1092,60 @@ Accept logits.
 // Depth to space and space to depth
 TVM_REGISTER_NODE_TYPE(SubPixelAttrs);
 
+// relay.nn.nll_loss
+TVM_REGISTER_NODE_TYPE(NLLLossAttrs);
+
+bool NLLLossRel(const Array<Type>& types, int num_inputs, const Attrs& attrs,
+                const TypeReporter& reporter) {
+  ICHECK_EQ(types.size(), 4);

Review comment:
       ```suggestion
     ICHECK_EQ(types.size(), 4) << "NLLLossRel expects 4 types, but " << types.size() << " were provided.";
   ```

##########
File path: src/relay/op/nn/nn.cc
##########
@@ -1091,6 +1092,60 @@ Accept logits.
 // Depth to space and space to depth
 TVM_REGISTER_NODE_TYPE(SubPixelAttrs);
 
+// relay.nn.nll_loss
+TVM_REGISTER_NODE_TYPE(NLLLossAttrs);
+
+bool NLLLossRel(const Array<Type>& types, int num_inputs, const Attrs& attrs,
+                const TypeReporter& reporter) {
+  ICHECK_EQ(types.size(), 4);
+  const auto* input = types[0].as<TensorTypeNode>();
+  const auto* target = types[1].as<TensorTypeNode>();
+  const auto* weight = types[2].as<TensorTypeNode>();
+  const NLLLossAttrs* param = attrs.as<NLLLossAttrs>();
+  if (input == nullptr || target == nullptr || weight == nullptr) return false;
+  ICHECK(input->shape.size() - target->shape.size() == 1)
+      << "NLLLossRel: input should be one dimension larger than target, "
+      << "input shape = " << input->shape << ", "
+      << "target shape = " << target->shape;
+  ICHECK(weight->shape.size() == 1);

Review comment:
       Please add an error message.

##########
File path: python/tvm/relay/op/nn/nn.py
##########
@@ -2973,6 +2973,34 @@ def cross_entropy_with_logits(predictions, targets):
     return _make.cross_entropy_with_logits(predictions, targets)
 
 
+def nll_loss(input, target, weight, reduction="mean", ignore_index=-100):
+    """Negative log likelihood loss.
+
+    Parameters
+    ----------
+    input : tvm.relay.Expr
+      The input.
+
+    target : tvm.relay.Expr
+      The target value of the input.
+
+    weight : tvm.relay.Expr
+      The weight of each target value.
+
+    reduction : string
+      The reduction method to apply to the output.

Review comment:
       Can you list the possible options for reduction.

##########
File path: src/relay/op/nn/nn.cc
##########
@@ -1091,6 +1092,60 @@ Accept logits.
 // Depth to space and space to depth
 TVM_REGISTER_NODE_TYPE(SubPixelAttrs);
 
+// relay.nn.nll_loss
+TVM_REGISTER_NODE_TYPE(NLLLossAttrs);
+
+bool NLLLossRel(const Array<Type>& types, int num_inputs, const Attrs& attrs,
+                const TypeReporter& reporter) {
+  ICHECK_EQ(types.size(), 4);
+  const auto* input = types[0].as<TensorTypeNode>();
+  const auto* target = types[1].as<TensorTypeNode>();
+  const auto* weight = types[2].as<TensorTypeNode>();
+  const NLLLossAttrs* param = attrs.as<NLLLossAttrs>();
+  if (input == nullptr || target == nullptr || weight == nullptr) return false;
+  ICHECK(input->shape.size() - target->shape.size() == 1)
+      << "NLLLossRel: input should be one dimension larger than target, "
+      << "input shape = " << input->shape << ", "
+      << "target shape = " << target->shape;
+  ICHECK(weight->shape.size() == 1);
+  ICHECK(reporter->AssertEQ(input->shape[1], weight->shape[0]))
+      << "NLLLossRel: the second dimension of input should be the number of classes, "
+      << "which is the length of weight, "
+      << "input shape = " << input->shape << ", "
+      << "weight shape = " << weight->shape;
+  ICHECK(input->dtype == weight->dtype && input->dtype.is_float());
+  ICHECK(target->dtype.is_int());

Review comment:
       Please add error messages

##########
File path: python/tvm/topi/nn/loss.py
##########
@@ -0,0 +1,53 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# pylint: disable=invalid-name,unused-argument
+"""TVM operator negative log likelihood loss compute."""

Review comment:
       We may add other losses in the future, so maybe this should read something like "Loss functions definitions".




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org