You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@apex.apache.org by Yogi Devendra <yo...@apache.org> on 2015/12/16 12:19:31 UTC

DAG with a single operator

Hi,

I am writing a Test application which does some validations on the HDFS
file.

Practically, I have single operator in my DAG which extends
AbstractFileInputOperator to do the necessary validation check.

public class OutputValidator extends AbstractFileInputOperator<String>

My output is just about writing log message saying SUCCESS or FAILURE
(which I can grep to find the status). I do not need any port, streams for
this operator.

Essentially, my DAG is just a single operator. Here is the code snippet for
populateDAG

 @Override
  public void populateDAG(DAG dag, Configuration conf)
  {
    dag.addOperator("OutputValidator", new OutputValidator());
  }

While launching this application; I am getting following error:

javax.validation.ValidationException: At least one output port must be
connected: OutputValidator
    at com.datatorrent.stram.plan.logical.LogicalPlan.validate(LogicalPlan.java:1720)
    at com.datatorrent.stram.StramClient.<init>(StramClient.java:140)
    at com.datatorrent.stram.client.StramAppLauncher.launchApp(StramAppLauncher.java:508)
    at com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:1928)
    at com.datatorrent.stram.cli.DTCli.launchAppPackage(DTCli.java:3382)
    at com.datatorrent.stram.cli.DTCli.access$7200(DTCli.java:109)
    at com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:1765)
    at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1329)

Looks like, this is invalid DAG as per the platform definition.

Obviously, I can do some work around by adding some dummy port, stream.

But, my question is:
*Why should we call a DAG with a single operator as invalid*?

~ Yogi

Re: DAG with a single operator

Posted by Yogi Devendra <yo...@apache.org>.

Question resolved.

I had one output port in OutputValidator which I forgot to remove.
Removing it allowed me to launch the application.

~ Yogi

On 16 December 2015 at 16:49, Yogi Devendra <yo...@apache.org> wrote:

> Hi,
>
> I am writing a Test application which does some validations on the HDFS
> file.
>
> Practically, I have single operator in my DAG which extends
> AbstractFileInputOperator to do the necessary validation check.
>
> public class OutputValidator extends AbstractFileInputOperator<String>
>
> My output is just about writing log message saying SUCCESS or FAILURE
> (which I can grep to find the status). I do not need any port, streams
> for this operator.
>
> Essentially, my DAG is just a single operator. Here is the code snippet
> for populateDAG
>
>  @Override
>   public void populateDAG(DAG dag, Configuration conf)
>   {
>     dag.addOperator("OutputValidator", new OutputValidator());
>   }
>
> While launching this application; I am getting following error:
>
> javax.validation.ValidationException: At least one output port must be connected: OutputValidator
>     at com.datatorrent.stram.plan.logical.LogicalPlan.validate(LogicalPlan.java:1720)
>     at com.datatorrent.stram.StramClient.<init>(StramClient.java:140)
>     at com.datatorrent.stram.client.StramAppLauncher.launchApp(StramAppLauncher.java:508)
>     at com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:1928)
>     at com.datatorrent.stram.cli.DTCli.launchAppPackage(DTCli.java:3382)
>     at com.datatorrent.stram.cli.DTCli.access$7200(DTCli.java:109)
>     at com.datatorrent.stram.cli.DTCli$LaunchCommand.execute(DTCli.java:1765)
>     at com.datatorrent.stram.cli.DTCli$3.run(DTCli.java:1329)
>
> Looks like, this is invalid DAG as per the platform definition.
>
> Obviously, I can do some work around by adding some dummy port, stream.
>
> But, my question is:
> *Why should we call a DAG with a single operator as invalid*?
>
> ~ Yogi
> 
>