[jira] [Comment Edited] (BIGTOP-1928) Implement BPS-Flink

["jay vyas (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BIGTOP-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14698668#comment-14698668 ] 

jay vyas edited comment on BIGTOP-1928 at 8/16/15 1:48 PM:
-----------------------------------------------------------

It looks to me like, when I run an operation flink does a System.exit(0).  Not sure why.  Just pasting the details here in case folks from the flink community have ideas.

{noformat}
SingleOutputStreamOperator<List<Transaction>,?> so = data.map(
        new MapFunction<Customer, List<Transaction>>() {
            public List<Transaction> map(Customer value) throws Exception {

                Collection<ProductCategory> products = id.getProductCategories();

                //TODO reus seedfactory variable above.
                PurchasingProfileGenerator profileGen = new PurchasingProfileGenerator(products, new SeedFactory(1));
                PurchasingProfile profile = profileGen.generate();
                TransactionGenerator transGen = new TransactionGenerator(value, profile, stores, products, new SeedFactory(1));
                List<Transaction> transactions = Lists.newArrayList();
                Transaction transaction = transGen.generate();
                transactions.add(transaction);
                //Create a list of this customer's transactions for the time period
                while (transaction.getDateTime() < simulationLength) {
                    //TODO implement burn in time like we do in bps-spark
                    transactions.add(transaction);
                    System.out.println("... "+transaction);
                    transaction = transGen.generate();
                }
                return transactions;
            }
        });

System.out.println("count "+ so.count());
so.writeAsText("/tmp/a");
{noformat}

The above program has the following output:  It simply exits, seemingly, when I try to run operations on the SingleOutputStreamOperator.

{noformat}
Reading zipcode data
Read 30891 zipcode entries
Reading name data
Read 86987 first names and 47819 last names
Reading product data
Read 4 product categories
from coll09:39:28,075 INFO  org.apache.flink.api.java.typeutils.TypeExtractor             - Class class com.github.rnowling.bps.datagenerator.datamodels.Customer is not a valid POJO type
now mapping....org.apache.flink.streaming.api.datastream.DataStreamSource@4b8466c2count org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator@5f1830bf

Process finished with exit code 0
{noformat}




was (Author: jayunit100):
It looks to me like, when I run a few operations flink does a System.exit(0).  Not sure why.  Just pasting the details here in case folks from the flink community have ideas.

{noformat}
SingleOutputStreamOperator<List<Transaction>,?> so = data.map(
        new MapFunction<Customer, List<Transaction>>() {
            public List<Transaction> map(Customer value) throws Exception {

                Collection<ProductCategory> products = id.getProductCategories();

                //TODO reus seedfactory variable above.
                PurchasingProfileGenerator profileGen = new PurchasingProfileGenerator(products, new SeedFactory(1));
                PurchasingProfile profile = profileGen.generate();
                TransactionGenerator transGen = new TransactionGenerator(value, profile, stores, products, new SeedFactory(1));
                List<Transaction> transactions = Lists.newArrayList();
                Transaction transaction = transGen.generate();
                transactions.add(transaction);
                //Create a list of this customer's transactions for the time period
                while (transaction.getDateTime() < simulationLength) {
                    //TODO implement burn in time like we do in bps-spark
                    transactions.add(transaction);
                    System.out.println("... "+transaction);
                    transaction = transGen.generate();
                }
                return transactions;
            }
        });

System.out.println("count "+ so.count());
so.writeAsText("/tmp/a");
{noformat}

The above program has the following output:  It simply exits, seemingly, when I try to run operations on the SingleOutputStreamOperator.

{noformat}
Reading zipcode data
Read 30891 zipcode entries
Reading name data
Read 86987 first names and 47819 last names
Reading product data
Read 4 product categories
from coll09:39:28,075 INFO  org.apache.flink.api.java.typeutils.TypeExtractor             - Class class com.github.rnowling.bps.datagenerator.datamodels.Customer is not a valid POJO type
now mapping....org.apache.flink.streaming.api.datastream.DataStreamSource@4b8466c2count org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator@5f1830bf

Process finished with exit code 0
{noformat}



> Implement BPS-Flink
> -------------------
>
>                 Key: BIGTOP-1928
>                 URL: https://issues.apache.org/jira/browse/BIGTOP-1928
>             Project: Bigtop
>          Issue Type: New Feature
>          Components: blueprints
>    Affects Versions: backlog
>            Reporter: jay vyas
>            Assignee: jay vyas
>             Fix For: backlog
>
>
> Have been discussing BPS flink with folks lately, some interest.  Heres a placeholder  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)