You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@bigtop.apache.org by "jay vyas (JIRA)" <ji...@apache.org> on 2015/08/16 15:48:45 UTC
[jira] [Commented] (BIGTOP-1928) Implement BPS-Flink
[ https://issues.apache.org/jira/browse/BIGTOP-1928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14698668#comment-14698668 ]
jay vyas commented on BIGTOP-1928:
----------------------------------
It looks to me like, when I run a few operations flink does a System.exit(0). Not sure why. Just pasting the details here in case folks from the flink community have ideas.
{noformat}
SingleOutputStreamOperator<List<Transaction>,?> so = data.map(
new MapFunction<Customer, List<Transaction>>() {
public List<Transaction> map(Customer value) throws Exception {
Collection<ProductCategory> products = id.getProductCategories();
//TODO reus seedfactory variable above.
PurchasingProfileGenerator profileGen = new PurchasingProfileGenerator(products, new SeedFactory(1));
PurchasingProfile profile = profileGen.generate();
TransactionGenerator transGen = new TransactionGenerator(value, profile, stores, products, new SeedFactory(1));
List<Transaction> transactions = Lists.newArrayList();
Transaction transaction = transGen.generate();
transactions.add(transaction);
//Create a list of this customer's transactions for the time period
while (transaction.getDateTime() < simulationLength) {
//TODO implement burn in time like we do in bps-spark
transactions.add(transaction);
System.out.println("... "+transaction);
transaction = transGen.generate();
}
return transactions;
}
});
System.out.println("count "+ so.count());
so.writeAsText("/tmp/a");
{noformat}
The above program has the following output: It simply exits, seemingly, when I try to run operations on the SingleOutputStreamOperator.
{noformat}
Reading zipcode data
Read 30891 zipcode entries
Reading name data
Read 86987 first names and 47819 last names
Reading product data
Read 4 product categories
from coll09:39:28,075 INFO org.apache.flink.api.java.typeutils.TypeExtractor - Class class com.github.rnowling.bps.datagenerator.datamodels.Customer is not a valid POJO type
now mapping....org.apache.flink.streaming.api.datastream.DataStreamSource@4b8466c2count org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator@5f1830bf
Process finished with exit code 0
{noformat}
> Implement BPS-Flink
> -------------------
>
> Key: BIGTOP-1928
> URL: https://issues.apache.org/jira/browse/BIGTOP-1928
> Project: Bigtop
> Issue Type: New Feature
> Components: blueprints
> Affects Versions: backlog
> Reporter: jay vyas
> Assignee: jay vyas
> Fix For: backlog
>
>
> Have been discussing BPS flink with folks lately, some interest. Heres a placeholder
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)