You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@reef.apache.org by Shouheng Yi <sh...@microsoft.com.INVALID> on 2017/04/12 17:04:55 UTC
Any Examples of Injecting Java Objects from C# Using Tang?
Hi REEF devs,
I've been going through our .Net code base, and I haven't find an example on how to inject Java objects from C# side using Tang. I'm wondering if anybody has a working example and willing to share to the dev list. Thanks!
Best,
Shouheng
RE: Any Examples of Injecting Java Objects from C# Using Tang?
Posted by Shouheng Yi <sh...@microsoft.com.INVALID>.
It is for [REEF-1771]. I wanted to find a solution that takes a dependency on Tang's Java injection.
However, to roll our parquet reader out, I just opened another process to call Java objects. It's kind of ugly, but it works. If people have established pattern feel free to jump in๐
Best,
Shouheng
-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de]
Sent: Wednesday, April 12, 2017 4:05 PM
To: REEF Developers Mailinglist <de...@reef.apache.org>
Subject: Re: Any Examples of Injecting Java Objects from C# Using Tang?
On Wed, Apr 12, 2017 at 10:04 AM, Shouheng Yi <sh...@microsoft.com.invalid> wrote:
> I've been going through our .Net code base, and I haven't find an example on how to inject Java objects from C# side using Tang. I'm wondering if anybody has a working example and willing to share to the dev list.
I don't have one, sorry! Is this for [REEF-1771]? If so, it might be easier to funnel the needed parameters through command line parameters instead, as the number seems to be limited (e.g. the path on HDFS and the path locally).
Markus
Re: Any Examples of Injecting Java Objects from C# Using Tang?
Posted by Gyewon Lee <st...@gmail.com>.
Hi, Shouheng.
I'm not familiar with parquet format, but I don't think there's a good way
for already created data to be deserialized into Avro using Tang. It is
because Tang serializes object configuration (before the object is
created), not the created object itself.
Best regards,
Gyewon
2017-04-13 10:33 GMT+09:00 Shouheng Yi <sh...@microsoft.com.invalid>:
> The original data is in parquet format. Then it is ingested in JVM and
> deserialized as Avro files (step 1). C# then ingest these Avro files and
> represent them as in-memory objects (step 2). My questions is if we can
> realize step 1 using Tang injection.
>
> -----Original Message-----
> From: Julia Wang (QIUHE) [mailto:Qiuhe.Wang@microsoft.com.INVALID]
> Sent: Wednesday, April 12, 2017 4:40 PM
> To: dev@reef.apache.org
> Subject: RE: Any Examples of Injecting Java Objects from C# Using Tang?
>
> If the data is in JSon or Avro format, then you can just write C# code to
> read the data.
>
> -----Original Message-----
> From: Shouheng Yi [mailto:shouyi@microsoft.com.INVALID]
> Sent: Wednesday, April 12, 2017 4:35 PM
> To: dev@reef.apache.org
> Subject: RE: Any Examples of Injecting Java Objects from C# Using Tang?
>
> Hi Julia,
>
> Essentially I have a Java object that deserializes data to disk. I wanted
> to call this java object from C# and read in the deserialized data on disk.
>
> Best,
> Shouheng
>
> -----Original Message-----
> From: Julia Wang (QIUHE) [mailto:Qiuhe.Wang@microsoft.com.INVALID]
> Sent: Wednesday, April 12, 2017 4:32 PM
> To: dev@reef.apache.org
> Subject: RE: Any Examples of Injecting Java Objects from C# Using Tang?
>
> Inject Java object in C#? Do you mean use Java configuration to inject an
> object in C#?
>
> Julia
>
> -----Original Message-----
> From: Markus Weimer [mailto:markus@weimo.de]
> Sent: Wednesday, April 12, 2017 4:05 PM
> To: REEF Developers Mailinglist <de...@reef.apache.org>
> Subject: Re: Any Examples of Injecting Java Objects from C# Using Tang?
>
> On Wed, Apr 12, 2017 at 10:04 AM, Shouheng Yi <sh...@microsoft.com.invalid>
> wrote:
> > I've been going through our .Net code base, and I haven't find an
> example on how to inject Java objects from C# side using Tang. I'm
> wondering if anybody has a working example and willing to share to the dev
> list.
>
> I don't have one, sorry! Is this for [REEF-1771]? If so, it might be
> easier to funnel the needed parameters through command line parameters
> instead, as the number seems to be limited (e.g. the path on HDFS and the
> path locally).
>
> Markus
>
RE: Any Examples of Injecting Java Objects from C# Using Tang?
Posted by Shouheng Yi <sh...@microsoft.com.INVALID>.
The original data is in parquet format. Then it is ingested in JVM and deserialized as Avro files (step 1). C# then ingest these Avro files and represent them as in-memory objects (step 2). My questions is if we can realize step 1 using Tang injection.
-----Original Message-----
From: Julia Wang (QIUHE) [mailto:Qiuhe.Wang@microsoft.com.INVALID]
Sent: Wednesday, April 12, 2017 4:40 PM
To: dev@reef.apache.org
Subject: RE: Any Examples of Injecting Java Objects from C# Using Tang?
If the data is in JSon or Avro format, then you can just write C# code to read the data.
-----Original Message-----
From: Shouheng Yi [mailto:shouyi@microsoft.com.INVALID]
Sent: Wednesday, April 12, 2017 4:35 PM
To: dev@reef.apache.org
Subject: RE: Any Examples of Injecting Java Objects from C# Using Tang?
Hi Julia,
Essentially I have a Java object that deserializes data to disk. I wanted to call this java object from C# and read in the deserialized data on disk.
Best,
Shouheng
-----Original Message-----
From: Julia Wang (QIUHE) [mailto:Qiuhe.Wang@microsoft.com.INVALID]
Sent: Wednesday, April 12, 2017 4:32 PM
To: dev@reef.apache.org
Subject: RE: Any Examples of Injecting Java Objects from C# Using Tang?
Inject Java object in C#? Do you mean use Java configuration to inject an object in C#?
Julia
-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de]
Sent: Wednesday, April 12, 2017 4:05 PM
To: REEF Developers Mailinglist <de...@reef.apache.org>
Subject: Re: Any Examples of Injecting Java Objects from C# Using Tang?
On Wed, Apr 12, 2017 at 10:04 AM, Shouheng Yi <sh...@microsoft.com.invalid> wrote:
> I've been going through our .Net code base, and I haven't find an example on how to inject Java objects from C# side using Tang. I'm wondering if anybody has a working example and willing to share to the dev list.
I don't have one, sorry! Is this for [REEF-1771]? If so, it might be easier to funnel the needed parameters through command line parameters instead, as the number seems to be limited (e.g. the path on HDFS and the path locally).
Markus
RE: Any Examples of Injecting Java Objects from C# Using Tang?
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com.INVALID>.
If the data is in JSon or Avro format, then you can just write C# code to read the data.
-----Original Message-----
From: Shouheng Yi [mailto:shouyi@microsoft.com.INVALID]
Sent: Wednesday, April 12, 2017 4:35 PM
To: dev@reef.apache.org
Subject: RE: Any Examples of Injecting Java Objects from C# Using Tang?
Hi Julia,
Essentially I have a Java object that deserializes data to disk. I wanted to call this java object from C# and read in the deserialized data on disk.
Best,
Shouheng
-----Original Message-----
From: Julia Wang (QIUHE) [mailto:Qiuhe.Wang@microsoft.com.INVALID]
Sent: Wednesday, April 12, 2017 4:32 PM
To: dev@reef.apache.org
Subject: RE: Any Examples of Injecting Java Objects from C# Using Tang?
Inject Java object in C#? Do you mean use Java configuration to inject an object in C#?
Julia
-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de]
Sent: Wednesday, April 12, 2017 4:05 PM
To: REEF Developers Mailinglist <de...@reef.apache.org>
Subject: Re: Any Examples of Injecting Java Objects from C# Using Tang?
On Wed, Apr 12, 2017 at 10:04 AM, Shouheng Yi <sh...@microsoft.com.invalid> wrote:
> I've been going through our .Net code base, and I haven't find an example on how to inject Java objects from C# side using Tang. I'm wondering if anybody has a working example and willing to share to the dev list.
I don't have one, sorry! Is this for [REEF-1771]? If so, it might be easier to funnel the needed parameters through command line parameters instead, as the number seems to be limited (e.g. the path on HDFS and the path locally).
Markus
RE: Any Examples of Injecting Java Objects from C# Using Tang?
Posted by Shouheng Yi <sh...@microsoft.com.INVALID>.
Hi Julia,
Essentially I have a Java object that deserializes data to disk. I wanted to call this java object from C# and read in the deserialized data on disk.
Best,
Shouheng
-----Original Message-----
From: Julia Wang (QIUHE) [mailto:Qiuhe.Wang@microsoft.com.INVALID]
Sent: Wednesday, April 12, 2017 4:32 PM
To: dev@reef.apache.org
Subject: RE: Any Examples of Injecting Java Objects from C# Using Tang?
Inject Java object in C#? Do you mean use Java configuration to inject an object in C#?
Julia
-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de]
Sent: Wednesday, April 12, 2017 4:05 PM
To: REEF Developers Mailinglist <de...@reef.apache.org>
Subject: Re: Any Examples of Injecting Java Objects from C# Using Tang?
On Wed, Apr 12, 2017 at 10:04 AM, Shouheng Yi <sh...@microsoft.com.invalid> wrote:
> I've been going through our .Net code base, and I haven't find an example on how to inject Java objects from C# side using Tang. I'm wondering if anybody has a working example and willing to share to the dev list.
I don't have one, sorry! Is this for [REEF-1771]? If so, it might be easier to funnel the needed parameters through command line parameters instead, as the number seems to be limited (e.g. the path on HDFS and the path locally).
Markus
RE: Any Examples of Injecting Java Objects from C# Using Tang?
Posted by "Julia Wang (QIUHE)" <Qi...@microsoft.com.INVALID>.
Inject Java object in C#? Do you mean use Java configuration to inject an object in C#?
Julia
-----Original Message-----
From: Markus Weimer [mailto:markus@weimo.de]
Sent: Wednesday, April 12, 2017 4:05 PM
To: REEF Developers Mailinglist <de...@reef.apache.org>
Subject: Re: Any Examples of Injecting Java Objects from C# Using Tang?
On Wed, Apr 12, 2017 at 10:04 AM, Shouheng Yi <sh...@microsoft.com.invalid> wrote:
> I've been going through our .Net code base, and I haven't find an example on how to inject Java objects from C# side using Tang. I'm wondering if anybody has a working example and willing to share to the dev list.
I don't have one, sorry! Is this for [REEF-1771]? If so, it might be easier to funnel the needed parameters through command line parameters instead, as the number seems to be limited (e.g. the path on HDFS and the path locally).
Markus
Re: Any Examples of Injecting Java Objects from C# Using Tang?
Posted by Markus Weimer <ma...@weimo.de>.
On Wed, Apr 12, 2017 at 10:04 AM, Shouheng Yi
<sh...@microsoft.com.invalid> wrote:
> I've been going through our .Net code base, and I haven't find an example on how to inject Java objects from C# side using Tang. I'm wondering if anybody has a working example and willing to share to the dev list.
I don't have one, sorry! Is this for [REEF-1771]? If so, it might be
easier to funnel the needed parameters through command line parameters
instead, as the number seems to be limited (e.g. the path on HDFS and
the path locally).
Markus