You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by Ernie Ostic <eo...@us.ibm.com> on 2016/05/09 14:49:21 UTC
Re: Questions about REST API and payloads...
Hi...
New to the list, so may have missed a response to the original note below,
but in the meantime, also had this question relative to the REST API. It
doesn't appear that there is currently a published API for declaring
lineage? (for connecting assets together). Is it just not yet
documented? Are there plans for exposing lineage definition via REST?
Thank you.
Ernie
Ernie Ostic
WW Product Specialist, Information Server
IBM Analytics
Cell: (617) 331 8238
---------------------------------------------------------------
Open IGC is here!
Extend the Catalog with custom objects and lineage definitions!
https://dsrealtime.wordpress.com/2015/07/29/open-igc-is-here/
From: Ernie Ostic/Newark/IBM
To: dev@atlas.incubator.apache.org
Date: 05/06/2016 01:39 PM
Subject: Question about REST payloads...
Hello...
Thanks to everyone for all the useful information on the Apache Atlas site
to-date. Recently I started experimenting with Atlas, using the REST API,
and establishing some of my own types and then reviewing their metadata.
I was able to get things working for the currently published REST calls,
but would like to know more about the formatting of the payloads. The
returned JSON payload is valid but embeds further JSON that "appears" to
have additional formatting --- is this something that is handled
automatically by some of the (presumably java) existing client
tooling? ...or maybe I should have passed another argument or made a
different call that would pass back a straight text response?
Here is a snippet of the sample response I received....
{"requestId":"qtp430472391-17 -
12f1518c-fdac-4a50-a23a-f56af47ee069","GUID":"1731e6e8-ee11-4fc0-b86a-e83f08d67981","definition":"{\n
\"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization
$_Reference\",\n \"id\":{\n \"jsonClass
\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Id\",\n
\"id\":\"1731e6e8-ee11-4fc0-b86a-e83f08d67981\"....
Is there a historical reason for the further formatting of "definition"?
Is there a parameter I should be passing that I accidentally missed (that
would avoid this formatting)?
Thanks. Many of our customers are considering using REST calls from a
variety of tools and languages. This structure will require additional
parsing and formatting after breaking out the initial elements.
Ernie
Ernie Ostic
WW Product Specialist, Information Server
IBM Analytics
Cell: (617) 331 8238
---------------------------------------------------------------
Open IGC is here!
Extend the Catalog with custom objects and lineage definitions!
https://dsrealtime.wordpress.com/2015/07/29/open-igc-is-here/
Re: Questions about REST API and payloads...
Posted by Fnu Neerju <gu...@us.ibm.com>.
This looks similar to ATLAS-586. That issue has been resolved as bug. If
you have the updated code with patch but still seeing the issue, it is
possible that fix needs to be applied elsewhere. Please create a jira
ticket for that.
Neeru
From: Ernie Ostic/Newark/IBM@IBMUS
To: dev@atlas.incubator.apache.org
Date: 05/09/2016 12:29 PM
Subject: Re: Questions about REST API and payloads...
Thank you for the direction on additional REST calls!
Anyone have thoughts about the payload structure (additional formatting in
the JSON response...such as '/n', etc.). Again, my apologies if I missed an
earlier response on this. ---ernie.
====== original email
=================================================================================================
Hello...
Thanks to everyone for all the useful information on the Apache Atlas site
to-date. Recently I started experimenting with Atlas, using the REST API,
and establishing some of my own types and then reviewing their metadata. I
was able to get things working for the currently published REST calls, but
would like to know more about the formatting of the payloads. The returned
JSON payload is valid but embeds further JSON that "appears" to have
additional formatting --- is this something that is handled automatically
by some of the (presumably java) existing client tooling? ...or maybe I
should have passed another argument or made a different call that would
pass back a straight text response?
Here is a snippet of the sample response I received....
{"requestId":"qtp430472391-17 -
12f1518c-fdac-4a50-a23a-f56af47ee069","GUID":"1731e6e8-ee11-4fc0-b86a-e83f08d67981","definition":"{\n
\"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization
$_Reference\",\n \"id\":{\n \"jsonClass
\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Id\",\n \"id
\":\"1731e6e8-ee11-4fc0-b86a-e83f08d67981\"....
Is there a historical reason for the further formatting of "definition"? Is
there a parameter I should be passing that I accidentally missed (that
would avoid this formatting)?
Thanks. Many of our customers are considering using REST calls from a
variety of tools and languages. This structure will require additional
parsing and formatting after breaking out the initial elements.
Ernie
====== original email
=================================================================================================
Ernie
Ernie Ostic
WW Product Specialist, Information Server
IBM Analytics
Cell: (617) 331 8238
---------------------------------------------------------------
Open IGC is here!
Extend the Catalog with custom objects and lineage definitions!
https://dsrealtime.wordpress.com/2015/07/29/open-igc-is-here/
Hemanth Yamijala ---05/09/2016 11:00:08 AM---The APIs in question are
atlas/api/lineage/hive/table/{tableName}/inputs/graph and atlas/api/lineage
From: Hemanth Yamijala <hy...@hortonworks.com>
To: "dev@atlas.incubator.apache.org" <de...@atlas.incubator.apache.org>
Date: 05/09/2016 11:00 AM
Subject: Re: Questions about REST API and payloads...
The APIs in question are
atlas/api/lineage/hive/table/{tableName}/inputs/graph and
atlas/api/lineage/hive/table/{tableName}/outputs/graph. The implementation
for the same is HiveLineageResource.java. (The ‘Hive’ in the names is an
unfortunate legacy – it applies to any type in Atlas that extends from a
DataSet). The UI code that shows the lineage graphs (in the older dashboard
module) should probably help see how the outputs of the two APIs can be
used for getting a lineage graph in place.
Thanks
Hemanth
From: Ernie Ostic <eo...@us.ibm.com>>
Reply-To: "dev@atlas.incubator.apache.org<
mailto:dev@atlas.incubator.apache.org>" <dev@atlas.incubator.apache.org<
mailto:dev@atlas.incubator.apache.org>>
Date: Monday, May 9, 2016 at 8:19 PM
To: "dev@atlas.incubator.apache.org<ma...@atlas.incubator.apache.org>"
<de...@atlas.incubator.apache.org>>
Subject: Re: Questions about REST API and payloads...
Hi...
New to the list, so may have missed a response to the original note below,
but in the meantime, also had this question relative to the REST API. It
doesn't appear that there is currently a published API for declaring
lineage? (for connecting assets together). Is it just not yet documented?
Are there plans for exposing lineage definition via REST?
Thank you.
Ernie
Ernie Ostic
WW Product Specialist, Information Server
IBM Analytics
Cell: (617) 331 8238
---------------------------------------------------------------
Open IGC is here!
Extend the Catalog with custom objects and lineage definitions!
https://dsrealtime.wordpress.com/2015/07/29/open-igc-is-here/
[Inactive hide details for Ernie Ostic---05/06/2016 01:39:04 PM---Hello...
Thanks to everyone for all the useful information on]Ernie
Ostic---05/06/2016 01:39:04 PM---Hello... Thanks to everyone for all the
useful information on the Apache Atlas site to-date. Recen
From: Ernie Ostic/Newark/IBM
To: dev@atlas.incubator.apache.org<ma...@atlas.incubator.apache.org>
Date: 05/06/2016 01:39 PM
Subject: Question about REST payloads...
________________________________
Hello...
Thanks to everyone for all the useful information on the Apache Atlas site
to-date. Recently I started experimenting with Atlas, using the REST API,
and establishing some of my own types and then reviewing their metadata. I
was able to get things working for the currently published REST calls, but
would like to know more about the formatting of the payloads. The returned
JSON payload is valid but embeds further JSON that "appears" to have
additional formatting --- is this something that is handled automatically
by some of the (presumably java) existing client tooling? ...or maybe I
should have passed another argument or made a different call that would
pass back a straight text response?
Here is a snippet of the sample response I received....
{"requestId":"qtp430472391-17 -
12f1518c-fdac-4a50-a23a-f56af47ee069","GUID":"1731e6e8-ee11-4fc0-b86a-e83f08d67981","definition":"{\n
\"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization
$_Reference\",\n \"id\":{\n \"jsonClass
\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Id\",\n \"id
\":\"1731e6e8-ee11-4fc0-b86a-e83f08d67981\"....
Is there a historical reason for the further formatting of "definition"? Is
there a parameter I should be passing that I accidentally missed (that
would avoid this formatting)?
Thanks. Many of our customers are considering using REST calls from a
variety of tools and languages. This structure will require additional
parsing and formatting after breaking out the initial elements.
Ernie
Ernie Ostic
WW Product Specialist, Information Server
IBM Analytics
Cell: (617) 331 8238
---------------------------------------------------------------
Open IGC is here!
Extend the Catalog with custom objects and lineage definitions!
https://dsrealtime.wordpress.com/2015/07/29/open-igc-is-here/
Re: Questions about REST API and payloads...
Posted by Ernie Ostic <eo...@us.ibm.com>.
Thank you for the direction on additional REST calls!
Anyone have thoughts about the payload structure (additional formatting in
the JSON response...such as '/n', etc.). Again, my apologies if I missed
an earlier response on this. ---ernie.
====== original email
=================================================================================================
Hello...
Thanks to everyone for all the useful information on the Apache Atlas site
to-date. Recently I started experimenting with Atlas, using the REST API,
and establishing some of my own types and then reviewing their metadata. I
was able to get things working for the currently published REST calls, but
would like to know more about the formatting of the payloads. The returned
JSON payload is valid but embeds further JSON that "appears" to have
additional formatting --- is this something that is handled automatically
by some of the (presumably java) existing client tooling? ...or maybe I
should have passed another argument or made a different call that would
pass back a straight text response?
Here is a snippet of the sample response I received....
{"requestId":"qtp430472391-17 -
12f1518c-fdac-4a50-a23a-f56af47ee069","GUID":"1731e6e8-ee11-4fc0-b86a-e83f08d67981","definition":"{\n
\"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization
$_Reference\",\n \"id\":{\n \"jsonClass
\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Id\",\n \"id
\":\"1731e6e8-ee11-4fc0-b86a-e83f08d67981\"....
Is there a historical reason for the further formatting of "definition"? Is
there a parameter I should be passing that I accidentally missed (that
would avoid this formatting)?
Thanks. Many of our customers are considering using REST calls from a
variety of tools and languages. This structure will require additional
parsing and formatting after breaking out the initial elements.
Ernie
====== original email
=================================================================================================
Ernie
Ernie Ostic
WW Product Specialist, Information Server
IBM Analytics
Cell: (617) 331 8238
---------------------------------------------------------------
Open IGC is here!
Extend the Catalog with custom objects and lineage definitions!
https://dsrealtime.wordpress.com/2015/07/29/open-igc-is-here/
From: Hemanth Yamijala <hy...@hortonworks.com>
To: "dev@atlas.incubator.apache.org"
<de...@atlas.incubator.apache.org>
Date: 05/09/2016 11:00 AM
Subject: Re: Questions about REST API and payloads...
The APIs in question are
atlas/api/lineage/hive/table/{tableName}/inputs/graph and
atlas/api/lineage/hive/table/{tableName}/outputs/graph. The implementation
for the same is HiveLineageResource.java. (The ‘Hive’ in the names is an
unfortunate legacy – it applies to any type in Atlas that extends from a
DataSet). The UI code that shows the lineage graphs (in the older dashboard
module) should probably help see how the outputs of the two APIs can be
used for getting a lineage graph in place.
Thanks
Hemanth
From: Ernie Ostic <eo...@us.ibm.com>>
Reply-To: "dev@atlas.incubator.apache.org<
mailto:dev@atlas.incubator.apache.org>" <dev@atlas.incubator.apache.org<
mailto:dev@atlas.incubator.apache.org>>
Date: Monday, May 9, 2016 at 8:19 PM
To: "dev@atlas.incubator.apache.org<ma...@atlas.incubator.apache.org>"
<de...@atlas.incubator.apache.org>>
Subject: Re: Questions about REST API and payloads...
Hi...
New to the list, so may have missed a response to the original note below,
but in the meantime, also had this question relative to the REST API. It
doesn't appear that there is currently a published API for declaring
lineage? (for connecting assets together). Is it just not yet documented?
Are there plans for exposing lineage definition via REST?
Thank you.
Ernie
Ernie Ostic
WW Product Specialist, Information Server
IBM Analytics
Cell: (617) 331 8238
---------------------------------------------------------------
Open IGC is here!
Extend the Catalog with custom objects and lineage definitions!
https://dsrealtime.wordpress.com/2015/07/29/open-igc-is-here/
[Inactive hide details for Ernie Ostic---05/06/2016 01:39:04 PM---Hello...
Thanks to everyone for all the useful information on]Ernie
Ostic---05/06/2016 01:39:04 PM---Hello... Thanks to everyone for all the
useful information on the Apache Atlas site to-date. Recen
From: Ernie Ostic/Newark/IBM
To: dev@atlas.incubator.apache.org<ma...@atlas.incubator.apache.org>
Date: 05/06/2016 01:39 PM
Subject: Question about REST payloads...
________________________________
Hello...
Thanks to everyone for all the useful information on the Apache Atlas site
to-date. Recently I started experimenting with Atlas, using the REST API,
and establishing some of my own types and then reviewing their metadata. I
was able to get things working for the currently published REST calls, but
would like to know more about the formatting of the payloads. The returned
JSON payload is valid but embeds further JSON that "appears" to have
additional formatting --- is this something that is handled automatically
by some of the (presumably java) existing client tooling? ...or maybe I
should have passed another argument or made a different call that would
pass back a straight text response?
Here is a snippet of the sample response I received....
{"requestId":"qtp430472391-17 -
12f1518c-fdac-4a50-a23a-f56af47ee069","GUID":"1731e6e8-ee11-4fc0-b86a-e83f08d67981","definition":"{\n
\"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization
$_Reference\",\n \"id\":{\n \"jsonClass
\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Id\",\n \"id
\":\"1731e6e8-ee11-4fc0-b86a-e83f08d67981\"....
Is there a historical reason for the further formatting of "definition"? Is
there a parameter I should be passing that I accidentally missed (that
would avoid this formatting)?
Thanks. Many of our customers are considering using REST calls from a
variety of tools and languages. This structure will require additional
parsing and formatting after breaking out the initial elements.
Ernie
Ernie Ostic
WW Product Specialist, Information Server
IBM Analytics
Cell: (617) 331 8238
---------------------------------------------------------------
Open IGC is here!
Extend the Catalog with custom objects and lineage definitions!
https://dsrealtime.wordpress.com/2015/07/29/open-igc-is-here/
Re: Questions about REST API and payloads...
Posted by Venkata R Madugundu <ve...@in.ibm.com>.
Hi Ernie,
Sometime last year, I had put up a small doc trying to understand the
Lineage support in Atlas.
See if this helps.
Thanks
Venkat
(See attached file: Atlas_Lineage_Support.pdf)
From: Hemanth Yamijala <hy...@hortonworks.com>
To: "dev@atlas.incubator.apache.org"
<de...@atlas.incubator.apache.org>
Date: 09/05/16 08:30 PM
Subject: Re: Questions about REST API and payloads...
The APIs in question are
atlas/api/lineage/hive/table/{tableName}/inputs/graph and
atlas/api/lineage/hive/table/{tableName}/outputs/graph. The implementation
for the same is HiveLineageResource.java. (The ‘Hive’ in the names is an
unfortunate legacy – it applies to any type in Atlas that extends from a
DataSet). The UI code that shows the lineage graphs (in the older dashboard
module) should probably help see how the outputs of the two APIs can be
used for getting a lineage graph in place.
Thanks
Hemanth
From: Ernie Ostic <eo...@us.ibm.com>>
Reply-To: "dev@atlas.incubator.apache.org<
mailto:dev@atlas.incubator.apache.org>" <dev@atlas.incubator.apache.org<
mailto:dev@atlas.incubator.apache.org>>
Date: Monday, May 9, 2016 at 8:19 PM
To: "dev@atlas.incubator.apache.org<ma...@atlas.incubator.apache.org>"
<de...@atlas.incubator.apache.org>>
Subject: Re: Questions about REST API and payloads...
Hi...
New to the list, so may have missed a response to the original note below,
but in the meantime, also had this question relative to the REST API. It
doesn't appear that there is currently a published API for declaring
lineage? (for connecting assets together). Is it just not yet documented?
Are there plans for exposing lineage definition via REST?
Thank you.
Ernie
Ernie Ostic
WW Product Specialist, Information Server
IBM Analytics
Cell: (617) 331 8238
---------------------------------------------------------------
Open IGC is here!
Extend the Catalog with custom objects and lineage definitions!
https://dsrealtime.wordpress.com/2015/07/29/open-igc-is-here/
[Inactive hide details for Ernie Ostic---05/06/2016 01:39:04 PM---Hello...
Thanks to everyone for all the useful information on]Ernie
Ostic---05/06/2016 01:39:04 PM---Hello... Thanks to everyone for all the
useful information on the Apache Atlas site to-date. Recen
From: Ernie Ostic/Newark/IBM
To: dev@atlas.incubator.apache.org<ma...@atlas.incubator.apache.org>
Date: 05/06/2016 01:39 PM
Subject: Question about REST payloads...
________________________________
Hello...
Thanks to everyone for all the useful information on the Apache Atlas site
to-date. Recently I started experimenting with Atlas, using the REST API,
and establishing some of my own types and then reviewing their metadata. I
was able to get things working for the currently published REST calls, but
would like to know more about the formatting of the payloads. The returned
JSON payload is valid but embeds further JSON that "appears" to have
additional formatting --- is this something that is handled automatically
by some of the (presumably java) existing client tooling? ...or maybe I
should have passed another argument or made a different call that would
pass back a straight text response?
Here is a snippet of the sample response I received....
{"requestId":"qtp430472391-17 -
12f1518c-fdac-4a50-a23a-f56af47ee069","GUID":"1731e6e8-ee11-4fc0-b86a-e83f08d67981","definition":"{\n
\"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization
$_Reference\",\n \"id\":{\n \"jsonClass
\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Id\",\n \"id
\":\"1731e6e8-ee11-4fc0-b86a-e83f08d67981\"....
Is there a historical reason for the further formatting of "definition"? Is
there a parameter I should be passing that I accidentally missed (that
would avoid this formatting)?
Thanks. Many of our customers are considering using REST calls from a
variety of tools and languages. This structure will require additional
parsing and formatting after breaking out the initial elements.
Ernie
Ernie Ostic
WW Product Specialist, Information Server
IBM Analytics
Cell: (617) 331 8238
---------------------------------------------------------------
Open IGC is here!
Extend the Catalog with custom objects and lineage definitions!
https://dsrealtime.wordpress.com/2015/07/29/open-igc-is-here/
Re: Questions about REST API and payloads...
Posted by Hemanth Yamijala <hy...@hortonworks.com>.
The APIs in question are atlas/api/lineage/hive/table/{tableName}/inputs/graph and atlas/api/lineage/hive/table/{tableName}/outputs/graph. The implementation for the same is HiveLineageResource.java. (The ‘Hive’ in the names is an unfortunate legacy – it applies to any type in Atlas that extends from a DataSet). The UI code that shows the lineage graphs (in the older dashboard module) should probably help see how the outputs of the two APIs can be used for getting a lineage graph in place.
Thanks
Hemanth
From: Ernie Ostic <eo...@us.ibm.com>>
Reply-To: "dev@atlas.incubator.apache.org<ma...@atlas.incubator.apache.org>" <de...@atlas.incubator.apache.org>>
Date: Monday, May 9, 2016 at 8:19 PM
To: "dev@atlas.incubator.apache.org<ma...@atlas.incubator.apache.org>" <de...@atlas.incubator.apache.org>>
Subject: Re: Questions about REST API and payloads...
Hi...
New to the list, so may have missed a response to the original note below, but in the meantime, also had this question relative to the REST API. It doesn't appear that there is currently a published API for declaring lineage? (for connecting assets together). Is it just not yet documented? Are there plans for exposing lineage definition via REST?
Thank you.
Ernie
Ernie Ostic
WW Product Specialist, Information Server
IBM Analytics
Cell: (617) 331 8238
---------------------------------------------------------------
Open IGC is here!
Extend the Catalog with custom objects and lineage definitions!
https://dsrealtime.wordpress.com/2015/07/29/open-igc-is-here/
[Inactive hide details for Ernie Ostic---05/06/2016 01:39:04 PM---Hello... Thanks to everyone for all the useful information on]Ernie Ostic---05/06/2016 01:39:04 PM---Hello... Thanks to everyone for all the useful information on the Apache Atlas site to-date. Recen
From: Ernie Ostic/Newark/IBM
To: dev@atlas.incubator.apache.org<ma...@atlas.incubator.apache.org>
Date: 05/06/2016 01:39 PM
Subject: Question about REST payloads...
________________________________
Hello...
Thanks to everyone for all the useful information on the Apache Atlas site to-date. Recently I started experimenting with Atlas, using the REST API, and establishing some of my own types and then reviewing their metadata. I was able to get things working for the currently published REST calls, but would like to know more about the formatting of the payloads. The returned JSON payload is valid but embeds further JSON that "appears" to have additional formatting --- is this something that is handled automatically by some of the (presumably java) existing client tooling? ...or maybe I should have passed another argument or made a different call that would pass back a straight text response?
Here is a snippet of the sample response I received....
{"requestId":"qtp430472391-17 - 12f1518c-fdac-4a50-a23a-f56af47ee069","GUID":"1731e6e8-ee11-4fc0-b86a-e83f08d67981","definition":"{\n \"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference\",\n \"id\":{\n \"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Id\",\n \"id\":\"1731e6e8-ee11-4fc0-b86a-e83f08d67981\"....
Is there a historical reason for the further formatting of "definition"? Is there a parameter I should be passing that I accidentally missed (that would avoid this formatting)?
Thanks. Many of our customers are considering using REST calls from a variety of tools and languages. This structure will require additional parsing and formatting after breaking out the initial elements.
Ernie
Ernie Ostic
WW Product Specialist, Information Server
IBM Analytics
Cell: (617) 331 8238
---------------------------------------------------------------
Open IGC is here!
Extend the Catalog with custom objects and lineage definitions!
https://dsrealtime.wordpress.com/2015/07/29/open-igc-is-here/