You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@atlas.apache.org by "Christian R (JIRA)" <ji...@apache.org> on 2017/06/14 20:13:02 UTC
[jira] [Updated] (ATLAS-1875) Gremlin id is no longer returned for
vertices in gremlin query
[ https://issues.apache.org/jira/browse/ATLAS-1875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Christian R updated ATLAS-1875:
-------------------------------
Description:
Hi,
while investigating a move from atlas 0.7 to 0.8 (HDP 2.5 to HDP2.6) our tests fail on gremlin queries. It turns out that the returned entities in a gremlin search in 0.8 does not include the 'id' attribute.
I've built commit a0bd93945cd45457bbf34a8cb819d4fa4ba72964 (0.8-rc1) on linux using berkely and elasticearch to test with.
The query
:21000/api/atlas/discovery/search/gremlin?g.V.has('__type.name', 'Infrastructure').collect()
on our 0.7 cluster gives
{code:json}
{
__type.name: "Infrastructure",
__type.category: "CLASS",
__type: "typeSystem",
id: "16640"
}{code}
while the same query on my 0.8-rc1 installation gives
{code:javascript}
{
__type.name: "Infrastructure",
__version: "1",
__type.category: "CLASS",
__type.version: "1.0",
__modificationTimestamp: "1497448424134",
__type: "typeSystem",
__type.options: "null",
__type.description: "Infrastructure",
__guid: "77d07283-7622-4305-9c0c-09ac5aee86c8",
__timestamp: "1497448424134"
}
{code}
Certainly more information, but id is missing.
The very poor DSL performance (see ATLAS-1868) and a need for advanced queries led us to base our queries on gremlin. This has worked very well so far. We include both edges and nodes in the result set and use the inVertex, outVertex and label info on the edges to rebuild our tree on the client side.
I also see that gremlin has disappeared from version two of the API. Since addE and addV lets you insert into the graph I can see how exposing a full gremlin endpoint might not be wanted.
As an example of the queries we run that I haven't been able to express in the DSL is
{noformat}
query = g.V.has('__guid','xxxx').copySplit(
_().out('track'),
_().as('x')
.out('functions', 'component')
.loop('x'){true}{true}
.copySplit(
_(),
_().in('part_of'),
_().outE('functions', 'component'),
_().inE('part_of')
)
.exhaustMerge.dedup
)
.exhaustMerge
.collect()
{noformat}
this might not be a normal usecase.
edit to add:
I looked at GraphBackedDiscoveryService.java and notice that:
0.7:
{code:java}
else if (r instanceof TitanVertex) {
Iterable<TitanProperty> ps = ((TitanVertex) r).getProperties();
for (TitanProperty tP : ps) {
String pName = tP.getPropertyKey().getName();
Object pValue = ((TitanVertex) r).getProperty(pName);
if (pValue != null) {
oRow.put(pName, pValue.toString());
}
}
{code}
Vs 0.8 code:
{code:java}
else if (value instanceof AtlasVertex) {
AtlasVertex<?,?> vertex = (AtlasVertex<?,?>)value;
for (String key : vertex.getPropertyKeys()) {
Object propertyValue = GraphHelper.getProperty(vertex, key);
if (propertyValue != null) {
oRow.put(key, propertyValue.toString());
}
}
{code}
look very similar.
However, 0.8 handles id in edges explicitly while the 0.7 doesn’t treat edges explicitly at all.
{code:java}
else if(value instanceof AtlasEdge) {
AtlasEdge edge = (AtlasEdge) value;
oRow.put("id", edge.getId().toString());
oRow.put("label", edge.getLabel());
oRow.put("inVertex", edge.getInVertex().getId().toString());
oRow.put("outVertex", edge.getOutVertex().getId().toString());
for (String propertyKey : edge.getPropertyKeys()) {
oRow.put(propertyKey, GraphHelper.getProperty(edge, propertyKey).toString());
}
{code}
Should a
oRow.put("id", vertex.getId().toString());
be added to the vertex-case?
was:
Hi,
while investigating a move from atlas 0.7 to 0.8 (HDP 2.5 to HDP2.6) our tests fail on gremlin queries. It turns out that the returned entities in a gremlin search in 0.8 does not include the 'id' attribute.
I've built commit a0bd93945cd45457bbf34a8cb819d4fa4ba72964 (0.8-rc1) on linux using berkely and elasticearch to test with.
The query
:21000/api/atlas/discovery/search/gremlin?g.V.has('__type.name', 'Infrastructure').collect()
on our 0.7 cluster gives
{code:json}
{
__type.name: "Infrastructure",
__type.category: "CLASS",
__type: "typeSystem",
id: "16640"
}{code}
while the same query on my 0.8-rc1 installation gives
{code:javascript}
{
__type.name: "Infrastructure",
__version: "1",
__type.category: "CLASS",
__type.version: "1.0",
__modificationTimestamp: "1497448424134",
__type: "typeSystem",
__type.options: "null",
__type.description: "Infrastructure",
__guid: "77d07283-7622-4305-9c0c-09ac5aee86c8",
__timestamp: "1497448424134"
}
{code}
Certainly more information, but id is missing.
The very poor DSL performance (see ATLAS-1868) and a need for advanced queries led us to base our queries on gremlin. This has worked very well so far. We include both edges and nodes in the result set and use the inVertex, outVertex and label info on the edges to rebuild our tree on the client side.
I also see that gremlin has disappeared from version two of the API. Since addE and addV lets you insert into the graph I can see how exposing a full gremlin endpoint might not be wanted.
As an example of the queries we run that I haven't been able to express in the DSL is
{noformat}
query = g.V.has('__guid','xxxx').copySplit(
_().out('track'),
_().as('x')
.out('functions', 'component')
.loop('x'){true}{true}
.copySplit(
_(),
_().in('part_of'),
_().outE('functions', 'component'),
_().inE('part_of')
)
.exhaustMerge.dedup
)
.exhaustMerge
.collect()
{noformat}
this might not be a normal usecase.
> Gremlin id is no longer returned for vertices in gremlin query
> --------------------------------------------------------------
>
> Key: ATLAS-1875
> URL: https://issues.apache.org/jira/browse/ATLAS-1875
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
> Affects Versions: trunk, 0.8-incubating
> Reporter: Christian R
> Labels: dsl, gremlin
>
> Hi,
> while investigating a move from atlas 0.7 to 0.8 (HDP 2.5 to HDP2.6) our tests fail on gremlin queries. It turns out that the returned entities in a gremlin search in 0.8 does not include the 'id' attribute.
> I've built commit a0bd93945cd45457bbf34a8cb819d4fa4ba72964 (0.8-rc1) on linux using berkely and elasticearch to test with.
> The query
> :21000/api/atlas/discovery/search/gremlin?g.V.has('__type.name', 'Infrastructure').collect()
> on our 0.7 cluster gives
> {code:json}
> {
> __type.name: "Infrastructure",
> __type.category: "CLASS",
> __type: "typeSystem",
> id: "16640"
> }{code}
> while the same query on my 0.8-rc1 installation gives
> {code:javascript}
> {
> __type.name: "Infrastructure",
> __version: "1",
> __type.category: "CLASS",
> __type.version: "1.0",
> __modificationTimestamp: "1497448424134",
> __type: "typeSystem",
> __type.options: "null",
> __type.description: "Infrastructure",
> __guid: "77d07283-7622-4305-9c0c-09ac5aee86c8",
> __timestamp: "1497448424134"
> }
> {code}
> Certainly more information, but id is missing.
> The very poor DSL performance (see ATLAS-1868) and a need for advanced queries led us to base our queries on gremlin. This has worked very well so far. We include both edges and nodes in the result set and use the inVertex, outVertex and label info on the edges to rebuild our tree on the client side.
> I also see that gremlin has disappeared from version two of the API. Since addE and addV lets you insert into the graph I can see how exposing a full gremlin endpoint might not be wanted.
> As an example of the queries we run that I haven't been able to express in the DSL is
> {noformat}
> query = g.V.has('__guid','xxxx').copySplit(
> _().out('track'),
> _().as('x')
> .out('functions', 'component')
> .loop('x'){true}{true}
> .copySplit(
> _(),
> _().in('part_of'),
> _().outE('functions', 'component'),
> _().inE('part_of')
> )
> .exhaustMerge.dedup
> )
> .exhaustMerge
> .collect()
> {noformat}
> this might not be a normal usecase.
> edit to add:
> I looked at GraphBackedDiscoveryService.java and notice that:
> 0.7:
> {code:java}
> else if (r instanceof TitanVertex) {
> Iterable<TitanProperty> ps = ((TitanVertex) r).getProperties();
> for (TitanProperty tP : ps) {
> String pName = tP.getPropertyKey().getName();
> Object pValue = ((TitanVertex) r).getProperty(pName);
> if (pValue != null) {
> oRow.put(pName, pValue.toString());
> }
> }
> {code}
> Vs 0.8 code:
> {code:java}
> else if (value instanceof AtlasVertex) {
> AtlasVertex<?,?> vertex = (AtlasVertex<?,?>)value;
> for (String key : vertex.getPropertyKeys()) {
> Object propertyValue = GraphHelper.getProperty(vertex, key);
> if (propertyValue != null) {
> oRow.put(key, propertyValue.toString());
> }
> }
> {code}
> look very similar.
> However, 0.8 handles id in edges explicitly while the 0.7 doesn’t treat edges explicitly at all.
> {code:java}
> else if(value instanceof AtlasEdge) {
> AtlasEdge edge = (AtlasEdge) value;
> oRow.put("id", edge.getId().toString());
> oRow.put("label", edge.getLabel());
> oRow.put("inVertex", edge.getInVertex().getId().toString());
> oRow.put("outVertex", edge.getOutVertex().getId().toString());
> for (String propertyKey : edge.getPropertyKeys()) {
> oRow.put(propertyKey, GraphHelper.getProperty(edge, propertyKey).toString());
> }
> {code}
> Should a
> oRow.put("id", vertex.getId().toString());
> be added to the vertex-case?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)