AsterixDB command line access

[sc...@informatik.hu-berlin.de]

Hello AsterixDB group,

I am a student currently working on a diploma thesis to execute AQL on
Apache Flink. Therefor I did setup AsterixDB on a cluster.

My problem is, that I am not in the same network as the cluster and only
have remote command line access, so I cannot reach Asterix's web
interface. I am sure it is possible to run queries via command line, too,
but I could not find anything on that topic on your official documentation
page. Do you have any document you can link me to, that provides a basic
explanation how to work with AsterixDB without the web interface?

Best Regards,
Max Schultze

Re: AsterixDB command line access

[Eldon Carman <ec...@ucr.edu>]

Apache VXQuery uses a command line interface. I wonder how much could be
reused for AsterixDB. Apache VXQuery cli builds the Hyracks job and sends
that to the cluster, where I think in the AsterixDB world you would want to
use the HTTP-based API and allow the server to compile the Hyracks job. The
cli for AsterixDB seems like a simple interface that connects to the
HTTP-based API.

On Mon, Sep 7, 2015 at 4:09 PM, Mike Carey <dt...@gmail.com> wrote:

> We have an HTTP-based API you could use - with curl - not a great answer -
> broadening the discussion to see if anyone on the dev list can offer you
> what they use.  (We need to add a nicer command line interface at some
> point.)
>
> Random side comment:  Hopefully you are doing this by translating from
> Algebricks physical plans to the Flink runtime?  (That would be the
> cleanest place to redirect queries, I would think, to be against Flink
> instead of against Hyracks.)
>
> Cheers,
> Mike
>
>
> On 9/7/15 2:03 AM, schultze@informatik.hu-berlin.de wrote:
>
>> Hello AsterixDB group,
>>
>> I am a student currently working on a diploma thesis to execute AQL on
>> Apache Flink. Therefor I did setup AsterixDB on a cluster.
>>
>> My problem is, that I am not in the same network as the cluster and only
>> have remote command line access, so I cannot reach Asterix's web
>> interface. I am sure it is possible to run queries via command line, too,
>> but I could not find anything on that topic on your official documentation
>> page. Do you have any document you can link me to, that provides a basic
>> explanation how to work with AsterixDB without the web interface?
>>
>> Best Regards,
>> Max Schultze
>>
>>
>>
>>
>>
>

Re: AsterixDB command line access

[Ian Maxon <im...@uci.edu>]

I'm certain more than one person has created a command-line utility at
some point (I think Chris had one at some point? There's one lurking
in asterix-tools, not sure if it's that one, or that is yet another
one). The HTTP API though isn't so complex (hopefully?) as to really
merit a special client in most cases.

If one just has a shell account, and port forwarding is available,
Till's suggestion is definitely a good one, and usually what I do.
However if for some reason that isn't tenable, 'curl' should also do
the trick. For example:

curl --data 'use dataverse issues;
{
"projects" : [ {
"name" : "asterixdb",
"key": "ASTERIXDB",
"description": "AsterixDB is a semi-structured, parallel data
management platform.",
"components": ["AsterixDB","Hyracks"],
"issues": for $y in dataset jiraIssues return $y
} ]
}' -H "Accept: application/x-adm" http://localhost:19002/aql >
/files/adb_jira_issues.json

Just replace the --data part with your query, and the host with the
proper one that's behind the firewall. I'm piping the output to a file
here, but stdout is fine too.

-Ian

On Tue, Sep 8, 2015 at 2:28 AM,  <sc...@informatik.hu-berlin.de> wrote:
> Hi Till,
>
> I probably have to do so. It would have just been easier to later have
> direct command line access for automatic execution of some queries I want
> to run to compare the two systems.
>
> @Mike I am directly translating the AQL Queries to Flink's Scala API.
>
> Regards, Max
>
>> Hi Max,
>>
>> If you have ssh-access to the cluster you could also tunnel the web
>> interface port (usually 19001) on the cluster machine that hosts
>> AsterixDB's cluster controller to your machine using ssh.
>> Of course that's only an option if the web interface works for you
>> otherwise.
>>
>> Cheers,
>> Till
>>
>> On 7 Sep 2015, at 16:09, Mike Carey wrote:
>>
>>> We have an HTTP-based API you could use - with curl - not a great
>>> answer - broadening the discussion to see if anyone on the dev list
>>> can offer you what they use.  (We need to add a nicer command line
>>> interface at some point.)
>>>
>>> Random side comment:  Hopefully you are doing this by translating from
>>> Algebricks physical plans to the Flink runtime?  (That would be the
>>> cleanest place to redirect queries, I would think, to be against Flink
>>> instead of against Hyracks.)
>>>
>>> Cheers,
>>> Mike
>>>
>>>
>>> On 9/7/15 2:03 AM, schultze@informatik.hu-berlin.de wrote:
>>>> Hello AsterixDB group,
>>>>
>>>> I am a student currently working on a diploma thesis to execute AQL
>>>> on
>>>> Apache Flink. Therefor I did setup AsterixDB on a cluster.
>>>>
>>>> My problem is, that I am not in the same network as the cluster and
>>>> only
>>>> have remote command line access, so I cannot reach Asterix's web
>>>> interface. I am sure it is possible to run queries via command line,
>>>> too,
>>>> but I could not find anything on that topic on your official
>>>> documentation
>>>> page. Do you have any document you can link me to, that provides a
>>>> basic
>>>> explanation how to work with AsterixDB without the web interface?
>>>>
>>>> Best Regards,
>>>> Max Schultze
>>>>
>>>>
>>>>
>>>>
>>
>
>

Re: AsterixDB command line access

[sc...@informatik.hu-berlin.de]

Hi Till,

I probably have to do so. It would have just been easier to later have
direct command line access for automatic execution of some queries I want
to run to compare the two systems.

@Mike I am directly translating the AQL Queries to Flink's Scala API.

Regards, Max

> Hi Max,
>
> If you have ssh-access to the cluster you could also tunnel the web
> interface port (usually 19001) on the cluster machine that hosts
> AsterixDB's cluster controller to your machine using ssh.
> Of course that's only an option if the web interface works for you
> otherwise.
>
> Cheers,
> Till
>
> On 7 Sep 2015, at 16:09, Mike Carey wrote:
>
>> We have an HTTP-based API you could use - with curl - not a great
>> answer - broadening the discussion to see if anyone on the dev list
>> can offer you what they use.  (We need to add a nicer command line
>> interface at some point.)
>>
>> Random side comment:  Hopefully you are doing this by translating from
>> Algebricks physical plans to the Flink runtime?  (That would be the
>> cleanest place to redirect queries, I would think, to be against Flink
>> instead of against Hyracks.)
>>
>> Cheers,
>> Mike
>>
>>
>> On 9/7/15 2:03 AM, schultze@informatik.hu-berlin.de wrote:
>>> Hello AsterixDB group,
>>>
>>> I am a student currently working on a diploma thesis to execute AQL
>>> on
>>> Apache Flink. Therefor I did setup AsterixDB on a cluster.
>>>
>>> My problem is, that I am not in the same network as the cluster and
>>> only
>>> have remote command line access, so I cannot reach Asterix's web
>>> interface. I am sure it is possible to run queries via command line,
>>> too,
>>> but I could not find anything on that topic on your official
>>> documentation
>>> page. Do you have any document you can link me to, that provides a
>>> basic
>>> explanation how to work with AsterixDB without the web interface?
>>>
>>> Best Regards,
>>> Max Schultze
>>>
>>>
>>>
>>>
>

Re: AsterixDB command line access

[sc...@informatik.hu-berlin.de]

Hi Till,

I probably have to do so. It would have just been easier to later have
direct command line access for automatic execution of some queries I want
to run to compare the two systems.

@Mike I am directly translating the AQL Queries to Flink's Scala API.

Regards, Max

> Hi Max,
>
> If you have ssh-access to the cluster you could also tunnel the web
> interface port (usually 19001) on the cluster machine that hosts
> AsterixDB's cluster controller to your machine using ssh.
> Of course that's only an option if the web interface works for you
> otherwise.
>
> Cheers,
> Till
>
> On 7 Sep 2015, at 16:09, Mike Carey wrote:
>
>> We have an HTTP-based API you could use - with curl - not a great
>> answer - broadening the discussion to see if anyone on the dev list
>> can offer you what they use.  (We need to add a nicer command line
>> interface at some point.)
>>
>> Random side comment:  Hopefully you are doing this by translating from
>> Algebricks physical plans to the Flink runtime?  (That would be the
>> cleanest place to redirect queries, I would think, to be against Flink
>> instead of against Hyracks.)
>>
>> Cheers,
>> Mike
>>
>>
>> On 9/7/15 2:03 AM, schultze@informatik.hu-berlin.de wrote:
>>> Hello AsterixDB group,
>>>
>>> I am a student currently working on a diploma thesis to execute AQL
>>> on
>>> Apache Flink. Therefor I did setup AsterixDB on a cluster.
>>>
>>> My problem is, that I am not in the same network as the cluster and
>>> only
>>> have remote command line access, so I cannot reach Asterix's web
>>> interface. I am sure it is possible to run queries via command line,
>>> too,
>>> but I could not find anything on that topic on your official
>>> documentation
>>> page. Do you have any document you can link me to, that provides a
>>> basic
>>> explanation how to work with AsterixDB without the web interface?
>>>
>>> Best Regards,
>>> Max Schultze
>>>
>>>
>>>
>>>
>

Re: AsterixDB command line access

[Till Westmann <ti...@apache.org>]

Hi Max,

If you have ssh-access to the cluster you could also tunnel the web 
interface port (usually 19001) on the cluster machine that hosts 
AsterixDB's cluster controller to your machine using ssh.
Of course that's only an option if the web interface works for you 
otherwise.

Cheers,
Till

On 7 Sep 2015, at 16:09, Mike Carey wrote:

> We have an HTTP-based API you could use - with curl - not a great 
> answer - broadening the discussion to see if anyone on the dev list 
> can offer you what they use.  (We need to add a nicer command line 
> interface at some point.)
>
> Random side comment:  Hopefully you are doing this by translating from 
> Algebricks physical plans to the Flink runtime?  (That would be the 
> cleanest place to redirect queries, I would think, to be against Flink 
> instead of against Hyracks.)
>
> Cheers,
> Mike
>
>
> On 9/7/15 2:03 AM, schultze@informatik.hu-berlin.de wrote:
>> Hello AsterixDB group,
>>
>> I am a student currently working on a diploma thesis to execute AQL 
>> on
>> Apache Flink. Therefor I did setup AsterixDB on a cluster.
>>
>> My problem is, that I am not in the same network as the cluster and 
>> only
>> have remote command line access, so I cannot reach Asterix's web
>> interface. I am sure it is possible to run queries via command line, 
>> too,
>> but I could not find anything on that topic on your official 
>> documentation
>> page. Do you have any document you can link me to, that provides a 
>> basic
>> explanation how to work with AsterixDB without the web interface?
>>
>> Best Regards,
>> Max Schultze
>>
>>
>>
>>

Re: AsterixDB command line access

[Eldon Carman <ec...@ucr.edu>]

Apache VXQuery uses a command line interface. I wonder how much could be
reused for AsterixDB. Apache VXQuery cli builds the Hyracks job and sends
that to the cluster, where I think in the AsterixDB world you would want to
use the HTTP-based API and allow the server to compile the Hyracks job. The
cli for AsterixDB seems like a simple interface that connects to the
HTTP-based API.

On Mon, Sep 7, 2015 at 4:09 PM, Mike Carey <dt...@gmail.com> wrote:

> We have an HTTP-based API you could use - with curl - not a great answer -
> broadening the discussion to see if anyone on the dev list can offer you
> what they use.  (We need to add a nicer command line interface at some
> point.)
>
> Random side comment:  Hopefully you are doing this by translating from
> Algebricks physical plans to the Flink runtime?  (That would be the
> cleanest place to redirect queries, I would think, to be against Flink
> instead of against Hyracks.)
>
> Cheers,
> Mike
>
>
> On 9/7/15 2:03 AM, schultze@informatik.hu-berlin.de wrote:
>
>> Hello AsterixDB group,
>>
>> I am a student currently working on a diploma thesis to execute AQL on
>> Apache Flink. Therefor I did setup AsterixDB on a cluster.
>>
>> My problem is, that I am not in the same network as the cluster and only
>> have remote command line access, so I cannot reach Asterix's web
>> interface. I am sure it is possible to run queries via command line, too,
>> but I could not find anything on that topic on your official documentation
>> page. Do you have any document you can link me to, that provides a basic
>> explanation how to work with AsterixDB without the web interface?
>>
>> Best Regards,
>> Max Schultze
>>
>>
>>
>>
>>
>

Re: Re: AsterixDB command line access

[李文海 <lw...@whu.edu.cn>]

Hi, Max
If you like, we also provide a key/value transactions http-api testbed for testing. Ping us if you want.
Best,
Wenhai

> -----原始邮件-----
> 发件人: "Mike Carey" <dt...@gmail.com>
> 发送时间: 2015年9月8日 星期二
> 收件人: users@asterixdb.incubator.apache.org, dev@asterixdb.incubator.apache.org
> 抄送: 
> 主题: Re: AsterixDB command line access
> 
> We have an HTTP-based API you could use - with curl - not a great answer 
> - broadening the discussion to see if anyone on the dev list can offer 
> you what they use.  (We need to add a nicer command line interface at 
> some point.)
> 
> Random side comment:  Hopefully you are doing this by translating from 
> Algebricks physical plans to the Flink runtime?  (That would be the 
> cleanest place to redirect queries, I would think, to be against Flink 
> instead of against Hyracks.)
> 
> Cheers,
> Mike
> 
> 
> On 9/7/15 2:03 AM, schultze@informatik.hu-berlin.de wrote:
> > Hello AsterixDB group,
> >
> > I am a student currently working on a diploma thesis to execute AQL on
> > Apache Flink. Therefor I did setup AsterixDB on a cluster.
> >
> > My problem is, that I am not in the same network as the cluster and only
> > have remote command line access, so I cannot reach Asterix's web
> > interface. I am sure it is possible to run queries via command line, too,
> > but I could not find anything on that topic on your official documentation
> > page. Do you have any document you can link me to, that provides a basic
> > explanation how to work with AsterixDB without the web interface?
> >
> > Best Regards,
> > Max Schultze
> >
> >
> >
> >
> 

Re: AsterixDB command line access

[Till Westmann <ti...@apache.org>]

Hi Max,

If you have ssh-access to the cluster you could also tunnel the web 
interface port (usually 19001) on the cluster machine that hosts 
AsterixDB's cluster controller to your machine using ssh.
Of course that's only an option if the web interface works for you 
otherwise.

Cheers,
Till

On 7 Sep 2015, at 16:09, Mike Carey wrote:

> We have an HTTP-based API you could use - with curl - not a great 
> answer - broadening the discussion to see if anyone on the dev list 
> can offer you what they use.  (We need to add a nicer command line 
> interface at some point.)
>
> Random side comment:  Hopefully you are doing this by translating from 
> Algebricks physical plans to the Flink runtime?  (That would be the 
> cleanest place to redirect queries, I would think, to be against Flink 
> instead of against Hyracks.)
>
> Cheers,
> Mike
>
>
> On 9/7/15 2:03 AM, schultze@informatik.hu-berlin.de wrote:
>> Hello AsterixDB group,
>>
>> I am a student currently working on a diploma thesis to execute AQL 
>> on
>> Apache Flink. Therefor I did setup AsterixDB on a cluster.
>>
>> My problem is, that I am not in the same network as the cluster and 
>> only
>> have remote command line access, so I cannot reach Asterix's web
>> interface. I am sure it is possible to run queries via command line, 
>> too,
>> but I could not find anything on that topic on your official 
>> documentation
>> page. Do you have any document you can link me to, that provides a 
>> basic
>> explanation how to work with AsterixDB without the web interface?
>>
>> Best Regards,
>> Max Schultze
>>
>>
>>
>>

Re: AsterixDB command line access

[Mike Carey <dt...@gmail.com>]

We have an HTTP-based API you could use - with curl - not a great answer 
- broadening the discussion to see if anyone on the dev list can offer 
you what they use.  (We need to add a nicer command line interface at 
some point.)

Random side comment:  Hopefully you are doing this by translating from 
Algebricks physical plans to the Flink runtime?  (That would be the 
cleanest place to redirect queries, I would think, to be against Flink 
instead of against Hyracks.)

Cheers,
Mike


On 9/7/15 2:03 AM, schultze@informatik.hu-berlin.de wrote:
> Hello AsterixDB group,
>
> I am a student currently working on a diploma thesis to execute AQL on
> Apache Flink. Therefor I did setup AsterixDB on a cluster.
>
> My problem is, that I am not in the same network as the cluster and only
> have remote command line access, so I cannot reach Asterix's web
> interface. I am sure it is possible to run queries via command line, too,
> but I could not find anything on that topic on your official documentation
> page. Do you have any document you can link me to, that provides a basic
> explanation how to work with AsterixDB without the web interface?
>
> Best Regards,
> Max Schultze
>
>
>
>

Re: AsterixDB command line access

[Mike Carey <dt...@gmail.com>]

We have an HTTP-based API you could use - with curl - not a great answer 
- broadening the discussion to see if anyone on the dev list can offer 
you what they use.  (We need to add a nicer command line interface at 
some point.)

Random side comment:  Hopefully you are doing this by translating from 
Algebricks physical plans to the Flink runtime?  (That would be the 
cleanest place to redirect queries, I would think, to be against Flink 
instead of against Hyracks.)

Cheers,
Mike


On 9/7/15 2:03 AM, schultze@informatik.hu-berlin.de wrote:
> Hello AsterixDB group,
>
> I am a student currently working on a diploma thesis to execute AQL on
> Apache Flink. Therefor I did setup AsterixDB on a cluster.
>
> My problem is, that I am not in the same network as the cluster and only
> have remote command line access, so I cannot reach Asterix's web
> interface. I am sure it is possible to run queries via command line, too,
> but I could not find anything on that topic on your official documentation
> page. Do you have any document you can link me to, that provides a basic
> explanation how to work with AsterixDB without the web interface?
>
> Best Regards,
> Max Schultze
>
>
>
>