You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@usergrid.apache.org by "Rod Simpson (JIRA)" <ji...@apache.org> on 2014/03/10 21:00:44 UTC
[jira] [Commented] (USERGRID-89) Add Export functionality
[ https://issues.apache.org/jira/browse/USERGRID-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13926126#comment-13926126 ]
Rod Simpson commented on USERGRID-89:
-------------------------------------
George has been working on this one and it is now complete. I created a ticket and assigned it to him. I realize this is taking place post-work, but I thought it was better to at least document it.
We will all be making a much better effort to put our work out in the open as we do it.
Rod
rod simpson
On March 10, 2014 at 1:55:20 PM, Rod Simpson (JIRA) (jira@apache.org) wrote:
Rod Simpson created USERGRID-89:
-----------------------------------
Summary: Add Export functionality
Key: USERGRID-89
URL: https://issues.apache.org/jira/browse/USERGRID-89
Project: Usergrid
Issue Type: Story
Components: Stack
Reporter: Rod Simpson
Assignee: George Reyes
We need to extend UGC to include data upload and data download capabilities.
1. Spec out how this will work (do design doc, get reviewed by Rod, Chris Von See, and Scott Ganyo).
2. Should include ability to upload CSV or JSON files.
3. Eventually we will extend functionality to include uploading file directly to server, but interface in UGC should work the same.
4. Download would be based on a path plus optional query.
4. Should be able to set file type for download.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
> Add Export functionality
> ------------------------
>
> Key: USERGRID-89
> URL: https://issues.apache.org/jira/browse/USERGRID-89
> Project: Usergrid
> Issue Type: Story
> Components: Stack
> Reporter: Rod Simpson
> Assignee: George Reyes
>
> We need to extend UGC to include data upload and data download capabilities.
> 1. Spec out how this will work (do design doc, get reviewed by Rod, Chris Von See, and Scott Ganyo).
> 2. Should include ability to upload CSV or JSON files.
> 3. Eventually we will extend functionality to include uploading file directly to server, but interface in UGC should work the same.
> 4. Download would be based on a path plus optional query.
> 4. Should be able to set file type for download.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: [jira] [Commented] (USERGRID-89) Add Export functionality
Posted by George Reyes <gr...@apigee.com>.
Hi All,
I'll be writing an overview of how the new export functionality works. At a
high level, this endpoint allows you to dump your collection(s) from a
specific org and app to a s3 bucket. The status of the export job is kept
track of in a new collection called 'exports'. The export format currently
only dumps the information in a Json data format.
I've added the following new endpoints.
POST
/management/orgs/<org_name>/apps/<app_name>/collection/<collection_name>/export
^ allows you to dump a specific collection to s3. Returns export_uuid.
POST /management/orgs/<org_name>/apps/<app_name>/export
^ allows you to dump the entire app's collections to a file. Returns
export_uuid.
GET /management/orgs/<org_name>/apps/<app_name>/export/export_uuid
^ gets the current status of export. Possible status are currently CREATED,
SCHEDULED, STARTED, FINISHED | FAILED.
You can query the endpoint to get one of the status returned, informing you
on the status of the job.
In order to call the POST endpoints you need to attach a json entity in the
following format.
"properties":{
"storage_provider":"s3",
"storage_info":{
"s3_access_id":"your s3 access id",
"s3_key":"<your s3 key here>",
"bucket_location":"<your bucket here>"
}
}
Properties stands in as a name for the json object. Storage provider gives
the export method context for what to expect in storage info. Currently
only amazon s3 is supported. Storage info represents the information that
the export tool will need to make the connection to your storage provider.
For instance, amazon needs: access_id, your secret key, and the bucket_name
Examples on how to call the endpoints:
curl -X POST -i -H 'Authorization: Bearer <your auth token here>'
http://localhost:8080/management/orgs/<organization_name>/apps/<application_name>/export'
-d '{"properties":{"storage_provider":"s3",
"storage_info":{"s3_access_id":<s3_access_id>, "s3_key":"<your s3 key
here>", "bucket_location":"<your bucket here>" }}}'
or
curl -X POST -i -H 'Authorization: Bearer <your auth token here>'
http://localhost:8080/management/orgs/<organization_name>/apps/<application_name>/collection/<collection_name>/export'
-d '{"properties":{"storage_provider":"s3",
"storage_info":{"s3_access_id":<s3_access_id>, "s3_key":"<your s3 key
here>", "bucket_location":"<your bucket here>" }}}'
To check the state
curl -X GET -i -H 'Authorization: Bearer <your auth token here>'
http://localhost:8080/management/orgs/
<organization_name>/apps/<application_name>/export/<uuid>
What output looks like:
{
"Metadata" : {
"uuid" : "3a44fd1a-89e6-11e3-aee0-0ff4d053e928",
"type" : "bacon",
"created" : 1391110766433,
"modified" : 1391110766433,
"temp" : "val"
}
}, {
"Metadata" : {
"uuid" : "70f39f36-1825-379d-8385-7a7fbe9ec74a",
"type" : "role",
"name" : "admin",
"created" : 1390247061613,
"modified" : 1390247061613,
"roleName" : "admin",
"title" : "Administrator",
"inactivity" : 0
},
"connections" : { },
"dictionaries" : { }
}, {
"Metadata" : {
"uuid" : "b8f8f336-30c9-3553-b447-6891f3e1e6bf",
"type" : "role",
"name" : "default",
"created" : 1390247061626,
"modified" : 1390247061626,
"roleName" : "default",
"title" : "Default",
"inactivity" : 0
},
"connections" : { },
"dictionaries" : {
"permissions" : {
"get,put,post,delete:/**" : null
}
}
}
On Mon, Mar 10, 2014 at 1:00 PM, Rod Simpson (JIRA) <ji...@apache.org> wrote:
>
> [
> https://issues.apache.org/jira/browse/USERGRID-89?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13926126#comment-13926126]
>
> Rod Simpson commented on USERGRID-89:
> -------------------------------------
>
> George has been working on this one and it is now complete. I created a
> ticket and assigned it to him. I realize this is taking place post-work,
> but I thought it was better to at least document it.
>
> We will all be making a much better effort to put our work out in the open
> as we do it.
>
> Rod
>
>
> rod simpson
>
> On March 10, 2014 at 1:55:20 PM, Rod Simpson (JIRA) (jira@apache.org)
> wrote:
>
> Rod Simpson created USERGRID-89:
> -----------------------------------
>
> Summary: Add Export functionality
> Key: USERGRID-89
> URL: https://issues.apache.org/jira/browse/USERGRID-89
> Project: Usergrid
> Issue Type: Story
> Components: Stack
> Reporter: Rod Simpson
> Assignee: George Reyes
>
>
> We need to extend UGC to include data upload and data download
> capabilities.
>
> 1. Spec out how this will work (do design doc, get reviewed by Rod, Chris
> Von See, and Scott Ganyo).
> 2. Should include ability to upload CSV or JSON files.
> 3. Eventually we will extend functionality to include uploading file
> directly to server, but interface in UGC should work the same.
> 4. Download would be based on a path plus optional query.
> 4. Should be able to set file type for download.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.2#6252)
>
>
> > Add Export functionality
> > ------------------------
> >
> > Key: USERGRID-89
> > URL: https://issues.apache.org/jira/browse/USERGRID-89
> > Project: Usergrid
> > Issue Type: Story
> > Components: Stack
> > Reporter: Rod Simpson
> > Assignee: George Reyes
> >
> > We need to extend UGC to include data upload and data download
> capabilities.
> > 1. Spec out how this will work (do design doc, get reviewed by Rod,
> Chris Von See, and Scott Ganyo).
> > 2. Should include ability to upload CSV or JSON files.
> > 3. Eventually we will extend functionality to include uploading file
> directly to server, but interface in UGC should work the same.
> > 4. Download would be based on a path plus optional query.
> > 4. Should be able to set file type for download.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.2#6252)
>