You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by xubo245 <gi...@git.apache.org> on 2018/05/23 13:48:17 UTC
[GitHub] carbondata pull request #2337: [CARBONDATA-2519] Add document for CarbonRead...
GitHub user xubo245 opened a pull request:
https://github.com/apache/carbondata/pull/2337
[CARBONDATA-2519] Add document for CarbonReader
Add document for CarbonReader
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
- [ ] Any interfaces changed?
No
- [ ] Any backward compatibility impacted?
No
- [ ] Document update required?
Yes
- [ ] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.
No need
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
No
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/xubo245/carbondata CARBONDATA-2519-carbonreaderDoc
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2337.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2337
----
commit 71c21c1c634dfdadab27da305241110187d35880
Author: xubo245 <xu...@...>
Date: 2018-05-23T13:45:49Z
[CARBONDATA-2519] Add document for CarbonReader
----
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2337
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5096/
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2337
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4918/
---
[GitHub] carbondata pull request #2337: [CARBONDATA-2519] Add document for CarbonRead...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2337#discussion_r190818765
--- Diff: store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonReaderBuilder.java ---
@@ -136,10 +160,17 @@ public CarbonReaderBuilder setEndPoint(String key, String value) {
* @return CarbonWriterBuilder
*/
public CarbonReaderBuilder setEndPoint(String value) {
- FileFactory.getConfiguration().set(Constants.ENDPOINT, value);
- return this;
+ return setEndPoint(Constants.ENDPOINT, value);
--- End diff --
There is one more implementation of setEndPoint. Please add that also
---
[GitHub] carbondata pull request #2337: [CARBONDATA-2519] Add document for CarbonRead...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2337#discussion_r190850065
--- Diff: store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonReaderBuilder.java ---
@@ -50,26 +50,50 @@
private String tableName;
private boolean isTransactionalTable = true;
+ /**
+ * Construct a CarbonReaderBuilder with table path and table name
+ *
+ * @param tablePath table path
+ * @param tableName table name
+ */
CarbonReaderBuilder(String tablePath, String tableName) {
this.tablePath = tablePath;
this.tableName = tableName;
}
+ /**
+ * Configure the projection column names of carbon reader
+ *
+ * @param projectionColumnNames projection column names
+ * @return CarbonReaderBuilder
+ */
public CarbonReaderBuilder projection(String[] projectionColumnNames) {
Objects.requireNonNull(projectionColumnNames);
this.projectionColumns = projectionColumnNames;
return this;
}
+ /**
+ * Configure the transactional status of table
+ *
+ * @param isTransactionalTable whether is transactional table or not
--- End diff --
update the default value and mention that if false, read from flat folder.
Else user will not know what is this API for.
---
[GitHub] carbondata pull request #2337: [CARBONDATA-2519] Add document for CarbonRead...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2337#discussion_r190819817
--- Diff: docs/sdk-guide.md ---
@@ -398,3 +399,160 @@ Reference : [list of carbon properties](http://carbondata.apache.org/configurati
*/
public static org.apache.carbondata.sdk.file.Schema getCarbonSchemaFromAvroSchema(String avroSchemaString);
```
+# SDK Reader
+This SDK reader reads CarbonData file and carbonindex file at a given path.
+External client can make use of this reader to read CarbonData files without CarbonSession.
+## Quick example
+```
+ // 1. Create carbon reader
+ String path = "./testWriteFiles";
+ CarbonReader reader = CarbonReader
+ .builder(path, "_temp")
+ .projection(new String[]{"name", "age"})
+ .build();
+
+ // 2. Read data
+ int i = 0;
+ while (reader.hasNext()) {
+ Object[] row = (Object[]) reader.readNextRow();
+ System.out.println(row[0] + "\t" + row[1]);
+ i++;
+ }
+
+ // 3. Close this reader
+ reader.close();
+```
+
+Find example code at [CarbonReaderExample](https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/java/org/apache/carbondata/examples/sdk/CarbonReaderExample.java) in the CarbonData repo.
+
+## API List
+
+### org.apache.carbondata.sdk.file.CarbonReader
--- End diff --
change it to below. Just to be same as above style
### org.apache.carbondata.sdk.file.CarbonReader
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2337
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4940/
---
[GitHub] carbondata pull request #2337: [CARBONDATA-2519] Add document for CarbonRead...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2337#discussion_r190819633
--- Diff: docs/sdk-guide.md ---
@@ -398,3 +399,160 @@ Reference : [list of carbon properties](http://carbondata.apache.org/configurati
*/
public static org.apache.carbondata.sdk.file.Schema getCarbonSchemaFromAvroSchema(String avroSchemaString);
```
+# SDK Reader
+This SDK reader reads CarbonData file and carbonindex file at a given path.
+External client can make use of this reader to read CarbonData files without CarbonSession.
+## Quick example
+```
+ // 1. Create carbon reader
+ String path = "./testWriteFiles";
+ CarbonReader reader = CarbonReader
+ .builder(path, "_temp")
+ .projection(new String[]{"name", "age"})
+ .build();
+
+ // 2. Read data
+ int i = 0;
+ while (reader.hasNext()) {
+ Object[] row = (Object[]) reader.readNextRow();
+ System.out.println(row[0] + "\t" + row[1]);
+ i++;
+ }
+
+ // 3. Close this reader
+ reader.close();
+```
+
+Find example code at [CarbonReaderExample](https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/java/org/apache/carbondata/examples/sdk/CarbonReaderExample.java) in the CarbonData repo.
+
+## API List
+
+### org.apache.carbondata.sdk.file.CarbonReader
+```
+ /**
+ * Return a new CarbonReaderBuilder instance
+ */
+ public static CarbonReaderBuilder builder(String tablePath, String tableName);
+```
+
+```
+ /**
+ * Read carbondata file and return the schema
+ */
+ public static List<ColumnSchema> readSchemaInDataFile(String dataFilePath);
+```
+
+```
+ /**
+ * Read schema file and return table info object
+ */
+ public static TableInfo readSchemaFile(String schemaFilePath);
+```
+
+```
+ /**
+ * Return true if has next row
+ */
+ public boolean hasNext();
+```
+
+```
+ /**
+ * Read and return next row object
+ */
+ public T readNextRow();
+```
+
+```
+ /**
+ * Close reader
+ */
+ public void close();
+```
+
+###org.apache.carbondata.sdk.file.CarbonReaderBuilder
--- End diff --
need to give space after ###, else formatting will not reflect
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2337
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5076/
---
[GitHub] carbondata pull request #2337: [CARBONDATA-2519] Add document for CarbonRead...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2337#discussion_r190825861
--- Diff: docs/sdk-guide.md ---
@@ -398,3 +399,160 @@ Reference : [list of carbon properties](http://carbondata.apache.org/configurati
*/
public static org.apache.carbondata.sdk.file.Schema getCarbonSchemaFromAvroSchema(String avroSchemaString);
```
+# SDK Reader
+This SDK reader reads CarbonData file and carbonindex file at a given path.
+External client can make use of this reader to read CarbonData files without CarbonSession.
+## Quick example
+```
+ // 1. Create carbon reader
+ String path = "./testWriteFiles";
+ CarbonReader reader = CarbonReader
+ .builder(path, "_temp")
+ .projection(new String[]{"name", "age"})
+ .build();
+
+ // 2. Read data
+ int i = 0;
+ while (reader.hasNext()) {
+ Object[] row = (Object[]) reader.readNextRow();
+ System.out.println(row[0] + "\t" + row[1]);
+ i++;
+ }
+
+ // 3. Close this reader
+ reader.close();
+```
+
+Find example code at [CarbonReaderExample](https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/java/org/apache/carbondata/examples/sdk/CarbonReaderExample.java) in the CarbonData repo.
+
+## API List
+
+### org.apache.carbondata.sdk.file.CarbonReader
+```
+ /**
+ * Return a new CarbonReaderBuilder instance
+ */
+ public static CarbonReaderBuilder builder(String tablePath, String tableName);
+```
+
+```
+ /**
+ * Read carbondata file and return the schema
+ */
+ public static List<ColumnSchema> readSchemaInDataFile(String dataFilePath);
+```
+
+```
+ /**
+ * Read schema file and return table info object
+ */
+ public static TableInfo readSchemaFile(String schemaFilePath);
+```
+
+```
+ /**
+ * Return true if has next row
+ */
+ public boolean hasNext();
+```
+
+```
+ /**
+ * Read and return next row object
+ */
+ public T readNextRow();
+```
+
+```
+ /**
+ * Close reader
+ */
+ public void close();
+```
+
+###org.apache.carbondata.sdk.file.CarbonReaderBuilder
--- End diff --
ok, done
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on the issue:
https://github.com/apache/carbondata/pull/2337
LGTM
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2337
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6075/
---
[GitHub] carbondata pull request #2337: [CARBONDATA-2519] Add document for CarbonRead...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/carbondata/pull/2337
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2337
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4928/
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2337
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4915/
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2337
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4946/
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/2337
retest this please
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2337
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6083/
---
[GitHub] carbondata pull request #2337: [CARBONDATA-2519] Add document for CarbonRead...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2337#discussion_r190825696
--- Diff: docs/sdk-guide.md ---
@@ -398,3 +399,160 @@ Reference : [list of carbon properties](http://carbondata.apache.org/configurati
*/
public static org.apache.carbondata.sdk.file.Schema getCarbonSchemaFromAvroSchema(String avroSchemaString);
```
+# SDK Reader
+This SDK reader reads CarbonData file and carbonindex file at a given path.
+External client can make use of this reader to read CarbonData files without CarbonSession.
+## Quick example
+```
+ // 1. Create carbon reader
+ String path = "./testWriteFiles";
+ CarbonReader reader = CarbonReader
+ .builder(path, "_temp")
+ .projection(new String[]{"name", "age"})
+ .build();
+
+ // 2. Read data
+ int i = 0;
+ while (reader.hasNext()) {
+ Object[] row = (Object[]) reader.readNextRow();
+ System.out.println(row[0] + "\t" + row[1]);
+ i++;
+ }
+
+ // 3. Close this reader
+ reader.close();
+```
+
+Find example code at [CarbonReaderExample](https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/java/org/apache/carbondata/examples/sdk/CarbonReaderExample.java) in the CarbonData repo.
+
+## API List
+
+### org.apache.carbondata.sdk.file.CarbonReader
+```
+ /**
+ * Return a new CarbonReaderBuilder instance
+ */
+ public static CarbonReaderBuilder builder(String tablePath, String tableName);
+```
+
+```
+ /**
+ * Read carbondata file and return the schema
+ */
+ public static List<ColumnSchema> readSchemaInDataFile(String dataFilePath);
+```
+
+```
+ /**
+ * Read schema file and return table info object
+ */
+ public static TableInfo readSchemaFile(String schemaFilePath);
+```
+
+```
+ /**
+ * Return true if has next row
+ */
+ public boolean hasNext();
+```
+
+```
+ /**
+ * Read and return next row object
+ */
+ public T readNextRow();
+```
+
+```
+ /**
+ * Close reader
+ */
+ public void close();
+```
+
+###org.apache.carbondata.sdk.file.CarbonReaderBuilder
+```
+ /**
+ * Construct a CarbonReaderBuilder with table path and table name
+ *
+ * @param tablePath table path
+ * @param tableName table name
+ */
+ CarbonReaderBuilder(String tablePath, String tableName);
+```
+
+```
+ /**
+ * Configure the projection column names of carbon reader
+ *
+ * @param projectionColumnNames projection column names
+ * @return CarbonReaderBuilder object
+ */
+ public CarbonReaderBuilder projection(String[] projectionColumnNames);
+```
+
+```
+ /**
+ * Configure the transactional status of table
+ *
+ * @param isTransactionalTable whether is transactional table or not
+ * @return CarbonReaderBuilder object
+ */
+ public CarbonReaderBuilder isTransactionalTable(boolean isTransactionalTable);
+```
+
+```
+ /**
+ * Configure the filter expression for carbon reader
+ *
+ * @param filterExpression filter expression
+ * @return CarbonReaderBuilder object
+ */
+ public CarbonReaderBuilder filter(Expression filterExpression);
+```
+
+```
+ /**
+ * Set the access key for S3
+ *
+ * @param key the string of access key for different S3 type,like: fs.s3a.access.key
+ * @param value the value of access key
+ * @return CarbonWriterBuilder
+ */
+ public CarbonReaderBuilder setAccessKey(String key, String value);
+```
+
+```
+ /**
+ * Set the secret key for S3
+ *
+ * @param key the string of secret key for different S3 type,like: fs.s3a.secret.key
+ * @param value the value of secret key
+ * @return CarbonWriterBuilder
+ */
+ public CarbonReaderBuilder setSecretKey(String key, String value);
+```
+
+```
+ /**
+ * Set the endpoint for S3
+ *
+ * @param key the string of endpoint for different S3 type,like: fs.s3a.endpoint
+ * @param value the value of endpoint
+ * @return CarbonWriterBuilder
+ */
+ public CarbonReaderBuilder setEndPoint(String key, String value);
--- End diff --
ok, AK and SK also have another interface
---
[GitHub] carbondata pull request #2337: [CARBONDATA-2519] Add document for CarbonRead...
Posted by ajantha-bhat <gi...@git.apache.org>.
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2337#discussion_r190819502
--- Diff: docs/sdk-guide.md ---
@@ -398,3 +399,160 @@ Reference : [list of carbon properties](http://carbondata.apache.org/configurati
*/
public static org.apache.carbondata.sdk.file.Schema getCarbonSchemaFromAvroSchema(String avroSchemaString);
```
+# SDK Reader
+This SDK reader reads CarbonData file and carbonindex file at a given path.
+External client can make use of this reader to read CarbonData files without CarbonSession.
+## Quick example
+```
+ // 1. Create carbon reader
+ String path = "./testWriteFiles";
+ CarbonReader reader = CarbonReader
+ .builder(path, "_temp")
+ .projection(new String[]{"name", "age"})
+ .build();
+
+ // 2. Read data
+ int i = 0;
+ while (reader.hasNext()) {
+ Object[] row = (Object[]) reader.readNextRow();
+ System.out.println(row[0] + "\t" + row[1]);
+ i++;
+ }
+
+ // 3. Close this reader
+ reader.close();
+```
+
+Find example code at [CarbonReaderExample](https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/java/org/apache/carbondata/examples/sdk/CarbonReaderExample.java) in the CarbonData repo.
+
+## API List
+
+### org.apache.carbondata.sdk.file.CarbonReader
+```
+ /**
+ * Return a new CarbonReaderBuilder instance
+ */
+ public static CarbonReaderBuilder builder(String tablePath, String tableName);
+```
+
+```
+ /**
+ * Read carbondata file and return the schema
+ */
+ public static List<ColumnSchema> readSchemaInDataFile(String dataFilePath);
+```
+
+```
+ /**
+ * Read schema file and return table info object
+ */
+ public static TableInfo readSchemaFile(String schemaFilePath);
+```
+
+```
+ /**
+ * Return true if has next row
+ */
+ public boolean hasNext();
+```
+
+```
+ /**
+ * Read and return next row object
+ */
+ public T readNextRow();
+```
+
+```
+ /**
+ * Close reader
+ */
+ public void close();
+```
+
+###org.apache.carbondata.sdk.file.CarbonReaderBuilder
+```
+ /**
+ * Construct a CarbonReaderBuilder with table path and table name
+ *
+ * @param tablePath table path
+ * @param tableName table name
+ */
+ CarbonReaderBuilder(String tablePath, String tableName);
+```
+
+```
+ /**
+ * Configure the projection column names of carbon reader
+ *
+ * @param projectionColumnNames projection column names
+ * @return CarbonReaderBuilder object
+ */
+ public CarbonReaderBuilder projection(String[] projectionColumnNames);
+```
+
+```
+ /**
+ * Configure the transactional status of table
+ *
+ * @param isTransactionalTable whether is transactional table or not
+ * @return CarbonReaderBuilder object
+ */
+ public CarbonReaderBuilder isTransactionalTable(boolean isTransactionalTable);
+```
+
+```
+ /**
+ * Configure the filter expression for carbon reader
+ *
+ * @param filterExpression filter expression
+ * @return CarbonReaderBuilder object
+ */
+ public CarbonReaderBuilder filter(Expression filterExpression);
+```
+
+```
+ /**
+ * Set the access key for S3
+ *
+ * @param key the string of access key for different S3 type,like: fs.s3a.access.key
+ * @param value the value of access key
+ * @return CarbonWriterBuilder
+ */
+ public CarbonReaderBuilder setAccessKey(String key, String value);
+```
+
+```
+ /**
+ * Set the secret key for S3
+ *
+ * @param key the string of secret key for different S3 type,like: fs.s3a.secret.key
+ * @param value the value of secret key
+ * @return CarbonWriterBuilder
+ */
+ public CarbonReaderBuilder setSecretKey(String key, String value);
+```
+
+```
+ /**
+ * Set the endpoint for S3
+ *
+ * @param key the string of endpoint for different S3 type,like: fs.s3a.endpoint
+ * @param value the value of endpoint
+ * @return CarbonWriterBuilder
+ */
+ public CarbonReaderBuilder setEndPoint(String key, String value);
--- End diff --
There is one more implementation of setEndPoint. Please add that also
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2337
Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/4924/
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/2337
retest this please
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2337
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5080/
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2337
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6078/
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2337
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6102/
---
[GitHub] carbondata pull request #2337: [CARBONDATA-2519] Add document for CarbonRead...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2337#discussion_r191036327
--- Diff: store/sdk/src/main/java/org/apache/carbondata/sdk/file/CarbonReaderBuilder.java ---
@@ -50,26 +50,50 @@
private String tableName;
private boolean isTransactionalTable = true;
+ /**
+ * Construct a CarbonReaderBuilder with table path and table name
+ *
+ * @param tablePath table path
+ * @param tableName table name
+ */
CarbonReaderBuilder(String tablePath, String tableName) {
this.tablePath = tablePath;
this.tableName = tableName;
}
+ /**
+ * Configure the projection column names of carbon reader
+ *
+ * @param projectionColumnNames projection column names
+ * @return CarbonReaderBuilder
+ */
public CarbonReaderBuilder projection(String[] projectionColumnNames) {
Objects.requireNonNull(projectionColumnNames);
this.projectionColumns = projectionColumnNames;
return this;
}
+ /**
+ * Configure the transactional status of table
+ *
+ * @param isTransactionalTable whether is transactional table or not
--- End diff --
ok, Done
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/2337
@ajantha-bhat CI pass， pease check
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by kunal642 <gi...@git.apache.org>.
Github user kunal642 commented on the issue:
https://github.com/apache/carbondata/pull/2337
LGTM
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/2337
@ajantha-bhat CI pass, please review it
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2337
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6089/
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2337
Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6085/
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by ravipesala <gi...@git.apache.org>.
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2337
SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5102/
---
[GitHub] carbondata issue #2337: [CARBONDATA-2519] Add document for CarbonReader
Posted by CarbonDataQA <gi...@git.apache.org>.
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2337
Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6108/
---
[GitHub] carbondata pull request #2337: [CARBONDATA-2519] Add document for CarbonRead...
Posted by xubo245 <gi...@git.apache.org>.
Github user xubo245 commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2337#discussion_r190826231
--- Diff: docs/sdk-guide.md ---
@@ -398,3 +399,160 @@ Reference : [list of carbon properties](http://carbondata.apache.org/configurati
*/
public static org.apache.carbondata.sdk.file.Schema getCarbonSchemaFromAvroSchema(String avroSchemaString);
```
+# SDK Reader
+This SDK reader reads CarbonData file and carbonindex file at a given path.
+External client can make use of this reader to read CarbonData files without CarbonSession.
+## Quick example
+```
+ // 1. Create carbon reader
+ String path = "./testWriteFiles";
+ CarbonReader reader = CarbonReader
+ .builder(path, "_temp")
+ .projection(new String[]{"name", "age"})
+ .build();
+
+ // 2. Read data
+ int i = 0;
+ while (reader.hasNext()) {
+ Object[] row = (Object[]) reader.readNextRow();
+ System.out.println(row[0] + "\t" + row[1]);
+ i++;
+ }
+
+ // 3. Close this reader
+ reader.close();
+```
+
+Find example code at [CarbonReaderExample](https://github.com/apache/carbondata/blob/master/examples/spark2/src/main/java/org/apache/carbondata/examples/sdk/CarbonReaderExample.java) in the CarbonData repo.
+
+## API List
+
+### org.apache.carbondata.sdk.file.CarbonReader
--- End diff --
ok, added
---