You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@carbondata.apache.org by ravipesala <gi...@git.apache.org> on 2018/09/23 16:23:12 UTC

[GitHub] carbondata pull request #2749: [CARBONDATA-2961] Simplify SDK API interfaces

Github user ravipesala commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2749#discussion_r219703810
  
    --- Diff: docs/sdk-guide.md ---
    @@ -377,91 +366,69 @@ public CarbonWriterBuilder withLoadOptions(Map<String, String> options);
     public CarbonWriterBuilder withTableProperties(Map<String, String> options);
     ```
     
    -
     ```
     /**
    -* this writer is not thread safe, use buildThreadSafeWriterForCSVInput in multi thread environment
    -* Build a {@link CarbonWriter}, which accepts row in CSV format object
    -* @param schema carbon Schema object {org.apache.carbondata.sdk.file.Schema}
    -* @param configuration hadoop configuration object.
    -* @return CSVCarbonWriter
    -* @throws IOException
    -* @throws InvalidLoadOptionException
    +* To make sdk writer thread safe.
    +*
    +* @param numOfThreads should number of threads in which writer is called in multi-thread scenario
    +*                     default sdk writer is not thread safe.
    +*                     can use one writer instance in one thread only.
    +* @return updated CarbonWriterBuilder
     */
    -public CarbonWriter buildWriterForCSVInput(org.apache.carbondata.sdk.file.Schema schema, Configuration configuration) throws IOException, InvalidLoadOptionException;
    +public CarbonWriterBuilder withThreadSafe(short numOfThreads);
     ```
     
     ```
     /**
    -* Can use this writer in multi-thread instance.
    -* Build a {@link CarbonWriter}, which accepts row in CSV format
    -* @param schema carbon Schema object {org.apache.carbondata.sdk.file.Schema}
    -* @param numOfThreads number of threads() in which .write will be called.    
    -* @param configuration hadoop configuration object          
    -* @return CSVCarbonWriter
    -* @throws IOException
    -* @throws InvalidLoadOptionException
    +* To support hadoop configuration
    +*
    +* @param conf hadoop configuration support, can set s3a AK,SK,end point and other conf with this
    +* @return updated CarbonWriterBuilder
     */
    -public CarbonWriter buildThreadSafeWriterForCSVInput(Schema schema, short numOfThreads, Configuration configuration)
    -  throws IOException, InvalidLoadOptionException;
    +public CarbonWriterBuilder withHadoopConf(Configuration conf)
     ```
     
    -
    -```  
    +```
     /**
    -* this writer is not thread safe, use buildThreadSafeWriterForAvroInput in multi thread environment
    -* Build a {@link CarbonWriter}, which accepts Avro format object
    -* @param avroSchema avro Schema object {org.apache.avro.Schema}
    -* @param configuration hadoop configuration object
    -* @return AvroCarbonWriter 
    -* @throws IOException
    -* @throws InvalidLoadOptionException
    +* to build a {@link CarbonWriter}, which accepts row in CSV format
    +*
    +* @param schema carbon Schema object {org.apache.carbondata.sdk.file.Schema}
    +* @return CarbonWriterBuilder
     */
    -public CarbonWriter buildWriterForAvroInput(org.apache.avro.Schema schema, Configuration configuration) throws IOException, InvalidLoadOptionException;
    +public CarbonWriterBuilder forCsvInput(Schema schema);
    --- End diff --
    
    Better change as `withCsvInput`


---