You are viewing a plain text version of this content. The canonical link for it is here.

Posted to common-user@hadoop.apache.org by xeonmailinglist-gmail <xe...@gmail.com> on 2015/04/04 17:15:51 UTC

How to create 3GB of compressed sequence file?

Hi,


I want to create 3GB of compressed sequence file in Hadoop. How can I do 
this?

Thanks,

-- 
--

RE: How to create 3GB of compressed sequence file?

Posted by Mich Talebzadeh <mi...@peridale.co.uk>.

Hi,

 

Do you mean you want to end up with a 3GB file in compressed sequence format?

 

If you have an RDBMS you can use Sqoop to import the data in sequence format. For example, this one imports table hddtester.t from Oracle as sequence file

 

    sqoop import \

        --connect "jdbc:oracle:thin:@rhes564:1521:mydb" --username hddtester -P \

                -query "select * from hddtester.t where \

                 \$CONDITIONS" \

                                       --split-by object_id \

                                      --as-sequencefile \

                                      --target-dir "/work/t" \

                                  --num-mappers 1

 

Now if I want to import it as a compressed sequential file I get an error

 

Error: java.lang.IllegalArgumentException: SequenceFile doesn't work with GzipCodec without native-hadoop code!

 

Which I am trying to resolve

 

HTH

 

Mich Talebzadeh

 

http://talebzadehmich.wordpress.com

 

Publications due shortly:

Creating in-memory Data Grid for Trading Systems with Oracle TimesTen and Coherence Cache

 

NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.

 

-----Original Message-----
From: xeonmailinglist-gmail [mailto:xeonmailinglist@gmail.com] 
Sent: 04 April 2015 16:16
To: user@hadoop.apache.org
Subject: How to create 3GB of compressed sequence file?

 

Hi,

 

 

I want to create 3GB of compressed sequence file in Hadoop. How can I do this?

 

Thanks,

 

--

--

RE: How to create 3GB of compressed sequence file?

Posted by Mich Talebzadeh <mi...@peridale.co.uk>.

Hi,

 

Do you mean you want to end up with a 3GB file in compressed sequence format?

 

If you have an RDBMS you can use Sqoop to import the data in sequence format. For example, this one imports table hddtester.t from Oracle as sequence file

 

    sqoop import \

        --connect "jdbc:oracle:thin:@rhes564:1521:mydb" --username hddtester -P \

                -query "select * from hddtester.t where \

                 \$CONDITIONS" \

                                       --split-by object_id \

                                      --as-sequencefile \

                                      --target-dir "/work/t" \

                                  --num-mappers 1

 

Now if I want to import it as a compressed sequential file I get an error

 

Error: java.lang.IllegalArgumentException: SequenceFile doesn't work with GzipCodec without native-hadoop code!

 

Which I am trying to resolve

 

HTH

 

Mich Talebzadeh

 

http://talebzadehmich.wordpress.com

 

Publications due shortly:

Creating in-memory Data Grid for Trading Systems with Oracle TimesTen and Coherence Cache

 

NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.

 

-----Original Message-----
From: xeonmailinglist-gmail [mailto:xeonmailinglist@gmail.com] 
Sent: 04 April 2015 16:16
To: user@hadoop.apache.org
Subject: How to create 3GB of compressed sequence file?

 

Hi,

 

 

I want to create 3GB of compressed sequence file in Hadoop. How can I do this?

 

Thanks,

 

--

--

RE: How to create 3GB of compressed sequence file?

Posted by Mich Talebzadeh <mi...@peridale.co.uk>.

Hi,

 

Do you mean you want to end up with a 3GB file in compressed sequence format?

 

If you have an RDBMS you can use Sqoop to import the data in sequence format. For example, this one imports table hddtester.t from Oracle as sequence file

 

    sqoop import \

        --connect "jdbc:oracle:thin:@rhes564:1521:mydb" --username hddtester -P \

                -query "select * from hddtester.t where \

                 \$CONDITIONS" \

                                       --split-by object_id \

                                      --as-sequencefile \

                                      --target-dir "/work/t" \

                                  --num-mappers 1

 

Now if I want to import it as a compressed sequential file I get an error

 

Error: java.lang.IllegalArgumentException: SequenceFile doesn't work with GzipCodec without native-hadoop code!

 

Which I am trying to resolve

 

HTH

 

Mich Talebzadeh

 

http://talebzadehmich.wordpress.com

 

Publications due shortly:

Creating in-memory Data Grid for Trading Systems with Oracle TimesTen and Coherence Cache

 

NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.

 

-----Original Message-----
From: xeonmailinglist-gmail [mailto:xeonmailinglist@gmail.com] 
Sent: 04 April 2015 16:16
To: user@hadoop.apache.org
Subject: How to create 3GB of compressed sequence file?

 

Hi,

 

 

I want to create 3GB of compressed sequence file in Hadoop. How can I do this?

 

Thanks,

 

--

--

RE: How to create 3GB of compressed sequence file?

Posted by Mich Talebzadeh <mi...@peridale.co.uk>.

Hi,

 

Do you mean you want to end up with a 3GB file in compressed sequence format?

 

If you have an RDBMS you can use Sqoop to import the data in sequence format. For example, this one imports table hddtester.t from Oracle as sequence file

 

    sqoop import \

        --connect "jdbc:oracle:thin:@rhes564:1521:mydb" --username hddtester -P \

                -query "select * from hddtester.t where \

                 \$CONDITIONS" \

                                       --split-by object_id \

                                      --as-sequencefile \

                                      --target-dir "/work/t" \

                                  --num-mappers 1

 

Now if I want to import it as a compressed sequential file I get an error

 

Error: java.lang.IllegalArgumentException: SequenceFile doesn't work with GzipCodec without native-hadoop code!

 

Which I am trying to resolve

 

HTH

 

Mich Talebzadeh

 

http://talebzadehmich.wordpress.com

 

Publications due shortly:

Creating in-memory Data Grid for Trading Systems with Oracle TimesTen and Coherence Cache

 

NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.

 

-----Original Message-----
From: xeonmailinglist-gmail [mailto:xeonmailinglist@gmail.com] 
Sent: 04 April 2015 16:16
To: user@hadoop.apache.org
Subject: How to create 3GB of compressed sequence file?

 

Hi,

 

 

I want to create 3GB of compressed sequence file in Hadoop. How can I do this?

 

Thanks,

 

--

--