Restore orc file help

[gf <gu...@126.com>]

Hi ORC team


Recently i have a case, not sure if it is possbile to restore the orc file. now ask for help, Steps to reproduce.


Step 1、we have a hive table A, which stored as orc, we have data（file a,file b,file c...) in tableA,
Step 2、we read data files（file a,file b,file c...）with text format by spark engine and rewrite data back (file x,file y,file z...) with text format to tableA,
Step 3、now tableA is serialized with orc, but the files are text format, which makes it impossble to read the data by text or orc. 


ps : data in table A is non-reproduce data and cannot be backtrack from origin source.


I wonder if it possible to restore the data, or we can do to restore the data by reconstruct the orc files.


Any help or suggestions will be appreciated.


Best Regards.

Re: Restore orc file help

[Owen O'Malley <ow...@gmail.com>]

Have you confirmed that the files in the table are in fact ORC files? I
believe the Hive syntax to fix that is:

ALTER TABLE ... [PARTITION partition_spec] SET FILEFORMAT ORC

.. Owen


On Sat, Aug 17, 2024 at 7:24 AM Gang Wu <us...@gmail.com> wrote:

> Sorry to hear that.
>
> Do you have access to Apache Spark where you can read them
> as text files and then write them into ORC files using the DataFrame
> API?
>
> Best,
> Gang
>
> On Sat, Aug 17, 2024 at 11:10 AM gf <gu...@126.com> wrote:
>
>> Hi ORC team
>>
>> Recently i have a case, not sure if it is possbile to restore the orc
>> file. now ask for help, Steps to reproduce.
>>
>> Step 1、we have a hive table A, which stored as orc, we have data（file
>> a,file b,file c...) in tableA,
>> Step 2、we read data files（file a,file b,file c...）with text format by
>> spark engine and rewrite data back (file x,file y,file z...) with text
>> format to tableA,
>> Step 3、now tableA is serialized with orc, but the files are text format,
>> which makes it impossble to read the data by text or orc.
>>
>> ps : data in table A is non-reproduce data and cannot be backtrack from
>> origin source.
>>
>> I wonder if it possible to restore the data, or we can do to restore the
>> data by reconstruct the orc files.
>>
>> Any help or suggestions will be appreciated.
>>
>> Best Regards.
>>
>

Re: Restore orc file help

[Gang Wu <us...@gmail.com>]

Sorry to hear that.

Do you have access to Apache Spark where you can read them
as text files and then write them into ORC files using the DataFrame
API?

Best,
Gang

On Sat, Aug 17, 2024 at 11:10 AM gf <gu...@126.com> wrote:

> Hi ORC team
>
> Recently i have a case, not sure if it is possbile to restore the orc
> file. now ask for help, Steps to reproduce.
>
> Step 1、we have a hive table A, which stored as orc, we have data（file
> a,file b,file c...) in tableA,
> Step 2、we read data files（file a,file b,file c...）with text format by
> spark engine and rewrite data back (file x,file y,file z...) with text
> format to tableA,
> Step 3、now tableA is serialized with orc, but the files are text format,
> which makes it impossble to read the data by text or orc.
>
> ps : data in table A is non-reproduce data and cannot be backtrack from
> origin source.
>
> I wonder if it possible to restore the data, or we can do to restore the
> data by reconstruct the orc files.
>
> Any help or suggestions will be appreciated.
>
> Best Regards.
>