You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Harsh J (Created) (JIRA)" <ji...@apache.org> on 2011/10/28 16:24:32 UTC
[jira] [Created] (MAPREDUCE-3303) MR part of removing RecordIO
(HADOOP-7781)
MR part of removing RecordIO (HADOOP-7781)
------------------------------------------
Key: MAPREDUCE-3303
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3303
Project: Hadoop Map/Reduce
Issue Type: Task
Affects Versions: 0.23.0
Reporter: Harsh J
Assignee: Harsh J
Priority: Minor
Fix For: 0.23.0
This is the MR part of removing deprecated RecordIO packages - parented by HADOOP-7781.
Basically, we need to remove {{/hadoop-mapreduce-project/src/c++/librecordio}} and all associated build helpers around it.
(For posterity, RecordIO has been replaced by Apache Avro http://avro.apache.org)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3303) MR part of removing RecordIO
(HADOOP-7781)
Posted by "Harsh J (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13139069#comment-13139069 ]
Harsh J commented on MAPREDUCE-3303:
------------------------------------
Ok I looked carefully again and there seems to be more:
- TypedBytes uses and depends on the record package to provide byte buffers. Streaming uses this. It will have to be shifted to use something else in place. I'll try to get Klaas to comment on this.
> MR part of removing RecordIO (HADOOP-7781)
> ------------------------------------------
>
> Key: MAPREDUCE-3303
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3303
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Affects Versions: 0.23.0
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Minor
> Fix For: 0.23.0
>
>
> This is the MR part of removing deprecated RecordIO packages - parented by HADOOP-7781.
> Basically, we need to remove {{/hadoop-mapreduce-project/src/c++/librecordio}} and all associated build helpers around it.
> (For posterity, RecordIO has been replaced by Apache Avro http://avro.apache.org)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-3303) MR part of removing RecordIO
(HADOOP-7781)
Posted by "Harsh J (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Harsh J resolved MAPREDUCE-3303.
--------------------------------
Resolution: Invalid
Per Owen on the parent JIRA, the deprecated recordio is still in use apparently and should not be removed.
We can pursue this once we have an action plan later (if we do).
> MR part of removing RecordIO (HADOOP-7781)
> ------------------------------------------
>
> Key: MAPREDUCE-3303
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3303
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Affects Versions: 0.23.0
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Minor
> Fix For: 0.24.0
>
>
> This is the MR part of removing deprecated RecordIO packages - parented by HADOOP-7781.
> Basically, we need to remove {{/hadoop-mapreduce-project/src/c++/librecordio}} and all associated build helpers around it.
> (For posterity, RecordIO has been replaced by Apache Avro http://avro.apache.org)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3303) MR part of removing RecordIO
(HADOOP-7781)
Posted by "Klaas Bosteels (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142546#comment-13142546 ]
Klaas Bosteels commented on MAPREDUCE-3303:
-------------------------------------------
Not sure I agree with the removal of RecordIO, but that's a different discussion.. :)
The TypedBytes functionality in Streaming doesn't really have an actual hard decency on RecordIO, it just supports it. RecordIO records are automatically converted to typed bytes when they are taken as input so that streaming programs can easily read sequence files that contain them, but removing the RecordIO support will not break typed bytes based Streaming altogether.
I guess we should add support for Avro (instead) though if that's going to be the new standard, but that could be a separate JIRA issue.
> MR part of removing RecordIO (HADOOP-7781)
> ------------------------------------------
>
> Key: MAPREDUCE-3303
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3303
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Affects Versions: 0.23.0
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Minor
> Fix For: 0.24.0
>
>
> This is the MR part of removing deprecated RecordIO packages - parented by HADOOP-7781.
> Basically, we need to remove {{/hadoop-mapreduce-project/src/c++/librecordio}} and all associated build helpers around it.
> (For posterity, RecordIO has been replaced by Apache Avro http://avro.apache.org)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3303) MR part of removing RecordIO
(HADOOP-7781)
Posted by "Harsh J (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167196#comment-13167196 ]
Harsh J commented on MAPREDUCE-3303:
------------------------------------
Klaas,
Thanks for commenting here! I noticed it too late, sorry.
I'll poke around the typedbytes source when I get some time and see if I can help with an avro transition.
Apparently, RecordIO is still in use by external users out there (?), and we're not removing it for now.
> MR part of removing RecordIO (HADOOP-7781)
> ------------------------------------------
>
> Key: MAPREDUCE-3303
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3303
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Affects Versions: 0.23.0
> Reporter: Harsh J
> Assignee: Harsh J
> Priority: Minor
> Fix For: 0.24.0
>
>
> This is the MR part of removing deprecated RecordIO packages - parented by HADOOP-7781.
> Basically, we need to remove {{/hadoop-mapreduce-project/src/c++/librecordio}} and all associated build helpers around it.
> (For posterity, RecordIO has been replaced by Apache Avro http://avro.apache.org)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira