You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tez.apache.org by Anton Petrov <an...@hadapt.com> on 2014/06/03 15:48:58 UTC
Debugging Tez release
All,
In the interest of learning more about Tez internals I am trying to debug a
running installation on a cluster. However, I am having trouble matching
the source on my local machine with the jars on the cluster, my source
seems to be off by three lines, making debugging impossible.
I installed HDP 2.1 and Tez as instructed in these docs
http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.1/bk_installing_manually_book/content/rpm-chap1.html
All the jars in the /usr/lib/tez directory are stamped with the following
suffix: 0.4.0.2.1.2.0-402. I assume the first three integers are for the
Tez version and the rest are the Hadoop version. A "yum list installed |
grep tez" returns the same version stamp.
Locally, I have checked out the Tez source and updated to
the branch-0.4.0-incubating branch. That branch has rc0 and rc1 tags, the
difference between which does not really touch any source files, so I don't
think it matters to which rc I update my source.
Despite updating to what I think the right version is, my DAGAppMaster.java
class is off by three lines with the jars on the cluster. I am attaching my
debugger to the AppMaster JVM.
Any help would be much appreciated!
-- Anton
Re: Debugging Tez release
Posted by Anton Petrov <an...@hadapt.com>.
Hitesh -- That makes sense. I think the best solution for me would be to
compile locally and scp the JARs up on the cluster.
For others reading this: if you don't have write access to the cluster and
can't modify the JARs then I would create a local branch and cherry pick
all the commits linked by Hitesh above. A little hacky, but it should do
the trick :)
Thank you very much for the help!
-- Anton
On Tue, Jun 3, 2014 at 4:06 PM, Hitesh Shah <hi...@apache.org> wrote:
> Hello Anton,
>
> The HDP release of Tez is not an exact match of the Apache release. You can
> look at
>
> http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.1/bk_releasenotes_hdp_2.1/content/ch_relnotes-hdp-2.1.1-tez.html
> for
> the full list of additional patches.
>
> Feel free to send any questions regd. Tez internals that you have to this
> mailing list.
>
> thanks
> — Hitesh
>
>
> On Tue, Jun 3, 2014 at 6:48 AM, Anton Petrov <an...@hadapt.com> wrote:
>
> > All,
> >
> > In the interest of learning more about Tez internals I am trying to
> debug a
> > running installation on a cluster. However, I am having trouble matching
> > the source on my local machine with the jars on the cluster, my source
> > seems to be off by three lines, making debugging impossible.
> >
> > I installed HDP 2.1 and Tez as instructed in these docs
> >
> >
> http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.1/bk_installing_manually_book/content/rpm-chap1.html
> >
> > All the jars in the /usr/lib/tez directory are stamped with the following
> > suffix: 0.4.0.2.1.2.0-402. I assume the first three integers are for the
> > Tez version and the rest are the Hadoop version. A "yum list installed |
> > grep tez" returns the same version stamp.
> >
> > Locally, I have checked out the Tez source and updated to
> > the branch-0.4.0-incubating branch. That branch has rc0 and rc1 tags, the
> > difference between which does not really touch any source files, so I
> don't
> > think it matters to which rc I update my source.
> >
> > Despite updating to what I think the right version is, my
> DAGAppMaster.java
> > class is off by three lines with the jars on the cluster. I am attaching
> my
> > debugger to the AppMaster JVM.
> >
> > Any help would be much appreciated!
> >
> > -- Anton
> >
>
Re: Debugging Tez release
Posted by Hitesh Shah <hi...@apache.org>.
Hello Anton,
The HDP release of Tez is not an exact match of the Apache release. You can
look at
http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.1/bk_releasenotes_hdp_2.1/content/ch_relnotes-hdp-2.1.1-tez.html
for
the full list of additional patches.
Feel free to send any questions regd. Tez internals that you have to this
mailing list.
thanks
— Hitesh
On Tue, Jun 3, 2014 at 6:48 AM, Anton Petrov <an...@hadapt.com> wrote:
> All,
>
> In the interest of learning more about Tez internals I am trying to debug a
> running installation on a cluster. However, I am having trouble matching
> the source on my local machine with the jars on the cluster, my source
> seems to be off by three lines, making debugging impossible.
>
> I installed HDP 2.1 and Tez as instructed in these docs
>
> http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.1/bk_installing_manually_book/content/rpm-chap1.html
>
> All the jars in the /usr/lib/tez directory are stamped with the following
> suffix: 0.4.0.2.1.2.0-402. I assume the first three integers are for the
> Tez version and the rest are the Hadoop version. A "yum list installed |
> grep tez" returns the same version stamp.
>
> Locally, I have checked out the Tez source and updated to
> the branch-0.4.0-incubating branch. That branch has rc0 and rc1 tags, the
> difference between which does not really touch any source files, so I don't
> think it matters to which rc I update my source.
>
> Despite updating to what I think the right version is, my DAGAppMaster.java
> class is off by three lines with the jars on the cluster. I am attaching my
> debugger to the AppMaster JVM.
>
> Any help would be much appreciated!
>
> -- Anton
>