You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by Dmitriy Lyubimov <dl...@gmail.com> on 2014/06/05 18:18:12 UTC
Re: git commit: MAHOUT-1541 still working on this, some refactoring
in the DSL for abstracting away Spark has moved access to rddsno Jira is
closed yet
* don't think we should be pushing this to apache git, I'd suggest to keep
individual issue branches strictly on github. I'd suggest to drop this
branch from apache git.*
On Wed, Jun 4, 2014 at 6:44 PM, <pa...@apache.org> wrote:
> Repository: mahout
> Updated Branches:
> refs/heads/mahout-1541 8a4b4347d -> 2f87f5433
>
>
> MAHOUT-1541 still working on this, some refactoring in the DSL for
> abstracting away Spark has moved access to rddsno Jira is closed yet
>
>
> Project: http://git-wip-us.apache.org/repos/asf/mahout/repo
> Commit: http://git-wip-us.apache.org/repos/asf/mahout/commit/2f87f543
> Tree: http://git-wip-us.apache.org/repos/asf/mahout/tree/2f87f543
> Diff: http://git-wip-us.apache.org/repos/asf/mahout/diff/2f87f543
>
> Branch: refs/heads/mahout-1541
> Commit: 2f87f5433f90fa2c49ef386ca245943e1fc73beb
> Parents: 8a4b434
> Author: pferrel <pa...@occamsmachete.com>
> Authored: Wed Jun 4 18:44:16 2014 -0700
> Committer: pferrel <pa...@occamsmachete.com>
> Committed: Wed Jun 4 18:44:16 2014 -0700
>
> ----------------------------------------------------------------------
> .../src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala | 4 ++++
> 1 file changed, 4 insertions(+)
> ----------------------------------------------------------------------
>
>
>
> http://git-wip-us.apache.org/repos/asf/mahout/blob/2f87f543/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> ----------------------------------------------------------------------
> diff --git
> a/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> b/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> index 1179eef..9201c81 100644
> --- a/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> +++ b/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> @@ -149,6 +149,10 @@ trait TDIndexedDatasetWriter extends
> Writer[IndexedDataset]{
> val matrix: DrmLike[Int] = indexedDataset.matrix
> val rowIDDictionary: BiMap[String, Int] = indexedDataset.rowIDs
> val columnIDDictionary: BiMap[String, Int] =
> indexedDataset.columnIDs
> + // below doesn't compile because the rdd is not in a
> CheckpointedDrmSpark also I don't know how to turn a
> + // CheckpointedDrmSpark[Int] into a DrmLike[Int], which I need to
> pass in the CooccurrenceAnalysis#cooccurrence
> + // This seems to be about the refacotring to abstract away from
> Spark but the Read and Write are Spark specific
> + // and the non-specific DrmLike is no longer attached to a
> CheckpointedDrmSpark, could be missing something though
> matrix.rdd.map({ case (rowID, itemVector) =>
> var line: String = rowIDDictionary.inverse.get(rowID) + outDelim1
> for (item <- itemVector.nonZeroes()) {
>
>
Re: git commit: MAHOUT-1541 still working on this, some refactoring
in the DSL for abstracting away Spark has moved access to rddsno Jira is
closed yet
Posted by Dmitriy Lyubimov <dl...@gmail.com>.
you need to do a PR of pferrel/mahout branch "MAHOUT-1541" for
apache/mahout "master"
On Thu, Jun 5, 2014 at 10:10 AM, Pat Ferrel <pa...@gmail.com> wrote:
> I deleted the apache.git version of the branch but it did not get mirrored
> to github.
>
> 'git remote show apache’ shows it removed but the github UI still has it,
> maybe it will disappear later.
>
> I’ll do a pr to github/apache/mahout, but it will by default try to merge
> with master. I can’t pick the branch the PR is targeted for. That makes me
> nervous but if you say so...
>
>
> On Jun 5, 2014, at 10:02 AM, Dmitriy Lyubimov <dl...@gmail.com> wrote:
>
> sorry for confusion
>
>
> On Thu, Jun 5, 2014 at 9:59 AM, Dmitriy Lyubimov <dl...@gmail.com>
> wrote:
>
> > I probably meant to do a PR to github's "apache/mahout" MASTER, not push
> > to it to git-wip-us.
> >
> >
> > On Thu, Jun 5, 2014 at 9:42 AM, Pat Ferrel <pa...@gmail.com> wrote:
> >
> >> Tried doing a PR to your repo and you asked for it to go to apache HEAD.
> >> I certainly didn’t want it to get into the master yet.
> >>
> >> Happy to delete it but isn’t the Apache git OK for WIP branches?
> >>
> >> On Jun 5, 2014, at 9:18 AM, Dmitriy Lyubimov <dl...@gmail.com> wrote:
> >>
> >> * don't think we should be pushing this to apache git, I'd suggest to
> keep
> >> individual issue branches strictly on github. I'd suggest to drop this
> >> branch from apache git.*
> >>
> >>
> >> On Wed, Jun 4, 2014 at 6:44 PM, <pa...@apache.org> wrote:
> >>
> >>> Repository: mahout
> >>> Updated Branches:
> >>> refs/heads/mahout-1541 8a4b4347d -> 2f87f5433
> >>>
> >>>
> >>> MAHOUT-1541 still working on this, some refactoring in the DSL for
> >>> abstracting away Spark has moved access to rddsno Jira is closed yet
> >>>
> >>>
> >>> Project: http://git-wip-us.apache.org/repos/asf/mahout/repo
> >>> Commit: http://git-wip-us.apache.org/repos/asf/mahout/commit/2f87f543
> >>> Tree: http://git-wip-us.apache.org/repos/asf/mahout/tree/2f87f543
> >>> Diff: http://git-wip-us.apache.org/repos/asf/mahout/diff/2f87f543
> >>>
> >>> Branch: refs/heads/mahout-1541
> >>> Commit: 2f87f5433f90fa2c49ef386ca245943e1fc73beb
> >>> Parents: 8a4b434
> >>> Author: pferrel <pa...@occamsmachete.com>
> >>> Authored: Wed Jun 4 18:44:16 2014 -0700
> >>> Committer: pferrel <pa...@occamsmachete.com>
> >>> Committed: Wed Jun 4 18:44:16 2014 -0700
> >>>
> >>> ----------------------------------------------------------------------
> >>> .../src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala | 4
> >> ++++
> >>> 1 file changed, 4 insertions(+)
> >>> ----------------------------------------------------------------------
> >>>
> >>>
> >>>
> >>>
> >>
> http://git-wip-us.apache.org/repos/asf/mahout/blob/2f87f543/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> >>> ----------------------------------------------------------------------
> >>> diff --git
> >>> a/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> >>> b/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> >>> index 1179eef..9201c81 100644
> >>> --- a/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> >>> +++ b/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> >>> @@ -149,6 +149,10 @@ trait TDIndexedDatasetWriter extends
> >>> Writer[IndexedDataset]{
> >>> val matrix: DrmLike[Int] = indexedDataset.matrix
> >>> val rowIDDictionary: BiMap[String, Int] = indexedDataset.rowIDs
> >>> val columnIDDictionary: BiMap[String, Int] =
> >>> indexedDataset.columnIDs
> >>> + // below doesn't compile because the rdd is not in a
> >>> CheckpointedDrmSpark also I don't know how to turn a
> >>> + // CheckpointedDrmSpark[Int] into a DrmLike[Int], which I need
> to
> >>> pass in the CooccurrenceAnalysis#cooccurrence
> >>> + // This seems to be about the refacotring to abstract away from
> >>> Spark but the Read and Write are Spark specific
> >>> + // and the non-specific DrmLike is no longer attached to a
> >>> CheckpointedDrmSpark, could be missing something though
> >>> matrix.rdd.map({ case (rowID, itemVector) =>
> >>> var line: String = rowIDDictionary.inverse.get(rowID) +
> >> outDelim1
> >>> for (item <- itemVector.nonZeroes()) {
> >>>
> >>>
> >>
> >>
> >
>
>
Re: git commit: MAHOUT-1541 still working on this, some refactoring in the DSL for abstracting away Spark has moved access to rddsno Jira is closed yet
Posted by Pat Ferrel <pa...@gmail.com>.
I deleted the apache.git version of the branch but it did not get mirrored to github.
'git remote show apache’ shows it removed but the github UI still has it, maybe it will disappear later.
I’ll do a pr to github/apache/mahout, but it will by default try to merge with master. I can’t pick the branch the PR is targeted for. That makes me nervous but if you say so...
On Jun 5, 2014, at 10:02 AM, Dmitriy Lyubimov <dl...@gmail.com> wrote:
sorry for confusion
On Thu, Jun 5, 2014 at 9:59 AM, Dmitriy Lyubimov <dl...@gmail.com> wrote:
> I probably meant to do a PR to github's "apache/mahout" MASTER, not push
> to it to git-wip-us.
>
>
> On Thu, Jun 5, 2014 at 9:42 AM, Pat Ferrel <pa...@gmail.com> wrote:
>
>> Tried doing a PR to your repo and you asked for it to go to apache HEAD.
>> I certainly didn’t want it to get into the master yet.
>>
>> Happy to delete it but isn’t the Apache git OK for WIP branches?
>>
>> On Jun 5, 2014, at 9:18 AM, Dmitriy Lyubimov <dl...@gmail.com> wrote:
>>
>> * don't think we should be pushing this to apache git, I'd suggest to keep
>> individual issue branches strictly on github. I'd suggest to drop this
>> branch from apache git.*
>>
>>
>> On Wed, Jun 4, 2014 at 6:44 PM, <pa...@apache.org> wrote:
>>
>>> Repository: mahout
>>> Updated Branches:
>>> refs/heads/mahout-1541 8a4b4347d -> 2f87f5433
>>>
>>>
>>> MAHOUT-1541 still working on this, some refactoring in the DSL for
>>> abstracting away Spark has moved access to rddsno Jira is closed yet
>>>
>>>
>>> Project: http://git-wip-us.apache.org/repos/asf/mahout/repo
>>> Commit: http://git-wip-us.apache.org/repos/asf/mahout/commit/2f87f543
>>> Tree: http://git-wip-us.apache.org/repos/asf/mahout/tree/2f87f543
>>> Diff: http://git-wip-us.apache.org/repos/asf/mahout/diff/2f87f543
>>>
>>> Branch: refs/heads/mahout-1541
>>> Commit: 2f87f5433f90fa2c49ef386ca245943e1fc73beb
>>> Parents: 8a4b434
>>> Author: pferrel <pa...@occamsmachete.com>
>>> Authored: Wed Jun 4 18:44:16 2014 -0700
>>> Committer: pferrel <pa...@occamsmachete.com>
>>> Committed: Wed Jun 4 18:44:16 2014 -0700
>>>
>>> ----------------------------------------------------------------------
>>> .../src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala | 4
>> ++++
>>> 1 file changed, 4 insertions(+)
>>> ----------------------------------------------------------------------
>>>
>>>
>>>
>>>
>> http://git-wip-us.apache.org/repos/asf/mahout/blob/2f87f543/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
>>> ----------------------------------------------------------------------
>>> diff --git
>>> a/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
>>> b/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
>>> index 1179eef..9201c81 100644
>>> --- a/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
>>> +++ b/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
>>> @@ -149,6 +149,10 @@ trait TDIndexedDatasetWriter extends
>>> Writer[IndexedDataset]{
>>> val matrix: DrmLike[Int] = indexedDataset.matrix
>>> val rowIDDictionary: BiMap[String, Int] = indexedDataset.rowIDs
>>> val columnIDDictionary: BiMap[String, Int] =
>>> indexedDataset.columnIDs
>>> + // below doesn't compile because the rdd is not in a
>>> CheckpointedDrmSpark also I don't know how to turn a
>>> + // CheckpointedDrmSpark[Int] into a DrmLike[Int], which I need to
>>> pass in the CooccurrenceAnalysis#cooccurrence
>>> + // This seems to be about the refacotring to abstract away from
>>> Spark but the Read and Write are Spark specific
>>> + // and the non-specific DrmLike is no longer attached to a
>>> CheckpointedDrmSpark, could be missing something though
>>> matrix.rdd.map({ case (rowID, itemVector) =>
>>> var line: String = rowIDDictionary.inverse.get(rowID) +
>> outDelim1
>>> for (item <- itemVector.nonZeroes()) {
>>>
>>>
>>
>>
>
Re: git commit: MAHOUT-1541 still working on this, some refactoring
in the DSL for abstracting away Spark has moved access to rddsno Jira is
closed yet
Posted by Dmitriy Lyubimov <dl...@gmail.com>.
sorry for confusion
On Thu, Jun 5, 2014 at 9:59 AM, Dmitriy Lyubimov <dl...@gmail.com> wrote:
> I probably meant to do a PR to github's "apache/mahout" MASTER, not push
> to it to git-wip-us.
>
>
> On Thu, Jun 5, 2014 at 9:42 AM, Pat Ferrel <pa...@gmail.com> wrote:
>
>> Tried doing a PR to your repo and you asked for it to go to apache HEAD.
>> I certainly didn’t want it to get into the master yet.
>>
>> Happy to delete it but isn’t the Apache git OK for WIP branches?
>>
>> On Jun 5, 2014, at 9:18 AM, Dmitriy Lyubimov <dl...@gmail.com> wrote:
>>
>> * don't think we should be pushing this to apache git, I'd suggest to keep
>> individual issue branches strictly on github. I'd suggest to drop this
>> branch from apache git.*
>>
>>
>> On Wed, Jun 4, 2014 at 6:44 PM, <pa...@apache.org> wrote:
>>
>> > Repository: mahout
>> > Updated Branches:
>> > refs/heads/mahout-1541 8a4b4347d -> 2f87f5433
>> >
>> >
>> > MAHOUT-1541 still working on this, some refactoring in the DSL for
>> > abstracting away Spark has moved access to rddsno Jira is closed yet
>> >
>> >
>> > Project: http://git-wip-us.apache.org/repos/asf/mahout/repo
>> > Commit: http://git-wip-us.apache.org/repos/asf/mahout/commit/2f87f543
>> > Tree: http://git-wip-us.apache.org/repos/asf/mahout/tree/2f87f543
>> > Diff: http://git-wip-us.apache.org/repos/asf/mahout/diff/2f87f543
>> >
>> > Branch: refs/heads/mahout-1541
>> > Commit: 2f87f5433f90fa2c49ef386ca245943e1fc73beb
>> > Parents: 8a4b434
>> > Author: pferrel <pa...@occamsmachete.com>
>> > Authored: Wed Jun 4 18:44:16 2014 -0700
>> > Committer: pferrel <pa...@occamsmachete.com>
>> > Committed: Wed Jun 4 18:44:16 2014 -0700
>> >
>> > ----------------------------------------------------------------------
>> > .../src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala | 4
>> ++++
>> > 1 file changed, 4 insertions(+)
>> > ----------------------------------------------------------------------
>> >
>> >
>> >
>> >
>> http://git-wip-us.apache.org/repos/asf/mahout/blob/2f87f543/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
>> > ----------------------------------------------------------------------
>> > diff --git
>> > a/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
>> > b/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
>> > index 1179eef..9201c81 100644
>> > --- a/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
>> > +++ b/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
>> > @@ -149,6 +149,10 @@ trait TDIndexedDatasetWriter extends
>> > Writer[IndexedDataset]{
>> > val matrix: DrmLike[Int] = indexedDataset.matrix
>> > val rowIDDictionary: BiMap[String, Int] = indexedDataset.rowIDs
>> > val columnIDDictionary: BiMap[String, Int] =
>> > indexedDataset.columnIDs
>> > + // below doesn't compile because the rdd is not in a
>> > CheckpointedDrmSpark also I don't know how to turn a
>> > + // CheckpointedDrmSpark[Int] into a DrmLike[Int], which I need to
>> > pass in the CooccurrenceAnalysis#cooccurrence
>> > + // This seems to be about the refacotring to abstract away from
>> > Spark but the Read and Write are Spark specific
>> > + // and the non-specific DrmLike is no longer attached to a
>> > CheckpointedDrmSpark, could be missing something though
>> > matrix.rdd.map({ case (rowID, itemVector) =>
>> > var line: String = rowIDDictionary.inverse.get(rowID) +
>> outDelim1
>> > for (item <- itemVector.nonZeroes()) {
>> >
>> >
>>
>>
>
Re: git commit: MAHOUT-1541 still working on this, some refactoring
in the DSL for abstracting away Spark has moved access to rddsno Jira is
closed yet
Posted by Dmitriy Lyubimov <dl...@gmail.com>.
I probably meant to do a PR to github's "apache/mahout" MASTER, not push to
it to git-wip-us.
On Thu, Jun 5, 2014 at 9:42 AM, Pat Ferrel <pa...@gmail.com> wrote:
> Tried doing a PR to your repo and you asked for it to go to apache HEAD. I
> certainly didn’t want it to get into the master yet.
>
> Happy to delete it but isn’t the Apache git OK for WIP branches?
>
> On Jun 5, 2014, at 9:18 AM, Dmitriy Lyubimov <dl...@gmail.com> wrote:
>
> * don't think we should be pushing this to apache git, I'd suggest to keep
> individual issue branches strictly on github. I'd suggest to drop this
> branch from apache git.*
>
>
> On Wed, Jun 4, 2014 at 6:44 PM, <pa...@apache.org> wrote:
>
> > Repository: mahout
> > Updated Branches:
> > refs/heads/mahout-1541 8a4b4347d -> 2f87f5433
> >
> >
> > MAHOUT-1541 still working on this, some refactoring in the DSL for
> > abstracting away Spark has moved access to rddsno Jira is closed yet
> >
> >
> > Project: http://git-wip-us.apache.org/repos/asf/mahout/repo
> > Commit: http://git-wip-us.apache.org/repos/asf/mahout/commit/2f87f543
> > Tree: http://git-wip-us.apache.org/repos/asf/mahout/tree/2f87f543
> > Diff: http://git-wip-us.apache.org/repos/asf/mahout/diff/2f87f543
> >
> > Branch: refs/heads/mahout-1541
> > Commit: 2f87f5433f90fa2c49ef386ca245943e1fc73beb
> > Parents: 8a4b434
> > Author: pferrel <pa...@occamsmachete.com>
> > Authored: Wed Jun 4 18:44:16 2014 -0700
> > Committer: pferrel <pa...@occamsmachete.com>
> > Committed: Wed Jun 4 18:44:16 2014 -0700
> >
> > ----------------------------------------------------------------------
> > .../src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala | 4 ++++
> > 1 file changed, 4 insertions(+)
> > ----------------------------------------------------------------------
> >
> >
> >
> >
> http://git-wip-us.apache.org/repos/asf/mahout/blob/2f87f543/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> > ----------------------------------------------------------------------
> > diff --git
> > a/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> > b/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> > index 1179eef..9201c81 100644
> > --- a/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> > +++ b/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> > @@ -149,6 +149,10 @@ trait TDIndexedDatasetWriter extends
> > Writer[IndexedDataset]{
> > val matrix: DrmLike[Int] = indexedDataset.matrix
> > val rowIDDictionary: BiMap[String, Int] = indexedDataset.rowIDs
> > val columnIDDictionary: BiMap[String, Int] =
> > indexedDataset.columnIDs
> > + // below doesn't compile because the rdd is not in a
> > CheckpointedDrmSpark also I don't know how to turn a
> > + // CheckpointedDrmSpark[Int] into a DrmLike[Int], which I need to
> > pass in the CooccurrenceAnalysis#cooccurrence
> > + // This seems to be about the refacotring to abstract away from
> > Spark but the Read and Write are Spark specific
> > + // and the non-specific DrmLike is no longer attached to a
> > CheckpointedDrmSpark, could be missing something though
> > matrix.rdd.map({ case (rowID, itemVector) =>
> > var line: String = rowIDDictionary.inverse.get(rowID) + outDelim1
> > for (item <- itemVector.nonZeroes()) {
> >
> >
>
>
Re: git commit: MAHOUT-1541 still working on this, some refactoring in the DSL for abstracting away Spark has moved access to rddsno Jira is closed yet
Posted by Pat Ferrel <pa...@gmail.com>.
Tried doing a PR to your repo and you asked for it to go to apache HEAD. I certainly didn’t want it to get into the master yet.
Happy to delete it but isn’t the Apache git OK for WIP branches?
On Jun 5, 2014, at 9:18 AM, Dmitriy Lyubimov <dl...@gmail.com> wrote:
* don't think we should be pushing this to apache git, I'd suggest to keep
individual issue branches strictly on github. I'd suggest to drop this
branch from apache git.*
On Wed, Jun 4, 2014 at 6:44 PM, <pa...@apache.org> wrote:
> Repository: mahout
> Updated Branches:
> refs/heads/mahout-1541 8a4b4347d -> 2f87f5433
>
>
> MAHOUT-1541 still working on this, some refactoring in the DSL for
> abstracting away Spark has moved access to rddsno Jira is closed yet
>
>
> Project: http://git-wip-us.apache.org/repos/asf/mahout/repo
> Commit: http://git-wip-us.apache.org/repos/asf/mahout/commit/2f87f543
> Tree: http://git-wip-us.apache.org/repos/asf/mahout/tree/2f87f543
> Diff: http://git-wip-us.apache.org/repos/asf/mahout/diff/2f87f543
>
> Branch: refs/heads/mahout-1541
> Commit: 2f87f5433f90fa2c49ef386ca245943e1fc73beb
> Parents: 8a4b434
> Author: pferrel <pa...@occamsmachete.com>
> Authored: Wed Jun 4 18:44:16 2014 -0700
> Committer: pferrel <pa...@occamsmachete.com>
> Committed: Wed Jun 4 18:44:16 2014 -0700
>
> ----------------------------------------------------------------------
> .../src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala | 4 ++++
> 1 file changed, 4 insertions(+)
> ----------------------------------------------------------------------
>
>
>
> http://git-wip-us.apache.org/repos/asf/mahout/blob/2f87f543/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> ----------------------------------------------------------------------
> diff --git
> a/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> b/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> index 1179eef..9201c81 100644
> --- a/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> +++ b/spark/src/main/scala/org/apache/mahout/drivers/ReaderWriter.scala
> @@ -149,6 +149,10 @@ trait TDIndexedDatasetWriter extends
> Writer[IndexedDataset]{
> val matrix: DrmLike[Int] = indexedDataset.matrix
> val rowIDDictionary: BiMap[String, Int] = indexedDataset.rowIDs
> val columnIDDictionary: BiMap[String, Int] =
> indexedDataset.columnIDs
> + // below doesn't compile because the rdd is not in a
> CheckpointedDrmSpark also I don't know how to turn a
> + // CheckpointedDrmSpark[Int] into a DrmLike[Int], which I need to
> pass in the CooccurrenceAnalysis#cooccurrence
> + // This seems to be about the refacotring to abstract away from
> Spark but the Read and Write are Spark specific
> + // and the non-specific DrmLike is no longer attached to a
> CheckpointedDrmSpark, could be missing something though
> matrix.rdd.map({ case (rowID, itemVector) =>
> var line: String = rowIDDictionary.inverse.get(rowID) + outDelim1
> for (item <- itemVector.nonZeroes()) {
>
>