You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by Claude Warren <cl...@xenei.com> on 2024/10/19 10:15:38 UTC
[DISCUSSs] new Iterator
I would like to introduce a new Iterator to the iterator library:
ExtendedIterator
This Iterator is an Iterator that extends the functionality of other
iterators and makes using some iterators in the library more idiomatic. It
is based on the ExtendedIterator found in the new Apache Rat core package
that was inspired by the Apache Jena ExtendedIterator interface.
I propose a class ExtendedIterator<T> that extends Iterator<T> and has the
following additional methods.
Static methods (all methods return an ExtendedIterator<T>)
- createNoRemove(final Iterator<T> it) -- prohibits removal on an
iterator that otherwise would allow removal.
- ofStream(final Stream<T> stream) -- convenience method to create
ExtendedIterator from stream.
- create(final Iterator<T> it)-- creates an ExtendedIterator from a
plain iterator. If the Iterator is already "extended" just return it.
- emptyIterator() -- convenience method to creat an empty
ExtendedIterator.
- unwind(final Iterator<Iterator<T>> woundIterator) -- creates an
Iterator<T> by iterating through each of the iterators in "woundIterator"
Instance methods (Unless otherwise noted all methods return an
ExtendedIterator<T>)
- T removeNext() -- extracts the next item from the iterator and calls
remove() to remove it.
- andThen(final Iterator<X> other)-- <X extends T> uses an IteratorChain
to create a chain of iterators.
- filter(final Predicate<T> predicate) -- uses a FilterIterator to
exclude the items from the iterator.
- ExtendedIterator<U> map(final Function<T, U> function) -- uses a
TransformerIterator to transform the items to a new type.
- <U extends Collection<T>> U addTo(final U collection) -- Adds the
remaining items in the iterator to the collection.
The advantage of the iterator is the idiomatic nature.
Iterator<Thing> iter;
List<OtherThing> lst =
ExtendedIteratot.create(iter).filter(Thing::isValid).map(thing -> new
OtherThing(thing)).addTo(new ArrayList<>());
Claude
--
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: [DISCUSSs] new Iterator
Posted by Claude Warren <cl...@xenei.com>.
I can see a future enhancement (or iterator type) that merges iterators
together in a round-robin strategy. While I don't have a need for this at
the moment and so will not provide an implementation. Consider a system
where multiple polled IoT devices only report abnormal values (e.g. only
when it detects particles of 2.5microns in the air). If the polling of
those devices were fronted by an iterator then a merging iterator that will
scan over a collection of iterators and stop on the first one that
indicates it has data would be very useful in a dashboard application.
Claude
On Sun, Oct 20, 2024 at 9:01 AM Claude Warren <cl...@xenei.com> wrote:
> @Gary Gregory <ga...@gmail.com>
>
> in response to your comments on "ofStream". I was thinking of renaming it
> to "fromStream" but then realized that it is really just another
> implementation of "create" so ExtendedIterator<T> create(Stream<T> stream)
> is what I am proposing now.
>
> In response to the comments on `addTo` Streams collect build instances of
> whatever they are putting the objects into. This method adds the objects
> to an existing collection. I have often found that when working with
> Streams I need to collect the objects into a container and then add them
> all to another container. This shortens that process.
>
> Finally, why not just use streams.
>
> - All instances of Streams that I have seen require a collection of
> the object being streamed. Thus all instances have to be in memory before
> the stream starts.
> - There are numerous performance tests that show that streams have a
> very high overhead for creation. Several projects prohibit the use of
> streams on the hot path for performance reasons. However, iterators are
> allowed.
> - There are cases where you just don't know how many of something
> there are, or there may be an unbounded number of objects (think streaming
> systems). My current example is reading from something like an S3 bucket
> where you make a REST call to get a list of objects. The status from the
> REST call tells you if there are more objects to process. Wrapping the
> code to create an iterator over the complete collection of objects is
> fairly trivial (lazy collection of iterator approach). Now you have an
> iterator that just handles getting all the objects from storage. As an
> added bonus (or problem if you are a stickler for iterators failing if the
> underlying collection changes) is that you can continue to poll the
> iterator after it has returnd false to "hasNext()" and if a new object is
> placed in storage it will find it and retrieve it.
>
>
>
>
>
> On Sat, Oct 19, 2024 at 11:19 PM Claude Warren <cl...@xenei.com> wrote:
>
>> @mdrob
>>
>> The reason to use an Iterator is that, from my experience, Stream
>> requires all the base objects to be in existence. In addition there are
>> times when you don't know how many objects there will be or that you have
>> an iterator or iterable to start with and not a stream. An example would
>> be reading from something like an S3 bucket where you make a call to get a
>> list of object summaries. The list of object summaries may not be complete
>> and so you have to make more calls to the API to get more and more object
>> summaries. This is very easy to wrap with an iterator that will allow you
>> to get all the object summaries and then you can do stream-like processing
>> on the object summaries to process all the files in the S3 bucket.
>>
>> @Gary Gregory <ga...@gmail.com>
>>
>> I have an implementation of this class in the a pull request for Rat
>> 0.17. It is effectively a wrapper on some of the other iterator classes in
>> commons-collections. With your example of how to do the
>> unwinding/flattening iterator I can now rewrite it so that it does not need
>> any other new classes in commons-collections.
>>
>> I am open to renaming methods as appropriate. I will put together a pull
>> request on Sunday that shows how to implement the methods and will make
>> method name changes as per your recommendations above.
>>
>> Claude
>>
>
>
> --
> LinkedIn: http://www.linkedin.com/in/claudewarren
>
--
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: [DISCUSSs] new Iterator
Posted by Claude Warren <cl...@xenei.com>.
@Gary Gregory <ga...@gmail.com>
in response to your comments on "ofStream". I was thinking of renaming it
to "fromStream" but then realized that it is really just another
implementation of "create" so ExtendedIterator<T> create(Stream<T> stream)
is what I am proposing now.
In response to the comments on `addTo` Streams collect build instances of
whatever they are putting the objects into. This method adds the objects
to an existing collection. I have often found that when working with
Streams I need to collect the objects into a container and then add them
all to another container. This shortens that process.
Finally, why not just use streams.
- All instances of Streams that I have seen require a collection of the
object being streamed. Thus all instances have to be in memory before the
stream starts.
- There are numerous performance tests that show that streams have a
very high overhead for creation. Several projects prohibit the use of
streams on the hot path for performance reasons. However, iterators are
allowed.
- There are cases where you just don't know how many of something there
are, or there may be an unbounded number of objects (think streaming
systems). My current example is reading from something like an S3 bucket
where you make a REST call to get a list of objects. The status from the
REST call tells you if there are more objects to process. Wrapping the
code to create an iterator over the complete collection of objects is
fairly trivial (lazy collection of iterator approach). Now you have an
iterator that just handles getting all the objects from storage. As an
added bonus (or problem if you are a stickler for iterators failing if the
underlying collection changes) is that you can continue to poll the
iterator after it has returnd false to "hasNext()" and if a new object is
placed in storage it will find it and retrieve it.
On Sat, Oct 19, 2024 at 11:19 PM Claude Warren <cl...@xenei.com> wrote:
> @mdrob
>
> The reason to use an Iterator is that, from my experience, Stream requires
> all the base objects to be in existence. In addition there are times when
> you don't know how many objects there will be or that you have an iterator
> or iterable to start with and not a stream. An example would be reading
> from something like an S3 bucket where you make a call to get a list of
> object summaries. The list of object summaries may not be complete and so
> you have to make more calls to the API to get more and more object
> summaries. This is very easy to wrap with an iterator that will allow you
> to get all the object summaries and then you can do stream-like processing
> on the object summaries to process all the files in the S3 bucket.
>
> @Gary Gregory <ga...@gmail.com>
>
> I have an implementation of this class in the a pull request for Rat
> 0.17. It is effectively a wrapper on some of the other iterator classes in
> commons-collections. With your example of how to do the
> unwinding/flattening iterator I can now rewrite it so that it does not need
> any other new classes in commons-collections.
>
> I am open to renaming methods as appropriate. I will put together a pull
> request on Sunday that shows how to implement the methods and will make
> method name changes as per your recommendations above.
>
> Claude
>
--
LinkedIn: http://www.linkedin.com/in/claudewarren
Re: [DISCUSSs] new Iterator
Posted by Claude Warren <cl...@xenei.com>.
@mdrob
The reason to use an Iterator is that, from my experience, Stream requires
all the base objects to be in existence. In addition there are times when
you don't know how many objects there will be or that you have an iterator
or iterable to start with and not a stream. An example would be reading
from something like an S3 bucket where you make a call to get a list of
object summaries. The list of object summaries may not be complete and so
you have to make more calls to the API to get more and more object
summaries. This is very easy to wrap with an iterator that will allow you
to get all the object summaries and then you can do stream-like processing
on the object summaries to process all the files in the S3 bucket.
@Gary Gregory <ga...@gmail.com>
I have an implementation of this class in the a pull request for Rat 0.17.
It is effectively a wrapper on some of the other iterator classes in
commons-collections. With your example of how to do the
unwinding/flattening iterator I can now rewrite it so that it does not need
any other new classes in commons-collections.
I am open to renaming methods as appropriate. I will put together a pull
request on Sunday that shows how to implement the methods and will make
method name changes as per your recommendations above.
Claude
Re: [DISCUSSs] new Iterator
Posted by "Gary D. Gregory" <gg...@apache.org>.
On 2024/10/19 18:44:17 "Gary D. Gregory" wrote:
> On 2024/10/19 16:16:09 "Gary D. Gregory" wrote:
> >
> > On 2024/10/19 16:12:45 "Gary D. Gregory" wrote:
> > > Hi Claude,
> > >
> > > Interesting stuff :-)
> > >
> > > I'll assume that you intend this discussion for Commons Collections. Note that there are some stream/iterator mix utilities in Commons Lang in the org.apache.commons.lang3.stream.Streams class.
> > >
> > > A few comments below.
> > >
> > > On 2024/10/19 10:15:38 Claude Warren wrote:
> > > > I would like to introduce a new Iterator to the iterator library:
> > > > ExtendedIterator
> > > >
> > > > This Iterator is an Iterator that extends the functionality of other
> > > > iterators and makes using some iterators in the library more idiomatic. It
> > > > is based on the ExtendedIterator found in the new Apache Rat core package
> > > > that was inspired by the Apache Jena ExtendedIterator interface.
> > > >
> > > > I propose a class ExtendedIterator<T> that extends Iterator<T> and has the
> > > > following additional methods.
> > >
> > > Assuming that other calls are delegated to the underlying Iterator, a name that matches the JDK that might be better is "FilterIterator", much like what a FilterInputStream does for an InputStream.
> >
> > Maybe there a plain FilterIterator like FilterInputStream and a FilterIterator subclass that has the additional methods, not sure.
>
> Doh, we already have org.apache.commons.collections4.iterators.FilterIterator.
FilterIterator suports Predicates. I would assume that this new 'extended' functionality would either extend FilterIterator or add new functionality to it.
Gary
>
> Gary
>
> >
> > Gary
> >
> > >
> > > >
> > > > Static methods (all methods return an ExtendedIterator<T>)
> > > >
> > > > - createNoRemove(final Iterator<T> it) -- prohibits removal on an
> > > > iterator that otherwise would allow removal.
> > >
> > > You'll need to specify what this means: Does the turn the call into a noop or provides the default Iterator implementation, throwing a UnsupportedOperationException
> > >
> > > > - ofStream(final Stream<T> stream) -- convenience method to create
> > > > ExtendedIterator from stream.
> > >
> > > I don't think the name should "double-up" the words "of" and "Stream". Either follow the JDKs, example of java.util.Arrays.stream(...) where the target type is the method name, so in this case "iterator", or since we know we're creating an Iterator, just "of", like java.util.stream.Stream.of(T...) and org.apache.commons.lang3.stream.Streams.of(Iterator<E>)
> > >
> > > > - create(final Iterator<T> it)-- creates an ExtendedIterator from a
> > > > plain iterator. If the Iterator is already "extended" just return it.
> > > > - emptyIterator() -- convenience method to creat an empty
> > > > ExtendedIterator.
> > > > - unwind(final Iterator<Iterator<T>> woundIterator) -- creates an
> > > > Iterator<T> by iterating through each of the iterators in "woundIterator"
> > >
> > > The wind/unwind terminology does not make sense to me. I think we should follow what the JDK does with Stream and call this operation "flat" or "flatten", following Stream.flatMap(Function) and other Stream "flat" methods.
> > >
> > > >
> > > > Instance methods (Unless otherwise noted all methods return an
> > > > ExtendedIterator<T>)
> > > >
> > > > - T removeNext() -- extracts the next item from the iterator and calls
> > > > remove() to remove it.
> > > > - andThen(final Iterator<X> other)-- <X extends T> uses an IteratorChain
> > > > to create a chain of iterators.
> > > > - filter(final Predicate<T> predicate) -- uses a FilterIterator to
> > > > exclude the items from the iterator.
> > > > - ExtendedIterator<U> map(final Function<T, U> function) -- uses a
> > > > TransformerIterator to transform the items to a new type.
> > > > - <U extends Collection<T>> U addTo(final U collection) -- Adds the
> > > > remaining items in the iterator to the collection.
> > > >
> > > > The advantage of the iterator is the idiomatic nature.
> > > >
> > > > Iterator<Thing> iter;
> > > > List<OtherThing> lst =
> > > > ExtendedIteratot.create(iter).filter(Thing::isValid).map(thing -> new
> > > > OtherThing(thing)).addTo(new ArrayList<>());
> > >
> > > Note that Streams use the "collect" name for this type of operation.
> > >
> > > I am somewhat concerned that this proposal is implementing what already in Streams but with Iterator. It might just be simpler to use Streams, _especially_ considering your last example, which is very stream-like.
> > >
> > > TY!
> > > Gary
> > >
> > > >
> > > > Claude
> > > > --
> > > > LinkedIn: http://www.linkedin.com/in/claudewarren
> > > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > > For additional commands, e-mail: dev-help@commons.apache.org
> > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [DISCUSSs] new Iterator
Posted by "Gary D. Gregory" <gg...@apache.org>.
On 2024/10/19 16:16:09 "Gary D. Gregory" wrote:
>
> On 2024/10/19 16:12:45 "Gary D. Gregory" wrote:
> > Hi Claude,
> >
> > Interesting stuff :-)
> >
> > I'll assume that you intend this discussion for Commons Collections. Note that there are some stream/iterator mix utilities in Commons Lang in the org.apache.commons.lang3.stream.Streams class.
> >
> > A few comments below.
> >
> > On 2024/10/19 10:15:38 Claude Warren wrote:
> > > I would like to introduce a new Iterator to the iterator library:
> > > ExtendedIterator
> > >
> > > This Iterator is an Iterator that extends the functionality of other
> > > iterators and makes using some iterators in the library more idiomatic. It
> > > is based on the ExtendedIterator found in the new Apache Rat core package
> > > that was inspired by the Apache Jena ExtendedIterator interface.
> > >
> > > I propose a class ExtendedIterator<T> that extends Iterator<T> and has the
> > > following additional methods.
> >
> > Assuming that other calls are delegated to the underlying Iterator, a name that matches the JDK that might be better is "FilterIterator", much like what a FilterInputStream does for an InputStream.
>
> Maybe there a plain FilterIterator like FilterInputStream and a FilterIterator subclass that has the additional methods, not sure.
Doh, we already have org.apache.commons.collections4.iterators.FilterIterator.
Gary
>
> Gary
>
> >
> > >
> > > Static methods (all methods return an ExtendedIterator<T>)
> > >
> > > - createNoRemove(final Iterator<T> it) -- prohibits removal on an
> > > iterator that otherwise would allow removal.
> >
> > You'll need to specify what this means: Does the turn the call into a noop or provides the default Iterator implementation, throwing a UnsupportedOperationException
> >
> > > - ofStream(final Stream<T> stream) -- convenience method to create
> > > ExtendedIterator from stream.
> >
> > I don't think the name should "double-up" the words "of" and "Stream". Either follow the JDKs, example of java.util.Arrays.stream(...) where the target type is the method name, so in this case "iterator", or since we know we're creating an Iterator, just "of", like java.util.stream.Stream.of(T...) and org.apache.commons.lang3.stream.Streams.of(Iterator<E>)
> >
> > > - create(final Iterator<T> it)-- creates an ExtendedIterator from a
> > > plain iterator. If the Iterator is already "extended" just return it.
> > > - emptyIterator() -- convenience method to creat an empty
> > > ExtendedIterator.
> > > - unwind(final Iterator<Iterator<T>> woundIterator) -- creates an
> > > Iterator<T> by iterating through each of the iterators in "woundIterator"
> >
> > The wind/unwind terminology does not make sense to me. I think we should follow what the JDK does with Stream and call this operation "flat" or "flatten", following Stream.flatMap(Function) and other Stream "flat" methods.
> >
> > >
> > > Instance methods (Unless otherwise noted all methods return an
> > > ExtendedIterator<T>)
> > >
> > > - T removeNext() -- extracts the next item from the iterator and calls
> > > remove() to remove it.
> > > - andThen(final Iterator<X> other)-- <X extends T> uses an IteratorChain
> > > to create a chain of iterators.
> > > - filter(final Predicate<T> predicate) -- uses a FilterIterator to
> > > exclude the items from the iterator.
> > > - ExtendedIterator<U> map(final Function<T, U> function) -- uses a
> > > TransformerIterator to transform the items to a new type.
> > > - <U extends Collection<T>> U addTo(final U collection) -- Adds the
> > > remaining items in the iterator to the collection.
> > >
> > > The advantage of the iterator is the idiomatic nature.
> > >
> > > Iterator<Thing> iter;
> > > List<OtherThing> lst =
> > > ExtendedIteratot.create(iter).filter(Thing::isValid).map(thing -> new
> > > OtherThing(thing)).addTo(new ArrayList<>());
> >
> > Note that Streams use the "collect" name for this type of operation.
> >
> > I am somewhat concerned that this proposal is implementing what already in Streams but with Iterator. It might just be simpler to use Streams, _especially_ considering your last example, which is very stream-like.
> >
> > TY!
> > Gary
> >
> > >
> > > Claude
> > > --
> > > LinkedIn: http://www.linkedin.com/in/claudewarren
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> > For additional commands, e-mail: dev-help@commons.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [DISCUSSs] new Iterator
Posted by "Gary D. Gregory" <gg...@apache.org>.
On 2024/10/19 16:12:45 "Gary D. Gregory" wrote:
> Hi Claude,
>
> Interesting stuff :-)
>
> I'll assume that you intend this discussion for Commons Collections. Note that there are some stream/iterator mix utilities in Commons Lang in the org.apache.commons.lang3.stream.Streams class.
>
> A few comments below.
>
> On 2024/10/19 10:15:38 Claude Warren wrote:
> > I would like to introduce a new Iterator to the iterator library:
> > ExtendedIterator
> >
> > This Iterator is an Iterator that extends the functionality of other
> > iterators and makes using some iterators in the library more idiomatic. It
> > is based on the ExtendedIterator found in the new Apache Rat core package
> > that was inspired by the Apache Jena ExtendedIterator interface.
> >
> > I propose a class ExtendedIterator<T> that extends Iterator<T> and has the
> > following additional methods.
>
> Assuming that other calls are delegated to the underlying Iterator, a name that matches the JDK that might be better is "FilterIterator", much like what a FilterInputStream does for an InputStream.
Maybe there a plain FilterIterator like FilterInputStream and a FilterIterator subclass that has the additional methods, not sure.
Gary
>
> >
> > Static methods (all methods return an ExtendedIterator<T>)
> >
> > - createNoRemove(final Iterator<T> it) -- prohibits removal on an
> > iterator that otherwise would allow removal.
>
> You'll need to specify what this means: Does the turn the call into a noop or provides the default Iterator implementation, throwing a UnsupportedOperationException
>
> > - ofStream(final Stream<T> stream) -- convenience method to create
> > ExtendedIterator from stream.
>
> I don't think the name should "double-up" the words "of" and "Stream". Either follow the JDKs, example of java.util.Arrays.stream(...) where the target type is the method name, so in this case "iterator", or since we know we're creating an Iterator, just "of", like java.util.stream.Stream.of(T...) and org.apache.commons.lang3.stream.Streams.of(Iterator<E>)
>
> > - create(final Iterator<T> it)-- creates an ExtendedIterator from a
> > plain iterator. If the Iterator is already "extended" just return it.
> > - emptyIterator() -- convenience method to creat an empty
> > ExtendedIterator.
> > - unwind(final Iterator<Iterator<T>> woundIterator) -- creates an
> > Iterator<T> by iterating through each of the iterators in "woundIterator"
>
> The wind/unwind terminology does not make sense to me. I think we should follow what the JDK does with Stream and call this operation "flat" or "flatten", following Stream.flatMap(Function) and other Stream "flat" methods.
>
> >
> > Instance methods (Unless otherwise noted all methods return an
> > ExtendedIterator<T>)
> >
> > - T removeNext() -- extracts the next item from the iterator and calls
> > remove() to remove it.
> > - andThen(final Iterator<X> other)-- <X extends T> uses an IteratorChain
> > to create a chain of iterators.
> > - filter(final Predicate<T> predicate) -- uses a FilterIterator to
> > exclude the items from the iterator.
> > - ExtendedIterator<U> map(final Function<T, U> function) -- uses a
> > TransformerIterator to transform the items to a new type.
> > - <U extends Collection<T>> U addTo(final U collection) -- Adds the
> > remaining items in the iterator to the collection.
> >
> > The advantage of the iterator is the idiomatic nature.
> >
> > Iterator<Thing> iter;
> > List<OtherThing> lst =
> > ExtendedIteratot.create(iter).filter(Thing::isValid).map(thing -> new
> > OtherThing(thing)).addTo(new ArrayList<>());
>
> Note that Streams use the "collect" name for this type of operation.
>
> I am somewhat concerned that this proposal is implementing what already in Streams but with Iterator. It might just be simpler to use Streams, _especially_ considering your last example, which is very stream-like.
>
> TY!
> Gary
>
> >
> > Claude
> > --
> > LinkedIn: http://www.linkedin.com/in/claudewarren
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [DISCUSSs] new Iterator
Posted by "Gary D. Gregory" <gg...@apache.org>.
Hi Claude,
Interesting stuff :-)
I'll assume that you intend this discussion for Commons Collections. Note that there are some stream/iterator mix utilities in Commons Lang in the org.apache.commons.lang3.stream.Streams class.
A few comments below.
On 2024/10/19 10:15:38 Claude Warren wrote:
> I would like to introduce a new Iterator to the iterator library:
> ExtendedIterator
>
> This Iterator is an Iterator that extends the functionality of other
> iterators and makes using some iterators in the library more idiomatic. It
> is based on the ExtendedIterator found in the new Apache Rat core package
> that was inspired by the Apache Jena ExtendedIterator interface.
>
> I propose a class ExtendedIterator<T> that extends Iterator<T> and has the
> following additional methods.
Assuming that other calls are delegated to the underlying Iterator, a name that matches the JDK that might be better is "FilterIterator", much like what a FilterInputStream does for an InputStream.
>
> Static methods (all methods return an ExtendedIterator<T>)
>
> - createNoRemove(final Iterator<T> it) -- prohibits removal on an
> iterator that otherwise would allow removal.
You'll need to specify what this means: Does the turn the call into a noop or provides the default Iterator implementation, throwing a UnsupportedOperationException
> - ofStream(final Stream<T> stream) -- convenience method to create
> ExtendedIterator from stream.
I don't think the name should "double-up" the words "of" and "Stream". Either follow the JDKs, example of java.util.Arrays.stream(...) where the target type is the method name, so in this case "iterator", or since we know we're creating an Iterator, just "of", like java.util.stream.Stream.of(T...) and org.apache.commons.lang3.stream.Streams.of(Iterator<E>)
> - create(final Iterator<T> it)-- creates an ExtendedIterator from a
> plain iterator. If the Iterator is already "extended" just return it.
> - emptyIterator() -- convenience method to creat an empty
> ExtendedIterator.
> - unwind(final Iterator<Iterator<T>> woundIterator) -- creates an
> Iterator<T> by iterating through each of the iterators in "woundIterator"
The wind/unwind terminology does not make sense to me. I think we should follow what the JDK does with Stream and call this operation "flat" or "flatten", following Stream.flatMap(Function) and other Stream "flat" methods.
>
> Instance methods (Unless otherwise noted all methods return an
> ExtendedIterator<T>)
>
> - T removeNext() -- extracts the next item from the iterator and calls
> remove() to remove it.
> - andThen(final Iterator<X> other)-- <X extends T> uses an IteratorChain
> to create a chain of iterators.
> - filter(final Predicate<T> predicate) -- uses a FilterIterator to
> exclude the items from the iterator.
> - ExtendedIterator<U> map(final Function<T, U> function) -- uses a
> TransformerIterator to transform the items to a new type.
> - <U extends Collection<T>> U addTo(final U collection) -- Adds the
> remaining items in the iterator to the collection.
>
> The advantage of the iterator is the idiomatic nature.
>
> Iterator<Thing> iter;
> List<OtherThing> lst =
> ExtendedIteratot.create(iter).filter(Thing::isValid).map(thing -> new
> OtherThing(thing)).addTo(new ArrayList<>());
Note that Streams use the "collect" name for this type of operation.
I am somewhat concerned that this proposal is implementing what already in Streams but with Iterator. It might just be simpler to use Streams, _especially_ considering your last example, which is very stream-like.
TY!
Gary
>
> Claude
> --
> LinkedIn: http://www.linkedin.com/in/claudewarren
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: [DISCUSSs] new Iterator
Posted by Mike Drob <md...@mdrob.com>.
What is the advantage of using an ExtendedIterator to
filter/chain/map/flatten instead of using Stream to do the same?
On Sat, Oct 19, 2024 at 5:16 AM Claude Warren <cl...@xenei.com> wrote:
> I would like to introduce a new Iterator to the iterator library:
> ExtendedIterator
>
> This Iterator is an Iterator that extends the functionality of other
> iterators and makes using some iterators in the library more idiomatic. It
> is based on the ExtendedIterator found in the new Apache Rat core package
> that was inspired by the Apache Jena ExtendedIterator interface.
>
> I propose a class ExtendedIterator<T> that extends Iterator<T> and has the
> following additional methods.
>
> Static methods (all methods return an ExtendedIterator<T>)
>
> - createNoRemove(final Iterator<T> it) -- prohibits removal on an
> iterator that otherwise would allow removal.
> - ofStream(final Stream<T> stream) -- convenience method to create
> ExtendedIterator from stream.
> - create(final Iterator<T> it)-- creates an ExtendedIterator from a
> plain iterator. If the Iterator is already "extended" just return it.
> - emptyIterator() -- convenience method to creat an empty
> ExtendedIterator.
> - unwind(final Iterator<Iterator<T>> woundIterator) -- creates an
> Iterator<T> by iterating through each of the iterators in
> "woundIterator"
>
> Instance methods (Unless otherwise noted all methods return an
> ExtendedIterator<T>)
>
> - T removeNext() -- extracts the next item from the iterator and calls
> remove() to remove it.
> - andThen(final Iterator<X> other)-- <X extends T> uses an IteratorChain
> to create a chain of iterators.
> - filter(final Predicate<T> predicate) -- uses a FilterIterator to
> exclude the items from the iterator.
> - ExtendedIterator<U> map(final Function<T, U> function) -- uses a
> TransformerIterator to transform the items to a new type.
> - <U extends Collection<T>> U addTo(final U collection) -- Adds the
> remaining items in the iterator to the collection.
>
> The advantage of the iterator is the idiomatic nature.
>
> Iterator<Thing> iter;
> List<OtherThing> lst =
> ExtendedIteratot.create(iter).filter(Thing::isValid).map(thing -> new
> OtherThing(thing)).addTo(new ArrayList<>());
>
> Claude
> --
> LinkedIn: http://www.linkedin.com/in/claudewarren
>