You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Niels Basjes <Ni...@basjes.nl> on 2020/07/09 12:21:27 UTC
Improvement idea: Naming the maven modules.
Hi,
I have the idea that naming the modules would make life easier for the
developers.
My main question: Do you agree?
If so I'll clean up my experiment (see below) and put up a pull request.
When I load the Flink code into IntelliJ I currently see this list of Maven
modules:
[image: image.png]
Now what happens is that the name of the maven module is in almost all
cases the name of the artifact.
Only one example in the current code base that deviates from this
(highlighted in the screenshot).
This list is shown by IntelliJ in alphabetical order.
I propose to give all modules a name that should make it a lot easier for
the developers to find the module they are looking for.
I've been playing with this idea and what I have now (local only) looks
like this:
[image: image.png]
They are still alphabetically sorted but because of the way I chose the
naming it becomes much easier to find the appropriate module.
Also this causes the build output from Maven to become more readable
(below).
First thing I noticed from this is that apparently the ordering of the
modules and their inter dependencies is off in certain places.
[INFO] Flink : Table : API Java bridge .................... SUCCESS [
0.004 s]
[INFO] Flink : Table : API Scala .......................... SUCCESS [
0.004 s]
[INFO] Flink : Table : API Scala bridge ................... SUCCESS [
0.004 s]
[INFO] Flink : Table : SQL Parser ......................... SUCCESS [
0.004 s]
[INFO] Flink : Libraries : ................................ SUCCESS [
0.002 s]
[INFO] Flink : Libraries : CEP ............................ SUCCESS [
0.003 s]
[INFO] Flink : Table : Planner ............................ SUCCESS [
0.003 s]
[INFO] Flink : Table : SQL Parser Hive .................... SUCCESS [
0.002 s]
[INFO] Flink : Table : Runtime Blink ...................... SUCCESS [
0.003 s]
[INFO] Flink : Table : Planner Blink ...................... SUCCESS [
0.004 s]
[INFO] Flink : Metrics : JMX .............................. SUCCESS [
0.004 s]
[INFO] Flink : Formats : .................................. SUCCESS [
0.004 s]
[INFO] Flink : Formats : Json ............................. SUCCESS [
0.004 s]
[INFO] Flink : Connectors : Kafka base .................... SUCCESS [
0.003 s]
[INFO] Flink : Formats : Avro ............................. SUCCESS [
0.005 s]
[INFO] Flink : Formats : Csv .............................. SUCCESS [
0.003 s]
[INFO] Flink : Connectors : Kafka 0.10 .................... SUCCESS [
0.009 s]
[INFO] Flink : Connectors : Kafka 0.11 .................... SUCCESS [
0.003 s]
[INFO] Flink : Connectors : Elasticsearch base ............ SUCCESS [
0.003 s]
[INFO] Flink : Connectors : Elasticsearch 5 ............... SUCCESS [
0.003 s]
[INFO] Flink : Connectors : Elasticsearch 6 ............... SUCCESS [
0.003 s]
[INFO] Flink : Connectors : Elasticsearch 7 ............... SUCCESS [
0.003 s]
[INFO] Flink : Connectors : HBase ......................... SUCCESS [
0.003 s]
[INFO] Flink : Formats : Hadoop bulk ...................... SUCCESS [
0.002 s]
[INFO] Flink : Formats : Orc .............................. SUCCESS [
0.003 s]
[INFO] Flink : Formats : Orc nohive ....................... SUCCESS [
0.002 s]
[INFO] Flink : Formats : Parquet .......................... SUCCESS [
0.003 s]
[INFO] Flink : Connectors : Hive .......................... SUCCESS [
0.002 s]
[INFO] Flink : Connectors : JDBC .......................... SUCCESS [
0.002 s]
[INFO] Flink : Connectors : RabbitMQ ...................... SUCCESS [
0.002 s]
[INFO] Flink : Connectors : Twitter ....................... SUCCESS [
0.002 s]
[INFO] Flink : Connectors : Nifi .......................... SUCCESS [
0.004 s]
[INFO] Flink : Connectors : Cassandra ..................... SUCCESS [
0.002 s]
[INFO] Flink : Connectors : Filesystem .................... SUCCESS [
0.002 s]
[INFO] Flink : Connectors : Kafka ......................... SUCCESS [
0.003 s]
[INFO] Flink : Connectors : Google PubSub ................. SUCCESS [
0.035 s]
[INFO] Flink : Connectors : Kinesis ....................... SUCCESS [
0.002 s]
[INFO] Flink : Connectors : Base .......................... SUCCESS [
0.003 s]
[INFO] Flink : Connectors : SQL : Elasticsearch 6 ......... SUCCESS [
0.002 s]
[INFO] Flink : Connectors : SQL : Elasticsearch 7 ......... SUCCESS [
0.002 s]
[INFO] Flink : Connectors : SQL : HBase ................... SUCCESS [
0.002 s]
[INFO] Flink : Connectors : SQL : Hive 1.2.2 .............. SUCCESS [
0.002 s]
[INFO] Flink : Connectors : SQL : Hive 2.2.0 .............. SUCCESS [
0.002 s]
[INFO] Flink : Connectors : SQL : Hive 2.3.6 .............. SUCCESS [
0.002 s]
[INFO] Flink : Connectors : SQL : Hive 3.1.2 .............. SUCCESS [
0.002 s]
[INFO] Flink : Connectors : SQL : Kafka 0.10 .............. SUCCESS [
0.002 s]
[INFO] Flink : Connectors : SQL : Kafka 0.11 .............. SUCCESS [
0.002 s]
[INFO] Flink : Connectors : SQL : Kafka ................... SUCCESS [
0.002 s]
[INFO] Flink : Formats : Avro confluent registry .......... SUCCESS [
0.002 s]
[INFO] Flink : Formats : Sequence file .................... SUCCESS [
0.002 s]
[INFO] Flink : Formats : Compress ......................... SUCCESS [
0.002 s]
[INFO] Flink : Formats : SQL Orc .......................... SUCCESS [
0.002 s]
[INFO] Flink : Formats : SQL Parquet ...................... SUCCESS [
0.002 s]
[INFO] Flink : Examples : Streaming ....................... SUCCESS [
0.002 s]
[INFO] Flink : Examples : Table ........................... SUCCESS [
0.003 s]
[INFO] Flink : Examples : Build-helper .................... SUCCESS [
0.003 s]
[INFO] Flink : Examples : Streaming Twitter ............... SUCCESS [
0.002 s]
[INFO] Flink : Examples : Streaming State machine ......... SUCCESS [
0.002 s]
[INFO] Flink : Examples : Streaming Google PubSub ......... SUCCESS [
0.002 s]
[INFO] Flink : Container .................................. SUCCESS [
0.002 s]
[INFO] Flink : Queryable state : Runtime .................. SUCCESS [
0.002 s]
[INFO] Flink : Mesos ...................................... SUCCESS [
0.002 s]
[INFO] Flink : Kubernetes ................................. SUCCESS [
0.002 s]
[INFO] Flink : Yarn ....................................... SUCCESS [
0.002 s]
[INFO] Flink : Libraries : Gelly .......................... SUCCESS [
0.002 s]
--
Best regards / Met vriendelijke groeten,
Niels Basjes
Re: Improvement idea: Naming the maven modules.
Posted by Kurt Young <yk...@gmail.com>.
+1, I also like this idea.
Best,
Kurt
On Wed, Jul 15, 2020 at 7:10 PM Niels Basjes <Ni...@basjes.nl> wrote:
> Ok,
>
> I'll put up a fix
> https://issues.apache.org/jira/browse/FLINK-18607
>
> Niels
>
> On Wed, Jul 15, 2020 at 11:23 AM Aljoscha Krettek <al...@apache.org>
> wrote:
>
> > Hi,
> >
> > I like the proposal! I remember that Beam also had more human-readable
> > names for the modules and found that helpful. Also, changing the names
> > shouldn't change anything for users because dependencies are referred to
> by
> > group/artifactId, it really just makes build output and IDE a bit more
> > parseable by humans.
> >
> > Best,
> > Aljoscha
> >
> > On Thu, Jul 9, 2020, at 16:03, Niels Basjes wrote:
> > > Yes,
> > > The tree of modules (usually on the left) shows the maven modules as a
> > tree.
> > > Yet I find that not very easy to use because of the readability of the
> > artifact names and because this becomes very wide (i.e. takes up a lot of
> > screen space).
> > > Having the maven modules with a name you'll get much better readable
> > names in the Maven page (usually on the right).
> > >
> > > Like I said: It's just an idea and in my opinion it makes certain parts
> > of the build more readable.
> > >
> > > Niels
> > >
> > >
> > > On Thu, Jul 9, 2020 at 2:55 PM Chesnay Schepler <ch...@apache.org>
> > wrote:
> > >> Couldn't you just use the tree view and get basically the same thing?
> > >>
> > >> On 09/07/2020 14:51, Niels Basjes wrote:
> > >>> Attempt 2 to get you the images.
> > >>> Now as attachments.
> > >>>
> > >>> Niels
> > >>>
> > >>> On Thu, Jul 9, 2020 at 2:38 PM Chesnay Schepler <ch...@apache.org>
> > wrote:
> > >>>> The images didn't go through.
> > >>>>
> > >>>> On 09/07/2020 14:21, Niels Basjes wrote:
> > >>>>> Hi,
> > >>>>>
> > >>>>> I have the idea that naming the modules would make life easier for
> > the developers.
> > >>>>> My main question: Do you agree?
> > >>>>> If so I'll clean up my experiment (see below) and put up a pull
> > request.
> > >>>>>
> > >>>>> When I load the Flink code into IntelliJ I currently see this list
> > of Maven modules:
> > >>>>>
> > >>>>> image.png
> > >>>>>
> > >>>>> Now what happens is that the name of the maven module is in almost
> > all cases the name of the artifact.
> > >>>>> Only one example in the current code base that deviates from this
> > (highlighted in the screenshot).
> > >>>>> This list is shown by IntelliJ in alphabetical order.
> > >>>>>
> > >>>>> I propose to give all modules a name that should make it a lot
> > easier for the developers to find the module they are looking for.
> > >>>>> I've been playing with this idea and what I have now (local only)
> > looks like this:
> > >>>>>
> > >>>>> image.png
> > >>>>>
> > >>>>> They are still alphabetically sorted but because of the way I chose
> > the naming it becomes much easier to find the appropriate module.
> > >>>>>
> > >>>>> Also this causes the build output from Maven to become more
> readable
> > (below).
> > >>>>> First thing I noticed from this is that apparently the ordering of
> > the modules and their inter dependencies is off in certain places.
> > >>>>>
> > >>>>>
> > >>>>> [INFO] Flink : Table : API Java bridge .................... SUCCESS
> > [ 0.004 s]
> > >>>>> [INFO] Flink : Table : API Scala .......................... SUCCESS
> > [ 0.004 s]
> > >>>>> [INFO] Flink : Table : API Scala bridge ................... SUCCESS
> > [ 0.004 s]
> > >>>>> [INFO] Flink : Table : SQL Parser ......................... SUCCESS
> > [ 0.004 s]
> > >>>>> [INFO] Flink : Libraries : ................................ SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Libraries : CEP ............................ SUCCESS
> > [ 0.003 s]
> > >>>>> [INFO] Flink : Table : Planner ............................ SUCCESS
> > [ 0.003 s]
> > >>>>> [INFO] Flink : Table : SQL Parser Hive .................... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Table : Runtime Blink ...................... SUCCESS
> > [ 0.003 s]
> > >>>>> [INFO] Flink : Table : Planner Blink ...................... SUCCESS
> > [ 0.004 s]
> > >>>>> [INFO] Flink : Metrics : JMX .............................. SUCCESS
> > [ 0.004 s]
> > >>>>> [INFO] Flink : Formats : .................................. SUCCESS
> > [ 0.004 s]
> > >>>>> [INFO] Flink : Formats : Json ............................. SUCCESS
> > [ 0.004 s]
> > >>>>> [INFO] Flink : Connectors : Kafka base .................... SUCCESS
> > [ 0.003 s]
> > >>>>> [INFO] Flink : Formats : Avro ............................. SUCCESS
> > [ 0.005 s]
> > >>>>> [INFO] Flink : Formats : Csv .............................. SUCCESS
> > [ 0.003 s]
> > >>>>> [INFO] Flink : Connectors : Kafka 0.10 .................... SUCCESS
> > [ 0.009 s]
> > >>>>> [INFO] Flink : Connectors : Kafka 0.11 .................... SUCCESS
> > [ 0.003 s]
> > >>>>> [INFO] Flink : Connectors : Elasticsearch base ............ SUCCESS
> > [ 0.003 s]
> > >>>>> [INFO] Flink : Connectors : Elasticsearch 5 ............... SUCCESS
> > [ 0.003 s]
> > >>>>> [INFO] Flink : Connectors : Elasticsearch 6 ............... SUCCESS
> > [ 0.003 s]
> > >>>>> [INFO] Flink : Connectors : Elasticsearch 7 ............... SUCCESS
> > [ 0.003 s]
> > >>>>> [INFO] Flink : Connectors : HBase ......................... SUCCESS
> > [ 0.003 s]
> > >>>>> [INFO] Flink : Formats : Hadoop bulk ...................... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Formats : Orc .............................. SUCCESS
> > [ 0.003 s]
> > >>>>> [INFO] Flink : Formats : Orc nohive ....................... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Formats : Parquet .......................... SUCCESS
> > [ 0.003 s]
> > >>>>> [INFO] Flink : Connectors : Hive .......................... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Connectors : JDBC .......................... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Connectors : RabbitMQ ...................... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Connectors : Twitter ....................... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Connectors : Nifi .......................... SUCCESS
> > [ 0.004 s]
> > >>>>> [INFO] Flink : Connectors : Cassandra ..................... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Connectors : Filesystem .................... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Connectors : Kafka ......................... SUCCESS
> > [ 0.003 s]
> > >>>>> [INFO] Flink : Connectors : Google PubSub ................. SUCCESS
> > [ 0.035 s]
> > >>>>> [INFO] Flink : Connectors : Kinesis ....................... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Connectors : Base .......................... SUCCESS
> > [ 0.003 s]
> > >>>>> [INFO] Flink : Connectors : SQL : Elasticsearch 6 ......... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Connectors : SQL : Elasticsearch 7 ......... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Connectors : SQL : HBase ................... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Connectors : SQL : Hive 1.2.2 .............. SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Connectors : SQL : Hive 2.2.0 .............. SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Connectors : SQL : Hive 2.3.6 .............. SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Connectors : SQL : Hive 3.1.2 .............. SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Connectors : SQL : Kafka 0.10 .............. SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Connectors : SQL : Kafka 0.11 .............. SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Connectors : SQL : Kafka ................... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Formats : Avro confluent registry .......... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Formats : Sequence file .................... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Formats : Compress ......................... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Formats : SQL Orc .......................... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Formats : SQL Parquet ...................... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Examples : Streaming ....................... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Examples : Table ........................... SUCCESS
> > [ 0.003 s]
> > >>>>> [INFO] Flink : Examples : Build-helper .................... SUCCESS
> > [ 0.003 s]
> > >>>>> [INFO] Flink : Examples : Streaming Twitter ............... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Examples : Streaming State machine ......... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Examples : Streaming Google PubSub ......... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Container .................................. SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Queryable state : Runtime .................. SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Mesos ...................................... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Kubernetes ................................. SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Yarn ....................................... SUCCESS
> > [ 0.002 s]
> > >>>>> [INFO] Flink : Libraries : Gelly .......................... SUCCESS
> > [ 0.002 s]
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>>
> > >>>>> Best regards / Met vriendelijke groeten,
> > >>>>>
> > >>>>> Niels Basjes
> > >>>>
> >
> > >>>
> > >>>
> > >>> --
> > >>>
> > >>> Best regards / Met vriendelijke groeten,
> > >>>
> > >>> Niels Basjes
> > >>
> >
> > >
> > >
> > > --
> > > Best regards / Met vriendelijke groeten,
> > >
> > > Niels Basjes
> >
>
>
> --
> Best regards / Met vriendelijke groeten,
>
> Niels Basjes
>
Re: Improvement idea: Naming the maven modules.
Posted by Niels Basjes <Ni...@basjes.nl>.
Ok,
I'll put up a fix
https://issues.apache.org/jira/browse/FLINK-18607
Niels
On Wed, Jul 15, 2020 at 11:23 AM Aljoscha Krettek <al...@apache.org>
wrote:
> Hi,
>
> I like the proposal! I remember that Beam also had more human-readable
> names for the modules and found that helpful. Also, changing the names
> shouldn't change anything for users because dependencies are referred to by
> group/artifactId, it really just makes build output and IDE a bit more
> parseable by humans.
>
> Best,
> Aljoscha
>
> On Thu, Jul 9, 2020, at 16:03, Niels Basjes wrote:
> > Yes,
> > The tree of modules (usually on the left) shows the maven modules as a
> tree.
> > Yet I find that not very easy to use because of the readability of the
> artifact names and because this becomes very wide (i.e. takes up a lot of
> screen space).
> > Having the maven modules with a name you'll get much better readable
> names in the Maven page (usually on the right).
> >
> > Like I said: It's just an idea and in my opinion it makes certain parts
> of the build more readable.
> >
> > Niels
> >
> >
> > On Thu, Jul 9, 2020 at 2:55 PM Chesnay Schepler <ch...@apache.org>
> wrote:
> >> Couldn't you just use the tree view and get basically the same thing?
> >>
> >> On 09/07/2020 14:51, Niels Basjes wrote:
> >>> Attempt 2 to get you the images.
> >>> Now as attachments.
> >>>
> >>> Niels
> >>>
> >>> On Thu, Jul 9, 2020 at 2:38 PM Chesnay Schepler <ch...@apache.org>
> wrote:
> >>>> The images didn't go through.
> >>>>
> >>>> On 09/07/2020 14:21, Niels Basjes wrote:
> >>>>> Hi,
> >>>>>
> >>>>> I have the idea that naming the modules would make life easier for
> the developers.
> >>>>> My main question: Do you agree?
> >>>>> If so I'll clean up my experiment (see below) and put up a pull
> request.
> >>>>>
> >>>>> When I load the Flink code into IntelliJ I currently see this list
> of Maven modules:
> >>>>>
> >>>>> image.png
> >>>>>
> >>>>> Now what happens is that the name of the maven module is in almost
> all cases the name of the artifact.
> >>>>> Only one example in the current code base that deviates from this
> (highlighted in the screenshot).
> >>>>> This list is shown by IntelliJ in alphabetical order.
> >>>>>
> >>>>> I propose to give all modules a name that should make it a lot
> easier for the developers to find the module they are looking for.
> >>>>> I've been playing with this idea and what I have now (local only)
> looks like this:
> >>>>>
> >>>>> image.png
> >>>>>
> >>>>> They are still alphabetically sorted but because of the way I chose
> the naming it becomes much easier to find the appropriate module.
> >>>>>
> >>>>> Also this causes the build output from Maven to become more readable
> (below).
> >>>>> First thing I noticed from this is that apparently the ordering of
> the modules and their inter dependencies is off in certain places.
> >>>>>
> >>>>>
> >>>>> [INFO] Flink : Table : API Java bridge .................... SUCCESS
> [ 0.004 s]
> >>>>> [INFO] Flink : Table : API Scala .......................... SUCCESS
> [ 0.004 s]
> >>>>> [INFO] Flink : Table : API Scala bridge ................... SUCCESS
> [ 0.004 s]
> >>>>> [INFO] Flink : Table : SQL Parser ......................... SUCCESS
> [ 0.004 s]
> >>>>> [INFO] Flink : Libraries : ................................ SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Libraries : CEP ............................ SUCCESS
> [ 0.003 s]
> >>>>> [INFO] Flink : Table : Planner ............................ SUCCESS
> [ 0.003 s]
> >>>>> [INFO] Flink : Table : SQL Parser Hive .................... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Table : Runtime Blink ...................... SUCCESS
> [ 0.003 s]
> >>>>> [INFO] Flink : Table : Planner Blink ...................... SUCCESS
> [ 0.004 s]
> >>>>> [INFO] Flink : Metrics : JMX .............................. SUCCESS
> [ 0.004 s]
> >>>>> [INFO] Flink : Formats : .................................. SUCCESS
> [ 0.004 s]
> >>>>> [INFO] Flink : Formats : Json ............................. SUCCESS
> [ 0.004 s]
> >>>>> [INFO] Flink : Connectors : Kafka base .................... SUCCESS
> [ 0.003 s]
> >>>>> [INFO] Flink : Formats : Avro ............................. SUCCESS
> [ 0.005 s]
> >>>>> [INFO] Flink : Formats : Csv .............................. SUCCESS
> [ 0.003 s]
> >>>>> [INFO] Flink : Connectors : Kafka 0.10 .................... SUCCESS
> [ 0.009 s]
> >>>>> [INFO] Flink : Connectors : Kafka 0.11 .................... SUCCESS
> [ 0.003 s]
> >>>>> [INFO] Flink : Connectors : Elasticsearch base ............ SUCCESS
> [ 0.003 s]
> >>>>> [INFO] Flink : Connectors : Elasticsearch 5 ............... SUCCESS
> [ 0.003 s]
> >>>>> [INFO] Flink : Connectors : Elasticsearch 6 ............... SUCCESS
> [ 0.003 s]
> >>>>> [INFO] Flink : Connectors : Elasticsearch 7 ............... SUCCESS
> [ 0.003 s]
> >>>>> [INFO] Flink : Connectors : HBase ......................... SUCCESS
> [ 0.003 s]
> >>>>> [INFO] Flink : Formats : Hadoop bulk ...................... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Formats : Orc .............................. SUCCESS
> [ 0.003 s]
> >>>>> [INFO] Flink : Formats : Orc nohive ....................... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Formats : Parquet .......................... SUCCESS
> [ 0.003 s]
> >>>>> [INFO] Flink : Connectors : Hive .......................... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Connectors : JDBC .......................... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Connectors : RabbitMQ ...................... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Connectors : Twitter ....................... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Connectors : Nifi .......................... SUCCESS
> [ 0.004 s]
> >>>>> [INFO] Flink : Connectors : Cassandra ..................... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Connectors : Filesystem .................... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Connectors : Kafka ......................... SUCCESS
> [ 0.003 s]
> >>>>> [INFO] Flink : Connectors : Google PubSub ................. SUCCESS
> [ 0.035 s]
> >>>>> [INFO] Flink : Connectors : Kinesis ....................... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Connectors : Base .......................... SUCCESS
> [ 0.003 s]
> >>>>> [INFO] Flink : Connectors : SQL : Elasticsearch 6 ......... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Connectors : SQL : Elasticsearch 7 ......... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Connectors : SQL : HBase ................... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Connectors : SQL : Hive 1.2.2 .............. SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Connectors : SQL : Hive 2.2.0 .............. SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Connectors : SQL : Hive 2.3.6 .............. SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Connectors : SQL : Hive 3.1.2 .............. SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Connectors : SQL : Kafka 0.10 .............. SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Connectors : SQL : Kafka 0.11 .............. SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Connectors : SQL : Kafka ................... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Formats : Avro confluent registry .......... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Formats : Sequence file .................... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Formats : Compress ......................... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Formats : SQL Orc .......................... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Formats : SQL Parquet ...................... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Examples : Streaming ....................... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Examples : Table ........................... SUCCESS
> [ 0.003 s]
> >>>>> [INFO] Flink : Examples : Build-helper .................... SUCCESS
> [ 0.003 s]
> >>>>> [INFO] Flink : Examples : Streaming Twitter ............... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Examples : Streaming State machine ......... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Examples : Streaming Google PubSub ......... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Container .................................. SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Queryable state : Runtime .................. SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Mesos ...................................... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Kubernetes ................................. SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Yarn ....................................... SUCCESS
> [ 0.002 s]
> >>>>> [INFO] Flink : Libraries : Gelly .......................... SUCCESS
> [ 0.002 s]
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>>
> >>>>> Best regards / Met vriendelijke groeten,
> >>>>>
> >>>>> Niels Basjes
> >>>>
>
> >>>
> >>>
> >>> --
> >>>
> >>> Best regards / Met vriendelijke groeten,
> >>>
> >>> Niels Basjes
> >>
>
> >
> >
> > --
> > Best regards / Met vriendelijke groeten,
> >
> > Niels Basjes
>
--
Best regards / Met vriendelijke groeten,
Niels Basjes
Re: Improvement idea: Naming the maven modules.
Posted by Aljoscha Krettek <al...@apache.org>.
Hi,
I like the proposal! I remember that Beam also had more human-readable names for the modules and found that helpful. Also, changing the names shouldn't change anything for users because dependencies are referred to by group/artifactId, it really just makes build output and IDE a bit more parseable by humans.
Best,
Aljoscha
On Thu, Jul 9, 2020, at 16:03, Niels Basjes wrote:
> Yes,
> The tree of modules (usually on the left) shows the maven modules as a tree.
> Yet I find that not very easy to use because of the readability of the artifact names and because this becomes very wide (i.e. takes up a lot of screen space).
> Having the maven modules with a name you'll get much better readable names in the Maven page (usually on the right).
>
> Like I said: It's just an idea and in my opinion it makes certain parts of the build more readable.
>
> Niels
>
>
> On Thu, Jul 9, 2020 at 2:55 PM Chesnay Schepler <ch...@apache.org> wrote:
>> Couldn't you just use the tree view and get basically the same thing?
>>
>> On 09/07/2020 14:51, Niels Basjes wrote:
>>> Attempt 2 to get you the images.
>>> Now as attachments.
>>>
>>> Niels
>>>
>>> On Thu, Jul 9, 2020 at 2:38 PM Chesnay Schepler <ch...@apache.org> wrote:
>>>> The images didn't go through.
>>>>
>>>> On 09/07/2020 14:21, Niels Basjes wrote:
>>>>> Hi,
>>>>>
>>>>> I have the idea that naming the modules would make life easier for the developers.
>>>>> My main question: Do you agree?
>>>>> If so I'll clean up my experiment (see below) and put up a pull request.
>>>>>
>>>>> When I load the Flink code into IntelliJ I currently see this list of Maven modules:
>>>>>
>>>>> image.png
>>>>>
>>>>> Now what happens is that the name of the maven module is in almost all cases the name of the artifact.
>>>>> Only one example in the current code base that deviates from this (highlighted in the screenshot).
>>>>> This list is shown by IntelliJ in alphabetical order.
>>>>>
>>>>> I propose to give all modules a name that should make it a lot easier for the developers to find the module they are looking for.
>>>>> I've been playing with this idea and what I have now (local only) looks like this:
>>>>>
>>>>> image.png
>>>>>
>>>>> They are still alphabetically sorted but because of the way I chose the naming it becomes much easier to find the appropriate module.
>>>>>
>>>>> Also this causes the build output from Maven to become more readable (below).
>>>>> First thing I noticed from this is that apparently the ordering of the modules and their inter dependencies is off in certain places.
>>>>>
>>>>>
>>>>> [INFO] Flink : Table : API Java bridge .................... SUCCESS [ 0.004 s]
>>>>> [INFO] Flink : Table : API Scala .......................... SUCCESS [ 0.004 s]
>>>>> [INFO] Flink : Table : API Scala bridge ................... SUCCESS [ 0.004 s]
>>>>> [INFO] Flink : Table : SQL Parser ......................... SUCCESS [ 0.004 s]
>>>>> [INFO] Flink : Libraries : ................................ SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Libraries : CEP ............................ SUCCESS [ 0.003 s]
>>>>> [INFO] Flink : Table : Planner ............................ SUCCESS [ 0.003 s]
>>>>> [INFO] Flink : Table : SQL Parser Hive .................... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Table : Runtime Blink ...................... SUCCESS [ 0.003 s]
>>>>> [INFO] Flink : Table : Planner Blink ...................... SUCCESS [ 0.004 s]
>>>>> [INFO] Flink : Metrics : JMX .............................. SUCCESS [ 0.004 s]
>>>>> [INFO] Flink : Formats : .................................. SUCCESS [ 0.004 s]
>>>>> [INFO] Flink : Formats : Json ............................. SUCCESS [ 0.004 s]
>>>>> [INFO] Flink : Connectors : Kafka base .................... SUCCESS [ 0.003 s]
>>>>> [INFO] Flink : Formats : Avro ............................. SUCCESS [ 0.005 s]
>>>>> [INFO] Flink : Formats : Csv .............................. SUCCESS [ 0.003 s]
>>>>> [INFO] Flink : Connectors : Kafka 0.10 .................... SUCCESS [ 0.009 s]
>>>>> [INFO] Flink : Connectors : Kafka 0.11 .................... SUCCESS [ 0.003 s]
>>>>> [INFO] Flink : Connectors : Elasticsearch base ............ SUCCESS [ 0.003 s]
>>>>> [INFO] Flink : Connectors : Elasticsearch 5 ............... SUCCESS [ 0.003 s]
>>>>> [INFO] Flink : Connectors : Elasticsearch 6 ............... SUCCESS [ 0.003 s]
>>>>> [INFO] Flink : Connectors : Elasticsearch 7 ............... SUCCESS [ 0.003 s]
>>>>> [INFO] Flink : Connectors : HBase ......................... SUCCESS [ 0.003 s]
>>>>> [INFO] Flink : Formats : Hadoop bulk ...................... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Formats : Orc .............................. SUCCESS [ 0.003 s]
>>>>> [INFO] Flink : Formats : Orc nohive ....................... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Formats : Parquet .......................... SUCCESS [ 0.003 s]
>>>>> [INFO] Flink : Connectors : Hive .......................... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Connectors : JDBC .......................... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Connectors : RabbitMQ ...................... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Connectors : Twitter ....................... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Connectors : Nifi .......................... SUCCESS [ 0.004 s]
>>>>> [INFO] Flink : Connectors : Cassandra ..................... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Connectors : Filesystem .................... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Connectors : Kafka ......................... SUCCESS [ 0.003 s]
>>>>> [INFO] Flink : Connectors : Google PubSub ................. SUCCESS [ 0.035 s]
>>>>> [INFO] Flink : Connectors : Kinesis ....................... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Connectors : Base .......................... SUCCESS [ 0.003 s]
>>>>> [INFO] Flink : Connectors : SQL : Elasticsearch 6 ......... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Connectors : SQL : Elasticsearch 7 ......... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Connectors : SQL : HBase ................... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Connectors : SQL : Hive 1.2.2 .............. SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Connectors : SQL : Hive 2.2.0 .............. SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Connectors : SQL : Hive 2.3.6 .............. SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Connectors : SQL : Hive 3.1.2 .............. SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Connectors : SQL : Kafka 0.10 .............. SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Connectors : SQL : Kafka 0.11 .............. SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Connectors : SQL : Kafka ................... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Formats : Avro confluent registry .......... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Formats : Sequence file .................... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Formats : Compress ......................... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Formats : SQL Orc .......................... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Formats : SQL Parquet ...................... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Examples : Streaming ....................... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Examples : Table ........................... SUCCESS [ 0.003 s]
>>>>> [INFO] Flink : Examples : Build-helper .................... SUCCESS [ 0.003 s]
>>>>> [INFO] Flink : Examples : Streaming Twitter ............... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Examples : Streaming State machine ......... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Examples : Streaming Google PubSub ......... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Container .................................. SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Queryable state : Runtime .................. SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Mesos ...................................... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Kubernetes ................................. SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Yarn ....................................... SUCCESS [ 0.002 s]
>>>>> [INFO] Flink : Libraries : Gelly .......................... SUCCESS [ 0.002 s]
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Best regards / Met vriendelijke groeten,
>>>>>
>>>>> Niels Basjes
>>>>
>>>
>>>
>>> --
>>>
>>> Best regards / Met vriendelijke groeten,
>>>
>>> Niels Basjes
>>
>
>
> --
> Best regards / Met vriendelijke groeten,
>
> Niels Basjes
Re: Improvement idea: Naming the maven modules.
Posted by Niels Basjes <Ni...@basjes.nl>.
Yes,
The tree of modules (usually on the left) shows the maven modules as a tree.
Yet I find that not very easy to use because of the readability of the
artifact names and because this becomes very wide (i.e. takes up a lot of
screen space).
Having the maven modules with a name you'll get much better readable names
in the Maven page (usually on the right).
Like I said: It's just an idea and in my opinion it makes certain parts of
the build more readable.
Niels
On Thu, Jul 9, 2020 at 2:55 PM Chesnay Schepler <ch...@apache.org> wrote:
> Couldn't you just use the tree view and get basically the same thing?
>
> On 09/07/2020 14:51, Niels Basjes wrote:
>
> Attempt 2 to get you the images.
> Now as attachments.
>
> Niels
>
> On Thu, Jul 9, 2020 at 2:38 PM Chesnay Schepler <ch...@apache.org>
> wrote:
>
>> The images didn't go through.
>>
>> On 09/07/2020 14:21, Niels Basjes wrote:
>>
>> Hi,
>>
>> I have the idea that naming the modules would make life easier for the
>> developers.
>> My main question: Do you agree?
>> If so I'll clean up my experiment (see below) and put up a pull request.
>>
>> When I load the Flink code into IntelliJ I currently see this list of
>> Maven modules:
>>
>> [image: image.png]
>>
>> Now what happens is that the name of the maven module is in almost all
>> cases the name of the artifact.
>> Only one example in the current code base that deviates from this
>> (highlighted in the screenshot).
>> This list is shown by IntelliJ in alphabetical order.
>>
>> I propose to give all modules a name that should make it a lot easier for
>> the developers to find the module they are looking for.
>> I've been playing with this idea and what I have now (local only) looks
>> like this:
>>
>> [image: image.png]
>>
>> They are still alphabetically sorted but because of the way I chose the
>> naming it becomes much easier to find the appropriate module.
>>
>> Also this causes the build output from Maven to become more readable
>> (below).
>> First thing I noticed from this is that apparently the ordering of the
>> modules and their inter dependencies is off in certain places.
>>
>>
>> [INFO] Flink : Table : API Java bridge .................... SUCCESS [
>> 0.004 s]
>> [INFO] Flink : Table : API Scala .......................... SUCCESS [
>> 0.004 s]
>> [INFO] Flink : Table : API Scala bridge ................... SUCCESS [
>> 0.004 s]
>> [INFO] Flink : Table : SQL Parser ......................... SUCCESS [
>> 0.004 s]
>> [INFO] Flink : Libraries : ................................ SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Libraries : CEP ............................ SUCCESS [
>> 0.003 s]
>> [INFO] Flink : Table : Planner ............................ SUCCESS [
>> 0.003 s]
>> [INFO] Flink : Table : SQL Parser Hive .................... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Table : Runtime Blink ...................... SUCCESS [
>> 0.003 s]
>> [INFO] Flink : Table : Planner Blink ...................... SUCCESS [
>> 0.004 s]
>> [INFO] Flink : Metrics : JMX .............................. SUCCESS [
>> 0.004 s]
>> [INFO] Flink : Formats : .................................. SUCCESS [
>> 0.004 s]
>> [INFO] Flink : Formats : Json ............................. SUCCESS [
>> 0.004 s]
>> [INFO] Flink : Connectors : Kafka base .................... SUCCESS [
>> 0.003 s]
>> [INFO] Flink : Formats : Avro ............................. SUCCESS [
>> 0.005 s]
>> [INFO] Flink : Formats : Csv .............................. SUCCESS [
>> 0.003 s]
>> [INFO] Flink : Connectors : Kafka 0.10 .................... SUCCESS [
>> 0.009 s]
>> [INFO] Flink : Connectors : Kafka 0.11 .................... SUCCESS [
>> 0.003 s]
>> [INFO] Flink : Connectors : Elasticsearch base ............ SUCCESS [
>> 0.003 s]
>> [INFO] Flink : Connectors : Elasticsearch 5 ............... SUCCESS [
>> 0.003 s]
>> [INFO] Flink : Connectors : Elasticsearch 6 ............... SUCCESS [
>> 0.003 s]
>> [INFO] Flink : Connectors : Elasticsearch 7 ............... SUCCESS [
>> 0.003 s]
>> [INFO] Flink : Connectors : HBase ......................... SUCCESS [
>> 0.003 s]
>> [INFO] Flink : Formats : Hadoop bulk ...................... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Formats : Orc .............................. SUCCESS [
>> 0.003 s]
>> [INFO] Flink : Formats : Orc nohive ....................... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Formats : Parquet .......................... SUCCESS [
>> 0.003 s]
>> [INFO] Flink : Connectors : Hive .......................... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Connectors : JDBC .......................... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Connectors : RabbitMQ ...................... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Connectors : Twitter ....................... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Connectors : Nifi .......................... SUCCESS [
>> 0.004 s]
>> [INFO] Flink : Connectors : Cassandra ..................... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Connectors : Filesystem .................... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Connectors : Kafka ......................... SUCCESS [
>> 0.003 s]
>> [INFO] Flink : Connectors : Google PubSub ................. SUCCESS [
>> 0.035 s]
>> [INFO] Flink : Connectors : Kinesis ....................... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Connectors : Base .......................... SUCCESS [
>> 0.003 s]
>> [INFO] Flink : Connectors : SQL : Elasticsearch 6 ......... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Connectors : SQL : Elasticsearch 7 ......... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Connectors : SQL : HBase ................... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Connectors : SQL : Hive 1.2.2 .............. SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Connectors : SQL : Hive 2.2.0 .............. SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Connectors : SQL : Hive 2.3.6 .............. SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Connectors : SQL : Hive 3.1.2 .............. SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Connectors : SQL : Kafka 0.10 .............. SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Connectors : SQL : Kafka 0.11 .............. SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Connectors : SQL : Kafka ................... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Formats : Avro confluent registry .......... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Formats : Sequence file .................... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Formats : Compress ......................... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Formats : SQL Orc .......................... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Formats : SQL Parquet ...................... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Examples : Streaming ....................... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Examples : Table ........................... SUCCESS [
>> 0.003 s]
>> [INFO] Flink : Examples : Build-helper .................... SUCCESS [
>> 0.003 s]
>> [INFO] Flink : Examples : Streaming Twitter ............... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Examples : Streaming State machine ......... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Examples : Streaming Google PubSub ......... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Container .................................. SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Queryable state : Runtime .................. SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Mesos ...................................... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Kubernetes ................................. SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Yarn ....................................... SUCCESS [
>> 0.002 s]
>> [INFO] Flink : Libraries : Gelly .......................... SUCCESS [
>> 0.002 s]
>>
>>
>>
>>
>>
>> --
>> Best regards / Met vriendelijke groeten,
>>
>> Niels Basjes
>>
>>
>>
>
> --
> Best regards / Met vriendelijke groeten,
>
> Niels Basjes
>
>
>
--
Best regards / Met vriendelijke groeten,
Niels Basjes
Re: Improvement idea: Naming the maven modules.
Posted by Chesnay Schepler <ch...@apache.org>.
Couldn't you just use the tree view and get basically the same thing?
On 09/07/2020 14:51, Niels Basjes wrote:
> Attempt 2 to get you the images.
> Now as attachments.
>
> Niels
>
> On Thu, Jul 9, 2020 at 2:38 PM Chesnay Schepler <chesnay@apache.org
> <ma...@apache.org>> wrote:
>
> The images didn't go through.
>
> On 09/07/2020 14:21, Niels Basjes wrote:
>> Hi,
>>
>> I have the idea that naming the modules would make life easier
>> for the developers.
>> My main question: Do you agree?
>> If so I'll clean up my experiment (see below) and put up a pull
>> request.
>>
>> When I load the Flink code into IntelliJ I currently see this
>> list of Maven modules:
>>
>> image.png
>>
>> Now what happens is that the name of the maven module is in
>> almost all cases the name of the artifact.
>> Only one example in the current code base that deviates from this
>> (highlighted in the screenshot).
>> This list is shown by IntelliJ in alphabetical order.
>>
>> I propose to give all modules a name that should make it a lot
>> easier for the developers to find the module they are looking for.
>> I've been playing with this idea and what I have now (local only)
>> looks like this:
>>
>> image.png
>>
>> They are still alphabetically sorted but because of the way I
>> chose the naming it becomes much easier to find the
>> appropriate module.
>>
>> Also this causes the build output from Maven to become more
>> readable (below).
>> First thing I noticed from this is that apparently the ordering
>> of the modules and their inter dependencies is off in certain places.
>>
>>
>> [INFO] Flink : Table : API Java bridge ....................
>> SUCCESS [ 0.004 s]
>> [INFO] Flink : Table : API Scala ..........................
>> SUCCESS [ 0.004 s]
>> [INFO] Flink : Table : API Scala bridge ...................
>> SUCCESS [ 0.004 s]
>> [INFO] Flink : Table : SQL Parser .........................
>> SUCCESS [ 0.004 s]
>> [INFO] Flink : Libraries : ................................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Libraries : CEP ............................
>> SUCCESS [ 0.003 s]
>> [INFO] Flink : Table : Planner ............................
>> SUCCESS [ 0.003 s]
>> [INFO] Flink : Table : SQL Parser Hive ....................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Table : Runtime Blink ......................
>> SUCCESS [ 0.003 s]
>> [INFO] Flink : Table : Planner Blink ......................
>> SUCCESS [ 0.004 s]
>> [INFO] Flink : Metrics : JMX ..............................
>> SUCCESS [ 0.004 s]
>> [INFO] Flink : Formats : ..................................
>> SUCCESS [ 0.004 s]
>> [INFO] Flink : Formats : Json .............................
>> SUCCESS [ 0.004 s]
>> [INFO] Flink : Connectors : Kafka base ....................
>> SUCCESS [ 0.003 s]
>> [INFO] Flink : Formats : Avro .............................
>> SUCCESS [ 0.005 s]
>> [INFO] Flink : Formats : Csv ..............................
>> SUCCESS [ 0.003 s]
>> [INFO] Flink : Connectors : Kafka 0.10 ....................
>> SUCCESS [ 0.009 s]
>> [INFO] Flink : Connectors : Kafka 0.11 ....................
>> SUCCESS [ 0.003 s]
>> [INFO] Flink : Connectors : Elasticsearch base ............
>> SUCCESS [ 0.003 s]
>> [INFO] Flink : Connectors : Elasticsearch 5 ...............
>> SUCCESS [ 0.003 s]
>> [INFO] Flink : Connectors : Elasticsearch 6 ...............
>> SUCCESS [ 0.003 s]
>> [INFO] Flink : Connectors : Elasticsearch 7 ...............
>> SUCCESS [ 0.003 s]
>> [INFO] Flink : Connectors : HBase .........................
>> SUCCESS [ 0.003 s]
>> [INFO] Flink : Formats : Hadoop bulk ......................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Formats : Orc ..............................
>> SUCCESS [ 0.003 s]
>> [INFO] Flink : Formats : Orc nohive .......................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Formats : Parquet ..........................
>> SUCCESS [ 0.003 s]
>> [INFO] Flink : Connectors : Hive ..........................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Connectors : JDBC ..........................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Connectors : RabbitMQ ......................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Connectors : Twitter .......................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Connectors : Nifi ..........................
>> SUCCESS [ 0.004 s]
>> [INFO] Flink : Connectors : Cassandra .....................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Connectors : Filesystem ....................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Connectors : Kafka .........................
>> SUCCESS [ 0.003 s]
>> [INFO] Flink : Connectors : Google PubSub .................
>> SUCCESS [ 0.035 s]
>> [INFO] Flink : Connectors : Kinesis .......................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Connectors : Base ..........................
>> SUCCESS [ 0.003 s]
>> [INFO] Flink : Connectors : SQL : Elasticsearch 6 .........
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Connectors : SQL : Elasticsearch 7 .........
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Connectors : SQL : HBase ...................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Connectors : SQL : Hive 1.2.2 ..............
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Connectors : SQL : Hive 2.2.0 ..............
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Connectors : SQL : Hive 2.3.6 ..............
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Connectors : SQL : Hive 3.1.2 ..............
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Connectors : SQL : Kafka 0.10 ..............
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Connectors : SQL : Kafka 0.11 ..............
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Connectors : SQL : Kafka ...................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Formats : Avro confluent registry ..........
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Formats : Sequence file ....................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Formats : Compress .........................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Formats : SQL Orc ..........................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Formats : SQL Parquet ......................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Examples : Streaming .......................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Examples : Table ...........................
>> SUCCESS [ 0.003 s]
>> [INFO] Flink : Examples : Build-helper ....................
>> SUCCESS [ 0.003 s]
>> [INFO] Flink : Examples : Streaming Twitter ...............
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Examples : Streaming State machine .........
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Examples : Streaming Google PubSub .........
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Container ..................................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Queryable state : Runtime ..................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Mesos ......................................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Kubernetes .................................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Yarn .......................................
>> SUCCESS [ 0.002 s]
>> [INFO] Flink : Libraries : Gelly ..........................
>> SUCCESS [ 0.002 s]
>>
>>
>>
>>
>>
>> --
>> Best regards / Met vriendelijke groeten,
>>
>> Niels Basjes
>
>
>
>
> --
> Best regards / Met vriendelijke groeten,
>
> Niels Basjes
Re: Improvement idea: Naming the maven modules.
Posted by Niels Basjes <Ni...@basjes.nl>.
Attempt 2 to get you the images.
Now as attachments.
Niels
On Thu, Jul 9, 2020 at 2:38 PM Chesnay Schepler <ch...@apache.org> wrote:
> The images didn't go through.
>
> On 09/07/2020 14:21, Niels Basjes wrote:
>
> Hi,
>
> I have the idea that naming the modules would make life easier for the
> developers.
> My main question: Do you agree?
> If so I'll clean up my experiment (see below) and put up a pull request.
>
> When I load the Flink code into IntelliJ I currently see this list of
> Maven modules:
>
> [image: image.png]
>
> Now what happens is that the name of the maven module is in almost all
> cases the name of the artifact.
> Only one example in the current code base that deviates from this
> (highlighted in the screenshot).
> This list is shown by IntelliJ in alphabetical order.
>
> I propose to give all modules a name that should make it a lot easier for
> the developers to find the module they are looking for.
> I've been playing with this idea and what I have now (local only) looks
> like this:
>
> [image: image.png]
>
> They are still alphabetically sorted but because of the way I chose the
> naming it becomes much easier to find the appropriate module.
>
> Also this causes the build output from Maven to become more readable
> (below).
> First thing I noticed from this is that apparently the ordering of the
> modules and their inter dependencies is off in certain places.
>
>
> [INFO] Flink : Table : API Java bridge .................... SUCCESS [
> 0.004 s]
> [INFO] Flink : Table : API Scala .......................... SUCCESS [
> 0.004 s]
> [INFO] Flink : Table : API Scala bridge ................... SUCCESS [
> 0.004 s]
> [INFO] Flink : Table : SQL Parser ......................... SUCCESS [
> 0.004 s]
> [INFO] Flink : Libraries : ................................ SUCCESS [
> 0.002 s]
> [INFO] Flink : Libraries : CEP ............................ SUCCESS [
> 0.003 s]
> [INFO] Flink : Table : Planner ............................ SUCCESS [
> 0.003 s]
> [INFO] Flink : Table : SQL Parser Hive .................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Table : Runtime Blink ...................... SUCCESS [
> 0.003 s]
> [INFO] Flink : Table : Planner Blink ...................... SUCCESS [
> 0.004 s]
> [INFO] Flink : Metrics : JMX .............................. SUCCESS [
> 0.004 s]
> [INFO] Flink : Formats : .................................. SUCCESS [
> 0.004 s]
> [INFO] Flink : Formats : Json ............................. SUCCESS [
> 0.004 s]
> [INFO] Flink : Connectors : Kafka base .................... SUCCESS [
> 0.003 s]
> [INFO] Flink : Formats : Avro ............................. SUCCESS [
> 0.005 s]
> [INFO] Flink : Formats : Csv .............................. SUCCESS [
> 0.003 s]
> [INFO] Flink : Connectors : Kafka 0.10 .................... SUCCESS [
> 0.009 s]
> [INFO] Flink : Connectors : Kafka 0.11 .................... SUCCESS [
> 0.003 s]
> [INFO] Flink : Connectors : Elasticsearch base ............ SUCCESS [
> 0.003 s]
> [INFO] Flink : Connectors : Elasticsearch 5 ............... SUCCESS [
> 0.003 s]
> [INFO] Flink : Connectors : Elasticsearch 6 ............... SUCCESS [
> 0.003 s]
> [INFO] Flink : Connectors : Elasticsearch 7 ............... SUCCESS [
> 0.003 s]
> [INFO] Flink : Connectors : HBase ......................... SUCCESS [
> 0.003 s]
> [INFO] Flink : Formats : Hadoop bulk ...................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Formats : Orc .............................. SUCCESS [
> 0.003 s]
> [INFO] Flink : Formats : Orc nohive ....................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Formats : Parquet .......................... SUCCESS [
> 0.003 s]
> [INFO] Flink : Connectors : Hive .......................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : JDBC .......................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : RabbitMQ ...................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : Twitter ....................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : Nifi .......................... SUCCESS [
> 0.004 s]
> [INFO] Flink : Connectors : Cassandra ..................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : Filesystem .................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : Kafka ......................... SUCCESS [
> 0.003 s]
> [INFO] Flink : Connectors : Google PubSub ................. SUCCESS [
> 0.035 s]
> [INFO] Flink : Connectors : Kinesis ....................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : Base .......................... SUCCESS [
> 0.003 s]
> [INFO] Flink : Connectors : SQL : Elasticsearch 6 ......... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : SQL : Elasticsearch 7 ......... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : SQL : HBase ................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : SQL : Hive 1.2.2 .............. SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : SQL : Hive 2.2.0 .............. SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : SQL : Hive 2.3.6 .............. SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : SQL : Hive 3.1.2 .............. SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : SQL : Kafka 0.10 .............. SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : SQL : Kafka 0.11 .............. SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : SQL : Kafka ................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Formats : Avro confluent registry .......... SUCCESS [
> 0.002 s]
> [INFO] Flink : Formats : Sequence file .................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Formats : Compress ......................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Formats : SQL Orc .......................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Formats : SQL Parquet ...................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Examples : Streaming ....................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Examples : Table ........................... SUCCESS [
> 0.003 s]
> [INFO] Flink : Examples : Build-helper .................... SUCCESS [
> 0.003 s]
> [INFO] Flink : Examples : Streaming Twitter ............... SUCCESS [
> 0.002 s]
> [INFO] Flink : Examples : Streaming State machine ......... SUCCESS [
> 0.002 s]
> [INFO] Flink : Examples : Streaming Google PubSub ......... SUCCESS [
> 0.002 s]
> [INFO] Flink : Container .................................. SUCCESS [
> 0.002 s]
> [INFO] Flink : Queryable state : Runtime .................. SUCCESS [
> 0.002 s]
> [INFO] Flink : Mesos ...................................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Kubernetes ................................. SUCCESS [
> 0.002 s]
> [INFO] Flink : Yarn ....................................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Libraries : Gelly .......................... SUCCESS [
> 0.002 s]
>
>
>
>
>
> --
> Best regards / Met vriendelijke groeten,
>
> Niels Basjes
>
>
>
--
Best regards / Met vriendelijke groeten,
Niels Basjes
Re: Improvement idea: Naming the maven modules.
Posted by Chesnay Schepler <ch...@apache.org>.
The images didn't go through.
On 09/07/2020 14:21, Niels Basjes wrote:
> Hi,
>
> I have the idea that naming the modules would make life easier for the
> developers.
> My main question: Do you agree?
> If so I'll clean up my experiment (see below) and put up a pull request.
>
> When I load the Flink code into IntelliJ I currently see this list of
> Maven modules:
>
> image.png
>
> Now what happens is that the name of the maven module is in almost all
> cases the name of the artifact.
> Only one example in the current code base that deviates from this
> (highlighted in the screenshot).
> This list is shown by IntelliJ in alphabetical order.
>
> I propose to give all modules a name that should make it a lot easier
> for the developers to find the module they are looking for.
> I've been playing with this idea and what I have now (local only)
> looks like this:
>
> image.png
>
> They are still alphabetically sorted but because of the way I chose
> the naming it becomes much easier to find the appropriate module.
>
> Also this causes the build output from Maven to become more readable
> (below).
> First thing I noticed from this is that apparently the ordering of the
> modules and their inter dependencies is off in certain places.
>
>
> [INFO] Flink : Table : API Java bridge .................... SUCCESS [
> 0.004 s]
> [INFO] Flink : Table : API Scala .......................... SUCCESS [
> 0.004 s]
> [INFO] Flink : Table : API Scala bridge ................... SUCCESS [
> 0.004 s]
> [INFO] Flink : Table : SQL Parser ......................... SUCCESS [
> 0.004 s]
> [INFO] Flink : Libraries : ................................ SUCCESS [
> 0.002 s]
> [INFO] Flink : Libraries : CEP ............................ SUCCESS [
> 0.003 s]
> [INFO] Flink : Table : Planner ............................ SUCCESS [
> 0.003 s]
> [INFO] Flink : Table : SQL Parser Hive .................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Table : Runtime Blink ...................... SUCCESS [
> 0.003 s]
> [INFO] Flink : Table : Planner Blink ...................... SUCCESS [
> 0.004 s]
> [INFO] Flink : Metrics : JMX .............................. SUCCESS [
> 0.004 s]
> [INFO] Flink : Formats : .................................. SUCCESS [
> 0.004 s]
> [INFO] Flink : Formats : Json ............................. SUCCESS [
> 0.004 s]
> [INFO] Flink : Connectors : Kafka base .................... SUCCESS [
> 0.003 s]
> [INFO] Flink : Formats : Avro ............................. SUCCESS [
> 0.005 s]
> [INFO] Flink : Formats : Csv .............................. SUCCESS [
> 0.003 s]
> [INFO] Flink : Connectors : Kafka 0.10 .................... SUCCESS [
> 0.009 s]
> [INFO] Flink : Connectors : Kafka 0.11 .................... SUCCESS [
> 0.003 s]
> [INFO] Flink : Connectors : Elasticsearch base ............ SUCCESS [
> 0.003 s]
> [INFO] Flink : Connectors : Elasticsearch 5 ............... SUCCESS [
> 0.003 s]
> [INFO] Flink : Connectors : Elasticsearch 6 ............... SUCCESS [
> 0.003 s]
> [INFO] Flink : Connectors : Elasticsearch 7 ............... SUCCESS [
> 0.003 s]
> [INFO] Flink : Connectors : HBase ......................... SUCCESS [
> 0.003 s]
> [INFO] Flink : Formats : Hadoop bulk ...................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Formats : Orc .............................. SUCCESS [
> 0.003 s]
> [INFO] Flink : Formats : Orc nohive ....................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Formats : Parquet .......................... SUCCESS [
> 0.003 s]
> [INFO] Flink : Connectors : Hive .......................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : JDBC .......................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : RabbitMQ ...................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : Twitter ....................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : Nifi .......................... SUCCESS [
> 0.004 s]
> [INFO] Flink : Connectors : Cassandra ..................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : Filesystem .................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : Kafka ......................... SUCCESS [
> 0.003 s]
> [INFO] Flink : Connectors : Google PubSub ................. SUCCESS [
> 0.035 s]
> [INFO] Flink : Connectors : Kinesis ....................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : Base .......................... SUCCESS [
> 0.003 s]
> [INFO] Flink : Connectors : SQL : Elasticsearch 6 ......... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : SQL : Elasticsearch 7 ......... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : SQL : HBase ................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : SQL : Hive 1.2.2 .............. SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : SQL : Hive 2.2.0 .............. SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : SQL : Hive 2.3.6 .............. SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : SQL : Hive 3.1.2 .............. SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : SQL : Kafka 0.10 .............. SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : SQL : Kafka 0.11 .............. SUCCESS [
> 0.002 s]
> [INFO] Flink : Connectors : SQL : Kafka ................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Formats : Avro confluent registry .......... SUCCESS [
> 0.002 s]
> [INFO] Flink : Formats : Sequence file .................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Formats : Compress ......................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Formats : SQL Orc .......................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Formats : SQL Parquet ...................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Examples : Streaming ....................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Examples : Table ........................... SUCCESS [
> 0.003 s]
> [INFO] Flink : Examples : Build-helper .................... SUCCESS [
> 0.003 s]
> [INFO] Flink : Examples : Streaming Twitter ............... SUCCESS [
> 0.002 s]
> [INFO] Flink : Examples : Streaming State machine ......... SUCCESS [
> 0.002 s]
> [INFO] Flink : Examples : Streaming Google PubSub ......... SUCCESS [
> 0.002 s]
> [INFO] Flink : Container .................................. SUCCESS [
> 0.002 s]
> [INFO] Flink : Queryable state : Runtime .................. SUCCESS [
> 0.002 s]
> [INFO] Flink : Mesos ...................................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Kubernetes ................................. SUCCESS [
> 0.002 s]
> [INFO] Flink : Yarn ....................................... SUCCESS [
> 0.002 s]
> [INFO] Flink : Libraries : Gelly .......................... SUCCESS [
> 0.002 s]
>
>
>
>
>
> --
> Best regards / Met vriendelijke groeten,
>
> Niels Basjes