Re: jdbc vs jdbc2 packages

[Denis Magda <dm...@apache.org>]

Ticket:
https://issues.apache.org/jira/browse/IGNITE-4922 <https://issues.apache.org/jira/browse/IGNITE-4922>

—
Denis

> On Mar 30, 2017, at 7:06 PM, Denis Magda <dm...@apache.org> wrote:
> 
> Folks,
> 
> I had a chance to validate usability of both Ignite JDBC drivers at PGConf USA today trying to connect to the cluster from the new JetBrains DataGrip SQL tool [1]. Together with JetBrains folks we agreed on the following or spotted the issues listed below.
> 
> 1) Default JDBC driver, that connects via a client node using a Spring XML config, is so counterintuitive and twisted that JetBrains folks couldn’t set it up w/o my assistance. We force SQL tools users, that accustomed to connect to DBs by hostname, to pass some weird Spring XML and add up to 15 (!) jars to the classpath of a tool rather than one. This is huge +1 for the thin client based driver that requires a hostname only. So, along with the thin client protocol optimizations DML has to be supported for it as well [2]. 
> 
> 2) Regardless of JDBC driver version if you execute a query like “SELECT * FROM PersonCache” you’ll get a deserialization exception for _val that is implicitly added to the result set. Luckily, this will be no longer the issue after this [3] improvement is released in 2.0.
> 
> 3) It would be more user-friendly if the JDBC drivers are packaged under single “ignite-jdbc.jar” and located in special directory of a release bundle. All the dependencies have to be in this jar. The ticket is ready [4].
> 
> So I would encourage us to wrap up this discussion creating additional tickets in order to renew the thin client based driver and promote it for tools and non transactional use cases.
> 
> [1] https://www.jetbrains.com/datagrip/ <https://www.jetbrains.com/datagrip/>
> [2] https://issues.apache.org/jira/browse/IGNITE-4892 <https://issues.apache.org/jira/browse/IGNITE-4892>
> [3] https://issues.apache.org/jira/browse/IGNITE-3487 <https://issues.apache.org/jira/browse/IGNITE-3487>
> [4] https://issues.apache.org/jira/browse/IGNITE-4893 <https://issues.apache.org/jira/browse/IGNITE-4893>
> 
> —
> Denis
> 
>> On Mar 30, 2017, at 1:09 AM, Denis Magda <dm...@apache.org> wrote:
>> 
>> Val, Igniters,
>> 
>> I still believe the thin client has a right for living. It’s ideal for use cases when someone attempts to connect to Ignite from a tool or some sort of interface and query the data or update it in non transactional fashion. A TCP/IP address as a connection string to the cluster is ideal for this scenario.
>> 
>> If someone decides to use JDBC programmatically and leverage from transactions then he will switch to the JDBC versions that starts a client node with a passed Ignite configuration.
>> 
>> How do you like this?
>> 
>> In general, it sounds like a right movement to replace REST with more flexible and, probably, unified protocol for thin client based JDBC implementation. But what if we extend REST a bit (supporting pagination for SELECTs and DML) at the first phase so that the thin client driver can be used right away in the scenarios I listed above? The rest of optimizations can be done after that.
>> 
>> —
>> Denis
>> 
>>> On Mar 23, 2017, at 7:24 PM, Valentin Kulichenko <va...@gmail.com> wrote:
>>> 
>>> I'm against keeping legacy thin client because:
>>> 
>>> - Having two ways to configure driver is unnecessary complication and very
>>> bad from usability standpoint.
>>> - It is much slower than client node, performance was the main driver
>>> behind its deprecation.
>>> 
>>> What we should do, is improve usability of the client node, this will be
>>> good improvement not only for JDBC driver. The list of required changes was
>>> covered earlier in the thread, I will check if we have tickets for all of
>>> them and provide a list.
>>> 
>>> -Val
>>> 
>>> On Thu, Mar 16, 2017 at 1:02 AM, Dmitriy Setrakyan <ds...@apache.org>
>>> wrote:
>>> 
>>>> Denis, I completely agree that we should have a thin JDBC client. Can you
>>>> file a ticket?
>>>> 
>>>> On Wed, Mar 15, 2017 at 4:58 PM, Denis Magda <dm...@apache.org> wrote:
>>>> 
>>>>> Frankly, during these two a couple of day I had a chance to play with Web
>>>>> Console and Ignite JDBC driver.
>>>>> 
>>>>> As many other users I referred to JDBC documentation in order to setup
>>>> the
>>>>> driver properly. However, then due to the recently reported issue I fell
>>>>> back to JDBC v 1 (thin client based):
>>>>> https://issues.apache.org/jira/browse/IGNITE-4829 <
>>>>> https://issues.apache.org/jira/browse/IGNITE-4829>
>>>>> 
>>>>> I was amazed how swift JDBC v 1 and sluggish JDBC v 2 which is default.
>>>>> Unfortunately, JDBC v 1 doesn’t support DML this is why I had hard time
>>>>> finding out a workaround for IGNITE-4829. But, in any case I fully
>>>> support
>>>>> reincarnation of JDBC v 1.
>>>>> 
>>>>> *Val*, as one of the most experienced folks who participated in this
>>>>> discussion, would you mind wrapping the discussion up in a form of a
>>>> ticket
>>>>> listing the design proposal?
>>>>> 
>>>>> —
>>>>> Denis
>>>>> 
>>>>>> On Feb 10, 2017, at 4:47 PM, Dmitriy Setrakyan <ds...@apache.org>
>>>>> wrote:
>>>>>> 
>>>>>> I generally like the idea of supporting thin JDBC clients. Often it is
>>>>> not
>>>>>> feasible to start a client node on JDBC side.
>>>>>> 
>>>>>> On Fri, Feb 10, 2017 at 1:04 PM, Valentin Kulichenko <
>>>>>> valentin.kulichenko@gmail.com> wrote:
>>>>>> 
>>>>>>> Yakov,
>>>>>>> 
>>>>>>> There are two separate aspects that we can improve:
>>>>>>> 
>>>>>>> 1. Message routing as you described. I agree it can be useful in some
>>>>>>> scenarios and I definitely not against this feature, but honestly I
>>>>> haven't
>>>>>>> seen a lot of requests for this so far.
>>>>>>> 2. Server can initiate connection with client. This really blows
>>>> users's
>>>>>>> minds :)
>>>>>>> 
>>>>>>> I only meant that the latter is much more critical in my view.
>>>>> Configuring
>>>>>>> port forwarding on the cluster can be complicated, but it absolutely
>>>>> makes
>>>>>>> sense, while doing the same on client just sounds crazy. Client should
>>>>> be
>>>>>>> able to just connect, without having public IP and without additional
>>>>>>> configuration on the router.
>>>>>>> 
>>>>>>> -Val
>>>>>>> 
>>>>>>> On Fri, Feb 10, 2017 at 1:58 AM, Yakov Zhdanov <yz...@apache.org>
>>>>>>> wrote:
>>>>>>> 
>>>>>>>> Val, can you please explain your statement. You suggest users have
>>>> port
>>>>>>>> forwarding for dozens of nodes in their topologies so client can
>>>>> initiate
>>>>>>>> connect to any of them? I don't think this is possible and routing
>>>>> seems
>>>>>>> to
>>>>>>>> be the only possible option.
>>>>>>>> 
>>>>>>>> --Yakov
>>>>>>>> 
>>>>>>>> 2017-02-10 0:04 GMT+03:00 Valentin Kulichenko <
>>>>>>>> valentin.kulichenko@gmail.com
>>>>>>>>> :
>>>>>>>> 
>>>>>>>>> Yakov,
>>>>>>>>> 
>>>>>>>>> 1. Yes, I will file a ticket.
>>>>>>>>> 
>>>>>>>>> 4. I meant that server can currently initiate connection with
>>>> client,
>>>>>>> and
>>>>>>>>> that's the main problem here. Is there a way to avoid this? Message
>>>>>>>> routing
>>>>>>>>> you're referring to can also be useful in some cases, but much less
>>>>>>>>> critical.
>>>>>>>>> 
>>>>>>>>> -Val
>>>>>>>>> 
>>>>>>>>> On Wed, Feb 8, 2017 at 9:20 PM, Yakov Zhdanov <
>>>> yzhdanov@gridgain.com>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Val,
>>>>>>>>>> 
>>>>>>>>>> 1. Our clients should stop require persistent store implementation
>>>> if
>>>>>>>>> they
>>>>>>>>>> do not need it. Can you please file a ticket? I know you fixed some
>>>>>>>>> places
>>>>>>>>>> already. As an idea I would keep everything in binary format until
>>>> we
>>>>>>>>>> really need it. Will that work?
>>>>>>>>>> 2. We can try adding the very first step to fetch the configuration
>>>>>>> and
>>>>>>>>>> then proceed with normal start.
>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-4675
>>>>>>>>>> 3. Agree, but user needs to define the closures then. I would think
>>>>>>> on
>>>>>>>>> how
>>>>>>>>>> to put this to a product.
>>>>>>>>>> 4. This needs to be implemented :) I mean we can communicate to a
>>>>>>>> client
>>>>>>>>>> through server it is connected to.
>>>>>>>>>> 
>>>>>>>>>> Thanks!
>>>>>>>>>> --
>>>>>>>>>> Yakov Zhdanov, Director R&D
>>>>>>>>>> *GridGain Systems*
>>>>>>>>>> www.gridgain.com
>>>>>>>>>> 
>>>>>>>>>> 2017-02-09 5:19 GMT+07:00 Valentin Kulichenko <
>>>>>>>>>> valentin.kulichenko@gmail.com
>>>>>>>>>>> :
>>>>>>>>>> 
>>>>>>>>>>> Yakov,
>>>>>>>>>>> 
>>>>>>>>>>> I agree that investing in the legacy client doesn't make sense -
>>>>>>> it's
>>>>>>>>>> slow
>>>>>>>>>>> and outdated. Regarding your points:
>>>>>>>>>>> 
>>>>>>>>>>> 1. This is just another build step, but the JAR is going to pretty
>>>>>>>> fat
>>>>>>>>> I
>>>>>>>>>>> think (it will have to include Spring). Not ideal, but definitely
>>>>>>>>> better
>>>>>>>>>>> than what we have now. However, our clients also often require
>>>> some
>>>>>>>>> user
>>>>>>>>>>> classes, like CacheStore implementations. This is also a problem.
>>>>>>>>>>> 
>>>>>>>>>>> 2. That's a great idea! Actually, I'm not sure why we require to
>>>>>>> have
>>>>>>>>>> full
>>>>>>>>>>> verbose config on client that is consistent with server. Why not
>>>>>>>> fetch
>>>>>>>>>> the
>>>>>>>>>>> configuration from cluster during join? Not sure how hard this
>>>>>>> change
>>>>>>>>> is,
>>>>>>>>>>> but it can be a very big usability improvement. And surely, JDBC
>>>>>>>> driver
>>>>>>>>>>> should be able to config with host:port without config file.
>>>>>>>>>>> 
>>>>>>>>>>> 3. This can be already achieved with Compute Grid, no? I don't
>>>>>>> think
>>>>>>>> we
>>>>>>>>>>> need to add anything here.
>>>>>>>>>>> 
>>>>>>>>>>> Another issue with clients is that they currently can't work
>>>> behind
>>>>>>>> NAT
>>>>>>>>>>> without additional config which is not very trivial
>>>>>>>> (AddressResolver).
>>>>>>>>> Is
>>>>>>>>>>> it possible to avoid server->client connections in communication
>>>>>>> SPI?
>>>>>>>>>>> 
>>>>>>>>>>> -Val
>>>>>>>>>>> 
>>>>>>>>>>> On Tue, Feb 7, 2017 at 9:09 PM, Yakov Zhdanov <
>>>> yzhdanov@apache.org
>>>>>>>> 
>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> "undeprecating" - lol :D
>>>>>>>>>>>> Consider introducing @Un annotation which negates all annotations
>>>>>>>> on
>>>>>>>>>> the
>>>>>>>>>>>> same level and below.
>>>>>>>>>>>> 
>>>>>>>>>>>> I would probably agree with Val and Vova, but adding features to
>>>>>>>>>>>> thin-client seems questionable to me.
>>>>>>>>>>>> 
>>>>>>>>>>>> Is these possible:
>>>>>>>>>>>> 1. avoid dependencies on client machine and require
>>>>>>> ignite-jdbc.jar
>>>>>>>>>> only
>>>>>>>>>>>> (e.g. gathering dependencies inside the jar).
>>>>>>>>>>>> 2. make it possible to provide just address and port to send join
>>>>>>>>>> request
>>>>>>>>>>>> to without providing the entire IgniteConfiguration. Client node
>>>>>>>>> sends
>>>>>>>>>>> join
>>>>>>>>>>>> request to the cluster with flag that this is jdbc-driver
>>>>>>>> connection
>>>>>>>>>> and
>>>>>>>>>>>> server-side topology omits configuration validation and forces
>>>>>>>> client
>>>>>>>>>> to
>>>>>>>>>>>> set some properties if this is necessary (e.g. CommunicationSpi
>>>>>>>>>>>> implementation class and settings)
>>>>>>>>>>>> 3. add possibility to offload complex reduce processing to
>>>>>>> server.
>>>>>>>>>> Which
>>>>>>>>>>>> may be very helpful for main client-server use cases when clients
>>>>>>>>> being
>>>>>>>>>>> run
>>>>>>>>>>>> on much weaker machines.
>>>>>>>>>>>> 
>>>>>>>>>>>> ?
>>>>>>>>>>>> 
>>>>>>>>>>>> --Yakov
>>>>>>>>>>>> 
>>>>>>>>>>>> 2017-02-07 14:30 GMT+07:00 Vladimir Ozerov <vozerov@gridgain.com
>>>>>>>> :
>>>>>>>>>>>> 
>>>>>>>>>>>>> Big +1 to Val, not only about JDBC, but about our overall
>>>>>>>> approach
>>>>>>>>> to
>>>>>>>>>>>>> clients. Starting a node with "client=true" is:
>>>>>>>>>>>>> + Very reach feature set, which is cool
>>>>>>>>>>>>> - Tons of dependencies
>>>>>>>>>>>>> - Tons of threads
>>>>>>>>>>>>> 
>>>>>>>>>>>>> It would be very cool if we have a true thin client with small
>>>>>>>>> single
>>>>>>>>>>>> JAR.
>>>>>>>>>>>>> It should have:
>>>>>>>>>>>>> - Failover
>>>>>>>>>>>>> - Load-balance
>>>>>>>>>>>>> - Optional server "stickyness"
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Once all these things are in place we will be able to provide
>>>>>>> the
>>>>>>>>>> same
>>>>>>>>>>>> API
>>>>>>>>>>>>> as in current client, but with predictable behavior and memory
>>>>>>>>>>> footprint.
>>>>>>>>>>>>> For instance our current client is not well-suited for running
>>>>>>>>>>> map-reduce
>>>>>>>>>>>>> (compute or SQL) because it moves large amount of data and
>>>>>>>>> processing
>>>>>>>>>>> to
>>>>>>>>>>>>> the client, which is potentially a slow desktop machine.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Tue, Feb 7, 2017 at 3:44 AM, Valentin Kulichenko <
>>>>>>>>>>>>> valentin.kulichenko@gmail.com> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> There are two implementations of JDBC driver - based on
>>>>>>> legacy
>>>>>>>>> thin
>>>>>>>>>>>>> client
>>>>>>>>>>>>>> (jdbc package) and on client node (jdbc2). The first one was
>>>>>>>>>>> deprecated
>>>>>>>>>>>>>> when we introduced the latter, but now I tend to think that
>>>>>>>> this
>>>>>>>>>> was
>>>>>>>>>>>> not
>>>>>>>>>>>>> a
>>>>>>>>>>>>>> right decision. Thin client driver provides worse
>>>>>>> performance,
>>>>>>>>> but
>>>>>>>>>>> it's
>>>>>>>>>>>>>> much easier to use, never requires additional dependencies
>>>>>>> like
>>>>>>>>>>> Spring
>>>>>>>>>>>>> and
>>>>>>>>>>>>>> can be used from any remote machine. Probably we can consider
>>>>>>>>>>>>> undeprecating
>>>>>>>>>>>>>> it.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> -Val
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Mon, Feb 6, 2017 at 2:02 AM, Sergi Vladykin <
>>>>>>>>>>>> sergi.vladykin@gmail.com
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Guys,
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> We have 2 different packages: jdbc and jdbc2. Everything in
>>>>>>>>> jdbc
>>>>>>>>>> is
>>>>>>>>>>>>>>> deprecated. Because of that new features like DML support
>>>>>>>> were
>>>>>>>>>> not
>>>>>>>>>>>>> added
>>>>>>>>>>>>>>> there.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> This seems to cause some problems to our users. Can someone
>>>>>>>>>>> clarify,
>>>>>>>>>>>>> did
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>> deprecated these classes wrongly and we have to continue
>>>>>>>>>> developing
>>>>>>>>>>>>> them
>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>> what?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Sergi
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> 
>>>> 
>> 
>