You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Igor Sapego <is...@apache.org> on 2018/07/02 11:41:10 UTC
Re: Thin Client lib: Python
By the way, guys,
I can see generated docs under source control. We do not store them
this way in Ignite. Instead, we have separate step during release, where
we generate them and include in binary distribution.
So you should remove documents from Git and provide us with instructions
on how to generate docs during release.
Best Regards,
Igor
On Fri, Jun 29, 2018 at 5:01 PM Dmitry Melnichuk <
dmitry.melnichuk@nobitlost.com> wrote:
> Dmitry,
>
> I have mentioned Jira task in my reply earlier today. Sorry, but the
> task description was a bit messy and less than informative. I just
> updated it. Please have a look.
>
> https://issues.apache.org/jira/browse/IGNITE-7782
>
> Please let me know if this information is supposed to be somewhere in
> Wiki. I'll post it right away.
>
> On 06/29/2018 07:53 AM, Dmitriy Setrakyan wrote:
> > Is there an IEP for it where the community can read about the feature set
> > that will be supported by the Python client?
> >
> > D.
>
Re: Thin Client lib: Python
Posted by Dmitry Melnichuk <dm...@nobitlost.com>.
Igor,
Thank you for your elaborated response. I think the examples you gave me
are valid for statically-typed languages only. At least not for Python.
If we add an extra parameter to key-value operation, we just add it to a
given API function as a named argument (“keyword argument” or “kwarg”)
with a sensible default value, so all the dependent code will run as it
did before the change, implicitly using the new default value. The newly
written code, in turn, will be able to take an advantage of the new
parameter by setting it explicitly. This behavior may be impossible in
Java or C++ without some additional abstraction layer for isolating
caller-supplied parameters from actually used ones. But for any
correctly written Python code such isolation level is only superfluous
and misleading.
The same thing is with the second example. If we stop using a string or
integer values as a cache identifiers and replace them with, say, floats
or objects, still no dependent code modification is required, taken that
the new cache identifiers remain unique and hashable in Pythonic sense
(i. e. it should be easy to test them for non-equality), that is only
natural for an identifier. All the existing code will continue to work
without hassle.
The above said is yet more relevant to the end user code, since my API
atm is mostly single-levelled and has very few internal codependencies
to maintain.
On 07/25/2018 08:00 PM, Igor Sapego wrote:
> Dmitriy,
>
> To me it seems that procedural approach makes it harder to maintaine
> code and add new features. For example, imagine that we are adding
> new feature for a thin client, and now we need to pass one more cache
> parameter when doing any key-value operation, such as put. In your
> approach, you will need to rewrite all the Cache API, adding new
> argument to all the functions.
>
> To give you another example, it is possible, that in future we will change
> the way cache is identified, so the "hash" argument will become
> deprecated in your approach.
>
> To put it simple, procedural approach lacks encapsulation, which is
> extremely important when you write library code and want to maintain
> backward compatibility. It removes the separation between public API,
> which we should not change from version to version, and private code,
> which we are free to change whenever and however we want.
>
> Best Regards,
> Igor
Re: Thin Client lib: Python
Posted by Igor Sapego <is...@apache.org>.
Dmitriy,
To me it seems that procedural approach makes it harder to maintaine
code and add new features. For example, imagine that we are adding
new feature for a thin client, and now we need to pass one more cache
parameter when doing any key-value operation, such as put. In your
approach, you will need to rewrite all the Cache API, adding new
argument to all the functions.
To give you another example, it is possible, that in future we will change
the way cache is identified, so the "hash" argument will become
deprecated in your approach.
To put it simple, procedural approach lacks encapsulation, which is
extremely important when you write library code and want to maintain
backward compatibility. It removes the separation between public API,
which we should not change from version to version, and private code,
which we are free to change whenever and however we want.
Best Regards,
Igor
On Wed, Jul 25, 2018 at 11:53 AM Dmitriy Setrakyan <ds...@apache.org>
wrote:
> Dmitriy,
>
> To be honest, I got a bit lost in such a big email. Can you tell us briefly
> - why do we not need the hash code on API in Java and we do need it in
> Python?
>
> D.
>
> On Wed, Jul 25, 2018 at 3:29 AM, Dmitry Melnichuk <
> dmitry.melnichuk@nobitlost.com> wrote:
>
> > Hello, Dmitriy, Igor!
> >
> > Dmitriy, as you have noted, most cache API functions are parameterized
> > with the hash codes, and the function for creating the hash codes is
> > documented. I personally see no harm in using hash codes as it is defined
> > by low-level client operations. But it may be better to enforce the use
> of
> > cache names throughout the Python library, or use names and hash codes
> > interchangeably where possible.
> >
> > Igor, you are right, due to the procedural nature of my client library,
> > there is no cache class in it. Caches in Ignite, from Python perspective,
> > are one-off, unique, immutable objects. The client code will not benefit
> > much from associating any additional context with them. On the contrary,
> > handling cache objects in its string or integer representation may
> > sometimes be advantageous.
> >
> > I guess, the implied question here is: why I chose a procedural approach
> > first of all? Well, I am glad to answer it too, although the answer will
> be
> > more verbose.
> >
> > Python is often referred as a multi-paradigm programming language, but at
> > heart it is a procedural scripting language. It has no obligatory object
> > orientation that could influence Java or C# clients' architecture.
> >
> > It is also worth noting, that Python has such an elaborate and versatile
> > built-in types, so its classes (`object` descendants) is never meant to
> be
> > used as plain data containers. That is unlike JavaScript, where `object`
> is
> > a legitimate default complex data type. Any combination of
> > list/set/dict/defaultdict/OrderedDict is always a better, more Pythonic
> > choice; getter/setter semantics is considered an anti-pattern.
> >
> > Considering the above, the client API architecture could be chosen
> > depending on the application field. Aiming for the most versatility,
> > however, I decided to implement lower-level procedural API.
> >
> > “Lower-level” in this case does not mean extra efforts for the end user.
> > Feel free to review the rest of the examples and make sure that those
> code
> > snippets actually take less and do more, comparing to the equivalent code
> > using other client APIs.
> >
> > If that is still not satisfying, any kind of object-oriented wrapper can
> > be built on top of the procedural implementation with little time and
> > effort. In fact, before taking the present course of actions, I
> considered
> > the following implementations of other services' clients:
> >
> > - key-value libraries, like redis-py, that have their methods
> incapsulated
> > in connection class,
> >
> > - RDBMS libraries like psycopg2 with their cursor classes − it could be
> an
> > architectural choice for SQL and scan queries-oriented client,
> >
> > - boto3 (Amazon S3 client) have its methods incapsulated in bucket
> > objects, somewhat analogous to Ignite caches, but notably different: a)
> S3
> > supports only one data type: file, b) bucket can be reconfigured on the
> > fly, unlike Ignite cache,
> >
> > - there could be other architectural choices too.
> >
> > I will be glad to answer your further questions. If you have suggestions
> > about higher-level abstractions and their use cases, please let me know.
> >
> > Dmitry
> >
> > On 07/24/2018 10:43 PM, Dmitriy Setrakyan wrote:> Yes, looks strange? Why
> > is the hash code part of API?
> >
> > >
> > > On Tue, Jul 24, 2018, 13:38 Igor Sapego <is...@apache.org> wrote:
> > >
> > >> Ok, I've quickly looked through API and I have a questions. Here
> > >> is the code from the example:
> > >>
> > >> cache_name = 'my cache'
> > >> hash_code = hashcode(cache_name)
> > >> cache_create(conn, cache_name)
> > >> result = cache_put(conn, hash_code, 'my key', 42)
> > >>
> > >> I'm not familiar with python, so here is the question. Why there is
> > >> no "cache" class, and functions with hash code are used?
> > >>
> > >> Best Regards,
> > >> Igor
> >
>
Re: Thin Client lib: Python
Posted by Sergey Kozlov <sk...@gridgain.com>.
Also I remember there are no hashcodes for NodeJS clients
On Wed, Jul 25, 2018 at 10:46 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:
> I am still confused. Let's work through an example. Suppose I have a cache
> named "my_cache" and I want to put an entry with key "a" and value "1".
>
> In Java, this code will look like this:
>
>
> > *IgniteCache<...> myCache = ignite.cache("my-cache");myCache.put("a",
> 1);*
>
>
> How will the same code look in Python?
>
> D.
>
> On Wed, Jul 25, 2018 at 5:08 PM, Dmitry Melnichuk <
> dmitry.melnichuk@nobitlost.com> wrote:
>
> > Igor,
> >
> > That is a very good point. It just did not cross my mind during the
> > implementation of this function, that the cache identifier can be
> abstract.
> > I will fix that.
> >
> >
> > On 07/26/2018 01:46 AM, Igor Sapego wrote:
> >
> >> Well, at least name should be changed, IMO, as the API function name
> >> should say what we do, and not how we do it. For example, cache_id()
> >> looks better to me than hashcode().
> >>
> >> Best Regards,
> >> Igor
> >>
> >
>
--
Sergey Kozlov
GridGain Systems
www.gridgain.com
Re: Thin Client lib: Python
Posted by Prachi Garg <pg...@gridgain.com>.
Thanks Dmitry. I'll look at the docs.
On Wed, Jul 25, 2018 at 8:11 PM, Dmitry Melnichuk <
dmitry.melnichuk@nobitlost.com> wrote:
> Hi Prachi!
>
> At the moment I already have my documents (temporarily) published at RTD.
> This is how they look like at a whole:
>
> https://apache-ignite-binary-protocol-client.readthedocs.io/
>
> I already have a separate section on examples:
>
> https://apache-ignite-binary-protocol-client.readthedocs.io/
> en/latest/examples.html
>
> My build process is also documented here
>
> https://apache-ignite-binary-protocol-client.readthedocs.io/
> en/latest/readme.html#documentation
>
> and there
>
> https://github.com/nobitlost/ignite/blob/ignite-7782/modules
> /platforms/python/README.md
>
> This instructions works for me, and I have at least one report of
> successful documentation build from elsewhere. And RTD is using basically
> the same way to put my docs online.
>
> My way of document building is pretty common for Python package
> developers, but if it needs some modifications to fit into Ignite process,
> please let me know.
>
> All the document sources (both autodoc'ed and hand-crafted) is available at
>
> https://github.com/nobitlost/ignite/tree/ignite-7782/modules
> /platforms/python/docs
>
> I will be glad to answer any questions.
>
>
> On 07/26/2018 06:25 AM, Prachi Garg wrote:
>
>> Hi Dmitry M,
>>
>> I am resposible for managing the Ignite documentation. At some point we
>> will merge the python documentation on github into the main Ignite
>> documentation. Currently, I am trying to restructure our thin client
>> documentation in a way that it (thin client documentation) is consistent
>> for all supported languages - Java, Node.js, Python etc.
>>
>> I looked at the python document on github. Under the :mod:`~pyignite.api`
>> section, I see all the components - cache config, key value, sql, binary
>> types - but there are no code snippets. Is it possible for you describe
>> these components with code examples?
>>
>> See for example -
>> https://apacheignite.readme.io/docs/java-thin-client-api#sec
>> tion-sql-queries
>> where the SQL Queries section explains, with example, how the thin client
>> SQL API can be used.
>>
>> Similarly, please see -
>> https://apacheignite.readme.io/docs/java-thin-client-security
>> https://apacheignite.readme.io/docs/java-thin-client-high-availability
>> https://apacheignite.readme.io/docs/java-thin-client-api
>>
>> Thanks,
>> -Prachi
>>
>
Re: Thin Client lib: Python
Posted by Dmitry Melnichuk <dm...@nobitlost.com>.
Hi Prachi!
At the moment I already have my documents (temporarily) published at
RTD. This is how they look like at a whole:
https://apache-ignite-binary-protocol-client.readthedocs.io/
I already have a separate section on examples:
https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/examples.html
My build process is also documented here
https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/readme.html#documentation
and there
https://github.com/nobitlost/ignite/blob/ignite-7782/modules/platforms/python/README.md
This instructions works for me, and I have at least one report of
successful documentation build from elsewhere. And RTD is using
basically the same way to put my docs online.
My way of document building is pretty common for Python package
developers, but if it needs some modifications to fit into Ignite
process, please let me know.
All the document sources (both autodoc'ed and hand-crafted) is available at
https://github.com/nobitlost/ignite/tree/ignite-7782/modules/platforms/python/docs
I will be glad to answer any questions.
On 07/26/2018 06:25 AM, Prachi Garg wrote:
> Hi Dmitry M,
>
> I am resposible for managing the Ignite documentation. At some point we
> will merge the python documentation on github into the main Ignite
> documentation. Currently, I am trying to restructure our thin client
> documentation in a way that it (thin client documentation) is consistent
> for all supported languages - Java, Node.js, Python etc.
>
> I looked at the python document on github. Under the :mod:`~pyignite.api`
> section, I see all the components - cache config, key value, sql, binary
> types - but there are no code snippets. Is it possible for you describe
> these components with code examples?
>
> See for example -
> https://apacheignite.readme.io/docs/java-thin-client-api#section-sql-queries
> where the SQL Queries section explains, with example, how the thin client
> SQL API can be used.
>
> Similarly, please see -
> https://apacheignite.readme.io/docs/java-thin-client-security
> https://apacheignite.readme.io/docs/java-thin-client-high-availability
> https://apacheignite.readme.io/docs/java-thin-client-api
>
> Thanks,
> -Prachi
Re: Thin Client lib: Python
Posted by Prachi Garg <pg...@gridgain.com>.
Hi Dmitry M,
I am resposible for managing the Ignite documentation. At some point we
will merge the python documentation on github into the main Ignite
documentation. Currently, I am trying to restructure our thin client
documentation in a way that it (thin client documentation) is consistent
for all supported languages - Java, Node.js, Python etc.
I looked at the python document on github. Under the :mod:`~pyignite.api`
section, I see all the components - cache config, key value, sql, binary
types - but there are no code snippets. Is it possible for you describe
these components with code examples?
See for example -
https://apacheignite.readme.io/docs/java-thin-client-api#section-sql-queries
where the SQL Queries section explains, with example, how the thin client
SQL API can be used.
Similarly, please see -
https://apacheignite.readme.io/docs/java-thin-client-security
https://apacheignite.readme.io/docs/java-thin-client-high-availability
https://apacheignite.readme.io/docs/java-thin-client-api
Thanks,
-Prachi
On Wed, Jul 25, 2018 at 12:46 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:
> I am still confused. Let's work through an example. Suppose I have a cache
> named "my_cache" and I want to put an entry with key "a" and value "1".
>
> In Java, this code will look like this:
>
>
> > *IgniteCache<...> myCache = ignite.cache("my-cache");myCache.put("a",
> 1);*
>
>
> How will the same code look in Python?
>
> D.
>
> On Wed, Jul 25, 2018 at 5:08 PM, Dmitry Melnichuk <
> dmitry.melnichuk@nobitlost.com> wrote:
>
> > Igor,
> >
> > That is a very good point. It just did not cross my mind during the
> > implementation of this function, that the cache identifier can be
> abstract.
> > I will fix that.
> >
> >
> > On 07/26/2018 01:46 AM, Igor Sapego wrote:
> >
> >> Well, at least name should be changed, IMO, as the API function name
> >> should say what we do, and not how we do it. For example, cache_id()
> >> looks better to me than hashcode().
> >>
> >> Best Regards,
> >> Igor
> >>
> >
>
Re: Thin Client lib: Python
Posted by Prachi Garg <pg...@gridgain.com>.
Thanks Dmitry!
On Wed, Sep 5, 2018 at 11:59 PM, Dmitry Melnichuk <
dmitry.melnichuk@nobitlost.com> wrote:
> Hello, Prachi!
>
> The section is a hand-written .rst-file, which contains links to some
> content, generated from code with `sphinx-apidoc`.
>
> All autogenerated content resides in `source` folder. For example, this is
> an autogenerated page:
>
> https://apache-ignite-binary-protocol-client.readthedocs.io/
> en/latest/source/pyignite.client.html
>
> And this is written by hand:
>
> https://apache-ignite-binary-protocol-client.readthedocs.io/
> en/latest/datatypes/parsers.html
>
>
> On 9/6/18 9:39 AM, Prachi Garg wrote:
>
>> Hi Dmitry M,
>>
>> Is the API Specification [1] section of the documentation auto-generated
>> from the code?
>>
>> [1]
>> https://apache-ignite-binary-protocol-client.readthedocs.io/
>> en/latest/modules.html
>>
>>
Re: Thin Client lib: Python
Posted by Dmitry Melnichuk <dm...@nobitlost.com>.
Hello, Prachi!
The section is a hand-written .rst-file, which contains links to some
content, generated from code with `sphinx-apidoc`.
All autogenerated content resides in `source` folder. For example, this
is an autogenerated page:
https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/source/pyignite.client.html
And this is written by hand:
https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/datatypes/parsers.html
On 9/6/18 9:39 AM, Prachi Garg wrote:
> Hi Dmitry M,
>
> Is the API Specification [1] section of the documentation auto-generated
> from the code?
>
> [1]
> https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/modules.html
>
Re: Thin Client lib: Python
Posted by Prachi Garg <pg...@gridgain.com>.
Hi Dmitry M,
Is the API Specification [1] section of the documentation auto-generated
from the code?
[1]
https://apache-ignite-binary-protocol-client.readthedocs.io/en/latest/modules.html
On Mon, Jul 30, 2018 at 5:05 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:
> Now we are talking. A much better and more user-friendly API.
>
> D.
>
> On Fri, Jul 27, 2018 at 7:19 AM, Dmitry Melnichuk <
> dmitry.melnichuk@nobitlost.com> wrote:
>
> > Dmitriy, Igor, Ilya, Sergey!
> >
> > Thank you for sharing your ideas, concerns and criticism with me. I do
> > appreciate it.
> >
> > I already made some changes in my API, influenced by your feedback. I
> also
> > plan to add a certain set of features, that will make my UX closer to
> what
> > you can see from other Ignite clients.
> >
> > I stopped using `hashcode` in my examples. Integer cache IDs and cache
> > names now can be used interchangeably, with primary focus on cache names.
> >
> > I will add a Cache class as a primary interface for cache operations, so
> > that earlier examples:
> >
> > ```
> > conn = Connection()
> > conn.connect('127.0.0.1', 10800)
> >
> > cache_create(conn, 'my cache')
> >
> > cache_put(conn, 'my cache', 'my key', 42)
> > result = my_cache.get('my key')
> >
> > cache_destroy(conn, 'my cache')
> > conn.close()
> > ```
> >
> > could be reiterated as:
> >
> > ```
> > conn = Connection()
> > conn.connect('127.0.0.1', 10800)
> >
> > my_cache = conn.create_cache('my cache')
> >
> > my_cache.put('my key', 42)
> > result = my_cache.get('my key')
> >
> > my_cache.destroy('my cache')
> > conn.close()
> > ```
> >
> > I will also make `Connection.connect()` accept any iterable (including
> > simple list) as a connection parameter. I will provide user with some
> basic
> > connection generators instead of what is done in my current connection
> > failover example.
> >
> >
> > On 07/27/2018 07:41 AM, Dmitriy Setrakyan wrote:
> >
> >> Dmitriy,
> >>
> >> I would stop using the word "hashcode" in this context. Hash code has a
> >> special meaning in Ignite and is used to determine key-to-node
> affinity. I
> >> agree that passing "cache_name" is the best option. I have no idea when
> >> "cache_name" is not going to be known and do not think we need to
> support
> >> this case at all. My suggestion is to drop the cache_id use case
> >> altogether.
> >>
> >> Also I am really surprised that we do not have a cache abstraction in
> >> python and need to pass cache name and connection into every method. To
> be
> >> honest, this smells really bad that such a popular modern language like
> >> Python forces us to have such a clumsy API. Can you please take a look
> at
> >> the Redis python clients and see if there is a better way to support
> this?
> >>
> >> https://redis.io/clients#python
> >>
> >> D.
> >>
> >
>
Re: Thin Client lib: Python
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Now we are talking. A much better and more user-friendly API.
D.
On Fri, Jul 27, 2018 at 7:19 AM, Dmitry Melnichuk <
dmitry.melnichuk@nobitlost.com> wrote:
> Dmitriy, Igor, Ilya, Sergey!
>
> Thank you for sharing your ideas, concerns and criticism with me. I do
> appreciate it.
>
> I already made some changes in my API, influenced by your feedback. I also
> plan to add a certain set of features, that will make my UX closer to what
> you can see from other Ignite clients.
>
> I stopped using `hashcode` in my examples. Integer cache IDs and cache
> names now can be used interchangeably, with primary focus on cache names.
>
> I will add a Cache class as a primary interface for cache operations, so
> that earlier examples:
>
> ```
> conn = Connection()
> conn.connect('127.0.0.1', 10800)
>
> cache_create(conn, 'my cache')
>
> cache_put(conn, 'my cache', 'my key', 42)
> result = my_cache.get('my key')
>
> cache_destroy(conn, 'my cache')
> conn.close()
> ```
>
> could be reiterated as:
>
> ```
> conn = Connection()
> conn.connect('127.0.0.1', 10800)
>
> my_cache = conn.create_cache('my cache')
>
> my_cache.put('my key', 42)
> result = my_cache.get('my key')
>
> my_cache.destroy('my cache')
> conn.close()
> ```
>
> I will also make `Connection.connect()` accept any iterable (including
> simple list) as a connection parameter. I will provide user with some basic
> connection generators instead of what is done in my current connection
> failover example.
>
>
> On 07/27/2018 07:41 AM, Dmitriy Setrakyan wrote:
>
>> Dmitriy,
>>
>> I would stop using the word "hashcode" in this context. Hash code has a
>> special meaning in Ignite and is used to determine key-to-node affinity. I
>> agree that passing "cache_name" is the best option. I have no idea when
>> "cache_name" is not going to be known and do not think we need to support
>> this case at all. My suggestion is to drop the cache_id use case
>> altogether.
>>
>> Also I am really surprised that we do not have a cache abstraction in
>> python and need to pass cache name and connection into every method. To be
>> honest, this smells really bad that such a popular modern language like
>> Python forces us to have such a clumsy API. Can you please take a look at
>> the Redis python clients and see if there is a better way to support this?
>>
>> https://redis.io/clients#python
>>
>> D.
>>
>
Re: Thin Client lib: Python
Posted by Dmitry Melnichuk <dm...@nobitlost.com>.
Dmitriy, Igor, Ilya, Sergey!
Thank you for sharing your ideas, concerns and criticism with me. I do
appreciate it.
I already made some changes in my API, influenced by your feedback. I
also plan to add a certain set of features, that will make my UX closer
to what you can see from other Ignite clients.
I stopped using `hashcode` in my examples. Integer cache IDs and cache
names now can be used interchangeably, with primary focus on cache names.
I will add a Cache class as a primary interface for cache operations, so
that earlier examples:
```
conn = Connection()
conn.connect('127.0.0.1', 10800)
cache_create(conn, 'my cache')
cache_put(conn, 'my cache', 'my key', 42)
result = my_cache.get('my key')
cache_destroy(conn, 'my cache')
conn.close()
```
could be reiterated as:
```
conn = Connection()
conn.connect('127.0.0.1', 10800)
my_cache = conn.create_cache('my cache')
my_cache.put('my key', 42)
result = my_cache.get('my key')
my_cache.destroy('my cache')
conn.close()
```
I will also make `Connection.connect()` accept any iterable (including
simple list) as a connection parameter. I will provide user with some
basic connection generators instead of what is done in my current
connection failover example.
On 07/27/2018 07:41 AM, Dmitriy Setrakyan wrote:
> Dmitriy,
>
> I would stop using the word "hashcode" in this context. Hash code has a
> special meaning in Ignite and is used to determine key-to-node affinity. I
> agree that passing "cache_name" is the best option. I have no idea when
> "cache_name" is not going to be known and do not think we need to support
> this case at all. My suggestion is to drop the cache_id use case altogether.
>
> Also I am really surprised that we do not have a cache abstraction in
> python and need to pass cache name and connection into every method. To be
> honest, this smells really bad that such a popular modern language like
> Python forces us to have such a clumsy API. Can you please take a look at
> the Redis python clients and see if there is a better way to support this?
>
> https://redis.io/clients#python
>
> D.
Re: Thin Client lib: Python
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Dmitriy,
I would stop using the word "hashcode" in this context. Hash code has a
special meaning in Ignite and is used to determine key-to-node affinity. I
agree that passing "cache_name" is the best option. I have no idea when
"cache_name" is not going to be known and do not think we need to support
this case at all. My suggestion is to drop the cache_id use case altogether.
Also I am really surprised that we do not have a cache abstraction in
python and need to pass cache name and connection into every method. To be
honest, this smells really bad that such a popular modern language like
Python forces us to have such a clumsy API. Can you please take a look at
the Redis python clients and see if there is a better way to support this?
https://redis.io/clients#python
D.
On Thu, Jul 26, 2018 at 9:51 AM, Dmitry Melnichuk <
dmitry.melnichuk@nobitlost.com> wrote:
> Hi, Ilya!
>
> I considered this option. Indeed, the code would look cleaner if only one
> kind of identifier (preferably the human-readable name) was used. But there
> can be a hypothetical situation, when the user is left with hash code only.
> (For example, obtained from some other API.) It would be sad to have an
> identifier and not be able to use it.
>
> Now I really think about using hash codes and names interchangeably, so
> both
>
> ```
> cache_put(conn, 'my-cache', value=1, key='a')
> ```
>
> and
>
>
> ```
> cache_put(conn, my_hash_code, value=1, key='a')
> ```
>
> will be allowed.
>
> This will be a minor complication on my side, and quite reasonable one.
>
>
> On 07/26/2018 10:44 PM, Ilya Kasnacheev wrote:
>
>> Hello!
>>
>> Why not use cache name as string here, instead of cache_id()?
>>
>> cache_put(conn, 'my-cache', value=1, key='a')
>>
>> Regards,
>>
>>
Re: Thin Client lib: Python
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I expect the effect to be neligible, and UX gain is well worth it.
In case it will ever become a sensitive issue, hashcode-based operation
might be retained as mentioned earlier.
Regards,
--
Ilya Kasnacheev
2018-07-26 17:39 GMT+03:00 Igor Sapego <is...@apache.org>:
> Ilya,
>
> This may affect performance in a negative way, as it requires
> additional hashcode calculation on every cache operation.
>
> Best Regards,
> Igor
>
>
> On Thu, Jul 26, 2018 at 5:02 PM Ilya Kasnacheev <ilya.kasnacheev@gmail.com
> >
> wrote:
>
> > Hello!
> >
> > I think that having both options is indeed preferable.
> >
> > Regards,
> >
> > --
> > Ilya Kasnacheev
> >
> > 2018-07-26 16:51 GMT+03:00 Dmitry Melnichuk <
> > dmitry.melnichuk@nobitlost.com>
> > :
> >
> > > Hi, Ilya!
> > >
> > > I considered this option. Indeed, the code would look cleaner if only
> one
> > > kind of identifier (preferably the human-readable name) was used. But
> > there
> > > can be a hypothetical situation, when the user is left with hash code
> > only.
> > > (For example, obtained from some other API.) It would be sad to have an
> > > identifier and not be able to use it.
> > >
> > > Now I really think about using hash codes and names interchangeably, so
> > > both
> > >
> > > ```
> > > cache_put(conn, 'my-cache', value=1, key='a')
> > > ```
> > >
> > > and
> > >
> > >
> > > ```
> > > cache_put(conn, my_hash_code, value=1, key='a')
> > > ```
> > >
> > > will be allowed.
> > >
> > > This will be a minor complication on my side, and quite reasonable one.
> > >
> > >
> > > On 07/26/2018 10:44 PM, Ilya Kasnacheev wrote:
> > >
> > >> Hello!
> > >>
> > >> Why not use cache name as string here, instead of cache_id()?
> > >>
> > >> cache_put(conn, 'my-cache', value=1, key='a')
> > >>
> > >> Regards,
> > >>
> > >>
> >
>
Re: Thin Client lib: Python
Posted by Igor Sapego <is...@apache.org>.
Ilya,
This may affect performance in a negative way, as it requires
additional hashcode calculation on every cache operation.
Best Regards,
Igor
On Thu, Jul 26, 2018 at 5:02 PM Ilya Kasnacheev <il...@gmail.com>
wrote:
> Hello!
>
> I think that having both options is indeed preferable.
>
> Regards,
>
> --
> Ilya Kasnacheev
>
> 2018-07-26 16:51 GMT+03:00 Dmitry Melnichuk <
> dmitry.melnichuk@nobitlost.com>
> :
>
> > Hi, Ilya!
> >
> > I considered this option. Indeed, the code would look cleaner if only one
> > kind of identifier (preferably the human-readable name) was used. But
> there
> > can be a hypothetical situation, when the user is left with hash code
> only.
> > (For example, obtained from some other API.) It would be sad to have an
> > identifier and not be able to use it.
> >
> > Now I really think about using hash codes and names interchangeably, so
> > both
> >
> > ```
> > cache_put(conn, 'my-cache', value=1, key='a')
> > ```
> >
> > and
> >
> >
> > ```
> > cache_put(conn, my_hash_code, value=1, key='a')
> > ```
> >
> > will be allowed.
> >
> > This will be a minor complication on my side, and quite reasonable one.
> >
> >
> > On 07/26/2018 10:44 PM, Ilya Kasnacheev wrote:
> >
> >> Hello!
> >>
> >> Why not use cache name as string here, instead of cache_id()?
> >>
> >> cache_put(conn, 'my-cache', value=1, key='a')
> >>
> >> Regards,
> >>
> >>
>
Re: Thin Client lib: Python
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I think that having both options is indeed preferable.
Regards,
--
Ilya Kasnacheev
2018-07-26 16:51 GMT+03:00 Dmitry Melnichuk <dm...@nobitlost.com>
:
> Hi, Ilya!
>
> I considered this option. Indeed, the code would look cleaner if only one
> kind of identifier (preferably the human-readable name) was used. But there
> can be a hypothetical situation, when the user is left with hash code only.
> (For example, obtained from some other API.) It would be sad to have an
> identifier and not be able to use it.
>
> Now I really think about using hash codes and names interchangeably, so
> both
>
> ```
> cache_put(conn, 'my-cache', value=1, key='a')
> ```
>
> and
>
>
> ```
> cache_put(conn, my_hash_code, value=1, key='a')
> ```
>
> will be allowed.
>
> This will be a minor complication on my side, and quite reasonable one.
>
>
> On 07/26/2018 10:44 PM, Ilya Kasnacheev wrote:
>
>> Hello!
>>
>> Why not use cache name as string here, instead of cache_id()?
>>
>> cache_put(conn, 'my-cache', value=1, key='a')
>>
>> Regards,
>>
>>
Re: Thin Client lib: Python
Posted by Dmitry Melnichuk <dm...@nobitlost.com>.
Hi, Ilya!
I considered this option. Indeed, the code would look cleaner if only
one kind of identifier (preferably the human-readable name) was used.
But there can be a hypothetical situation, when the user is left with
hash code only. (For example, obtained from some other API.) It would be
sad to have an identifier and not be able to use it.
Now I really think about using hash codes and names interchangeably, so both
```
cache_put(conn, 'my-cache', value=1, key='a')
```
and
```
cache_put(conn, my_hash_code, value=1, key='a')
```
will be allowed.
This will be a minor complication on my side, and quite reasonable one.
On 07/26/2018 10:44 PM, Ilya Kasnacheev wrote:
> Hello!
>
> Why not use cache name as string here, instead of cache_id()?
>
> cache_put(conn, 'my-cache', value=1, key='a')
>
> Regards,
>
Re: Thin Client lib: Python
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Why not use cache name as string here, instead of cache_id()?
cache_put(conn, 'my-cache', value=1, key='a')
Regards,
--
Ilya Kasnacheev
2018-07-26 5:11 GMT+03:00 Dmitry Melnichuk <dm...@nobitlost.com>:
> Either
>
> ```
> conn = Connection('example.com', 10800)
> cache_put(conn, cache_id('my-cache'), 'a', 1)
> ```
>
> or
>
> ```
> conn = Connection('example.com', 10800)
> my_cache_id = cache_id('my-cache')
> cache_put(conn, my_cache_id, 'a', 1)
> ```
>
> It is also possible to give parameters names, if you like to.
>
> ```
> conn = Connection('example.com', 10800)
> cache_put(conn, cache_id('my-cache'), key='a', value=1)
> ```
>
> This should also work, but not recommended:
>
> ```
> conn = Connection('example.com', 10800)
> cache_put(conn, cache_id('my-cache'), value=1, key='a')
> ```
>
> All variants can coexist in one user program.
>
>
> On 07/26/2018 05:46 AM, Dmitriy Setrakyan wrote:
>
>> I am still confused. Let's work through an example. Suppose I have a cache
>> named "my_cache" and I want to put an entry with key "a" and value "1".
>>
>> In Java, this code will look like this:
>>
>>
>> *IgniteCache<...> myCache = ignite.cache("my-cache");myCache.put("a",
>>> 1);*
>>>
>>
>>
>> How will the same code look in Python?
>>
>> D.
>>
>
Re: Thin Client lib: Python
Posted by Dmitry Melnichuk <dm...@nobitlost.com>.
Either
```
conn = Connection('example.com', 10800)
cache_put(conn, cache_id('my-cache'), 'a', 1)
```
or
```
conn = Connection('example.com', 10800)
my_cache_id = cache_id('my-cache')
cache_put(conn, my_cache_id, 'a', 1)
```
It is also possible to give parameters names, if you like to.
```
conn = Connection('example.com', 10800)
cache_put(conn, cache_id('my-cache'), key='a', value=1)
```
This should also work, but not recommended:
```
conn = Connection('example.com', 10800)
cache_put(conn, cache_id('my-cache'), value=1, key='a')
```
All variants can coexist in one user program.
On 07/26/2018 05:46 AM, Dmitriy Setrakyan wrote:
> I am still confused. Let's work through an example. Suppose I have a cache
> named "my_cache" and I want to put an entry with key "a" and value "1".
>
> In Java, this code will look like this:
>
>
>> *IgniteCache<...> myCache = ignite.cache("my-cache");myCache.put("a", 1);*
>
>
> How will the same code look in Python?
>
> D.
Re: Thin Client lib: Python
Posted by Dmitriy Setrakyan <ds...@apache.org>.
I am still confused. Let's work through an example. Suppose I have a cache
named "my_cache" and I want to put an entry with key "a" and value "1".
In Java, this code will look like this:
> *IgniteCache<...> myCache = ignite.cache("my-cache");myCache.put("a", 1);*
How will the same code look in Python?
D.
On Wed, Jul 25, 2018 at 5:08 PM, Dmitry Melnichuk <
dmitry.melnichuk@nobitlost.com> wrote:
> Igor,
>
> That is a very good point. It just did not cross my mind during the
> implementation of this function, that the cache identifier can be abstract.
> I will fix that.
>
>
> On 07/26/2018 01:46 AM, Igor Sapego wrote:
>
>> Well, at least name should be changed, IMO, as the API function name
>> should say what we do, and not how we do it. For example, cache_id()
>> looks better to me than hashcode().
>>
>> Best Regards,
>> Igor
>>
>
Re: Thin Client lib: Python
Posted by Dmitry Melnichuk <dm...@nobitlost.com>.
Igor,
That is a very good point. It just did not cross my mind during the
implementation of this function, that the cache identifier can be
abstract. I will fix that.
On 07/26/2018 01:46 AM, Igor Sapego wrote:
> Well, at least name should be changed, IMO, as the API function name
> should say what we do, and not how we do it. For example, cache_id()
> looks better to me than hashcode().
>
> Best Regards,
> Igor
Re: Thin Client lib: Python
Posted by Igor Sapego <is...@apache.org>.
Well, at least name should be changed, IMO, as the API function name
should say what we do, and not how we do it. For example, cache_id()
looks better to me than hashcode().
Best Regards,
Igor
On Wed, Jul 25, 2018 at 6:22 PM Dmitry Melnichuk <
dmitry.melnichuk@nobitlost.com> wrote:
> Dmitriy,
>
> The short answer is: Python client uses hash code (or any cache
> identifier) instead of Java Cache object, since such an abstraction is
> not necessary in Python.
>
> As for why (IMO) Cache object may be needed in Java, but not in Python −
> the main reason is the difference in typing systems of both languages.
>
> On 07/25/2018 06:52 PM, Dmitriy Setrakyan wrote:
> > Dmitriy,
> >
> > To be honest, I got a bit lost in such a big email. Can you tell us
> briefly
> > - why do we not need the hash code on API in Java and we do need it in
> > Python?
> >
> > D.
> >
>
Re: Thin Client lib: Python
Posted by Dmitry Melnichuk <dm...@nobitlost.com>.
Dmitriy,
The short answer is: Python client uses hash code (or any cache
identifier) instead of Java Cache object, since such an abstraction is
not necessary in Python.
As for why (IMO) Cache object may be needed in Java, but not in Python −
the main reason is the difference in typing systems of both languages.
On 07/25/2018 06:52 PM, Dmitriy Setrakyan wrote:
> Dmitriy,
>
> To be honest, I got a bit lost in such a big email. Can you tell us briefly
> - why do we not need the hash code on API in Java and we do need it in
> Python?
>
> D.
>
Re: Thin Client lib: Python
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Dmitriy,
To be honest, I got a bit lost in such a big email. Can you tell us briefly
- why do we not need the hash code on API in Java and we do need it in
Python?
D.
On Wed, Jul 25, 2018 at 3:29 AM, Dmitry Melnichuk <
dmitry.melnichuk@nobitlost.com> wrote:
> Hello, Dmitriy, Igor!
>
> Dmitriy, as you have noted, most cache API functions are parameterized
> with the hash codes, and the function for creating the hash codes is
> documented. I personally see no harm in using hash codes as it is defined
> by low-level client operations. But it may be better to enforce the use of
> cache names throughout the Python library, or use names and hash codes
> interchangeably where possible.
>
> Igor, you are right, due to the procedural nature of my client library,
> there is no cache class in it. Caches in Ignite, from Python perspective,
> are one-off, unique, immutable objects. The client code will not benefit
> much from associating any additional context with them. On the contrary,
> handling cache objects in its string or integer representation may
> sometimes be advantageous.
>
> I guess, the implied question here is: why I chose a procedural approach
> first of all? Well, I am glad to answer it too, although the answer will be
> more verbose.
>
> Python is often referred as a multi-paradigm programming language, but at
> heart it is a procedural scripting language. It has no obligatory object
> orientation that could influence Java or C# clients' architecture.
>
> It is also worth noting, that Python has such an elaborate and versatile
> built-in types, so its classes (`object` descendants) is never meant to be
> used as plain data containers. That is unlike JavaScript, where `object` is
> a legitimate default complex data type. Any combination of
> list/set/dict/defaultdict/OrderedDict is always a better, more Pythonic
> choice; getter/setter semantics is considered an anti-pattern.
>
> Considering the above, the client API architecture could be chosen
> depending on the application field. Aiming for the most versatility,
> however, I decided to implement lower-level procedural API.
>
> “Lower-level” in this case does not mean extra efforts for the end user.
> Feel free to review the rest of the examples and make sure that those code
> snippets actually take less and do more, comparing to the equivalent code
> using other client APIs.
>
> If that is still not satisfying, any kind of object-oriented wrapper can
> be built on top of the procedural implementation with little time and
> effort. In fact, before taking the present course of actions, I considered
> the following implementations of other services' clients:
>
> - key-value libraries, like redis-py, that have their methods incapsulated
> in connection class,
>
> - RDBMS libraries like psycopg2 with their cursor classes − it could be an
> architectural choice for SQL and scan queries-oriented client,
>
> - boto3 (Amazon S3 client) have its methods incapsulated in bucket
> objects, somewhat analogous to Ignite caches, but notably different: a) S3
> supports only one data type: file, b) bucket can be reconfigured on the
> fly, unlike Ignite cache,
>
> - there could be other architectural choices too.
>
> I will be glad to answer your further questions. If you have suggestions
> about higher-level abstractions and their use cases, please let me know.
>
> Dmitry
>
> On 07/24/2018 10:43 PM, Dmitriy Setrakyan wrote:> Yes, looks strange? Why
> is the hash code part of API?
>
> >
> > On Tue, Jul 24, 2018, 13:38 Igor Sapego <is...@apache.org> wrote:
> >
> >> Ok, I've quickly looked through API and I have a questions. Here
> >> is the code from the example:
> >>
> >> cache_name = 'my cache'
> >> hash_code = hashcode(cache_name)
> >> cache_create(conn, cache_name)
> >> result = cache_put(conn, hash_code, 'my key', 42)
> >>
> >> I'm not familiar with python, so here is the question. Why there is
> >> no "cache" class, and functions with hash code are used?
> >>
> >> Best Regards,
> >> Igor
>
Re: Thin Client lib: Python
Posted by Dmitry Melnichuk <dm...@nobitlost.com>.
Hello, Dmitriy, Igor!
Dmitriy, as you have noted, most cache API functions are parameterized
with the hash codes, and the function for creating the hash codes is
documented. I personally see no harm in using hash codes as it is
defined by low-level client operations. But it may be better to enforce
the use of cache names throughout the Python library, or use names and
hash codes interchangeably where possible.
Igor, you are right, due to the procedural nature of my client library,
there is no cache class in it. Caches in Ignite, from Python
perspective, are one-off, unique, immutable objects. The client code
will not benefit much from associating any additional context with them.
On the contrary, handling cache objects in its string or integer
representation may sometimes be advantageous.
I guess, the implied question here is: why I chose a procedural approach
first of all? Well, I am glad to answer it too, although the answer will
be more verbose.
Python is often referred as a multi-paradigm programming language, but
at heart it is a procedural scripting language. It has no obligatory
object orientation that could influence Java or C# clients' architecture.
It is also worth noting, that Python has such an elaborate and versatile
built-in types, so its classes (`object` descendants) is never meant to
be used as plain data containers. That is unlike JavaScript, where
`object` is a legitimate default complex data type. Any combination of
list/set/dict/defaultdict/OrderedDict is always a better, more Pythonic
choice; getter/setter semantics is considered an anti-pattern.
Considering the above, the client API architecture could be chosen
depending on the application field. Aiming for the most versatility,
however, I decided to implement lower-level procedural API.
“Lower-level” in this case does not mean extra efforts for the end user.
Feel free to review the rest of the examples and make sure that those
code snippets actually take less and do more, comparing to the
equivalent code using other client APIs.
If that is still not satisfying, any kind of object-oriented wrapper can
be built on top of the procedural implementation with little time and
effort. In fact, before taking the present course of actions, I
considered the following implementations of other services' clients:
- key-value libraries, like redis-py, that have their methods
incapsulated in connection class,
- RDBMS libraries like psycopg2 with their cursor classes − it could be
an architectural choice for SQL and scan queries-oriented client,
- boto3 (Amazon S3 client) have its methods incapsulated in bucket
objects, somewhat analogous to Ignite caches, but notably different: a)
S3 supports only one data type: file, b) bucket can be reconfigured on
the fly, unlike Ignite cache,
- there could be other architectural choices too.
I will be glad to answer your further questions. If you have suggestions
about higher-level abstractions and their use cases, please let me know.
Dmitry
On 07/24/2018 10:43 PM, Dmitriy Setrakyan wrote:> Yes, looks strange?
Why is the hash code part of API?
>
> On Tue, Jul 24, 2018, 13:38 Igor Sapego <is...@apache.org> wrote:
>
>> Ok, I've quickly looked through API and I have a questions. Here
>> is the code from the example:
>>
>> cache_name = 'my cache'
>> hash_code = hashcode(cache_name)
>> cache_create(conn, cache_name)
>> result = cache_put(conn, hash_code, 'my key', 42)
>>
>> I'm not familiar with python, so here is the question. Why there is
>> no "cache" class, and functions with hash code are used?
>>
>> Best Regards,
>> Igor
Re: Thin Client lib: Python
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Yes, looks strange? Why is the hash code part of API?
On Tue, Jul 24, 2018, 13:38 Igor Sapego <is...@apache.org> wrote:
> Ok, I've quickly looked through API and I have a questions. Here
> is the code from the example:
>
> cache_name = 'my cache'
> hash_code = hashcode(cache_name)
> cache_create(conn, cache_name)
> result = cache_put(conn, hash_code, 'my key', 42)
>
> I'm not familiar with python, so here is the question. Why there is
> no "cache" class, and functions with hash code are used?
>
> Best Regards,
> Igor
>
>
> On Mon, Jul 2, 2018 at 9:01 PM Dmitry Melnichuk <
> dmitry.melnichuk@nobitlost.com> wrote:
>
> > Hi Igor,
> >
> > I totally agree with the point that auto-generated things does not
> > belong to the source tree.
> >
> > I already made short instructions on how to generate HTML documents with
> > Sphinx in README file. The instructions assume that the user is able to
> > use the common tools (Python, pip, virtualenv) and have them installed.
> > From the beginning of the development we found it inconvenient for
> > those who wish to just read the documents and not fire up the whole
> > development process. That's why I included the prebuilt documents in the
> > repository.
> >
> > Note also, that Python programs, unlike Java ones, do not have the
> > additional build step and always distributed as text. So each user will
> > have to build the documents from the source to read it.
> >
> > To leverage this condition, most of Python package developers use
> > readthedocs.org service (freemium, engine is under MIT license) to
> > publish their documentation online. Unfortunately, the service can not
> > be used by our project because of the directory structure. RTD treats
> > the git root as a home to `docs` directory to build Sphinx docs out of
> > it. In our case it would build anything but the documents we need from
> > upstream git source. This situation can be fixed by developing Python
> > thin client in a separate git repository and including it in upstream as
> > a git sub-module, but I see this solution is too radical.
> >
> > Anyway, I am removing the prebuilt docs from now on, just as you
> > suggested. Hope that potential usability problems might be addressed in
> > the future.
> >
> > On 07/02/2018 09:41 PM, Igor Sapego wrote:
> > > By the way, guys,
> > >
> > > I can see generated docs under source control. We do not store them
> > > this way in Ignite. Instead, we have separate step during release,
> where
> > > we generate them and include in binary distribution.
> > >
> > > So you should remove documents from Git and provide us with
> instructions
> > > on how to generate docs during release.
> > >
> > > Best Regards,
> > > Igor
> > >
> >
>
Re: Thin Client lib: Python
Posted by Igor Sapego <is...@apache.org>.
Ok, I've quickly looked through API and I have a questions. Here
is the code from the example:
cache_name = 'my cache'
hash_code = hashcode(cache_name)
cache_create(conn, cache_name)
result = cache_put(conn, hash_code, 'my key', 42)
I'm not familiar with python, so here is the question. Why there is
no "cache" class, and functions with hash code are used?
Best Regards,
Igor
On Mon, Jul 2, 2018 at 9:01 PM Dmitry Melnichuk <
dmitry.melnichuk@nobitlost.com> wrote:
> Hi Igor,
>
> I totally agree with the point that auto-generated things does not
> belong to the source tree.
>
> I already made short instructions on how to generate HTML documents with
> Sphinx in README file. The instructions assume that the user is able to
> use the common tools (Python, pip, virtualenv) and have them installed.
> From the beginning of the development we found it inconvenient for
> those who wish to just read the documents and not fire up the whole
> development process. That's why I included the prebuilt documents in the
> repository.
>
> Note also, that Python programs, unlike Java ones, do not have the
> additional build step and always distributed as text. So each user will
> have to build the documents from the source to read it.
>
> To leverage this condition, most of Python package developers use
> readthedocs.org service (freemium, engine is under MIT license) to
> publish their documentation online. Unfortunately, the service can not
> be used by our project because of the directory structure. RTD treats
> the git root as a home to `docs` directory to build Sphinx docs out of
> it. In our case it would build anything but the documents we need from
> upstream git source. This situation can be fixed by developing Python
> thin client in a separate git repository and including it in upstream as
> a git sub-module, but I see this solution is too radical.
>
> Anyway, I am removing the prebuilt docs from now on, just as you
> suggested. Hope that potential usability problems might be addressed in
> the future.
>
> On 07/02/2018 09:41 PM, Igor Sapego wrote:
> > By the way, guys,
> >
> > I can see generated docs under source control. We do not store them
> > this way in Ignite. Instead, we have separate step during release, where
> > we generate them and include in binary distribution.
> >
> > So you should remove documents from Git and provide us with instructions
> > on how to generate docs during release.
> >
> > Best Regards,
> > Igor
> >
>
Re: Thin Client lib: Python
Posted by Dmitry Melnichuk <dm...@nobitlost.com>.
Hi Igor,
I totally agree with the point that auto-generated things does not
belong to the source tree.
I already made short instructions on how to generate HTML documents with
Sphinx in README file. The instructions assume that the user is able to
use the common tools (Python, pip, virtualenv) and have them installed.
From the beginning of the development we found it inconvenient for
those who wish to just read the documents and not fire up the whole
development process. That's why I included the prebuilt documents in the
repository.
Note also, that Python programs, unlike Java ones, do not have the
additional build step and always distributed as text. So each user will
have to build the documents from the source to read it.
To leverage this condition, most of Python package developers use
readthedocs.org service (freemium, engine is under MIT license) to
publish their documentation online. Unfortunately, the service can not
be used by our project because of the directory structure. RTD treats
the git root as a home to `docs` directory to build Sphinx docs out of
it. In our case it would build anything but the documents we need from
upstream git source. This situation can be fixed by developing Python
thin client in a separate git repository and including it in upstream as
a git sub-module, but I see this solution is too radical.
Anyway, I am removing the prebuilt docs from now on, just as you
suggested. Hope that potential usability problems might be addressed in
the future.
On 07/02/2018 09:41 PM, Igor Sapego wrote:
> By the way, guys,
>
> I can see generated docs under source control. We do not store them
> this way in Ignite. Instead, we have separate step during release, where
> we generate them and include in binary distribution.
>
> So you should remove documents from Git and provide us with instructions
> on how to generate docs during release.
>
> Best Regards,
> Igor
>