You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@paimon.apache.org by "klboke (via GitHub)" <gi...@apache.org> on 2025/08/25 07:03:12 UTC
[I] [Feature] pypaimon supports Python 3.6.x and other lower versions. [paimon]
klboke opened a new issue, #6140:
URL: https://github.com/apache/paimon/issues/6140
### Search before asking
- [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Motivation
In the field of algorithm training, there are still a large number of industrial-level training tasks using the combination of TensorFlow 1.12.x and Python 3.6. Due to inconsistencies in algorithm performance, it is very difficult to upgrade these legacy training tasks. Therefore, I hope that the pypaimon package can be compatible with and support these older versions.
### Solution
_No response_
### Anything else?
_No response_
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature] pypaimon supports Python 3.6.x and other lower versions. [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi commented on issue #6140:
URL: https://github.com/apache/paimon/issues/6140#issuecomment-3254389718
Fix this in https://github.com/apache/paimon/pull/6194
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature] pypaimon supports Python 3.6.x and other lower versions. [paimon]
Posted by "klboke (via GitHub)" <gi...@apache.org>.
klboke commented on issue #6140:
URL: https://github.com/apache/paimon/issues/6140#issuecomment-3265946344
pyarrow == 0.6.1 .
```python
Traceback (most recent call last):
File "/Users/kl/Applications/PyCharm Professional Edition.app/Contents/plugins/python-ce/helpers/pycharm/_jb_unittest_runner.py", line 39, in <module>
buffer=not JB_DISABLE_BUFFERING))
File "/opt/homebrew/anaconda3/envs/py36/lib/python3.6/unittest/main.py", line 94, in __init__
self.parseArgs(argv)
File "/opt/homebrew/anaconda3/envs/py36/lib/python3.6/unittest/main.py", line 141, in parseArgs
self.createTests()
File "/opt/homebrew/anaconda3/envs/py36/lib/python3.6/unittest/main.py", line 148, in createTests
self.module)
File "/opt/homebrew/anaconda3/envs/py36/lib/python3.6/unittest/loader.py", line 219, in loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
File "/opt/homebrew/anaconda3/envs/py36/lib/python3.6/unittest/loader.py", line 219, in <listcomp>
suites = [self.loadTestsFromName(name, module) for name in names]
File "/opt/homebrew/anaconda3/envs/py36/lib/python3.6/unittest/loader.py", line 153, in loadTestsFromName
module = __import__(module_name)
File "/Users/kl/githubnamespace/paimon/paimon-python/pypaimon/tests/reader_test_kl.py", line 28, in <module>
from pypaimon.api.auth import BearTokenAuthProvider
File "/Users/kl/githubnamespace/paimon/paimon-python/pypaimon/__init__.py", line 18, in <module>
from pypaimon.filesystem.pvfs import PaimonVirtualFileSystem
File "/Users/kl/githubnamespace/paimon/paimon-python/pypaimon/filesystem/pvfs.py", line 32, in <module>
from pypaimon.api.api_response import GetTableTokenResponse, GetTableResponse
File "/Users/kl/githubnamespace/paimon/paimon-python/pypaimon/api/api_response.py", line 24, in <module>
from pypaimon.schema.schema import Schema
File "/Users/kl/githubnamespace/paimon/paimon-python/pypaimon/schema/schema.py", line 24, in <module>
from pypaimon.schema.data_types import DataField, PyarrowFieldParser
File "/Users/kl/githubnamespace/paimon/paimon-python/pypaimon/schema/data_types.py", line 384, in <module>
class PyarrowFieldParser:
File "/Users/kl/githubnamespace/paimon/paimon-python/pypaimon/schema/data_types.py", line 387, in PyarrowFieldParser
def from_paimon_type(data_type: DataType) -> pyarrow.DataType:
AttributeError: module 'pyarrow' has no attribute 'DataType'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature] pypaimon supports Python 3.6.x and other lower versions. [paimon]
Posted by "JingsongLi (via GitHub)" <gi...@apache.org>.
JingsongLi closed issue #6140: [Feature] pypaimon supports Python 3.6.x and other lower versions.
URL: https://github.com/apache/paimon/issues/6140
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Feature] pypaimon supports Python 3.6.x and other lower versions. [paimon]
Posted by "klboke (via GitHub)" <gi...@apache.org>.
klboke commented on issue #6140:
URL: https://github.com/apache/paimon/issues/6140#issuecomment-3265891481
```python
def test_reader(self):
"""Example usage of RESTCatalogServer"""
options = {
'metastore': 'rest',
'uri': "https://cn-shanghai-vpc.dlf.aliyuncs.com",
'warehouse': 'lakehouse_rnd',
'dlf.region': 'cn-shanghai',
"token.provider": "dlf",
'dlf.access-key-id': os.getenv("DLF_ACCESS_KEY_ID"),
'dlf.access-key-secret': os.getenv("DLF_ACCESS_KEY_SECRET"),
'data.token.enabled': 'true'
}
rest_catalog = RESTCatalog(CatalogContext.create_from_options(Options(options)))
table = rest_catalog.get_table('adn.wide_table_200cols')
predicate_builder = table.new_read_builder().new_predicate_builder()
read_builder = (table.new_read_builder()
.with_projection(['f0', 'f1'])
.with_filter(
predicate=predicate_builder.equal("dt", "2025-09-01")))
table_read = read_builder.new_read()
splits = read_builder.new_scan().plan().splits()
print(table_read.to_arrow(splits).num_rows)
```
The code above raises the following exception — could this be caused by mismatched software versions? Could you list the compatible dependencies, for example the pyarrow version?
``` python
Error
Traceback (most recent call last):
File "/Users/kl/githubnamespace/paimon/paimon-python/pypaimon/tests/reader_test_kl.py", line 66, in test_reader
print(table_read.to_arrow(splits).num_rows)
File "/Users/kl/githubnamespace/paimon/paimon-python/pypaimon/read/table_read.py", line 64, in to_arrow
arrow_table = batch_reader.read_all()
File "pyarrow/ipc.pxi", line 556, in pyarrow.lib.RecordBatchReader.read_all
File "/Users/kl/githubnamespace/paimon/paimon-python/pypaimon/read/table_read.py", line 74, in _arrow_batch_generator
yield from iter(reader.read_arrow_batch, None)
File "/Users/kl/githubnamespace/paimon/paimon-python/pypaimon/read/reader/concat_batch_reader.py", line 43, in read_arrow_batch
self.current_reader = supplier()
File "/Users/kl/githubnamespace/paimon/paimon-python/pypaimon/read/split_read.py", line 81, in file_reader_supplier
self._get_final_read_data_fields(), self.push_down_predicate)
File "/Users/kl/githubnamespace/paimon/paimon-python/pypaimon/read/reader/format_pyarrow_reader.py", line 37, in __init__
self.reader = self.dataset.scanner(
AttributeError: 'pyarrow._dataset.FileSystemDataset' object has no attribute 'scanner'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@paimon.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org