You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@bahir.apache.org by ScrapCodes <gi...@git.apache.org> on 2018/05/10 11:54:38 UTC
[GitHub] bahir pull request #65: [Bahir-164] Port Mqtt sql source to datasource v2 AP...
GitHub user ScrapCodes opened a pull request:
https://github.com/apache/bahir/pull/65
[Bahir-164] Port Mqtt sql source to datasource v2 API.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ScrapCodes/bahir BAHIR-164
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/bahir/pull/65.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #65
----
commit efde0e9b778d5dff6602a2e18d47c06108136157
Author: Prashant Sharma <pr...@...>
Date: 2018-04-27T07:09:35Z
Porting some changes from spark build, to fix my local bahir build
commit 37487c131f602c2879daf5760396f40cab306f11
Author: Prashant Sharma <pr...@...>
Date: 2018-05-09T07:24:17Z
Migrating Mqtt spark structured streaming connector to DatasourceV2 API.
commit bd1125a96adbe47ebc4d6e51eb4fc623d4fc861c
Author: Prashant Sharma <pr...@...>
Date: 2018-05-10T11:22:39Z
Fixed tests and added mechanism for deecting redelivery.
commit 4786d3b791c16beb591ed2059af4c4abbb822c70
Author: Prashant Sharma <pr...@...>
Date: 2018-05-10T11:28:08Z
Removed support for detecting redelivery,
because Mqtt does not give a way to distinguish a normal message
from a re-attempted delivery.
On each reattempt seqid of the received message is changed and duplicate is set
to false. This way there is no way to say a message was a normal message or a
redelivery.
This patch also removes the support for fetching last processed offset from disk.
Because, doing so does not serve a purpose, as we do not support complete replay of
messages anyway.
closes #19 and BAHIR-46
----
---
[GitHub] bahir issue #65: [BAHIR-164][BAHIR-165] Port Mqtt sql source to datasource v...
Posted by lresende <gi...@git.apache.org>.
Github user lresende commented on the issue:
https://github.com/apache/bahir/pull/65
Thank you for these updates @ScrapCodes. Just clarifying, this should not be applied until we do the Bahir release that supports Spark 2.3, correct?
---
[GitHub] bahir issue #65: [BAHIR-164][BAHIR-165] Port Mqtt sql source to datasource v...
Posted by ScrapCodes <gi...@git.apache.org>.
Github user ScrapCodes commented on the issue:
https://github.com/apache/bahir/pull/65
Yes, this is step one in supporting spark 2.3. The other connectors need to be updated before bahir fully supports spark 2.3+
---
[GitHub] bahir issue #65: [BAHIR-164][BAHIR-165] Port Mqtt sql source to datasource v...
Posted by ScrapCodes <gi...@git.apache.org>.
Github user ScrapCodes commented on the issue:
https://github.com/apache/bahir/pull/65
ok to test
---
[GitHub] bahir issue #65: [BAHIR-164][BAHIR-165] Port Mqtt sql source to datasource v...
Posted by ScrapCodes <gi...@git.apache.org>.
Github user ScrapCodes commented on the issue:
https://github.com/apache/bahir/pull/65
@lresende and @ckadner please take a look.
---
[GitHub] bahir pull request #65: [BAHIR-164][BAHIR-165] Port Mqtt sql source to datas...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/bahir/pull/65
---