You are viewing a plain text version of this content. The canonical link for it is here.

Posted to users@kafka.apache.org by Pratik Shah <ti...@gmail.com> on 2017/12/13 10:38:40 UTC

Need help in storing key along with value of _schemas topic using s3 sink connector.

Hi All,
          I am using kafka s3 sink connector ( version 4.0) to backup the
_schemas topic for its backup.
          The issue is that the content stored in S3 is only the value part
of the kafka message stored in _schemas topic.
          How can I configure the connector to store the key as well . I
have configured the key and value convertor to JSONConverter.
          But I am unable to get the expected output in the S3 object ?
Kindly provide your inputs.

Thank you in advance.
- Pratik

Re: Need help in storing key along with value of _schemas topic using s3 sink connector.

Posted by Pratik Shah <ti...@gmail.com>.

Hi Ted,
           Yes after I posted this query in this group , I also found an
open issue related to this.

https://github.com/confluentinc/kafka-connect-storage-cloud/issues/119
Thanks,
Pratik

On Wed, Dec 13, 2017 at 9:25 PM, Ted Yu <yu...@gmail.com> wrote:

> Have you checked s3 connector issue listing ?
>
> Cheers
>
> On Wed, Dec 13, 2017 at 2:38 AM, Pratik Shah <ti...@gmail.com> wrote:
>
> > Hi All,
> >           I am using kafka s3 sink connector ( version 4.0) to backup the
> > _schemas topic for its backup.
> >           The issue is that the content stored in S3 is only the value
> part
> > of the kafka message stored in _schemas topic.
> >           How can I configure the connector to store the key as well . I
> > have configured the key and value convertor to JSONConverter.
> >           But I am unable to get the expected output in the S3 object ?
> > Kindly provide your inputs.
> >
> > Thank you in advance.
> > - Pratik
> >
>

Re: Need help in storing key along with value of _schemas topic using s3 sink connector.

Posted by Ted Yu <yu...@gmail.com>.

Have you checked s3 connector issue listing ?

Cheers

On Wed, Dec 13, 2017 at 2:38 AM, Pratik Shah <ti...@gmail.com> wrote:

> Hi All,
>           I am using kafka s3 sink connector ( version 4.0) to backup the
> _schemas topic for its backup.
>           The issue is that the content stored in S3 is only the value part
> of the kafka message stored in _schemas topic.
>           How can I configure the connector to store the key as well . I
> have configured the key and value convertor to JSONConverter.
>           But I am unable to get the expected output in the S3 object ?
> Kindly provide your inputs.
>
> Thank you in advance.
> - Pratik
>