You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by ss...@gmail.com on 2024/01/09 06:59:05 UTC
Re: HDFS-17221: Pls review and merge NNThroughputBenchmark adoptation for RBF benchmarking PR
Hi Ayush!
Gentle reminder to review my PR:)
Please let me know if you have any questions, suggestions.
Alex.
> 28 дек. 2023 г., в 10:53, Александр Богданов <ss...@gmail.com> написал(а):
>
> Hi Ayush,
>
> I've updated my PR:
> * Removed RBF specifics.
> * Merged with HDFS-17272.
> * Updated MD file.
>
> Appreciate if you could review!:)
>
> https://github.com/apache/hadoop/pull/6230
>
> Alex.
>
> вт, 19 дек. 2023 г. в 14:41, <ss...@gmail.com>:
>>
>>
>> Hi Ayush,
>>
>> Thank you for swift response!
>>
>> Agree with your comments. HDFS-17272 partially solves the problem, but my change in addition allows to run different operations in different subdirs in parallel (as you said). Also I see that I’ve forgotten to update md file.
>>
>> Will update my PR tomorrow and drop you an email.
>>
>> Alex.
>>
>>>> 19 дек. 2023 г., в 11:13, Ayush Saxena <ay...@gmail.com> написал(а):
>>>
>>> Hi Alex,
>>> I haven't gone through the PR, I can check that over the weekend or
>>> next week, if nobody volunteers till then...
>>>
>>> I just had a quick pass, I don't think there is any need to mention
>>> RBF or something like that in the core HDFS code, if something is
>>> specific to RBF, that goes to RBF module, If I got your patch or
>>> intentions correct, you want to specify individual paths for each
>>> operations, create, mkdir. rather than all going to one single base
>>> dir.
>>>
>>> Firstly I feel this could have been achieved by playing with the mount
>>> table, mount /baseDir/<op> to your specific cluster, & it should work,
>>> right? RBF allows nested mount points, Anyway If I catch you right,
>>> you want to overwrite the /baseDir part with a custom param in
>>> /baseDir/<op> for each operation?
>>>
>>> If yes, then it is pretty generic, nothing related to RBF nor sub
>>> cluster, just take baseDir as param per operation and use it instead
>>> of /baseDir/<op>, BaseDir is now configurable post HDFS-17272, give a
>>> check to the ticket, & if it is global not per operation, make it work
>>> per operation
>>>
>>> -Ayush
>>>
>>>> On Tue, 19 Dec 2023 at 13:12, <ss...@gmail.com> wrote:
>>>>
>>>> Hi Ayush, team,
>>>>
>>>> Could you please have a look at my pull request?
>>>>
>>>> Any suggestion you could give to get it merged?
>>>>
>>>> Alex.
>>>>
>>>>
>>>>>> 16 нояб. 2023 г., в 15:29, Александр Богданов <ss...@gmail.com> написал(а):
>>>>>
>>>>> Dear HDFS devs, Ayush,
>>>>>
>>>>> Pipe line is green now, could you please review and merge if everything is OK?
>>>>>
>>>>> https://issues.apache.org/jira/browse/HDFS-17221
>>>>>
>>>>> https://github.com/apache/hadoop/pull/6230
>>>>>
>>>>> --
>>>>> Kind regards,
>>>>> Alexander Bogdanov
>
>
>
> --
> Kind regards,
> Alexander Bogdanov
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: HDFS-17221: Pls review and merge NNThroughputBenchmark adoptation for RBF benchmarking PR
Posted by Ayush Saxena <ay...@gmail.com>.
hey,
limited bandwidth, please feel free to reach out other folks
thnx
-Ayush
On Tue, 9 Jan 2024 at 12:29, <ss...@gmail.com> wrote:
>
> Hi Ayush!
>
> Gentle reminder to review my PR:)
> Please let me know if you have any questions, suggestions.
>
> Alex.
>
> > 28 дек. 2023 г., в 10:53, Александр Богданов <ss...@gmail.com> написал(а):
> >
> > Hi Ayush,
> >
> > I've updated my PR:
> > * Removed RBF specifics.
> > * Merged with HDFS-17272.
> > * Updated MD file.
> >
> > Appreciate if you could review!:)
> >
> > https://github.com/apache/hadoop/pull/6230
> >
> > Alex.
> >
> > вт, 19 дек. 2023 г. в 14:41, <ss...@gmail.com>:
> >>
> >>
> >> Hi Ayush,
> >>
> >> Thank you for swift response!
> >>
> >> Agree with your comments. HDFS-17272 partially solves the problem, but my change in addition allows to run different operations in different subdirs in parallel (as you said). Also I see that I’ve forgotten to update md file.
> >>
> >> Will update my PR tomorrow and drop you an email.
> >>
> >> Alex.
> >>
> >>>> 19 дек. 2023 г., в 11:13, Ayush Saxena <ay...@gmail.com> написал(а):
> >>>
> >>> Hi Alex,
> >>> I haven't gone through the PR, I can check that over the weekend or
> >>> next week, if nobody volunteers till then...
> >>>
> >>> I just had a quick pass, I don't think there is any need to mention
> >>> RBF or something like that in the core HDFS code, if something is
> >>> specific to RBF, that goes to RBF module, If I got your patch or
> >>> intentions correct, you want to specify individual paths for each
> >>> operations, create, mkdir. rather than all going to one single base
> >>> dir.
> >>>
> >>> Firstly I feel this could have been achieved by playing with the mount
> >>> table, mount /baseDir/<op> to your specific cluster, & it should work,
> >>> right? RBF allows nested mount points, Anyway If I catch you right,
> >>> you want to overwrite the /baseDir part with a custom param in
> >>> /baseDir/<op> for each operation?
> >>>
> >>> If yes, then it is pretty generic, nothing related to RBF nor sub
> >>> cluster, just take baseDir as param per operation and use it instead
> >>> of /baseDir/<op>, BaseDir is now configurable post HDFS-17272, give a
> >>> check to the ticket, & if it is global not per operation, make it work
> >>> per operation
> >>>
> >>> -Ayush
> >>>
> >>>> On Tue, 19 Dec 2023 at 13:12, <ss...@gmail.com> wrote:
> >>>>
> >>>> Hi Ayush, team,
> >>>>
> >>>> Could you please have a look at my pull request?
> >>>>
> >>>> Any suggestion you could give to get it merged?
> >>>>
> >>>> Alex.
> >>>>
> >>>>
> >>>>>> 16 нояб. 2023 г., в 15:29, Александр Богданов <ss...@gmail.com> написал(а):
> >>>>>
> >>>>> Dear HDFS devs, Ayush,
> >>>>>
> >>>>> Pipe line is green now, could you please review and merge if everything is OK?
> >>>>>
> >>>>> https://issues.apache.org/jira/browse/HDFS-17221
> >>>>>
> >>>>> https://github.com/apache/hadoop/pull/6230
> >>>>>
> >>>>> --
> >>>>> Kind regards,
> >>>>> Alexander Bogdanov
> >
> >
> >
> > --
> > Kind regards,
> > Alexander Bogdanov
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org