[jira] [Work logged] (GOBBLIN-1558) Make BasePublisher record detailed folders in state when parent-folder doesn't exist

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GOBBLIN-1558?focusedWorklogId=661137&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-661137 ]

ASF GitHub Bot logged work on GOBBLIN-1558:
-------------------------------------------

                Author: ASF GitHub Bot
            Created on: 06/Oct/21 19:06
            Start Date: 06/Oct/21 19:06
    Worklog Time Spent: 10m 
      Work Description: jobar opened a new pull request #3409:
URL: https://github.com/apache/gobblin/pull/3409


   Dear Gobblin maintainers,
   
   This is my second PR, I hope I'm doing it in a way that works for you folks :)
   
   
   ### JIRA
   - [ ] My PR addresses the following [Gobblin JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references them in the PR title.
       - https://issues.apache.org/jira/browse/GOBBLIN-1558
   
   
   ### Description
   - [ ] Make BaseDataPublisher publish to new folders in a way that preserve detailed subfolder information in PUBLISHER_DIRS state value. In previous version, when publishing to non-existing parent directory (first run of job type for instance), the BaseDataPublisher wouldn't recurse into written folders but moves the top written folder at once, leading to the state's PUBLISHER_DIRS value containing the top-level written folder, not the contain the detailed published folder.
   
   
   
   ### Tests
   - [ ] Not yet done, will come soon.
   
   
   ### Commits
   - [ ] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)":
       1. Subject is separated from body by a blank line
       2. Subject is limited to 50 characters
       3. Subject does not end with a period
       4. Subject uses the imperative mood ("add", not "adding")
       5. Body wraps at 72 characters
       6. Body explains "what" and "why", not "how"
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscribe@gobblin.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 661137)
    Remaining Estimate: 0h
            Time Spent: 10m

> Make BasePublisher record detailed folders in state when parent-folder doesn't exist
> ------------------------------------------------------------------------------------
>
>                 Key: GOBBLIN-1558
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-1558
>             Project: Apache Gobblin
>          Issue Type: Improvement
>          Components: gobblin-core
>            Reporter: Joseph Allemandou
>            Assignee: Abhishek Tiwari
>            Priority: Minor
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> When publishing to non-existing parent directory (first run of job type for instance), the BaseDataPublisher doesn't recurse into written folders but moves the top written folder at once (line 429). While being correct, this prevent the state's PUBLISHER_DIRS value to contain the detailed published folder.
> The update consists of a new method
> {code:java}
> addWriterOutputToNewDir{code}
> that mimic the already existing
> {code:java}
> addWriterOutputToExistingDir{code}
> and is applied when the parent folder is not present, making the PUBLISHER_DIRS value being filled with detailed folder value.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)