You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "davidvoit (via GitHub)" <gi...@apache.org> on 2023/05/17 12:08:31 UTC
[GitHub] [camel] davidvoit opened a new pull request, #10131: File Changed ReadLock Strategy with minAge only looks for lastModified
davidvoit opened a new pull request, #10131:
URL: https://github.com/apache/camel/pull/10131
If a file is copied with the original lastModified date, files are processes instantly. Also the file change detection is disabled if a copy takes more than minAge time. Instead of comparing startTime with lastModified on each iteration we are comparing it with now(). If files are created anew this should be the same behaviour, and files copied with preserved lastModifed are still correctly checked.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] davidvoit commented on pull request #10131: CAMEL-19378: File Changed ReadLock Strategy with minAge only looks for lastModified
Posted by "davidvoit (via GitHub)" <gi...@apache.org>.
davidvoit commented on PR #10131:
URL: https://github.com/apache/camel/pull/10131#issuecomment-1557618201
Sorry I only checked mvn test for the component, now I learned something new here and will also run mvn test on atleast core or a full mvn test (which takes ages on my setup :-( ) next time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] davsclaus commented on pull request #10131: File Changed ReadLock Strategy with minAge only looks for lastModified
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus commented on PR #10131:
URL: https://github.com/apache/camel/pull/10131#issuecomment-1553403772
In light of this I think your patch is better than the current code.
For current time, you can likely use System.currentTimeMillis() instead
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] davidvoit commented on pull request #10131: File Changed ReadLock Strategy with minAge only looks for lastModified
Posted by "davidvoit (via GitHub)" <gi...@apache.org>.
davidvoit commented on PR #10131:
URL: https://github.com/apache/camel/pull/10131#issuecomment-1551632497
The use case that a file needs to be in the folder for n seconds (to avoid
slow transfers which would not be noticed by the change lock) could also be
achieved with increasing the interval and not decreasing like its hinted in
the documentation. I will play with this stuff on monday
But non theless I would recommend to at least patch the filesize
comparision into the this strategy. This way we would have at least two
stat calls on the input file, instead of directly loading it
Otavio Rodolfo Piske ***@***.***> schrieb am Mi., 17. Mai
2023, 16:51:
> I wonder if implementing a separate lock strategy would do the trick here?
> That would give users a choice ... IMHO
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/camel/pull/10131#issuecomment-1551546451>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAADDHWURE4MNQLNOCB3GXTXGTQXZANCNFSM6AAAAAAYE7VNPM>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] davsclaus commented on pull request #10131: File Changed ReadLock Strategy with minAge only looks for lastModified
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus commented on PR #10131:
URL: https://github.com/apache/camel/pull/10131#issuecomment-1553401565
Yeah I can see the point with having length check in that 2nd OR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] davidvoit commented on pull request #10131: CAMEL-19378: File Changed ReadLock Strategy with minAge only looks for lastModified
Posted by "davidvoit (via GitHub)" <gi...@apache.org>.
davidvoit commented on PR #10131:
URL: https://github.com/apache/camel/pull/10131#issuecomment-1557592147
Sorry about that i look into this tomorrow
Claus Ibsen ***@***.***> schrieb am Mo., 22. Mai 2023, 19:06:
> Just a heads up that one of the core tests is failing
>
> FileChangedReadLockMinAgeShortCircuitTest.testChangedReadLockMinAge
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/camel/pull/10131#issuecomment-1557589510>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAADDHRT7KLQOJRALFKPUXDXHOMITANCNFSM6AAAAAAYE7VNPM>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] davsclaus merged pull request #10131: CAMEL-19378: File Changed ReadLock Strategy with minAge only looks for lastModified
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus merged PR #10131:
URL: https://github.com/apache/camel/pull/10131
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] davidvoit commented on pull request #10131: File Changed ReadLock Strategy with minAge only looks for lastModified
Posted by "davidvoit (via GitHub)" <gi...@apache.org>.
davidvoit commented on PR #10131:
URL: https://github.com/apache/camel/pull/10131#issuecomment-1551512120
Yeah but this ist not only a Windows thing.
A "cp -p" could also lead to broken files on unix systems as the length
comparision is skipped and files are instantly triggered
Claus Ibsen ***@***.***> schrieb am Mi., 17. Mai 2023, 14:56:
> ***@***.**** requested changes on this pull request.
>
> as you say this PR needs more investigations.
> As the "windows file hack" should not cause problems for everyone else.
>
> Marking "req for change" to avoid merging this too soon
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/camel/pull/10131#pullrequestreview-1430621131>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAADDHVHXM2IRNJCHNE3BOTXGTDGXANCNFSM6AAAAAAYE7VNPM>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] davidvoit commented on pull request #10131: File Changed ReadLock Strategy with minAge only looks for lastModified
Posted by "davidvoit (via GitHub)" <gi...@apache.org>.
davidvoit commented on PR #10131:
URL: https://github.com/apache/camel/pull/10131#issuecomment-1553409837
Sure. I will be back in the office on monday and then will modify the patch
and create a jira issue.
Claus Ibsen ***@***.***> schrieb am Do., 18. Mai 2023, 19:46:
> I wonder if you could create a JIRA ticket for this issue also. This helps
> to backport to patch branches as we need to track changes in ticket system
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/camel/pull/10131#issuecomment-1553405249>, or
> unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAADDHRHUC2AFN2TNS3IA6LXGZN7LANCNFSM6AAAAAAYE7VNPM>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] davsclaus commented on pull request #10131: CAMEL-19378: File Changed ReadLock Strategy with minAge only looks for lastModified
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus commented on PR #10131:
URL: https://github.com/apache/camel/pull/10131#issuecomment-1557589510
Just a heads up that one of the core tests is failing
FileChangedReadLockMinAgeShortCircuitTest.testChangedReadLockMinAge
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] davidvoit commented on pull request #10131: File Changed ReadLock Strategy with minAge only looks for lastModified
Posted by "davidvoit (via GitHub)" <gi...@apache.org>.
davidvoit commented on PR #10131:
URL: https://github.com/apache/camel/pull/10131#issuecomment-1551320364
I know the patch is not optimal, it's a first draft to more cleary show the changes. Let's see this first draft as an RFC.
In the end we don't need to differentiate between minage != 0 and minage == 0. We can just always do the currentTime > minTriggerTime.
Just a short word why we are having some problem with the current implemntation. We are loading files from Windows clients. Windows client save files without modifing lastModifed. As our input files are quite big, the camel route get's triggered before the copying is finished.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] davsclaus commented on pull request #10131: CAMEL-19378: File Changed ReadLock Strategy with minAge only looks for lastModified
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus commented on PR #10131:
URL: https://github.com/apache/camel/pull/10131#issuecomment-1557611515
The unit test is using the old behaviour, and with the new code, then the read lock cannot be acquired on first polll. I am changing the unit test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] github-actions[bot] commented on pull request #10131: CAMEL-19378: File Changed ReadLock Strategy with minAge only looks for lastModified
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10131:
URL: https://github.com/apache/camel/pull/10131#issuecomment-1557248001
:leftwards_arrow_with_hook: There are either **too many** changes to be tested in this PR or the code **needs be rebased**: (50 components likely to be affected)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] github-actions[bot] commented on pull request #10131: File Changed ReadLock Strategy with minAge only looks for lastModified
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10131:
URL: https://github.com/apache/camel/pull/10131#issuecomment-1551276679
:star2: Thank you for your contribution to the Apache Camel project! :star2:
:camel: Maintainers, please note that first-time contributors *require manual approval* for the GitHub Actions to run.
:warning: Please note that the changes on this PR may be **tested automatically** if they change components.
If necessary Apache Camel Committers may access logs and test results in the job summaries!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] github-actions[bot] commented on pull request #10131: File Changed ReadLock Strategy with minAge only looks for lastModified
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10131:
URL: https://github.com/apache/camel/pull/10131#issuecomment-1551360512
### Components tested:
| Total | Tested | Failed :x: | Passed :white_check_mark: |
| --- | --- | --- | --- |
| 1 | 1 | 0 | 1 |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] davidvoit commented on pull request #10131: CAMEL-19378: File Changed ReadLock Strategy with minAge only looks for lastModified
Posted by "davidvoit (via GitHub)" <gi...@apache.org>.
davidvoit commented on PR #10131:
URL: https://github.com/apache/camel/pull/10131#issuecomment-1557155789
As discussed: Simplified the commit and created a jira issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] davsclaus commented on pull request #10131: File Changed ReadLock Strategy with minAge only looks for lastModified
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus commented on PR #10131:
URL: https://github.com/apache/camel/pull/10131#issuecomment-1553405249
I wonder if you could create a JIRA ticket for this issue also. This helps to backport to patch branches as we need to track changes in ticket system
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] orpiske commented on pull request #10131: File Changed ReadLock Strategy with minAge only looks for lastModified
Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske commented on PR #10131:
URL: https://github.com/apache/camel/pull/10131#issuecomment-1551546451
I wonder if implementing a separate lock strategy would do the trick here? That would give users a choice ... IMHO
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org