You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@lucene.apache.org by "velee (via GitHub)" <gi...@apache.org> on 2025/07/23 10:30:09 UTC
[I] FilterDirectory not delegate method copyFrom [lucene]
velee opened a new issue, #14985:
URL: https://github.com/apache/lucene/issues/14985
### Description
`org.apache.lucene.store.FilterDirectory` not delegate method `copyFrom`, which causes `org.apache.lucene.store.HardlinkCopyDirectoryWrapper` to be ineffective in the middle layer of the delegate chain.
such as delegate chain:
org.elasticsearch.index.store.Store$StoreDirectory
org.elasticsearch.index.store.ByteSizeCachingDirectory
org.apache.lucene.store.HardlinkCopyDirectoryWrapper
org.elasticsearch.index.store.FsDirectoryFactory$HybridDirectory
the wapper `org.apache.lucene.store.HardlinkCopyDirectoryWrapper` not effect,because `org.elasticsearch.index.store.Store$StoreDirectory`‘s method `copyFrom` extends from `org.apache.lucene.store.Directory`
### Version and environment details
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] FilterDirectory not delegate method copyFrom [lucene]
Posted by "velee (via GitHub)" <gi...@apache.org>.
velee commented on issue #14985:
URL: https://github.com/apache/lucene/issues/14985#issuecomment-3111662205
eg. delegate chain: `org.apache.lucene.misc.store.ByteWritesTrackingDirectoryWrapper` -> `org.apache.lucene.misc.store.HardlinkCopyDirectoryWrapper` -> `org.apache.lucene.store.MMapDirectory`
The expectation is to use hardlink instead of copy, but in fact copy is still used, because the call method `copyFrom` from `org.apache.lucene.misc.store.ByteWritesTrackingDirectoryWrapper` (it extend from `org.apache.lucene.store.Directory`).
If the `org.apache.lucene.store.FilterDirectory`'s method `copyFrom` is decorated, the `org.apache.lucene.misc.store.HardlinkCopyDirectoryWrapper`'s method `copyFrom` will be used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] FilterDirectory not delegate method copyFrom [lucene]
Posted by "velee (via GitHub)" <gi...@apache.org>.
velee commented on issue #14985:
URL: https://github.com/apache/lucene/issues/14985#issuecomment-3115980036
Yes, your thinking is correct, maybe should make `Directory#copyFrom` abstract, and implementation on `BaseDirectory`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] FilterDirectory not delegate method copyFrom [lucene]
Posted by "velee (via GitHub)" <gi...@apache.org>.
velee commented on issue #14985:
URL: https://github.com/apache/lucene/issues/14985#issuecomment-3111677746
eg. delegate chain: `org.apache.lucene.misc.store.ByteWritesTrackingDirectoryWrapper` -> `org.apache.lucene.store.TrackingDirectoryWrapper` -> `org.apache.lucene.store.LockValidatingDirectoryWrapper` -> `org.apache.lucene.store.MMapDirectory`
The `org.apache.lucene.store.TrackingDirectoryWrapper` and `org.apache.lucene.store.LockValidatingDirectoryWrapper`'s method `copyFrom` will have no effect.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] FilterDirectory not delegate method copyFrom [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on issue #14985:
URL: https://github.com/apache/lucene/issues/14985#issuecomment-3113735282
Maybe we should make `Directory#copyFrom` abstract in 11.0 so that `FilterDirectory#copyFrom` can delegate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] FilterDirectory not delegate method copyFrom [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on issue #14985:
URL: https://github.com/apache/lucene/issues/14985#issuecomment-3114138600
Yes, like I wrote above we try to only delegate abstract methods in our `Filter*` classes. For instance, imagine a `FilterDirectory` that adds some magic bytes to the beginning of every file. It overrides `createOutput` to write these bytes and `openInput` to skip these bytes. The default implementation of `copyFrom` only relies on `createOutput`, so it works. If now you start delegating `copyFrom` in `FilterDirectory`, it would copy files as-is without adding the magic bytes at the beginning. In general, this causes two problems:
- this may break existing implementations,
- to be correct, all `FilterDirectory` implementations should review whether they need to override `copyFrom`, while `copyFrom` is always correct today.
Your point is valid though, `copyFrom` being correct is one thing, but if it's inefficient, there's still a bug. In similar situations in the past, we made such methods abstract to convey the expectation that implementations should carefully consider whether/how to implement it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] FilterDirectory not delegate method copyFrom [lucene]
Posted by "jpountz (via GitHub)" <gi...@apache.org>.
jpountz commented on issue #14985:
URL: https://github.com/apache/lucene/issues/14985#issuecomment-3109844947
The guidance is that `Filter*` classes should delegate abstract methods, and not methods that have a default implementation. So I'd rather fix `HardlinkCopyDirectoryWrapper` than `FilterDirectory`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org
Re: [I] FilterDirectory not delegate method copyFrom [lucene]
Posted by "velee (via GitHub)" <gi...@apache.org>.
velee commented on issue #14985:
URL: https://github.com/apache/lucene/issues/14985#issuecomment-3113806511
Is there any problem delegate with the implementation method? If it is not override in the middle chain, it will finally execute to latest chain of `Directory#copyFrom`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@lucene.apache.org
For additional commands, e-mail: issues-help@lucene.apache.org