You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@accumulo.apache.org by "cshannon (via GitHub)" <gi...@apache.org> on 2023/02/21 15:44:18 UTC
[GitHub] [accumulo-classloaders] cshannon opened a new issue, #18: Enabling the VFS classloader as the system classloader with debug logging causes Accumulo start up errors
cshannon opened a new issue, #18:
URL: https://github.com/apache/accumulo-classloaders/issues/18
Currently debug logging in the AccumuloVFSClassLoader dumps everything into System.out because it can't use log4j as it's not available for the system classloader. So when the [ClusterConfigParser](https://github.com/apache/accumulo/blob/main/core/src/main/java/org/apache/accumulo/core/conf/cluster/ClusterConfigParser.java) is used by the accumulo command to [parse](https://github.com/apache/accumulo/blob/main/assemble/bin/accumulo-cluster#L69) the yaml it breaks as extra debug statements make it into the config file that the scripts are not expecting later when the file is sourced by bash.
To fix this we should allow setting a file as a system property that can be used to write log statements to instead of standard out. We can just create a PrintWriter and write to the file directly so that we don't interfere with standard out at all. The debug flag that exists now could just be changed to a file location instead.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-classloaders] ctubbsii commented on issue #18: Enabling the VFS classloader as the system classloader with debug logging causes Accumulo start up errors
Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on issue #18:
URL: https://github.com/apache/accumulo-classloaders/issues/18#issuecomment-1443962850
> isn't super helpful
I actually find this very helpful to quickly visualize what's happening. Thank you.
I don't think changing it to write to a file is a good solution. We don't want to be leaving a bunch of files around. The command itself is supposed to just emit the requested node list to the console.
I think if it can't load log4j or other `java.class.path` classes, that's a bug in the VFS classloader. The VFS classloader, even though it is taking over for the system classloader, should still defer to things found in `java.class.path`, as its parent, so it can still load its own dependencies.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-classloaders] cshannon commented on issue #18: Enabling the VFS classloader as the system classloader with debug logging causes Accumulo start up errors
Posted by "cshannon (via GitHub)" <gi...@apache.org>.
cshannon commented on issue #18:
URL: https://github.com/apache/accumulo-classloaders/issues/18#issuecomment-1451020131
This has been resolved by https://github.com/apache/accumulo/pull/3205
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-classloaders] cshannon closed issue #18: Enabling the VFS classloader as the system classloader with debug logging causes Accumulo start up errors
Posted by "cshannon (via GitHub)" <gi...@apache.org>.
cshannon closed issue #18: Enabling the VFS classloader as the system classloader with debug logging causes Accumulo start up errors
URL: https://github.com/apache/accumulo-classloaders/issues/18
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-classloaders] cshannon commented on issue #18: Enabling the VFS classloader as the system classloader with debug logging causes Accumulo start up errors
Posted by "cshannon (via GitHub)" <gi...@apache.org>.
cshannon commented on issue #18:
URL: https://github.com/apache/accumulo-classloaders/issues/18#issuecomment-1444590114
So it looks like we can't use SLF4J because it can't use the service provider during VM init. Instead of I will have a PR shortly with a small modification to ClusterConfigParser that allows us to write the a file directly instead of piping Standard out so we only capture the config lines we want and not the rest of System.out in the accumulo-cluster script.
```
Error occurred during initialization of VM
java.util.ServiceConfigurationError: org.slf4j.spi.SLF4JServiceProvider: not accessible to unnamed module @6504e3b2 during VM init
at java.util.ServiceLoader.fail(java.base@11.0.18/ServiceLoader.java:589)
at java.util.ServiceLoader.<init>(java.base@11.0.18/ServiceLoader.java:517)
at java.util.ServiceLoader.load(java.base@11.0.18/ServiceLoader.java:1692)
at org.slf4j.LoggerFactory.findServiceProviders(LoggerFactory.java:102)
at org.slf4j.LoggerFactory.bind(LoggerFactory.java:147)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:139)
at org.slf4j.LoggerFactory.getProvider(LoggerFactory.java:422)
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:408)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
at org.apache.accumulo.classloader.vfs.AccumuloVFSClassLoader.<clinit>(AccumuloVFSClassLoader.java:82)
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(java.base@11.0.18/Native Method)
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(java.base@11.0.18/NativeConstructorAccessorImpl.java:62)
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(java.base@11.0.18/DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(java.base@11.0.18/Constructor.java:490)
at java.lang.ClassLoader.initSystemClassLoader(java.base@11.0.18/ClassLoader.java:1978)
at java.lang.System.initPhase3(java.base@11.0.18/System.java:2074)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-classloaders] dlmarion commented on issue #18: Enabling the VFS classloader as the system classloader with debug logging causes Accumulo start up errors
Posted by "dlmarion (via GitHub)" <gi...@apache.org>.
dlmarion commented on issue #18:
URL: https://github.com/apache/accumulo-classloaders/issues/18#issuecomment-1444628973
I remember something like this when I was working on the classloader as the system classloader. There are some things you can't do because the VM is not initialized yet.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-classloaders] ctubbsii commented on issue #18: Enabling the VFS classloader as the system classloader with debug logging causes Accumulo start up errors
Posted by "ctubbsii (via GitHub)" <gi...@apache.org>.
ctubbsii commented on issue #18:
URL: https://github.com/apache/accumulo-classloaders/issues/18#issuecomment-1443775457
In future, please paste the error observed whenever possible, rather than just stating something like "causes ... start up errors". If the actual error message is included in the ticket, then it is easier for others to understand what's going on without trying to reproduce it from scratch. It also helps them know whether or not the error they are seeing is the same one you're seeing, if they do try to reproduce it. This lowers the bar to contributing and makes it easier to participate in the discussion of potential solutions.
If the stack trace is long, you can either truncate it to just the meaningful bits, or put it in [a collapsible details block](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/organizing-information-with-collapsed-sections#creating-a-collapsed-section) or both, to make it easier to read the text surrounding it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-classloaders] cshannon commented on issue #18: Enabling the VFS classloader as the system classloader with debug logging causes Accumulo start up errors
Posted by "cshannon (via GitHub)" <gi...@apache.org>.
cshannon commented on issue #18:
URL: https://github.com/apache/accumulo-classloaders/issues/18#issuecomment-1443816223
I didn't post the error as there isn't a stack trace as it's a bash error because it can't parse the config file. It's just an error to the output on start up and isn't super helpful but it looks like the following (I stripped out my IP and paths, etc)
```
$ ./bin/uno start accumulo
ZooKeeper JMX enabled by default
Using config:<pathhidden>conf/zoo.cfg
Starting zookeeper ... STARTED
Starting namenodes on [localhost]
Starting datanodes
Starting secondary namenodes [localhost]
Starting resourcemanager
Starting nodemanagers
OpenJDK 64-Bit Server VM warning: Archived non-system classes are disabled because the java.system.class.loader property is specified (value = "org.apache.accumulo.classloader.vfs.AccumuloVFSClassLoader"). To use archived non-system classes, this property must be not be set
WARN: 1677250840499 AccumuloVFSClassLoader: vfs.cache.dir system property not set, using default of java.io.tmpdir
WARN: 1677250841475 AccumuloVFSClassLoader: vfs.cache.dir system property not set, using default of java.io.tmpdir
/tmp/tmp.Lg3ibqz8wc: line 1: DEBUG:: command not found
/tmp/tmp.Lg3ibqz8wc: line 2: DEBUG:: command not found
/tmp/tmp.Lg3ibqz8wc: line 3: DEBUG:: command not found
/tmp/tmp.Lg3ibqz8wc: line 4: DEBUG:: command not found
/tmp/tmp.Lg3ibqz8wc: line 5: DEBUG:: command not found
/tmp/tmp.Lg3ibqz8wc: line 6: syntax error near unexpected token `('
/tmp/tmp.Lg3ibqz8wc: line 6: `DEBUG: 1677250840018 AccumuloVFSClassLoader: getDelegateClassLoader() initializing VFS.'
CONFIG: /tmp/tmp.Lg3ibqz8wc
ERROR: managers not found in <pathhidden>/conf/cluster.yaml
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [accumulo-classloaders] cshannon commented on issue #18: Enabling the VFS classloader as the system classloader with debug logging causes Accumulo start up errors
Posted by "cshannon (via GitHub)" <gi...@apache.org>.
cshannon commented on issue #18:
URL: https://github.com/apache/accumulo-classloaders/issues/18#issuecomment-1444188107
I played around with the prototype a bit and I didn't like writing to a file either from the classloader. Another option that I'm working on is to update the ClusterConfigParser to allow giving a file to output. Right now it always [writes](https://github.com/apache/accumulo/blob/cc44add8bcf9d69057f18d19e6fe9d6b9328575f/core/src/main/java/org/apache/accumulo/core/conf/cluster/ClusterConfigParser.java#L143) to System.out but we could optionally provide an argument for a file that way the configs get written to a file instead so that nothing from Standard.out would interfere. The way it is now if anything (whether it's the VFS classloader or something else) were to write to System.out it could break parsing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@accumulo.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org