You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@flink.apache.org by aljoscha <gi...@git.apache.org> on 2016/07/05 16:25:24 UTC

[GitHub] flink pull request #2202: [FLINK-4149] Fix Serialization of NFA in AbstractK...

GitHub user aljoscha opened a pull request:

    https://github.com/apache/flink/pull/2202

    [FLINK-4149] Fix Serialization of NFA in AbstractKeyedCEPPatternOperator

    NFA is Serializable and has readObject()/writeObject() methods. In
    AbstractKeyedCEPPatternOperator a KryoSerializer was used as the
    TypeSerializer for the ValueState that holds NFA instances. Kryo does
    not call readObject()/writeObject() therefore the state of the NFA was
    invalid after deserialization.
    
    This change adds a new TypeSerializer for NFA that uses
    Java Serialization. In the long run it will be better to get rid of the
    readObject()/writeObject() methods and instead efficiently serialize
    using a specialized TypeSerializer.
    
    This also adds a test that verifies that checkpoint/restore work as expected.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/aljoscha/flink cep/fix-serialization

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/flink/pull/2202.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2202
    
----
commit ba1deb8cae7223f3ca9def1b09b311e7d7889a8a
Author: Aljoscha Krettek <al...@gmail.com>
Date:   2016-07-05T15:56:49Z

    Replace StreamEvent by StreamRecord in CEP Tests

commit 19aaa005e0389b512a563cf28e82d8ad07c4a194
Author: Aljoscha Krettek <al...@gmail.com>
Date:   2016-07-05T15:58:45Z

    [FLINK-4149] Fix Serialization of NFA in AbstractKeyedCEPPatternOperator
    
    NFA is Serializable and has readObject()/writeObject() methods. In
    AbstractKeyedCEPPatternOperator a KryoSerializer was used as the
    TypeSerializer for the ValueState that holds NFA instances. Kryo does
    not call readObject()/writeObject() therefore the state of the NFA was
    invalid after deserialization.
    
    This change adds a new TypeSerializer for NFA that uses
    Java Serialization. In the long run it will be better to get rid of the
    readObject()/writeObject() methods and instead efficiently serialize
    using a specialized TypeSerializer.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #2202: [FLINK-4149] Fix Serialization of NFA in AbstractK...

Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/2202#discussion_r70449753
  
    --- Diff: flink-libraries/flink-cep/src/main/java/org/apache/flink/cep/nfa/NFA.java ---
    @@ -454,4 +458,126 @@ static String generateStateName(final String name, final int index) {
     			return name + "_" + index;
     		}
     	}
    +
    +	/**
    +	 * {@link TypeSerializer} for {@link NFA} that uses Java Serialization.
    +	 */
    +	public static class Serializer<T> extends TypeSerializer<NFA<T>> {
    +		private static final long serialVersionUID = 1L;
    +
    +		@Override
    +		public boolean isImmutableType() {
    +			return false;
    +		}
    +
    +		@Override
    +		public TypeSerializer<NFA<T>> duplicate() {
    +			return this;
    +		}
    +
    +		@Override
    +		public NFA<T> createInstance() {
    +			return null;
    +		}
    +
    +		@Override
    +		public NFA<T> copy(NFA<T> from) {
    +			try {
    +				ByteArrayOutputStream baos = new ByteArrayOutputStream();
    +				ObjectOutputStream oos = new ObjectOutputStream(baos);
    +
    +				oos.writeObject(from);
    +
    +				oos.close();
    +				baos.close();
    +
    +				byte[] data = baos.toByteArray();
    +
    +				ByteArrayInputStream bais = new ByteArrayInputStream(data);
    +				ObjectInputStream ois = new ObjectInputStream(bais);
    +
    +				@SuppressWarnings("unchecked")
    +				NFA<T> copy = (NFA<T>) ois.readObject();
    +				return copy;
    +			} catch (IOException|ClassNotFoundException e) {
    +				throw new RuntimeException("Could not copy NFA.", e);
    +			}
    +		}
    +
    +		@Override
    +		public NFA<T> copy(NFA<T> from, NFA<T> reuse) {
    +			return copy(from);
    +		}
    +
    +		@Override
    +		public int getLength() {
    +			return 0;
    +		}
    +
    +		@Override
    +		public void serialize(NFA<T> record, DataOutputView target) throws IOException {
    +			try {
    +				ByteArrayOutputStream baos = new ByteArrayOutputStream();
    +				ObjectOutputStream oos = new ObjectOutputStream(baos);
    +
    +				oos.writeObject(record);
    +
    +				oos.close();
    +				baos.close();
    +
    +				byte[] data = baos.toByteArray();
    +
    +				target.writeInt(data.length);
    +				target.write(data);
    +			} catch (IOException e) {
    +				throw new RuntimeException("Could not serialize NFA.", e);
    +			}
    +		}
    +
    +		@Override
    +		public NFA<T> deserialize(DataInputView source) throws IOException {
    +			try {
    +				int size = source.readInt();
    +
    +				byte[] data = new byte[size];
    +
    +				source.readFully(data);
    +
    +				ByteArrayInputStream bais = new ByteArrayInputStream(data);
    +				ObjectInputStream ois = new ObjectInputStream(bais);
    +
    +				@SuppressWarnings("unchecked")
    +				NFA<T> copy = (NFA<T>) ois.readObject();
    +				return copy;
    --- End diff --
    
    Same here with the `DataInputViewStream`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #2202: [FLINK-4149] Fix Serialization of NFA in AbstractK...

Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/2202#discussion_r70449634
  
    --- Diff: flink-libraries/flink-cep/src/main/java/org/apache/flink/cep/nfa/NFA.java ---
    @@ -454,4 +458,126 @@ static String generateStateName(final String name, final int index) {
     			return name + "_" + index;
     		}
     	}
    +
    +	/**
    +	 * {@link TypeSerializer} for {@link NFA} that uses Java Serialization.
    +	 */
    +	public static class Serializer<T> extends TypeSerializer<NFA<T>> {
    +		private static final long serialVersionUID = 1L;
    +
    +		@Override
    +		public boolean isImmutableType() {
    +			return false;
    +		}
    +
    +		@Override
    +		public TypeSerializer<NFA<T>> duplicate() {
    +			return this;
    +		}
    +
    +		@Override
    +		public NFA<T> createInstance() {
    +			return null;
    +		}
    +
    +		@Override
    +		public NFA<T> copy(NFA<T> from) {
    +			try {
    +				ByteArrayOutputStream baos = new ByteArrayOutputStream();
    +				ObjectOutputStream oos = new ObjectOutputStream(baos);
    +
    +				oos.writeObject(from);
    +
    +				oos.close();
    +				baos.close();
    +
    +				byte[] data = baos.toByteArray();
    +
    +				ByteArrayInputStream bais = new ByteArrayInputStream(data);
    +				ObjectInputStream ois = new ObjectInputStream(bais);
    +
    +				@SuppressWarnings("unchecked")
    +				NFA<T> copy = (NFA<T>) ois.readObject();
    +				return copy;
    +			} catch (IOException|ClassNotFoundException e) {
    +				throw new RuntimeException("Could not copy NFA.", e);
    +			}
    +		}
    +
    +		@Override
    +		public NFA<T> copy(NFA<T> from, NFA<T> reuse) {
    +			return copy(from);
    +		}
    +
    +		@Override
    +		public int getLength() {
    +			return 0;
    +		}
    +
    +		@Override
    +		public void serialize(NFA<T> record, DataOutputView target) throws IOException {
    +			try {
    +				ByteArrayOutputStream baos = new ByteArrayOutputStream();
    +				ObjectOutputStream oos = new ObjectOutputStream(baos);
    +
    +				oos.writeObject(record);
    +
    +				oos.close();
    +				baos.close();
    +
    +				byte[] data = baos.toByteArray();
    +
    +				target.writeInt(data.length);
    +				target.write(data);
    --- End diff --
    
    Can't we use the `DataOutputViewStream`?
    
    ```
    ObjectOutputStream oos = new ObjectOutputStream(new DataOutputViewStream(target));
    oos.writeObject(record);
    oos.close();
    ```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #2202: [FLINK-4149] Fix Serialization of NFA in AbstractK...

Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/2202#discussion_r70448543
  
    --- Diff: flink-libraries/flink-cep/src/main/java/org/apache/flink/cep/nfa/NFA.java ---
    @@ -19,12 +19,16 @@
     package org.apache.flink.cep.nfa;
     
     import com.google.common.collect.LinkedHashMultimap;
    +import org.apache.commons.io.output.ByteArrayOutputStream;
    --- End diff --
    
    Why do we use once the `org.apache.commons.io.output.BAOS` and then the `java.io.BAIS`?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink issue #2202: [FLINK-4149] Fix Serialization of NFA in AbstractKeyedCEP...

Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the issue:

    https://github.com/apache/flink/pull/2202
  
    R: @tillrohrmann for review


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink issue #2202: [FLINK-4149] Fix Serialization of NFA in AbstractKeyedCEP...

Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the issue:

    https://github.com/apache/flink/pull/2202
  
    Thanks for the review @tillrohrmann! The `commons.io` was just an IDE mess-up. The `equals` I will fix.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink issue #2202: [FLINK-4149] Fix Serialization of NFA in AbstractKeyedCEP...

Posted by shashank734 <gi...@git.apache.org>.
Github user shashank734 commented on the issue:

    https://github.com/apache/flink/pull/2202
  
    Again getting this error. After changing state backend from Rocksdb it's working fine.
    
    ```
    04/12/2017 10:05:04	Job execution switched to status FAILING.
    java.lang.RuntimeException: Could not deserialize NFA.
    	at org.apache.flink.cep.nfa.NFA$Serializer.deserialize(NFA.java:538)
    	at org.apache.flink.cep.nfa.NFA$Serializer.deserialize(NFA.java:469)
    	at org.apache.flink.contrib.streaming.state.RocksDBValueState.value(RocksDBValueState.java:81)
    	at org.apache.flink.cep.operator.AbstractKeyedCEPPatternOperator.getNFA(AbstractKeyedCEPPatternOperator.java:124)
    	at org.apache.flink.cep.operator.AbstractCEPBasePatternOperator.processElement(AbstractCEPBasePatternOperator.java:72)
    	at org.apache.flink.cep.operator.AbstractKeyedCEPPatternOperator.processElement(AbstractKeyedCEPPatternOperator.java:162)
    	at org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:185)
    	at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:63)
    	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:272)
    	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:655)
    	at java.lang.Thread.run(Thread.java:745)
    Caused by: java.lang.ClassNotFoundException: co.ronak.nto.Job$$anon$18$$anon$21$$anon$3
    	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    	at java.lang.Class.forName0(Native Method)
    	at java.lang.Class.forName(Class.java:348)
    	at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:626)
    	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1613)
    	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1518)
    	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1774)
    	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
    	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
    	at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:501)
    	at org.apache.flink.api.scala.typeutils.TraversableSerializer.readObject(TraversableSerializer.scala:53)
    	at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:497)
    	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
    	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900)
    	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
    	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
    	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
    	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924)
    	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
    	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
    	at java.io.ObjectInputStream.readArray(ObjectInputStream.java:1707)
    	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1345)
    	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
    	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924)
    	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
    	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
    	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
    	at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:501)
    	at org.apache.flink.cep.NonDuplicatingTypeSerializer.readObject(NonDuplicatingTypeSerializer.java:190)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:497)
    	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
    	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900)
    	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
    	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
    	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
    	at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:501)
    	at org.apache.flink.cep.nfa.NFA.readObject(NFA.java:394)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:497)
    	at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1017)
    	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1900)
    	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
    	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
    	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:371)
    	at org.apache.flink.cep.nfa.NFA$Serializer.deserialize(NFA.java:535)
    	... 10 more
    ```



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #2202: [FLINK-4149] Fix Serialization of NFA in AbstractK...

Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/2202#discussion_r70450213
  
    --- Diff: flink-libraries/flink-cep/src/main/java/org/apache/flink/cep/nfa/NFA.java ---
    @@ -454,4 +458,126 @@ static String generateStateName(final String name, final int index) {
     			return name + "_" + index;
     		}
     	}
    +
    +	/**
    +	 * {@link TypeSerializer} for {@link NFA} that uses Java Serialization.
    +	 */
    +	public static class Serializer<T> extends TypeSerializer<NFA<T>> {
    +		private static final long serialVersionUID = 1L;
    +
    +		@Override
    +		public boolean isImmutableType() {
    +			return false;
    +		}
    +
    +		@Override
    +		public TypeSerializer<NFA<T>> duplicate() {
    +			return this;
    +		}
    +
    +		@Override
    +		public NFA<T> createInstance() {
    +			return null;
    +		}
    +
    +		@Override
    +		public NFA<T> copy(NFA<T> from) {
    +			try {
    +				ByteArrayOutputStream baos = new ByteArrayOutputStream();
    +				ObjectOutputStream oos = new ObjectOutputStream(baos);
    +
    +				oos.writeObject(from);
    +
    +				oos.close();
    +				baos.close();
    +
    +				byte[] data = baos.toByteArray();
    +
    +				ByteArrayInputStream bais = new ByteArrayInputStream(data);
    +				ObjectInputStream ois = new ObjectInputStream(bais);
    +
    +				@SuppressWarnings("unchecked")
    +				NFA<T> copy = (NFA<T>) ois.readObject();
    +				return copy;
    +			} catch (IOException|ClassNotFoundException e) {
    +				throw new RuntimeException("Could not copy NFA.", e);
    +			}
    +		}
    +
    +		@Override
    +		public NFA<T> copy(NFA<T> from, NFA<T> reuse) {
    +			return copy(from);
    +		}
    +
    +		@Override
    +		public int getLength() {
    +			return 0;
    +		}
    +
    +		@Override
    +		public void serialize(NFA<T> record, DataOutputView target) throws IOException {
    +			try {
    +				ByteArrayOutputStream baos = new ByteArrayOutputStream();
    +				ObjectOutputStream oos = new ObjectOutputStream(baos);
    +
    +				oos.writeObject(record);
    +
    +				oos.close();
    +				baos.close();
    +
    +				byte[] data = baos.toByteArray();
    +
    +				target.writeInt(data.length);
    +				target.write(data);
    +			} catch (IOException e) {
    +				throw new RuntimeException("Could not serialize NFA.", e);
    +			}
    +		}
    +
    +		@Override
    +		public NFA<T> deserialize(DataInputView source) throws IOException {
    +			try {
    +				int size = source.readInt();
    +
    +				byte[] data = new byte[size];
    +
    +				source.readFully(data);
    +
    +				ByteArrayInputStream bais = new ByteArrayInputStream(data);
    +				ObjectInputStream ois = new ObjectInputStream(bais);
    +
    +				@SuppressWarnings("unchecked")
    +				NFA<T> copy = (NFA<T>) ois.readObject();
    +				return copy;
    +			} catch (IOException|ClassNotFoundException e) {
    +				throw new RuntimeException("Could not deserialize NFA.", e);
    +			}		}
    +
    +		@Override
    +		public NFA<T> deserialize(NFA<T> reuse, DataInputView source) throws IOException {
    +			return deserialize(source);
    +		}
    +
    +		@Override
    +		public void copy(DataInputView source, DataOutputView target) throws IOException {
    +			int size = source.readInt();
    +			target.writeInt(size);
    +			target.write(source, size);
    +		}
    +
    +		@Override
    +		public boolean equals(Object obj) {
    +			return obj instanceof Serializer;
    --- End diff --
    
    In order to make the equals relation symmetric, we should `return obj instanceof Serializer && ((Serializer) obj).canEqual(this);`.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink pull request #2202: [FLINK-4149] Fix Serialization of NFA in AbstractK...

Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha closed the pull request at:

    https://github.com/apache/flink/pull/2202


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink issue #2202: [FLINK-4149] Fix Serialization of NFA in AbstractKeyedCEP...

Posted by aljoscha <gi...@git.apache.org>.
Github user aljoscha commented on the issue:

    https://github.com/apache/flink/pull/2202
  
    Thanks @shashank734 I created another issue since this is a different (but somewhat related) problem: https://issues.apache.org/jira/browse/FLINK-6318


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] flink issue #2202: [FLINK-4149] Fix Serialization of NFA in AbstractKeyedCEP...

Posted by tillrohrmann <gi...@git.apache.org>.
Github user tillrohrmann commented on the issue:

    https://github.com/apache/flink/pull/2202
  
    Changes look good to me. Thanks for fixing the CEP issues @aljoscha :-). I had some minor comments concerning the NFA serializer.
    
    Before merging it would be good to rebase on the latest master and rerun Travis, because many builds failed due to a problem with the Kinesis connector. Thus, there might be some problems shadowed by the failing Kinesis tests (not really expecting this).


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---