You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kafka.apache.org by Jun Rao <ju...@confluent.io> on 2017/06/27 05:26:44 UTC
Re: [kafka-clients] Re: [VOTE] 0.11.0.0 RC2
Hi, Ismael,
Thanks for running the release. +1. Verified quickstart on the 2.11 binary.
Jun
On Mon, Jun 26, 2017 at 3:53 PM, Ismael Juma <is...@gmail.com> wrote:
> Hi Vahid,
>
> There are a few known issues when running Kafka on Windows. A PR with some
> fixes is: https://github.com/apache/kafka/pull/3283. The fact that the
> index cannot be accessed indicates that it may be a similar issue. I
> suggest we move this discussion to the relevant JIRAs instead of the
> release thread.
>
> Ismael
>
> On Mon, Jun 26, 2017 at 11:25 PM, Vahid S Hashemian <
> vahidhashemian@us.ibm.com> wrote:
>
>> Hi Ismael,
>>
>> This is the output of core tests from the start until the first failed
>> test.
>>
>> kafka.admin.AdminRackAwareTest > testAssignmentWithRackAwareWithUnevenRacks
>> PASSED
>>
>> kafka.admin.AdminRackAwareTest > testAssignmentWith2ReplicasRackAware
>> PASSED
>>
>> kafka.admin.AdminRackAwareTest > testAssignmentWithRackAwareWithUnevenReplicas
>> PASSED
>>
>> kafka.admin.AdminRackAwareTest > testSkipBrokerWithReplicaAlreadyAssigned
>> PASSED
>>
>> kafka.admin.AdminRackAwareTest > testAssignmentWithRackAware PASSED
>>
>> kafka.admin.AdminRackAwareTest > testRackAwareExpansion PASSED
>>
>> kafka.admin.AdminRackAwareTest > testAssignmentWith2ReplicasRackAwareWith6Partitions
>> PASSED
>>
>> kafka.admin.AdminRackAwareTest > testAssignmentWith2ReplicasRac
>> kAwareWith6PartitionsAnd3Brokers PASSED
>>
>> kafka.admin.AdminRackAwareTest > testGetRackAlternatedBrokerListAndAssignReplicasToBrokers
>> PASSED
>>
>> kafka.admin.AdminRackAwareTest > testMoreReplicasThanRacks PASSED
>>
>> kafka.admin.AdminRackAwareTest > testSingleRack PASSED
>>
>> kafka.admin.AdminRackAwareTest > testAssignmentWithRackAwareWithRandomStartIndex
>> PASSED
>>
>> kafka.admin.AdminRackAwareTest > testLargeNumberPartitionsAssignment
>> PASSED
>>
>> kafka.admin.AdminRackAwareTest > testLessReplicasThanRacks PASSED
>>
>> kafka.admin.AclCommandTest > testInvalidAuthorizerProperty PASSED
>>
>> kafka.admin.ConfigCommandTest > testScramCredentials PASSED
>>
>> kafka.admin.ConfigCommandTest > shouldParseArgumentsForTopicsEntityType
>> PASSED
>>
>> kafka.admin.ConfigCommandTest > testUserClientQuotaOpts PASSED
>>
>> kafka.admin.ConfigCommandTest > shouldAddTopicConfig PASSED
>>
>> kafka.admin.ConfigCommandTest > shouldAddClientConfig PASSED
>>
>> kafka.admin.ConfigCommandTest > shouldDeleteBrokerConfig PASSED
>>
>> kafka.admin.DeleteConsumerGroupTest > testGroupWideDeleteInZKDoesNothingForActiveConsumerGroup
>> PASSED
>>
>> kafka.admin.ConfigCommandTest > testQuotaConfigEntity PASSED
>>
>> kafka.admin.ConfigCommandTest > shouldNotUpdateBrokerConfigIfMalformedBracketConfig
>> PASSED
>>
>> kafka.admin.ConfigCommandTest > shouldFailIfUnrecognisedEntityType PASSED
>>
>> kafka.admin.AdminTest > testBasicPreferredReplicaElection PASSED
>>
>> kafka.admin.ConfigCommandTest > shouldNotUpdateBrokerConfigIfNonExistingConfigIsDeleted
>> PASSED
>>
>> kafka.admin.AdminTest > testPreferredReplicaJsonData PASSED
>>
>> kafka.admin.BrokerApiVersionsCommandTest > checkBrokerApiVersionCommandOutput
>> PASSED
>>
>> kafka.admin.ReassignPartitionsCommandTest >
>> shouldRemoveThrottleReplicaListBasedOnProposedAssignment PASSED
>>
>> kafka.admin.ReassignPartitionsCommandTest >
>> shouldFindMovingReplicasMultipleTopics PASSED
>>
>> kafka.admin.ReassignPartitionsCommandTest >
>> shouldNotOverwriteExistingPropertiesWhenLimitIsAdded PASSED
>>
>> kafka.admin.ReassignPartitionsCommandTest >
>> shouldFindMovingReplicasMultipleTopicsAndPartitions PASSED
>>
>> kafka.admin.ReassignPartitionsCommandTest >
>> shouldRemoveThrottleLimitFromAllBrokers PASSED
>>
>> kafka.admin.ReassignPartitionsCommandTest > shouldFindMovingReplicas
>> PASSED
>>
>> kafka.admin.ReassignPartitionsCommandTest >
>> shouldFindMovingReplicasMultiplePartitions PASSED
>>
>> kafka.admin.ReassignPartitionsCommandTest > shouldSetQuotaLimit PASSED
>>
>> kafka.admin.ReassignPartitionsCommandTest >
>> shouldFindMovingReplicasWhenProposedIsSubsetOfExisting PASSED
>>
>> kafka.admin.ReassignPartitionsCommandTest > shouldUpdateQuotaLimit PASSED
>>
>> kafka.admin.ReassignPartitionsCommandTest >
>> shouldFindTwoMovingReplicasInSamePartition PASSED
>>
>> kafka.admin.ReassignPartitionsCommandTest >
>> shouldNotOverwriteEntityConfigsWhenUpdatingThrottledReplicas PASSED
>>
>> kafka.admin.ConfigCommandTest > shouldNotUpdateBrokerConfigIfMalformedEntityName
>> PASSED
>>
>> kafka.admin.ConfigCommandTest > shouldSupportCommaSeparatedValues PASSED
>>
>> kafka.admin.ConfigCommandTest > shouldNotUpdateBrokerConfigIfMalformedConfig
>> PASSED
>>
>> kafka.admin.DeleteConsumerGroupTest > testGroupTopicWideDeleteInZKDo
>> esNothingForActiveGroupConsumingMultipleTopics PASSED
>>
>> kafka.admin.AddPartitionsTest > testReplicaPlacementAllServers PASSED
>>
>> kafka.admin.ConfigCommandTest > shouldParseArgumentsForBrokersEntityType
>> PASSED
>>
>> kafka.admin.ConfigCommandTest > shouldAddBrokerConfig PASSED
>>
>> kafka.admin.AdminTest > testReassigningNonExistingPartition PASSED
>>
>> kafka.admin.ConfigCommandTest > testQuotaDescribeEntities PASSED
>>
>> kafka.admin.AdminTest > testGetBrokerMetadatas PASSED
>>
>> kafka.admin.ConfigCommandTest > shouldParseArgumentsForClientsEntityType
>> PASSED
>>
>> kafka.admin.AclCommandTest > testAclCli PASSED
>>
>> kafka.admin.ReassignPartitionsIntegrationTest > testRackAwareReassign
>> PASSED
>>
>> kafka.admin.AdminTest > testBootstrapClientIdConfig PASSED
>>
>> kafka.admin.ReassignPartitionsClusterTest >
>> shouldExecuteThrottledReassignment FAILED
>> java.nio.file.FileSystemException: C:\Users\IBM_AD~1\AppData\Loca
>> l\Temp\kafka-719085320148197500\my-topic-0\00000000000000000000.index:
>> The process cannot access the file because it is being used by another
>> process.
>>
>>
>> From the error message, it sounds like one of the prior tests does not do
>> a proper clean-up?!
>>
>> Thanks.
>> --Vahid
>>
>>
>>
>>
>> From: Ismael Juma <is...@juma.me.uk>
>> To: dev@kafka.apache.org
>> Cc: kafka-clients <ka...@googlegroups.com>, Kafka Users <
>> users@kafka.apache.org>
>> Date: 06/26/2017 01:54 PM
>> Subject: Re: [VOTE] 0.11.0.0 RC2
>> Sent by: ismaelj@gmail.com
>> ------------------------------
>>
>>
>>
>> Hi Vahid,
>>
>> Can you please check which test fails first? The errors you mentioned can
>> happen if a test fails and doesn't clean-up properly.
>>
>> Ismael
>>
>> On Mon, Jun 26, 2017 at 8:41 PM, Vahid S Hashemian <
>> vahidhashemian@us.ibm.com> wrote:
>>
>> > Hi Ismael,
>> >
>> > To answer your questions:
>> >
>> > 1. Yes, the issues exists in trunk too.
>> >
>> > 2. I haven't checked with Cygwin, but I can give it a try.
>> >
>> > And thanks for addressing this issue. I can confirm with your PR I no
>> > longer see it.
>> > But now that the tests progress I see quite a few errors like this in
>> > core:
>> >
>> > kafka.server.ReplicaFetchTest > classMethod FAILED
>> > java.lang.AssertionError: Found unexpected threads,
>> > allThreads=Set(ZkClient-EventThread-268-127.0.0.1:56565,
>> > ProcessThread(sid:0 cport:56565):, metrics-mete
>> > r-tick-thread-2, SessionTracker, Signal Dispatcher, main, Reference
>> > Handler, ForkJoinPool-1-worker-1, Attach Listener, ProcessThread(sid:0
>> > cport:59720):, ZkClie
>> > nt-EventThread-1347-127.0.0.1:59720, kafka-producer-network-thread |
>> > producer-1, Test worker-SendThread(127.0.0.1:56565), /127.0.0.1:54942
>> to
>> > /127.0.0.1:54926 w
>> > orkers Thread 2, Test worker, SyncThread:0,
>> > NIOServerCxn.Factory:/127.0.0.1:0, Test worker-EventThread, Test
>> > worker-SendThread(127.0.0.1:59720), /127.0.0.1:5494
>> > 2 to /127.0.0.1:54926 workers Thread 3,
>> > ZkClient-EventThread-22-127.0.0.1:54976, ProcessThread(sid:0
>> > cport:54976):, Test worker-SendThread(127.0.0.1:54976), Fin
>> > alizer, metrics-meter-tick-thread-1)
>> >
>> > I tested on a VM and a physical machine, and both give me a lot of
>> errors
>> > like this.
>> >
>> > Thanks.
>> > --Vahid
>> >
>> >
>> >
>> >
>> > From: Ismael Juma <is...@gmail.com>
>> > To: Vahid S Hashemian <va...@us.ibm.com>
>> > Cc: dev@kafka.apache.org, kafka-clients
>> > <ka...@googlegroups.com>, Kafka Users <us...@kafka.apache.org>
>> > Date: 06/26/2017 03:53 AM
>> > Subject: Re: [VOTE] 0.11.0.0 RC2
>> >
>> >
>> >
>> > Hi Vahid,
>> >
>> > Sorry for not replying to the previous email, I had missed it. A couple
>> of
>> > questions:
>> >
>> > 1. Is this also happening in trunk? Seems like it should be the case for
>> > months and seemingly no-one reported it until the RC stage.
>> > 2. Is it correct that this only happens when compiling on Windows
>> without
>> > Cygwin?
>> >
>> > I believe the following PR should fix it, please verify:
>> >
>> > https://github.com/apache/kafka/pull/3431
>> >
>> > Ismael
>> >
>> > On Fri, Jun 23, 2017 at 8:25 PM, Vahid S Hashemian <
>> > vahidhashemian@us.ibm.com> wrote:
>> >
>> > > Hi Ismael,
>> > >
>> > > Not sure if my response on RC1 was lost or this issue is not a
>> > > show-stopper:
>> > >
>> > > I checked again and with RC2, tests still fail in my Windown 64 bit
>> > > environment.
>> > >
>> > > :clients:checkstyleMain
>> > > [ant:checkstyle] [ERROR] C:\Users\User\Downloads\kafka-
>> > >
>> > 0.11.0.0-src\clients\src\main\java\org\apache\kafka\common\
>> > protocol\Errors.java:89:1:
>> > > Class Data Abstraction Coupling is 57 (max allowed is 20) classes
>> > > [ApiExceptionBuilder, BrokerNotAvailableException,
>> > > ClusterAuthorizationException, ConcurrentTransactionsException,
>> > > ControllerMovedException, CoordinatorLoadInProgressException,
>> > > CoordinatorNotAvailableException, CorruptRecordException,
>> > > DuplicateSequenceNumberException, GroupAuthorizationException,
>> > > IllegalGenerationException, IllegalSaslStateException,
>> > > InconsistentGroupProtocolException, InvalidCommitOffsetSizeException,
>> > > InvalidConfigurationException, InvalidFetchSizeException,
>> > > InvalidGroupIdException, InvalidPartitionsException,
>> > > InvalidPidMappingException, InvalidReplicaAssignmentException,
>> > > InvalidReplicationFactorException, InvalidRequestException,
>> > > InvalidRequiredAcksException, InvalidSessionTimeoutException,
>> > > InvalidTimestampException, InvalidTopicException,
>> > InvalidTxnStateException,
>> > > InvalidTxnTimeoutException, LeaderNotAvailableException,
>> > NetworkException,
>> > > NotControllerException, NotCoordinatorException,
>> > > NotEnoughReplicasAfterAppendException, NotEnoughReplicasException,
>> > > NotLeaderForPartitionException, OffsetMetadataTooLarge,
>> > > OffsetOutOfRangeException, OperationNotAttemptedException,
>> > > OutOfOrderSequenceException, PolicyViolationException,
>> > > ProducerFencedException, RebalanceInProgressException,
>> > > RecordBatchTooLargeException, RecordTooLargeException,
>> > > ReplicaNotAvailableException, SecurityDisabledException,
>> > TimeoutException,
>> > > TopicAuthorizationException, TopicExistsException,
>> > > TransactionCoordinatorFencedException,
>> > TransactionalIdAuthorizationException,
>> > > UnknownMemberIdException, UnknownServerException,
>> > > UnknownTopicOrPartitionException, UnsupportedForMessageFormatExc
>> eption,
>> > > UnsupportedSaslMechanismException, UnsupportedVersionException].
>> > > [ClassDataAbstractionCoupling]
>> > > [ant:checkstyle] [ERROR] C:\Users\User\Downloads\kafka-
>> > >
>> > 0.11.0.0-src\clients\src\main\java\org\apache\kafka\common\
>> > protocol\Errors.java:89:1:
>> > > Class Fan-Out Complexity is 60 (max allowed is 40).
>> > [ClassFanOutComplexity]
>> > > [ant:checkstyle] [ERROR] C:\Users\User\Downloads\kafka-
>> > > 0.11.0.0-src\clients\src\main\java\org\apache\kafka\common\
>> > > requests\AbstractRequest.java:26:1: Class Fan-Out Complexity is 43
>> (max
>> > > allowed is 40). [ClassFanOutComplexity]
>> > > [ant:checkstyle] [ERROR] C:\Users\User\Downloads\kafka-
>> > > 0.11.0.0-src\clients\src\main\java\org\apache\kafka\common\
>> > > requests\AbstractResponse.java:26:1: Class Fan-Out Complexity is 42
>> (max
>> > > allowed is 40). [ClassFanOutComplexity]
>> > > :clients:checkstyleMain FAILED
>> > >
>> > > FAILURE: Build failed with an exception.
>> > >
>> > > Thanks.
>> > > --Vahid
>> > >
>> > >
>> > >
>> > > From: Ismael Juma <is...@juma.me.uk>
>> > > To: dev@kafka.apache.org, Kafka Users <users@kafka.apache.org
>> >,
>> > > kafka-clients <ka...@googlegroups.com>
>> > > Date: 06/22/2017 06:16 PM
>> > > Subject: [VOTE] 0.11.0.0 RC2
>> > > Sent by: ismaelj@gmail.com
>> > > ------------------------------
>> > >
>> > >
>> > >
>> > > Hello Kafka users, developers and client-developers,
>> > >
>> > > This is the third candidate for release of Apache Kafka 0.11.0.0.
>> > >
>> > > This is a major version release of Apache Kafka. It includes 32 new
>> > KIPs.
>> > > See the release notes and release plan (
>> > > https://cwiki.apache.org/confluence/display/KAFKA/Release+
>> Plan+0.11.0.0)
>> > > for more details. A few feature highlights:
>> > >
>> > > * Exactly-once delivery and transactional messaging
>> > > * Streams exactly-once semantics
>> > > * Admin client with support for topic, ACLs and config management
>> > > * Record headers
>> > > * Request rate quotas
>> > > * Improved resiliency: replication protocol improvement and
>> > single-threaded
>> > > controller
>> > > * Richer and more efficient message format
>> > >
>> > > Release notes for the 0.11.0.0 release:
>> > > http://home.apache.org/~ijuma/kafka-0.11.0.0-rc2/RELEASE_NOTES.html
>> > >
>> > > *** Please download, test and vote by Tuesday, June 27, 6pm PT
>> > >
>> > > Kafka's KEYS file containing PGP keys we use to sign the release:
>> > > http://kafka.apache.org/KEYS
>> > >
>> > > * Release artifacts to be voted upon (source and binary):
>> > > http://home.apache.org/~ijuma/kafka-0.11.0.0-rc2/
>> > >
>> > > * Maven artifacts to be voted upon:
>> > > https://repository.apache.org/content/groups/staging/org/apa
>> che/kafka/
>> > >
>> > > * Javadoc:
>> > > http://home.apache.org/~ijuma/kafka-0.11.0.0-rc2/javadoc/
>> > >
>> > > * Tag to be voted upon (off 0.11.0 branch) is the 0.11.0.0 tag:
>> > > https://git-wip-us.apache.org/repos/asf?p=kafka.git;a=tag;h=
>> > > 8698fa1f41102f1664b05baa4d6953fc9564d91e
>> > >
>> > > * Documentation:
>> > > http://kafka.apache.org/0110/documentation.html
>> > >
>> > > * Protocol:
>> > > http://kafka.apache.org/0110/protocol.html
>> > >
>> > > * Successful Jenkins builds for the 0.11.0 branch:
>> > > Unit/integration tests: https://builds.apache.org/job/
>> > > kafka-0.11.0-jdk7/187/
>> > > System tests: pending (will send an update tomorrow)
>> > >
>> > > /**************************************
>> > >
>> > > Thanks,
>> > > Ismael
>> > >
>> > >
>> > >
>> > >
>> >
>> >
>> >
>> >
>> >
>>
>>
>>
>>
> --
> You received this message because you are subscribed to the Google Groups
> "kafka-clients" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kafka-clients+unsubscribe@googlegroups.com.
> To post to this group, send email to kafka-clients@googlegroups.com.
> Visit this group at https://groups.google.com/group/kafka-clients.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/kafka-clients/CAD5tkZYey%3DLSri2NpGnh1RNGfCTtZRJyCNMOU7
> cF0nZW5Ec38g%40mail.gmail.com
> <https://groups.google.com/d/msgid/kafka-clients/CAD5tkZYey%3DLSri2NpGnh1RNGfCTtZRJyCNMOU7cF0nZW5Ec38g%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
> For more options, visit https://groups.google.com/d/optout.
>