Entry point to start reading the code

[Nelson Bighetti <ba...@gmail.com>]

Hello everyone! :)

I've just started reading the code but it appears a little overwhelming to
me. Is there any documentation describing the directory structure? Or maybe
could anybody suggest a good starting point to read the code?

Thanks in advance.

Re: Entry point to start reading the code

[ziming deng <de...@gmail.com>]

Hello,
I think you can start by reading `clients` module, starting from Producer and Consumer and AdminClient, there are many related blogs which can be found by google but may be outdated since Kafka are updating very quickly, a good new is that Kafka have very comprehensive test cases and you can run them locally. Then you can read `core` and other modules.

Here are some of my experience:
1. Build kafka using gradle build tool
2. KafkaProducer
  2.1 KafkaProducer API
  2.2 KafkaProducer class
    2.2.1 ProducerInterceptor
    2.2.2 ProcuderMetadata
    2.2.3 Serializer and Deserializer
    2.2.4 Partitioner
  2.3 RecordAccumulator
    2.3.1 MemoryRecords
    2.3.4 RecordBatch
    2.3.3 BufferPool
    2.3.4 RecordAccumulator
  2.4 Sender
    2.4.1 Request
    2.4.2 Selector
    2.4.3 InFlightRequests
    2.4.4 MetadataUpdater
    2.4.5 NetworkClient
3. KafkaConsumer
  3.1 KafkaConsumer API
  3.2 delivery guarantee semantic
  3.3 Consumer group rebalance
  3.4 KafkaConsumer
    3.4.1 ConsumerNetworkClient
    3.4.2 SubscriptionState
    3.4.3 ConsumerCoordinator
    3.4.4 PartitionAssignor
    3.4.5 Heartbeat Request
    3.4.6 Reblance request
    3.4.7 offset request
    3.4.8 Fetcher
    3.4.9 overall
4. Core module
  4.1 network
    4.1.1 reactor pattern
    4.1.2 SockerServer
    4.1.3 AbstractServerThread
    4.1.4 Acceptor
    4.1.5 Processor
    4.1.6 RequestChannel
  4.2 API 
    4.2.1 KafkaRequestHandler
    4.2.2 KafkaApis
  4.3 Log
    4.3.1 document
    4.3.2 FileMessageSet
    4.3.3 ButeBufferMessageSet
    4.3.4 OffsetIndex
    4.3.5 LogSegment
    4.3.6 Log
    4.3.7 LogManager
  4.4. DelayedOperationPugatory 
    4.4.1 TimingWheel
    4.4.2 SystemTimer
    4.4.3 DelayedOperation
    4.4.4 DelayedOperationPugatory
    4.4.5 DelayedFetch
    4.4.6 DelayedProcuce
  4.5 replica
    4.5.1 Replica
    4.5.2 Partition
    4.5.3 ReplicaManager
  4.6 kafkaController (deprecated, using KRaft instead)
    4.6.1 ControllerChannelManager
    4.6.2 ControllerContext
    4.6.3 ControllerBrokerRequestBatch
    4.6.4 PartitionStateMachine
    4.6.5 PartitionLeaderSelectir
    4.6.6 REplicateStateMachine
    4.6.7 Zookeeper Listener
    4.6.8 KafkaController  startup and failover 
    4.6.9 ControlledShutdownRequest
  4.7 GroupCoordinator
    4.7.1 GroupMetadataManager
    4.7.2 GroupCoordinator
5. Acl and authentication
6. Jmx
7. Kafka tool
8. raft/metadata/stream/connect  modules

Some classes may have deprecated or renamed, you can using git to check their change history.
--
Best,
Ziming

> On Jan 29, 2023, at 21:35, Nelson Bighetti <ba...@gmail.com> wrote:
> 
> Hello everyone! :)
> 
> I've just started reading the code but it appears a little overwhelming to
> me. Is there any documentation describing the directory structure? Or maybe
> could anybody suggest a good starting point to read the code?
> 
> Thanks in advance.