Zookeeper Is an open source distributed application coordination service , From Google An open source implementation , yes Hadoop and HBase Important components of .Zookeeper
It can provide consistency services for distributed applications , Features include ： Configuration services , Name Service , Distributed synchronization , Group services and so on .Zookeeper
The goal is to encapsulate complex and error prone key services , Will be easy to use interface and performance efficient , Stable function of the system to provide users .Zookeeper Contains a simple set of primitives , provide Java and C Interface .
Zookeeper Application scenarios —— Matching management
In distributed environment , Profile management and synchronization is a common problem ：
1. One cluster , The configuration information of all nodes is consistent , such as Hadoop 2. After modifying the configuration file , Hope to be able to quickly synchronize to each node
Zookeeper Application scenarios —— Unified naming service
In distributed environment , It is often necessary to apply / Services are named uniformly , Easy to identify different services ： Similar to domain name and IP
Corresponding relationship between , Domain names are easy to remember ; Get the address of a resource or server by name , Information such as providers .
Organize services according to hierarchy / apply name ： The service name and address information can be written to Zookeeper upper , Client through Zookeeper Get available service list class .
Configuration management can be left to Zookeeper realization ： Can write configuration information to Zookeeper One of the Znode upper , Each node listens to this Znode , once Znode
The data in is modified Zookeeper The nodes are notified
Zookeeper Introduction to the main concepts of
zookeeper There are three main types of roles in ：
role describe （ The number of machines is odd ）
Leader It is mainly responsible for the initiation and resolution of voting , Update system status
Learner Follower Follower Receive the client request and the client returns the result , Voting in the election process
Observer OBserver Receive client connections , Forward write request to Leader node ,Observer Not participating in the voting process , Sync only Leader
Status of ,Observer Is to extend the system , Increase reading speed
Client Access request originator
The system model is as follows ：
Zookeeper The design objectives include ：
1. Final consistency ：Client No matter which one you connect to Server , It's all shown the same view , This is Zookeeper The most important feature .
2. reliability ： If a message is received by a server , Then it will be connected by all servers in the cluster .
3. Real time ：Zookeeper Ensure that the client will get the server update or server failure information within a time range .
4. It's nothing to wait for （wait-free）： Slow or ineffective Client Had to intervene Client Request for .
5. Atomicity ： Updates can only succeed or fail , There is no intermediate state .
6. Sequence ： It includes global order and partial order ： Global ordering means that if a message is sent on a server a In the news b Pre release , At all Server On the news a All will be in the news b
Previously released ; Partial ordering means that if a message b In the news a It was published by the same sender ,a Will be in the b front .
each Server Inside, it stores a piece of data ,Zookeeper On startup , One will be elected from the instance Leader
Leader Responsible for data update and other operations
An update operation was successful , If and only if the majority Server Data modified successfully in memory
zookeeper How it works
zookeeper At its core is atomic broadcasting , This mechanism ensures that Server Synchronization between . The protocol that implements this mechanism is called zab agreement .zab
There are two modes of protocol , They are recovery mode and broadcast mode . When the server starts up or after the leader runs down ,zab It's in recovery mode , When leaders are elected , And most of them Server
Completed and Leader After status synchronization of , The recovery mode is over . State synchronization ensures Leader and Server Has the same system state .
In order to ensure the order consistency of transactions ,Zookeeper Something incremental was used id Number (zxid) To identify the transaction . All proposals are added when they are put forward zxid, In implementation zxid
It's a 64 Bit data , Its height 32 Is it epoch, Used to identify Leader Is the relationship changing , Whenever one Leader
When elected , It's going to have a new one epoch, Used to identify the current belonging to the current Leader The period of ,zxid Low of 32 Bits are used to increment the count .
each Server Three states in the working process ：
1.LOOKING： current Server hear nothing of Leader Who is it? , Searching
2.LEADER: current Server They are elected Leader
3.FOLLOWER：Leader It has been elected , current Server Keep pace with it
Election process ：
When Leader Run down or Leader Losing most of the Follower Time ,Zookeeper
You can enter recovery mode , The recovery model needs to be re elected with a new one Leader, So that all the Server All returned to a correct state .Zookeeper There are two states in the election rules ：
One is basic paxos realization ; The other is basic paxos technological process
1. The election thread is created by the Server Thread as , Its main function is to count the voting results , And elect the recommended ones Server
2. Election thread receives all Server Initiate an inquiry
3. After the election thread receives a reply , Verify whether it is a self initiated inquiry , And get the other person's id , And stored in the current query object list , Finally get the other party's proposal Leader
Information about , And set this information to the next vote Server
4. On receipt Server After reply , Election thread accounting zxid The biggest one Server , And put the Server For the next vote Server
5. The election thread will be current zxid maximal Server Set to the currently recommended Leader , If there are Server get n/2+1
Number of votes , Will be set to the current recommendation Leader To win for Server , Otherwise, continue the process , until leader Be elected .
adopt basic paxos Process analysis can be concluded ： To make Leader Gain the majority Server Support for , be Server
The total number must be odd n/2+1 The number of votes , And survived Server The number of n+1. each Server The above process will be repeated after starting .
In recovery mode , If you're recovering from a crash or just starting Server Data and session information are also recovered from disk snapshots ,Zookeeper
You will remember the transaction log and take periodic snapshots , In order to facilitate state recovery in case of failure .
Synchronization process
The election is over Leader in the future ,Zookeeper It enters the state synchronization process ;
1.Leader wait for Server connect
2.Follower connect Leader , Will be the largest zxid Send to Leader
3.Leader according to Follower Of zxid Determine synchronization point
4. Notify when synchronization is complete Follower Has become updated state
5.Follower received updated After the news , You can accept it again Client Request to service
Technology