A few days ago, some water friends asked questions ：
Through message queuing （MsgQueue,MQ） Send tasks and messages , In case MQ What if you restart it ? Can it be guaranteed MQ Don't lose news ?
Let's talk about it today MQ Architecture and process of message delivery .
Don't lose news ,MQ What is the core direction of architecture design ?
MQ If you want information, you will get it , There are two core design points in the architecture ：
（1） News landing ;
（2） Message timeout , Retransmission , confirm ;
In order to achieve the above two core points ,MQ What is the architecture ?
The image above is a MQ Core architecture of , It can be divided into three parts ：
（1） Sender -> Pink part on the left ;
（2）MQ Core cluster -> Middle blue part ;
（3） Receiver -> The yellow part of the excrement on the right ;
The pink sender consists of two parts ：
（1） Business caller ;
（2）MQ-client-sender;
The latter provides two cores for the former API：
（1）SendMsg(bytes[] msg);
（2）SendCallback();
blue MQ The core cluster is divided into four parts ：
（1）MQ-server
（2）zk;
（3）db;
（4） Management backstage web;
The Yellow receiver also consists of two parts ：
（1） Business receiver ;
（2）MQ-client-receiver;
The latter provides two cores for the former API：
（1）RecvCallback(bytes[] msg);
（2）SendAck();
MQ It is a powerful tool for decoupling between systems , It can decouple publishing subscribers , It decouples the upstream and downstream message delivery into two parts , As in the frame composition 1 Arrow and 2 arrow ：
arrow 1： The sender delivers the message to MQ, first half ;
arrow 2：MQ Post message to receiver , second half ;
MQ What is the core process of reliable message delivery ?
MQ Since we split the delivery of the message into the first and second half , In order to ensure the reliable delivery of the message , In the first half and the second half, the news must be delivered .
MQ First half of message delivery ,MQ-client-sender reach MQ-server The process is shown in the figure above 1-3：
（1）MQ-client Send message to MQ-server;
Voice over ： At this point, the business side calls API：SendMsg.
（2）MQ-server Drop the news , After landing, the system is sent successfully ;
（3）MQ-server Send response to MQ-client;
Voice over ： At this time, call back the business API：SendCallback.
MQ Message delivery second half ,MQ-server reach MQ-client-receiver The process is shown in the figure above 4-6：
（4）MQ-server Send message to MQ-client;
Voice over ： At this time, call back the business API：RecvCallback.
（5）MQ-client Reply to MQ-server;
Voice over ： At this time, the business side calls the API：SendAck.
（6）MQ-server received ack, Delete the message that has landed before , Complete the reliable delivery of messages ;
What if the news is lost ?
MQ The first and second half of message delivery , Message loss can occur , To ensure message accessibility ,MQ Timeout and retransmission are required .
How to implement overtime and retransmission in the first half ?
MQ First half 1 perhaps 2 perhaps 3 If lost or timed out ,MQ-client-sender Internal timer The message will be retransmitted
, Until you expect it 3, If retransmission N I haven't received it yet , be SendCallback Callback send failed , It should be noted that , In the process MQ-server Multiple retransmissions of the same message may be received .
How to implement overtime and retransmission in the second half ?
MQ In the second half 4 perhaps 5 perhaps 6 If lost or timed out ,MQ-server Internal timer The message will be retransmitted , Until we get it 5 And it was successfully implemented 6, This process can be repeated many times .
Voice over ： Generally, the strategy of exponential backoff is adopted , Septum propria x Second retransmission ,2x Second retransmission ,4x Second retransmission , and so on .
It should be noted that , In the process MQ-client-receiver You may also receive multiple retransmissions of the same message .
summary
MQ It is a powerful tool for decoupling between systems ,MQ In order to ensure that the news will arrive , The direction of architecture design is ：
（1） When the message is received, it will land first ;
（2） Message timeout , Retransmission , Confirm to ensure that the message will arrive ;
remaining problems ：
first half ,MQ-server Duplicate messages may be received ; second half ,MQ-client-receiver, That is, the message receiver may receive duplicate messages , What should I do? ?
Voice over ： How to remove duplicate , How to design idempotent , Listen to the live broadcast of sister Xuan .
event ：《RocketMQ Enterprise application and customized development practice 》
character ： Naixue Education CEO, Sister Xuan
time ：6.4,6.5,6.6 Three days ,20:00-22:00
Voice over ： Two hours a day , Free Admission , Is there sincerity ?
How to participate ?
Scan code and add assistant , Get live address , Send architect growth map
What about the content outline ?
（1）MQ Analysis of enterprise application scenarios ;
（2） common MQ Selection comparison ;
（3）MQ High availability and horizontal expansion architecture ;
（4）MQ In depth analysis of storage principle ;
（5）MQ Deep analysis of service registration and service discovery ;
（6）MQ Deep analysis of load balancing architecture ;
（7）MQ Principle of transaction message implementation ;
（8）MQ Principle of delay message architecture ;
（9）RocketMQ Customized development practice ;
（10） Homework and explanation ;
continuity 3 day (6.4/6.5/6.6), Every day 2 hour (20:00-22:00), Free live broadcast of sister Xuan ,MQ, A lot of dry goods , Read the original , Get live address , I can only help you get here , Let's learn together .
Technology