Python Explain the principle of execution
I am a Python thread , My job is to explain what the executive programmer wrote Python code .
The reason is to explain and implement , Because Python It's a high-level language ,CPU I don't know that guy Python code , Dynamic translation when running CPU instructions .
I put Python Source code process “ compile ” in the future , Into a bytecode file ：.pyc, This is a binary file , Human beings can't understand , Only I can understand .
Then my job is simple , Constantly take out the data in the bytecode file “ instructions ” Interpretation and execution , Until all instructions are executed , I can rest .
GIL
later , Multithreaded programming technology has become popular , I'm not the only thread in the process , Some new partners have come .
I thought we could live in peace , I didn't expect to be in trouble , We'll go our separate ways , Often make problems with memory space , I can't find anyone to carry the pot .
Finally one day , I can't stand it anymore , Call on everyone to discuss a solution .
“ Guys , We can't go our separate ways anymore , We are one Team, We should cooperate with each other , A thread can go very fast , But a group of threads can go further together !”
“ boss , Just say what you think ”, Another thread said .
“ or , Let's put a lock on it ! Simple and fast , Each thread executes code , Have to apply for this lock , The application cannot be executed until it is received , Or you'll have to wait ”, I said .
“ When will it be released ?”
“ Why don't you get a counter , Count to per thread 100 Just release , This ensures that others have the opportunity to perform ?”
“ How do you count ? Do you count every byte ?”
“ sure , But don't be so rigid , Some instructions are relatively simple , It will be finished soon , You don't have to count ”
“ Good is good , But if you haven't counted enough 100, But it's executing I/O Operation blocked , If you still occupy the lock , Isn't that a waste of resources ?”
The little friends were talking about it .
I thought about it and wanted to say ：“ That's it , Combine the two situations , Normally count to 100 Just release the lock once , But if you encounter a blockage , Also release the lock in advance , how ?”
Everyone nodded , An agreement was reached , Then we gave the lock a name ： Global interpreter lock GIL.
Since I used it GIL, Our big guys work more regularly , There is no longer any case of damaging public resources .
Upgraded version GIL
Later , Multi core technology began to rise , One CPU It can execute multiple threads at the same time . The friends spread the news happily .
“ boss , Now? CPU How many cores are there , We can do it together , Can you take that GIL Get rid of it , In this way, we can take advantage of multi-core .”
“ Yeah , next door Java Threads always laugh at us for looking crowded , There is actually only one thread executing ”
Easier said than done , That's how we've worked for so many years , It should be removed suddenly , If something goes wrong, no one dares to be responsible .
“ But boss , Now this GIL Lock unfair ”, The new thread complained .
“ What's unfair ?”
“ I'm going to execute the code , Found it locked in your hand , I had to wait in place , I fell asleep after waiting for a long time , It's hard to wait until you're released , The operating system woke me up , Ready to apply for a lock , It turns out you grabbed it again , What a waste of time ”, The new thread is full of grievances .
“ Yeah , boss , I found out, too , This is not an accident , I've been watching for a long time , It happens all the time ! Frequently awakened , But I found that I was busy in vain , Waste in vain CPU resources , Everyone is very angry ”, Another thread partner also said .
I'm a little embarrassed ,“ Um , This is really a problem ”
“ It's not over yet ”, The new thread continues ：“ Now count according to the number of bytecode instructions , But some scripts are simple , Some are complicated , Cause the same number to 100, Some threads can run for a long time , And some of them run quickly , It's not fair ”
The questions raised by the partners are very important , Looks like it's time, right GIL An upgrade has been made .
After a heated discussion , We improved the original GIL, With a new strategy ： No longer count , Switch to time slice ： The execution time slice of each thread is 5000 Microsecond .
To ensure release GIL after , Don't be robbed by yourself right away , A lock is added to force thread switching
After improvement , Now it's fair , You guys have nothing to say , You can work at ease again .
epilogue
Python It is a language of interpretation and execution , It has powerful third-party libraries and cross platform capabilities , In recent years Python A second spring , Sweep reptile ,web development , Machine learning and many other fields .
But for a long time ,Python What is most criticized is that it has a lock ：GIL, This lock makes Python Can't really implement multi-threaded execution , Unable to utilize multi-core CPU High performance .
actually , This lock heel Python It has nothing to do with half a dime , But responsible for interpretation and implementation Python Interpreter for ：CPython Pot .
CPython Yes C Written in Python interpreter , It is also the most widely used Python interpreter , Generally, when there is no special instruction , say Python That's it CPython interpreter .
Python At the beginning of its birth , Multithreading technology is far less popular than it is today , Even multicore CPU Also Python Many years after its birth . In the early interpreters, in order to support multiple threads , Used rough GIL To control , Convenient and simple at the same time , Also became CPython A huge historical burden .
stay Python3.2 before ,Python Use a simple counting method to count and control the execution time of each thread . After that , A fairer time slice method is introduced to upgrade and replace .
The past two decades , There were many big cattle trying to get rid of it completely GIL, But there was no perfect success .
although Python Not completely removed GIL, Fortunately , Several other ways are provided “ save the nation by a devious path ” Achieve concurrency ：Ctypes
By writing C Language extension and Python interactive , stay C Language level bypass GIL Realize multi-core utilization .
MultiProcess Python Provided MultiProcess, Bypass through multiple processes GIL
Synergetic process Co process is also called user thread ,Python3.4 Support for collaborative process is added after version , It also provides a choice for performance improvement .
This article tells the story in first person vernacular GIL stay CPython Working principle of interpreter , Does everyone understand ?
Technology