What is GIL in Python and examples
The Python Global Interpreter Lock (GIL) is a process lock that ensures only one thread executes Python byte code at a time within a single process.
- In Python, the performance of single-threaded and multi-threaded processes is the same, this is due to the GIL in Python.
- We can’t implement multithreading in Python because we have the Global Interpreter Lock, which limits threads to work as single threaded.
- GIL may be the bottleneck in multithreaded Python programs, especially when parallel execution of CPU-intensive tasks is required. This is because only one thread can execute Python byte code at any given time.
- GIL mainly affects parallelism (simultaneous execution of multiple tasks), but does not affect concurrency (appearance of simultaneous execution of multiple tasks). Python threads are still useful for concurrent I/O intensive tasks or tasks involving waiting for external resources.
In 1992, Guido van Rossum added a well-known lock, called the Global Interpreter Lock (GIL), to resolve race conditions in the CPython interpreter. Twenty-five years later, it is one of the main disadvantages of the Python interpreter that we all love and use.
What problems does GIL solve for Python:
Python has features that other languages do not have, which is the reference counter. Through the reference counter, we can calculate the total number of references allocated by Python internally for data objects. With this counter, we can count references, and when the count drops to zero, the corresponding variables or data objects will be automatically released. For example:
# Python program showing
# use of reference counter
import sys
geek_var = "Geek"
print(sys.getrefcount(geek_var))
string_gfg = geek_var
print(sys.getrefcount(string_gfg))
# output
# 4
# 5
This reference counter variable needs to be protected, because sometimes two threads increase or decrease its value at the same time, which may lead to memory leaks. Therefore, to protect the thread, we add locks to all data structures shared across threads, but sometimes adding locks will lead to another problem, that is, deadlock. To avoid the problems of memory leaks and deadlock, we use a single lock on the interpreter, namely the Global Interpreter Lock (GIL).
Why choose GIL as a solution:
Python’s backend supports C language, and most of the related libraries owned by Python are written in C and C++. Due to GIL, Python provides a better way to deal with thread-safe memory management. The Global Interpreter Lock is easy to implement in Python because it only needs to provide a lock for the processing threads in Python. GIL is simple to implement and easy to add to Python. It enhances the performance of single-threaded programs because only one lock needs to be managed.
The impact on multi-threaded Python programs:
When users write Python programs or any computer programs, there is a performance difference between CPU-bound programs and I/O-bound programs. The CPU pushes the program to the limit by executing many operations at the same time, while the I/O program must spend time waiting for input/output.
Code 1: A CPU-intensive program that performs a simple countdown
# Python program showing
# CPU bound program
import time
from threadingimport Thread
COUNT = 50000000
def countdown(n):
while n>0:
n -= 1
start = time.time()
countdown(COUNT)
end = time.time()
print('Time taken in seconds -', end - start)
Output: Time used (in seconds) - 2.5236213207244873
Code 2: Two threads running in parallel
# Python program showing
# two threads running parallel
import time
from threadingimport Thread
COUNT = 50000000
def countdown(n):
while n>0:
n -= 1
t1 = Thread(target = countdown, args =(COUNT//2, ))
t2 = Thread(target = countdown, args =(COUNT//2, ))
start = time.time()
t1.start()
t2.start()
t1.join()
t2.join()
end = time.time()
print('Time taken in seconds -', end - start)
Output: Time used (in seconds) - 2.183610439300537
In the above code, the two sections of code for the CPU-intensive process and the multi-threaded process have the same performance, because in the CPU-intensive program, the GIL limits the CPU to only use a single thread. The impact of CPU-intensive threads and multi-threading in python is the same.
Why GIL has not been removed yet:
GIL has not been improved so far because python 2 has a GIL implementation, if we change it in python 3, it will bring us problems. So we are not removing GIL, but improving the concept of GIL.
One of the reasons why GIL has not been removed yet is that the python backend heavily relies on C, and the C extensions heavily rely on the GIL implementation method.
Even though there are many ways to solve the problems that GIL solves, most methods are difficult to implement and will slow down system speed.
How to handle Python’s GIL:
Most of the time we use multiprocessing to prevent the program from experiencing GIL. In this implementation, python provides different interpreters to run for each process, so in this case, a single thread is provided for each process in multiprocessing.
# Python program showing
# multiprocessing
import multiprocessing
import time
COUNT = 50000000
def countdown(n):
while n>0:
n -= 1
if __name__ == "__main__":
# creating processes
start = time.time()
p1 = multiprocessing.Process(target = countdown, args =(COUNT//2, ))
p2 = multiprocessing.Process(target = countdown, args =(COUNT//2, ))
# starting process 1
p1.start()
# starting process 2
p2.start()
# wait until process 1 is finished
p1.join()
# wait until process 2 is finished
p2.join()
end = time.time()
print('Time taken in seconds -', end - start)
Output:
Time taken (in seconds) - 2.5148496627807617
As you can see, there is no difference in the time spent by multi-threaded systems and multi-processing systems. This is because multi-processing systems have their own problems to solve. So this does not solve the problem, but it provides a solution allowed by GIL executed by python.