python Pool

Creator
Creator
Seonglae ChoSeonglae Cho
Created
Created
2024 Apr 8 1:35
Editor
Edited
Edited
2024 Nov 29 21:42
Refs
Refs

Simplest without
python multiprocessing Manger

from multiprocessing import Pool with Pool() as pool: results = pool.map(worker, tasks)

Sequential bad

from multiprocessing import Pool import time def process_request(request): time.sleep(1) return f"Processed {request}" requests = ['req1', 'req2', 'req3', 'req4', 'req5'] # Sequential processing start = time.time() results_seq = [process_request(req) for req in requests] print(f"Sequential: {time.time() - start:.2f} seconds") # Output: Sequential: 5.00 seconds

Concurrent multiprocessing

# Concurrent processing with multiprocessing Pool start = time.time() with Pool(5) as p:     results_pool = p.map(process_request, requests) print(f"Pool: {time.time() - start:.2f} seconds") # Output: Pool: 1.11 seconds
 
 
 
 

Recommendations