English
English
When handling high-concurrency requests—particularly in scenarios like concurrent data scraping or API calls using PHP—the overhead from repeated resource loading and connection establishment can’t be ignored. The curl_share_init function is an advanced feature provided by libcurl that allows multiple cURL sessions to share certain resources, such as DNS cache, SSL sessions, and cookies, thereby improving performance in multi-threaded environments.
This article will explain how to use curl_share_init in PHP to implement resource sharing in cURL, along with practical examples to demonstrate its usage.
curl_share_init is a function provided by libcurl used to initialize a shared handle (cURL Share Handle). This handle can be used across multiple cURL requests to share data, helping to avoid redundant initialization, parsing, or negotiation processes.
High-concurrency request scenarios, such as multi-threaded or multi-process crawlers;
Multiple requests accessing the same host, requiring shared DNS cache;
Multiple requests using the same Cookie or SSL session.
In PHP, we can implement resource sharing using a combination of curl_share_init(), curl_share_setopt(), and curl_setopt(). Here’s a complete example:
<?php
<p>// Initialize shared handle<br>
$sh = curl_share_init();</p>
<p>// Set share options – in this case, sharing Cookie and DNS<br>
curl_share_setopt($sh, CURLSHOPT_SHARE, CURL_LOCK_DATA_COOKIE);<br>
curl_share_setopt($sh, CURLSHOPT_SHARE, CURL_LOCK_DATA_DNS);</p>
<p>// Create multiple cURL sessions<br>
$urls = [<br>
'<a rel="noopener" target="_new" class="" href="https://m66.net/api/data1">https://m66.net/api/data1</a>',<br>
'<a rel="noopener" target="_new" class="" href="https://m66.net/api/data2">https://m66.net/api/data2</a>',<br>
];</p>
<p>$chs = [];<br>
foreach ($urls as $url) {<br>
$ch = curl_init($url);<br>
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);<br>
// Attach shared handle<br>
curl_setopt($ch, CURLOPT_SHARE, $sh);<br>
$chs[] = $ch;<br>
}</p>
<p>// Initialize multi handle<br>
$mh = curl_multi_init();</p>
<p>// Add all sessions to multi handle<br>
foreach ($chs as $ch) {<br>
curl_multi_add_handle($mh, $ch);<br>
}</p>
<p>// Execute all requests<br>
$running = null;<br>
do {<br>
curl_multi_exec($mh, $running);<br>
curl_multi_select($mh);<br>
} while ($running > 0);</p>
<p>// Retrieve results<br>
foreach ($chs as $ch) {<br>
$content = curl_multi_getcontent($ch);<br>
echo "Response Content:\n" . $content . "\n\n";<br>
curl_multi_remove_handle($mh, $ch);<br>
curl_close($ch);<br>
}</p>
<p>// Cleanup<br>
curl_share_close($sh);<br>
curl_multi_close($mh);<br>
?><br>
Shared Lock Type Limitations: Only one lock type (such as CURL_LOCK_DATA_COOKIE or CURL_LOCK_DATA_DNS) can be set at a time. To share multiple types, call curl_share_setopt multiple times.
Thread Safety Concerns: Although curl_share is designed for multi-threaded use in C, PHP itself is not thread-safe. Thus, this feature is mainly useful for simulated concurrency (e.g., with curl_multi_*) to optimize resource usage.
Error Handling: In real-world applications, include curl_errno and curl_error checks to ensure requests execute properly.
curl_share_init is a powerful tool that can significantly enhance performance in scenarios involving multiple requests. By sharing resources across multiple cURL requests, we reduce network overhead and improve overall system efficiency. Combined with curl_multi in PHP, it lays a solid foundation for building high-performance concurrent systems.
If you need to integrate with third-party APIs or build efficient scraping systems, it's worth trying the optimization capabilities offered by curl_share.
