How to combine curl_multi_select() and curl_upkeep() to optimize connection

M66 2025-06-06

When making large-scale HTTP requests, PHP's cURL library is a very powerful tool that can help us make concurrent requests and improve performance. However, simply placing multiple requests does not necessarily lead to optimal performance, as we need to manage the connection efficiently. In order to achieve more efficient connection optimization, you can use curl_upkeep() and curl_multi_select() functions to reduce the connection overhead.

1. What are curl_upkeep() and curl_multi_select() ?

curl_upkeep() : This function is mainly used to manage initialized cURL connections, ensuring that they remain valid when used, and can save resources by keeping the reuse of the connections, reducing duplicate connection establishment processes.
curl_multi_select() : This function is used with curl_multi_exec() , which can help us optimize the execution efficiency of multitasking cURL requests. Its function is to block the waiting for at least one cURL handle to be ready, suitable for efficient management of large numbers of concurrent requests.

By combining these two functions, the delay during each request can be greatly reduced and the concurrent processing efficiency between multiple requests can be improved.

2. Basic cURL multitasking

Before introducing how to use curl_upkeep() and curl_multi_select() , let's review how to perform basic multitasking. The following code demonstrates how to use curl_multi_init() and curl_multi_exec() to make concurrent requests.

 // Initialize multiplecURLHandle
$mh = curl_multi_init();
$handles = [];

$urls = [
    'https://m66.net/api/data1',
    'https://m66.net/api/data2',
    'https://m66.net/api/data3'
];

// Create one for each requestcURLHandle
foreach ($urls as $url) {
    $ch = curl_init($url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_multi_add_handle($mh, $ch);
    $handles[] = $ch;
}

// Perform multitasking requests
do {
    $status = curl_multi_exec($mh, $active);
    if ($active) {
        curl_multi_select($mh); // Block until there is an active request
    }
} while ($active);

// Process the results and clean them
foreach ($handles as $ch) {
    $response = curl_multi_getcontent($ch);
    echo $response;
    curl_multi_remove_handle($mh, $ch);
    curl_close($ch);
}

curl_multi_close($mh);

This code multitasking requests via curl_multi_exec() and calls curl_multi_select() to wait for the active connection every time the loop, thus avoiding unnecessary CPU resource usage.

3. Use curl_upkeep() to implement connection multiplexing

In order to avoid frequent establishment and destruction of connections, we can implement connection multiplexing through the curl_upkeep() function. Although PHP does not directly provide curl_upkeep() function, we can use the curl_multi_* function combination to achieve similar functions. Specifically, we use a persistent connection pool to manage these connections instead of recreating the connection every time we request.

Implementation steps:

Connection pool initialization : We first create a cURL handle pool, and the connections in the pool will be reused, instead of initializing a new connection for each request.
Connection multiplexing : During multitasking request execution, we can check the connection in the connection pool, and if there are free connections, we can reuse it.

Sample code:

 // Create a persistent connection pool
$mh = curl_multi_init();
$handles = [];
$connectionPool = [];

// RequestedURLs
$urls = [
    'https://m66.net/api/data1',
    'https://m66.net/api/data2',
    'https://m66.net/api/data3'
];

// initializationcURLHandle并存储
foreach ($urls as $url) {
    if (empty($connectionPool[$url])) {
        $ch = curl_init($url);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
        $connectionPool[$url] = $ch; // Save new connections into the connection pool
    } else {
        $ch = $connectionPool[$url]; // Use an existing connection in the connection pool
    }

    curl_multi_add_handle($mh, $ch);
    $handles[] = $ch;
}

// Perform multitasking requests
do {
    $status = curl_multi_exec($mh, $active);
    if ($active) {
        curl_multi_select($mh); // Block until there is an active request
    }
} while ($active);

// Process the results and clean them
foreach ($handles as $ch) {
    $response = curl_multi_getcontent($ch);
    echo $response;
    curl_multi_remove_handle($mh, $ch);
    // Here we do not close the connection，Keep multiplexing in connection pool
}

curl_multi_close($mh);

In the above code, we create a simple connection pool connectionPool and multiplex it with connections in the pool when multitasking requests are executed. This can significantly improve connection efficiency, especially when multiple requests point to the same domain name.

4. Summary

By combining curl_upkeep() (connection pool management) and curl_multi_select() (blocking and waiting for active requests), we can optimize cURL multitasking requests in PHP to improve overall system performance. The connection pooling mechanism effectively reduces the frequent creation and destruction of connections and saves system resources, while curl_multi_select() ensures efficient concurrent execution.

In this way, you can manage connections more efficiently when handling large amounts of HTTP requests, improve program response speed, reduce latency, and achieve higher performance.

curl_upkeep