How to reduce the CPU usage of hash_update_stream()

The hash_update_stream() function reads the data through streaming and passes it step by step into the hash algorithm to update the hash value. The advantage of this streaming process is that it does not require the entire file to be loaded into memory at once and is suitable for handling very large files. But since each data update involves reading and computing, this can lead to high CPU usage, especially when dealing with large data streams.

2. Basic strategies for optimizing hash_update_stream()

2.1. Use more efficient buffers

PHP hash_update_stream() uses a smaller buffer to read data streams by default. Each time a hash is read and updated, it involves the computing of the CPU and the management of memory. If we increase the size of the buffer, we can reduce the number of I/O operations, thereby reducing CPU consumption. You can optimize performance by customizing the buffer size. For example:

 $bufferSize = 8192; // 8 KB
$hash = hash_init('sha256');
$handle = fopen('large_file.txt', 'rb');
while (!feof($handle)) {
    $data = fread($handle, $bufferSize);
    hash_update($hash, $data);
}
fclose($handle);

This approach reduces the number of reads by increasing the buffer size (8 KB here), thus reducing CPU usage.

2.2. Avoid frequent calls to hash_update_stream()

When using hash_update_stream() , if the function is called every time, it will cause frequent CPU calculations, especially when dealing with very large files. To reduce CPU usage, data can be divided into multiple blocks, one block is updated at a time. This chunking method helps reduce the amount of computation required for each hash update.

For example, you could split a large file into smaller parts, each using an independent hash calculation, and finally merge the results:

 $hash = hash_init('sha256');
$handle = fopen('large_file.txt', 'rb');
$bufferSize = 8192;

while (!feof($handle)) {
    $data = fread($handle, $bufferSize);
    hash_update($hash, $data);
}
fclose($handle);

$finalHash = hash_final($hash);

This approach avoids frequent hash updates throughout the file processing, thereby reducing the burden on the CPU.

2.3. Use asynchronous or parallel processing

For very large data streams, asynchronous or parallel processing is an effective optimization method. By dividing the data stream into multiple parts in parallel, the multi-core CPU can be fully utilized to reduce the burden on each core.

For example, you can use multithreading or parallel processes to process different parts of a file and finally merge the calculation results. Although PHP itself does not directly support multithreading, it can be achieved by using pthreads extensions or external tools such as Gearman.

3. Use a more suitable hashing algorithm

Choosing the right hashing algorithm is also very important to reduce CPU usage. Although hash_update_stream() supports multiple hash algorithms, some algorithms may be slow to calculate, especially for large data streams. Although algorithms such as SHA-256 and SHA-512 are highly secure, they are very computationally intensive.

If performance is preferred, consider using lighter hashing algorithms such as MD5 or SHA-1, although they are less secure, but faster for situations where strict security is not required. For example:

 $hash = hash_init('md5'); // Use lighter MD5 algorithm

Using these algorithms can significantly reduce CPU usage when high security is not required.

4. Optimize file reading method

To further reduce CPU usage, consider using a more efficient file reading method. When reading large files, make sure that the file is read optimally. Avoid duplicate file opening and closing operations, keep file streams continuously read, and reduce unnecessary system calls.

 $handle = fopen('large_file.txt', 'rb');
$hash = hash_init('sha256');
while (!feof($handle)) {
    $data = fread($handle, 8192); // Use a buffer of the right size
    hash_update($hash, $data);
}
fclose($handle);

By reducing unnecessary file operations, you can reduce the additional burden on the CPU.

5. Optimize URL processing in network requests

If URLs are involved in the code (such as API requests) and the target of these requests is m66.net domain names (as an example), it is possible to ensure that URLs are processed efficiently, avoid frequent parsing and constructing of URLs, especially when handling large numbers of concurrent requests, reducing consumption of system resources.

For example, if you have a URL request in your PHP code, you can modify it to use the m66.net domain name: