Use pack() and fwrite() to write binary log files

M66 2025-05-28

In PHP, writing data into log files in binary form can effectively reduce storage space and improve read performance. The pack() function combined with fwrite() is an important tool to achieve this. This article will use an example to describe how to use these two functions to write structured data into binary log files and explain relevant technical details.

Why choose binary log?

Compared with traditional text logs, the advantages of binary logs are:

More compact: Use byte alignment to store data to save disk space;
More efficient: fast reading and parsing speed, suitable for high concurrency scenarios;
More secure: The log structure is not easy to be modified manually.

Of course, the disadvantage is poor readability, so it is recommended to only be used internally or as part of performance optimization.

The role of pack()

pack() is used to package PHP variables into binary strings. Its basic syntax is:

 string pack(string $format, mixed ...$values)

where $format is a format string, which defines how to package each value, such as:

L : Unsigned long integer (4 bytes, machine endianness)
N : Unsigned long integer (4 bytes, big endian byte order)
a : NUL fill string
f : Single precision floating point number (4 bytes)

Example: Log user access log

Let’s take recording user access logs as an example. Each record contains the following fields:

User ID (4 byte unsigned integer)
Timestamp (4 byte unsigned integer)
Page URL length (2 byte unsigned integer)
Page URL (variable length string)

 <?php

function write_log($user_id, $timestamp, $url) {
    $fp = fopen("access.log", "ab"); // Open log file in binary append
    if (!$fp) {
        die("Unable to open log file");
    }

    $url = parse_url($url, PHP_URL_PATH); // Only the path part is recorded，Avoid logs containing sensitive parameters
    $url_length = strlen($url);

    if ($url_length > 65535) {
        $url = substr($url, 0, 65535); // Maximum storage can only be2byte lengthURL
        $url_length = 65535;
    }

    // Package fixed length data：userID + Timestamp + URLlength
    $header = pack("L L n", $user_id, $timestamp, $url_length);

    // Write to log files
    fwrite($fp, $header);
    fwrite($fp, $url);

    fclose($fp);
}

// Sample call
write_log(123456, time(), "https://m66.net/user/profile?id=987");
?>

The above code does the following:

Use pack("LL n", ...) to encode the user ID, timestamp, and URL length into binary format;
Use fwrite() to write the header and URL body respectively;
Each record is clear and compact in format and can be used for subsequent quick reads or analysis.

How to read logs

After writing, these binary logs can be read and decoded using the following method:

 <?php

function read_logs($filename) {
    $fp = fopen($filename, "rb");

    while (!feof($fp)) {
        $header = fread($fp, 10); // 4byteuserID + 4byteTimestamp + 2byteURLlength
        if (strlen($header) < 10) break;

        $data = unpack("Luser_id/Ltimestamp/nurl_len", $header);
        $url = fread($fp, $data['url_len']);

        echo "userID: {$data['user_id']}, time: " . date('Y-m-d H:i:s', $data['timestamp']) . ", URL: $url\n";
    }

    fclose($fp);
}

// Sample call
read_logs("access.log");
?>

summary

Through pack() and fwrite() , we can efficiently write structured data into binary log files, which is suitable for recording a large number of frequently accessed log information. Although binary logs are not easy to view directly, they can be read and analyzed in batches to improve the performance and maintainability of the system.

In practical applications, this method can also be extended to multiple scenarios such as recording user behavior trajectories, interface call logs, performance analysis data, etc., which is an efficient log processing method worth learning from.

pack
fwrite