English
English
Data cleaning and standardization are an extremely important task when developing web crawlers, especially when we need to extract HTML tables, JSON interfaces, or API data from multiple sources, there may be case inconsistent for different field key names. In order to facilitate subsequent data processing and logical judgment, we often need to unify the key names in the array to be all lowercase or all uppercase.
In PHP, array_change_key_case() is an efficient and practical function that can help us complete this task quickly. This article will start from the actual Web crawler scenario and introduce the use of this function and its efficient application in crawler data processing.
array_change_key_case(array $array, int $case = CASE_LOWER): array
$array : the original array of input;
$case : Specifies the target case, optional values are CASE_LOWER (default) or CASE_UPPER .
This function returns a new array whose key names have been converted to the specified case.
Suppose we grab a JSON interface from the website https://api.m66.net/data/products and get an array of the following structure:
$data = [
"ProductID" => 123,
"ProductName" => "USB Cable",
"PRICE" => 9.99,
"currency" => "USD"
];
As you can see, the case of these key names is very inconsistent. When processing this type of data, if we use $data['price'] directly, we cannot get the value because the real key is PRICE . To solve this problem, we can use array_change_key_case() :
$normalizedData = array_change_key_case($data, CASE_LOWER);
echo $normalizedData['price']; // Output：9.99
In this way, we can handle key names in a unified manner, and no longer need to write multiple judgment branches for different upper and lower cases.
If the crawled data is a nested array, for example:
$data = [
"ProductID" => 123,
"Details" => [
"Manufacturer" => "XYZ Corp",
"Warranty" => "1 year"
]
];
At this time array_change_key_case() will only process the top-level key names and will not process the subarray recursively. If you need to recursively unify the key name, you can use the following custom function:
function array_change_key_case_recursive(array $array, int $case = CASE_LOWER): array {
$result = [];
foreach ($array as $key => $value) {
$key = is_string($key) ? ($case === CASE_UPPER ? strtoupper($key) : strtolower($key)) : $key;
$result[$key] = is_array($value) ? array_change_key_case_recursive($value, $case) : $value;
}
return $result;
}
$normalizedData = array_change_key_case_recursive($data, CASE_LOWER);
echo $normalizedData['details']['manufacturer']; // Output：XYZ Corp
In web crawlers, we often use json_decode() to get JSON data, and this function returns an object or an array. To make it easier to use key name unified operations, we recommend returning the associative array directly when decoding:
$json = file_get_contents("https://api.m66.net/data/products");
$data = json_decode($json, true); // The second parameter is true，Returns the associative array
$normalizedData = array_change_key_case_recursive($data, CASE_LOWER);
In this way, the processed array structure will be more consistent and it will be easier to perform subsequent field extraction and logical judgment.
Unified case of array key names is a very practical requirement in web crawlers, especially when merging, searching, mapping or storing multi-source data. PHP's built-in array_change_key_case() provides an efficient solution that can easily deal with complex data structures with recursive implementation.
In actual development, it is recommended to encapsulate this function as a tool method and be uniformly used in the data cleaning stage of all crawler modules, greatly improving the robustness and maintenance of the code. With just one line of code, the key name case problem will be solved easily.
