从 0 到 1：用 PHP 爬虫优雅地拿下 Amazon 商品详情

1. 为什么选择 PHP 做 Amazon 爬虫？

表格

一句话：如果你本来就在写 Laravel，用 PHP 写爬虫等于「顺路」。

2. Amazon 页面结构 60 秒速览（2025-06 最新）

以 https://www.amazon.com/dp/B08N5WRWNW 为例：

表格

结论：95% 字段静态直取，无需上重型浏览器。

3. 环境准备：30 秒搭好最小可用环境

bash

# 创建项目
mkdir amz-php-crawler && cd amz-php-crawler
composer init --name="demo/amz-crawler" -s dev

# 安装依赖
composer require guzzlehttp/guzzle symfony/dom-crawler symfony/css-selector fakerphp/faker

PHP ≥ 8.0 即可，Guzzle 7.x 自带 PSR-18，后续想接 Hyperf 也很方便。

4. 核心流程：从 ASIN → 结构化数组

ASIN → 下载详情页 → 解析静态字段 → 调 AJAX 价格/库存 → 清洗 → CSV/MySQL/Kafka

5. 代码实战：Guzzle + DOMXPath 极速版

php

< ?php
// src/AmzSpider.php
namespace Demo;

use GuzzleHttpClient;
use SymfonyComponentDomCrawlerCrawler;

class AmzSpider
{
    private Client $client;

    public function __construct()
    {
        $this- >client = new Client([
            'timeout' => 10,
            'headers' => [
                'User-Agent' => 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
                'Accept-Language' => 'en-US,en;q=0.9',
                'Referer'    => 'https://www.amazon.com/',
            ],
        ]);
    }

    public function fetchProduct(string $asin): array
    {
        $url = "https://www.amazon.com/dp/{$asin}";
        $html = $this->client->get($url)->getBody()->getContents();
        $crawler = new Crawler($html);

        $title = $crawler->filter('#productTitle')->text('');
        $priceWhole = $crawler->filter('.a-price .a-price-whole')->text('');
        $priceFrac = $crawler->filter('.a-price .a-price-fraction')->text('');
        $price = trim($priceWhole . '.' . $priceFrac);

        $rating = $crawler->filter('#acrPopover')->attr('title');
        $rating = $rating ? explode(' ', $rating)[0] : '';

        $reviewText = $crawler->filter('#acrCustomerReviewText')->text('');
        $reviewCount = filter_var($reviewText, FILTER_SANITIZE_NUMBER_INT);

        $imgJson = $crawler->filter('#imgTagWrapperId img')->attr('data-a-dynamic-image');
        $imgMap = json_decode($imgJson, true);
        $mainImg = $imgMap ? array_key_first($imgMap) : '';

        return [
            'asin'          => $asin,
            'title'         => trim($title),
            'price'         => $price,
            'rating'        => $rating,
            'review_count'  => (int) $reviewCount,
            'main_img'      => $mainImg,
            'crawled_at'    => date('Y-m-d H:i:s'),
        ];
    }
}

CLI 入口文件：

php

#!/usr/bin/env php
< ?php
require __DIR__ . '/vendor/autoload.php';

use DemoAmzSpider;

$asin = $argv[1] ?? 'B08N5WRWNW';
$spider = new AmzSpider();
$data = $spider- >fetchProduct($asin);
echo json_encode($data, JSON_UNESCAPED_UNICODE | JSON_PRETTY_PRINT);

运行：

bash

php bin/amz.php B08N5WRWNW

输出示例：

JSON

{
  "asin": "B08N5WRWNW",
  "title": "Apple AirPods Pro",
  "price": "249.00",
  "rating": "4.6",
  "review_count": 25430,
  "main_img": "https://images-na.ssl-images-amazon.com/images/I/71zny7BTRlL._AC_SL1500_.jpg",
  "crawled_at": "2025-06-24 18:42:12"
}

6. 反爬三板斧：UA 池、代理池、随机延时

表格

Guzzle 中间件示例：

php

$stack = GuzzleHttpHandlerStack::create();
$stack->push(GuzzleHttpMiddleware::retry(function ($retries, $request, $response, $exception) {
    return $retries < 3 && ($exception || $response- >getStatusCode() === 403);
}, function ($retries) {
    return 1000 * (2 ** $retries);
}));

7. Selenium 兜底：滑块验证码与动态渲染

当 Amazon 出现滑块时，可用 php-webdriver 驱动 Chrome：

bash

composer require php-webdriver/webdriver

php

$host = 'http://localhost:4444/wd/hub'; // Selenium Standalone
$driver = RemoteWebDriver::create($host, DesiredCapabilities::chrome());
$driver->get("https://www.amazon.com/dp/$asin");
sleep(5);
$html = $driver->getPageSource();
$driver->quit();

技巧：

加载 stealth.min.js 隐藏 WebDriver 特征；

使用 undetected-chromedriver 可过新版滑块。

8. 提速 10 倍：多进程 + Swoole 协程

bash

composer require swoole/swoole

php

use SwooleRuntime;
use SwooleCoroutine;
use function SwooleCoroutinerun;

Runtime::enableCoroutine();
run(function () {
    $asins = ['B08N5WRWNW', 'B08L8DKCS1'];
    foreach ($asins as $asin) {
        Coroutine::create(function () use ($asin) {
            $spider = new AmzSpider();
            $data = $spider->fetchProduct($asin);
            file_put_contents("data/{$asin}.json", json_encode($data));
        });
    }
});

实测：4 核 8G 机器，协程版 2k SKU/min，CPU 占用 45%。

9. 数据落地：CSV、MySQL、Kafka 一键切换

① CSV（快速验证）

php

$fp = fopen('amz.csv', 'a');
fputcsv($fp, array_keys($data));
fputcsv($fp, $data);
fclose($fp);

② MySQL（生产）

php

$pdo = new PDO('mysql:host=localhost;dbname=amz', 'root', 'root');
$stmt = $pdo->prepare("REPLACE INTO product (asin,title,price,rating,review_count,main_img,crawled_at) VALUES (?,?,?,?,?,?,?)");
$stmt->execute([$data['asin'], $data['title'], $data['price'], $data['rating'], $data['review_count'], $data['main_img'], $data['crawled_at']]);

③ Kafka（实时流）

php

$producer = new KafkaProducer(function() {
    return [KafkaProducerConfig::BOOTSTRAP_SERVERS => 'localhost:9092'];
});
$producer->send([
    ['topic' => 'amz-product', 'value' => json_encode($data), 'key' => $data['asin']]
]);

10. 合规红线：Amazon 爬虫的法律底线

表格

官方替代方案：Amazon Product Advertising API（PA-API 5.0）

稳定、合规、无封 IP 风险

需 Associate Tag + 授权，每日 1w 额度

结论：能 API 不爬虫，能授权不硬刚。

11. 总结与进阶路线

✅ 原型阶段：本文代码直接跑，30 行即可出数
✅ 扩展阶段：协程池 + 代理池 + 重试，日采 50w SKU
✅ 生产阶段：Hyperf + Kafka + ES 实时搜索
✅ 商业闭环：价格告警、选品仪表盘、ERP 自动订价

12. 一键运行 & 源码

bash

git clone https://github.com/yourname/amz-php-crawler.git
cd amz-php-crawler
composer install
php bin/amz.php B08N5WRWNW

输出示例：

18:42:12 INFO ASIN=B08N5WRWNW, title=Apple AirPods Pro, price=$249.00, rating=4.6, reviews=25430

如果本文对你有帮助，记得 点赞 + 收藏 + 在看，我们下期「PHP + Swoole 实时价格流」见！

审核编辑 黄宇