使用Python获取1688商品详情的教程

​

如何使用Python如何使用Python上的商品详情信息。通过本教程，读者将学会使用Python的请求库和解析库来爬取商品数据。

目录

简介

环境准备

编写爬虫代码

处理和存储数据

快速获取

遵守法律法规和网站政策

1. 简介

1688是中国领先的B2B电子商务平台，提供海量的商品信息。指导你如何使用Python来获取这些商品的详细信息。

2. 环境准备

Python 3.x

网络请求库：requests

HTML解析库：BeautifulSoup

可选：pandas用于数据处理和lxml作为BeautifulSoup的解析器

安装所需库：

pip install requests beautifulsoup4 pandas lxml

3. 编写爬虫代码

3.1 请求网页

首先，你需要使用requests库来发送HTTP请求，获取商品页面的HTML内容。

import requests def get_page(url): headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3' } response = requests.get(url, headers=headers) return response.text

3.2 解析HTML

使用BeautifulSoup来解析获取到的HTML内容，提取商品的详细信息。

from bs4 import BeautifulSoup def parse_page(html): soup = BeautifulSoup(html, 'lxml') # 假设商品名称在

标签中 title = soup.find('h1').text # 假设商品价格在标签中 price = soup.find('span', class_='price').text # 其他信息按需提取 # ... return { 'title': title, 'price': price, # ... }

3.3 整合代码

将上述功能整合到一个函数中，实现自动化爬取。

def fetch_product_details(url): html = get_page(url) product_details = parse_page(html) return product_details

4. 处理和存储数据

使用pandas库来处理和存储爬取的数据。

import pandas as pd def save_to_csv(data, filename): df = pd.DataFrame([data]) df.to_csv(filename, index=False, encoding='utf-8')

5.快速获取商品详情

快速获取1688商品详情

java（请求源码）

————————————————————

import java.io.BufferedReader;

import java.io.IOException;

import java.io.InputStream;

import java.io.InputStreamReader;

import java.io.Reader;

import java.net.URL;

import java.nio.charset.Charset;

import org.json.JSONException;

import org.json.JSONObject;

import java.io.PrintWriter;

import java.net.URLConnection;

public class Example {

private static String readAll(Reader rd) throws IOException {

StringBuilder sb = new StringBuilder();

int cp;

while ((cp = rd.read()) != -1) {

sb.append((char) cp);

}

return sb.toString();

}

public static JSONObject postRequestFromUrl(String url, String body) throws IOException, JSONException {

URL realUrl = new URL(url);

URLConnection conn = realUrl.openConnection();

conn.setDoOutput(true);

conn.setDoInput(true);

PrintWriter out = new PrintWriter(conn.getOutputStream());

out.print(body);

out.flush();

InputStream instream = conn.getInputStream();

try {

BufferedReader rd = new BufferedReader(new InputStreamReader(instream, Charset.forName("UTF-8")));

String jsonText = readAll(rd);

JSONObject json = new JSONObject(jsonText);

return json;

} finally {

instream.close();

}

}

public static JSONObject getRequestFromUrl(String url) throws IOException, JSONException {

URL realUrl = new URL(url);

URLConnection conn = realUrl.openConnection();

InputStream instream = conn.getInputStream();

try {

BufferedReader rd = new BufferedReader(new InputStreamReader(instream, Charset.forName("UTF-8")));

String jsonText = readAll(rd);

JSONObject json = new JSONObject(jsonText);

return json;

} finally {

instream.close();

}

}

public static void main(String[] args) throws IOException, JSONException {

// 请求示例 url 默认请求参数已经URL编码处理

String url = "https://api-gw.onebound.cn/1688/item_get/?key=<您自己的apiKey>&secret=<您自己的apiSecret>&num_iid=610947572360";

JSONObject json = getRequestFromUrl(url);

System.out.println(json.toString());

}

————————————————————

快速获取key和秘钥：https://o0b.cn/lin

返回商品格式

————————————————————

{

"item":{

"num_iid":601824709263,

"title":"韩版仿兔毛围巾秋冬毛毛围巾学生毛毛绒围脖冬季女套头防寒户外",

"desc_short":"",

"price":"2.2",

"total_price":0,

"suggestive_price":0,

"orginal_price":"2.2",

"nick":"_sopid@BBBEbRNm7OS0uGCXPJucMBy2w",

"num":3786174,

"min_num":2,

"detail_url":"https://detail.1688.com/offer/601824709263.html",

"pic_url":"https://cbu01.alicdn.com/img/ibank/11921650503_1818272533.jpg",

"brand":"",

"brandId":"",

"rootCatId":0,

"cid":325,

"crumbs":[

],

"created_time":"",

"modified_time":"",

"delist_time":"",

"desc":"

rn

材质：双面仿兔毛毛绒面料，

rn

不掉毛（不是薄薄的单面哦）

rn

厚款3.39元

rn

普通款2.38元

rn

每条围巾都是独立OPP袋子包装，放心选购！

rn

现货供应，厂家直销，没有中间商赚差价，有大量可以谈价，有量有价

rn

（抖音，快手，直播电商福利品，天猫，淘宝，PDD，跨境电商供应商）

rn

这款围巾是不掉毛的，有些激光高温裁剪残留的浮毛，抖抖就好了哦。

rn

rn

此图拉拉袋包装的为客户定做款（大货是OPP透明袋独立包装）

rn

rn

rn

rn

..........

",

"desc_img":[

"https://cbu01.alicdn.com/img/ibank/O1CN01emutXa27pyd6vYZzm_!!2206475177847-0-cib.jpg",

"https://file1.elecfans.com/web3/M00/2E/96/wKgZO2i_nCOAF0iuAAFOLd1ZnWU225.jpg",

"https://img.alicdn.com/imgextra/i3/2272041426/TB2lZSUX7iHJuJjSZFjXXcGvXXa_!!2272041426.jpg",

"https://img.alicdn.com/imgextra/i2/2272041426/TB2rvmVXYSGJuJjSZFqXXXo2pXa_!!2272041426.jpg",

"https://img.alicdn.com/imgextra/i3/2272041426/O1CN01EKuyER1MP9LVqRODf_!!2272041426.jpg",

"https://img.alicdn.com/imgextra/i3/2272041426/O1CN01fH7zeu1MP9LXYMDb9_!!2272041426.jpg",

"https://img.alicdn.com/imgextra/i3/2272041426/O1CN01o03kWB1MP9LUPh2vD_!!2272041426.jpg",

"https://img.alicdn.com/imgextra/i4/2272041426/O1CN01zAl3p11MP9LTi5nZh_!!2272041426.jpg",

"https://img.alicdn.com/imgextra/i3/2272041426/O1CN01Zv9KBT1MP9LUIKRow_!!2272041426.jpg",

"https://img.alicdn.com/imgextra/i3/2272041426/O1CN01VlbKPj1MP9LU0NVFN_!!2272041426.jpg",

"https://img.alicdn.com/imgextra/i4/2272041426/O1CN01MjaoVZ1MP9LTiFjny_!!2272041426.jpg",

"https://img.alicdn.com/imgextra/i4/2272041426/O1CN01junsnm1MP9LX5bOdM_!!2272041426.jpg",

"https://img.alicdn.com/imgextra/i2/2272041426/O1CN014Dv2oq1MP9LU0KgIb_!!2272041426.jpg",

"https://img.alicdn.com/imgextra/i4/2272041426/O1CN01jNA7KZ1MP9LUHdDBF_!!2272041426.jpg",

"https://img.alicdn.com/imgextra/i3/2272041426/O1CN01JZW4Hn1MP9LUHd91D_!!2272041426.jpg",

"https://img.alicdn.com/imgextra/i3/2272041426/O1CN01nSbj2H1MP9LVqZ2mg_!!2272041426.jpg",

"https://img.alicdn.com/imgextra/i4/2272041426/O1CN01HmPSpT1MP9LSDyrsr_!!2272041426.jpg"

],

"item_imgs":[

{

"url":"https://cbu01.alicdn.com/img/ibank/11921650503_1818272533.jpg"

},

{

"url":"https://cbu01.alicdn.com/img/ibank/11921665153_1818272533.jpg"

},

{

"url":"https://cbu01.alicdn.com/img/ibank/11884953047_1818272533.jpg"

},

{

"url":"https://cbu01.alicdn.com/img/ibank/11954549050_1818272533.jpg"

},

{

"url":"https://cbu01.alicdn.com/img/ibank/12483429103_1818272533.jpg"

}

],

"item_weight":"",

"item_size":"",

"location":"浙江省 金华市",

"post_fee":"",

"express_fee":"",

"ems_fee":"",

"shipping_to":"",

"has_discount":"",

"video":"https://cloud.video.taobao.com/play/u/2206475177847/p/1/e/6/t/1/238604472723.mp4",

"is_virtual":"",

"sample_id":"",

"is_promotion":"",

"props_name":"0:0:颜色:红色;0:1:颜色:黑色;0:2:颜色:白色;0:3:颜色:灰色;0:4:颜色:杏色;0:5:颜色:粉色;0:6:颜色:咖啡色;0:7:颜色:米色;0:8:颜色:卡其色",

"prop_imgs":{

"prop_img":[

{

"properties":"0:0",

"url":"https://cbu01.alicdn.com/img/ibank/11884977080_1818272533.jpg"

},

{

"properties":"0:1",

"url":"https://cbu01.alicdn.com/img/ibank/11954549680_1818272533.jpg"

},

{

"properties":"0:2",

"url":"https://cbu01.alicdn.com/img/ibank/11921650503_1818272533.jpg"

},

{

"properties":"0:3",

"url":"https://cbu01.alicdn.com/img/ibank/11921680795_1818272533.jpg"

},

{

"properties":"0:4",

"url":"https://cbu01.alicdn.com/img/ibank/11954549050_1818272533.jpg"

},

{

"properties":"0:5",

"url":"https://cbu01.alicdn.com/img/ibank/11921689268_1818272533.jpg"

},

{

"properties":"0:6",

"url":"https://cbu01.alicdn.com/img/ibank/11954552765_1818272533.jpg"

},

{

"properties":"0:7",

"url":"https://cbu01.alicdn.com/img/ibank/O1CN01PtW77W27pyekEhcAt_!!2206475177847-0-cib.jpg"

},

{

"properties":"0:8",

"url":"https://cbu01.alicdn.com/img/ibank/O1CN01t2Q1ci27pyjXB0fxo_!!2206475177847-0-cib.jpg"

}

]

},

"property_alias":"0:0:红色;0:1:黑色;0:2:白色;0:3:灰色;0:4:杏色;0:5:粉色;0:6:咖啡色;0:7:米色;0:8:卡其色",

"props":[

{

"name":"品牌",

"value":"高着"

},

{

"name":"编织方法",

"value":"经编"

},

{

"name":"功能",

"value":"保暖"

},

{

"name":"适合季节",

"value":"冬季"

},

{

"name":"货号",

"value":"WJ618"

},

{

"name":"产地",

"value":"浙江义乌"

},

{

"name":"款式",

"value":"中长"

},

{

"name":"颜色",

"value":"红色"

},

{

"name":"颜色",

"value":"黑色"

},

{

"name":"颜色",

"value":"白色"

},

{

"name":"颜色",

"value":"灰色"

},

{

"name":"颜色",

"value":"杏色"

},

{

"name":"颜色",

"value":"粉色"

},

{

"name":"颜色",

"value":"咖啡色"

},

{

"name":"颜色",

"value":"米色"

},

{

"name":"颜色",

"value":"卡其色"

},

{

"name":"主要下游平台",

"value":"淘宝"

},

{

"name":"主要下游平台",

"value":"京东"

},

{

"name":"主要下游平台",

"value":"ebay"

},

{

"name":"主要下游平台",

"value":"亚马逊"

},

{

"name":"主要下游平台",

"value":"wish"

},

{

"name":"主要下游平台",

"value":"快手"

},

{

"name":"主要下游平台",

"value":"速卖通"

},

{

"name":"主要下游平台",

"value":"天猫"

},

{

"name":"主要下游平台",

"value":"独立站"

},

{

"name":"主要下游平台",

"value":"LAZADA"

},

{

"name":"主要下游平台",

"value":"拼多多"

},

{

"name":"主要下游平台",

"value":"抖音"

},

{

"name":"有可授权的自有品牌",

"value":"否"

},

{

"name":"是否跨境出口专供货源",

"value":"是"

},

{

"name":"上市年份/季节",

"value":"2021年秋季"

},

{

"name":"宽度(CM)",

"value":"12cm"

},

{

"name":"风格",

"value":"百搭"

},

{

"name":"风格",

"value":"优雅"

},

{

"name":"风格",

"value":"极简"

},

{

"name":"风格",

"value":"休闲风"

},

{

"name":"风格",

"value":"时尚OL"

},

{

"name":"风格",

"value":"简约"

},

{

"name":"风格",

"value":"欧美"

},

{

"name":"风格",

"value":"轻奢风"

},

{

"name":"风格",

"value":"chic风"

},

{

"name":"风格",

"value":"高级感"

},

{

"name":"面料名称",

"value":"毛绒"

},

{

"name":"风格分类",

"value":"时尚通勤"

},

{

"name":"处理工艺",

"value":"经编"

},

{

"name":"流行元素分类",

"value":"无"

}

],

"total_sold":3786174,

"skus":{

"sku":[

{

"price":4.99,

"total_price":4.99,

"orginal price":4.99,

"properties":"0:0",

"properties_name":"0:0:颜色:红色",

"quantity":0,

"sku_id":5330445855404,

"spec_id":"87b07b1e746bd546e97a0f5888726ae3"

},

{

"price":4.99,

"total_price":4.99,

"orginal price":4.99,

"properties":"0:1",

"properties_name":"0:1:颜色:黑色",

"quantity":0,

"sku_id":5330445855410,

"spec_id":"e664b2e7c4adae52d35767f51a9219f6"

},

{

"price":4.99,

"total_price":4.99,

"orginal price":4.99,

"properties":"0:2",

"properties_name":"0:2:颜色:白色",

"quantity":0,

"sku_id":5330445855398,

"spec_id":"18dc5ba8c7dc3df0eb8b731977df2372"

},

{

"price":4.99,

"total_price":4.99,

"orginal price":4.99,

"properties":"0:3",

"properties_name":"0:3:颜色:灰色",

"quantity":8020,

"sku_id":5330445855408,

"spec_id":"05808df6001a6ffa6fa13ae7df588b36"

},

{

"price":4.99,

"total_price":4.99,

"orginal price":4.99,

"properties":"0:4",

"properties_name":"0:4:颜色:杏色",

"quantity":0,

"sku_id":5330445855412,

"spec_id":"1ab2b4cd3cc2b51ed4055978ca4b8579"

},

{

"price":4.99,

"total_price":4.99,

"orginal price":4.99,

"properties":"0:5",

"properties_name":"0:5:颜色:粉色",

"quantity":0,

"sku_id":5330445855402,

"spec_id":"7a394d5d7d8560fae263cd2ffe9ee444"

},

{

"price":4.99,

"total_price":4.99,

"orginal price":4.99,

"properties":"0:6",

"properties_name":"0:6:颜色:咖啡色",

"quantity":0,

"sku_id":5330445855396,

"spec_id":"423651c0792f75de75484ae9bd5fe535"

},

{

"price":4.99,

"total_price":4.99,

"orginal price":4.99,

"properties":"0:7",

"properties_name":"0:7:颜色:米色",

"quantity":0,

"sku_id":5330445855400,

"spec_id":"45b9c574ac45662aba79f2cb72d20112"

},

{

"price":4.99,

"total_price":4.99,

"orginal price":4.99,

"properties":"0:8",

"properties_name":"0:8:颜色:卡其色",

"quantity":0,

"sku_id":5330445855406,

"spec_id":"f7931f4b9892a8c41b76205cb1d50627"

}

]

},

"seller_id":null,

"sales":0,

"shop_id":null,

"props_list":{

"0:0":"颜色:红色",

"0:1":"颜色:黑色",

"0:2":"颜色:白色",

"0:3":"颜色:灰色",

"0:4":"颜色:杏色",

"0:5":"颜色:粉色",

"0:6":"颜色:咖啡色",

"0:7":"颜色:米色",

"0:8":"颜色:卡其色"

},

"seller_info":{

"nick":null,

"user_num_id":null,

"sid":null,

"title":null,

"zhuy":null,

"shop_name":null

},

"tmall":"false",

"error":"",

"warning":"",

"url_log":[

],

"unit":null,

"is_support_mix":null,

"mix_amount":null,

"mix_begin":null,

"mix_number":null,

"priceRange":[

[

2,

"2.2"

]

],

"priceRangeOriginal":[

],

"_ddf":"sop",

"props_img":{

"0:0":"https://cbu01.alicdn.com/img/ibank/11884977080_1818272533.jpg",

"0:1":"https://cbu01.alicdn.com/img/ibank/11954549680_1818272533.jpg",

"0:2":"https://cbu01.alicdn.com/img/ibank/11921650503_1818272533.jpg",

"0:3":"https://cbu01.alicdn.com/img/ibank/11921680795_1818272533.jpg",

"0:4":"https://cbu01.alicdn.com/img/ibank/11954549050_1818272533.jpg",

"0:5":"https://cbu01.alicdn.com/img/ibank/11921689268_1818272533.jpg",

"0:6":"https://cbu01.alicdn.com/img/ibank/11954552765_1818272533.jpg",

"0:7":"https://cbu01.alicdn.com/img/ibank/O1CN01PtW77W27pyekEhcAt_!!2206475177847-0-cib.jpg",

"0:8":"https://cbu01.alicdn.com/img/ibank/O1CN01t2Q1ci27pyjXB0fxo_!!2206475177847-0-cib.jpg"

},

"sales_info":{

"seller_num":"",

"repeat_rate_purchase":"",

"per_capita_purchases":"",

"comment_num":"",

"comment_url":""

},

"format_check":"ok",

"shop_item":[

],

"relate_items":[

]

},

"error":"",

"secache":"a1950f8cc13611671db129a5c7ae864a",

6. 遵守法律法规和网站政策

在进行网页爬取时，必须遵守相关的法律法规和网站的爬虫政策。不要频繁发送请求以免对网站服务器造成压力。

​审核编辑 黄宇