Executando verificação de segurança...
1
marcuth
5 min de leitura ·

[Help] Web Scraping: AliExpress

Dificudades em extrair os dados corretos

Olá pessoal estou enfrentando dificudades em raspar os dados do AliExpress, ví que eles tem uma API fussando as requisições da red, porém há um parâmetro na hora de fazer a requisição no qual ele afeta completamente o resultado dessa API, olhem só:

URL da API

https://pt.aliexpress.com/aeglodetailweb/api/store/header

Exemplo

Ou versão fetch()

fetch("https://pt.aliexpress.com/aeglodetailweb/api/store/header?fields=installment&itemId=1005004336772445&skuId=12000028808253571&minPriceCurrency=BRL&minPrice=114.55", {
  "headers": {
    "accept": "*/*",
    "accept-language": "pt-BR,pt;q=0.9,en-US;q=0.8,en;q=0.7",
    "cache-control": "no-cache",
    "pragma": "no-cache",
    "sec-ch-ua": "\"Not.A/Brand\";v=\"8\", \"Chromium\";v=\"114\", \"Google Chrome\";v=\"114\"",
    "sec-ch-ua-mobile": "?0",
    "sec-ch-ua-platform": "\"Windows\"",
    "sec-fetch-dest": "empty",
    "sec-fetch-mode": "cors",
    "sec-fetch-site": "same-origin"
  },
  "referrer": "https://pt.aliexpress.com/item/1005004336772445.html",
  "referrerPolicy": "strict-origin-when-cross-origin",
  "body": null,
  "method": "GET",
  "mode": "cors",
  "credentials": "include"
})

Os query parameters da URL:

{
 "fields": "installment",
 "itemId": "1005004336772445",
 "skuId": "12000028808253571",
 "minPriceCurrency" :"BRL",
 "minPrice": "114.55"
}

O problema

O grande vilão dessa história é a query minPrice, pois quando eu a removo retorma uma página HTML mas quando eu deixo eu tenho que colocar o valor certo pois ela inpacta totalmente no resultado da API

O meu código em Python

Com minPrice de "0":

{"displayMultipleFreight": false, "hideFreight": true}

Com minPrice de "50":

{"displayMultipleFreight": false, "hideFreight": true, "installmentInfo": {"defaultInstallmentForm": {"hasInterestFee": true, "installmentTerm": 2, "termAmount": {"currency": "BRL", "formatedAmount": "R$25,25", "value": 25.25}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$50,50", "value": 50.5}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$0,50", "value": 0.5}}, "installmentFormList": [{"hasInterestFee": true, "installmentTerm": 2, "termAmount": {"currency": "BRL", "formatedAmount": "R$25,25", "value": 25.25}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$50,50", "value": 50.5}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$0,50", "value": 0.5}}, {"hasInterestFee": true, "installmentTerm": 3, "termAmount": {"currency": "BRL", "formatedAmount": "R$16,91", "value": 16.91}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$50,75", "value": 50.75}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$0,75", "value": 0.75}}, {"hasInterestFee": true, "installmentTerm": 4, "termAmount": {"currency": "BRL", 
"formatedAmount": "R$12,81", "value": 12.81}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$51,25", "value": 51.25}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$1,25", "value": 1.25}}, {"hasInterestFee": true, "installmentTerm": 5, "termAmount": {"currency": "BRL", "formatedAmount": "R$10,50", "value": 10.5}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$52,50", "value": 52.5}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$2,50", "value": 2.5}}, {"hasInterestFee": true, "installmentTerm": 6, "termAmount": {"currency": "BRL", "formatedAmount": "R$8,76", "value": 8.76}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$52,60", "value": 52.6}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$2,60", "value": 2.6}}, {"hasInterestFee": true, "installmentTerm": 7, "termAmount": {"currency": "BRL", "formatedAmount": "R$7,57", "value": 7.57}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$52,99", "value": 52.99}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$2,99", "value": 2.99}}, {"hasInterestFee": true, "installmentTerm": 8, "termAmount": {"currency": "BRL", "formatedAmount": "R$6,65", "value": 6.65}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$53,25", "value": 53.25}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$3,25", "value": 3.25}}, {"hasInterestFee": true, "installmentTerm": 9, "termAmount": {"currency": "BRL", "formatedAmount": "R$5,94", "value": 5.94}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$53,50", "value": 53.5}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$3,50", "value": 3.5}}, {"hasInterestFee": true, "installmentTerm": 10, "termAmount": {"currency": "BRL", "formatedAmount": "R$5,37", "value": 5.37}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$53,75", "value": 53.75}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$3,75", "value": 3.75}}, {"hasInterestFee": true, "installmentTerm": 11, "termAmount": {"currency": "BRL", "formatedAmount": "R$4,90", "value": 4.9}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$53,99", "value": 53.99}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$3,99", "value": 3.99}}, {"hasInterestFee": true, "installmentTerm": 12, "termAmount": {"currency": "BRL", "formatedAmount": "R$4,54", "value": 4.54}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$54,50", "value": 54.5}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$4,50", "value": 4.5}}], "priceTagText": "2x R$25,25"}}

Com minPrice de "100":

{"displayMultipleFreight": false, "hideFreight": true, "installmentInfo": {"defaultInstallmentForm": {"hasInterestFee": true, "installmentTerm": 2, "termAmount": {"currency": "BRL", "formatedAmount": "R$25,25", "value": 25.25}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$50,50", "value": 50.5}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$0,50", "value": 0.5}}, "installmentFormList": [{"hasInterestFee":rrency": "BRL", "formatedAmount": "R$0,50", "value": 0.5}}, {"hasInterestFee": true, "installmentTerm": 3, "termAmount": {"currency": "BRL", "formatedAmount": "R$16,91", "value": 16.91}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$50,75", "value": 50.75}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$0,75", "value": 0.75}}, {"hasInterestFee": true, "installmentTerm": 4, "termAmount": {"currency": "BRL", 
"formatedAmount": "R$12,81", "value": 12.81}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$51,25", "value": 51.25}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$1,25", "value": 1.25}}, {"hasInterestFee": true, "installmentTerm": 5, "termAmount": {"currency": "BRL", "formatedAmount": "R$10,50", "value": 10.5}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$52,50", "value": 52.5}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$2,50", "value": 2.5}}, {"hasInterestFee": true, "installmentTerm": 6, "termAmount": {"currency": "BRL", "formatedAmount": "R$8,76", "value": 8.76}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$52,60", "value": 52.6}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$2,60", "value": 2.6}}, {"hasInterestFee": true, "installmentTerm": 7, "termAmount": {"currency": "BRL", "formatedAmount": "R$7,57", "value": 7.57}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$52,99", "value": 52.99}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$2,99", "value": 2.99}}, {"hasInterestFee": true, "installmentTerm": 8, "termAmount": {"currency": "BRL", "formatedAmount": "R$6,65", "value": 6.65}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$53,25", "value": 53.25}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$3,25", "value": 3.25}}, {"hasInterestFee": true, "installmentTerm": 9, "termAmount": {"currency": "BRL", "formatedAmount": "R$5,94", "value": 5.94}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$53,50", "value": 53.5}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$3,50", "value": 3.5}}, {"hasInterestFee": true, "installmentTerm": 10, "termAmount": {"currency": "BRL", "formatedAmount": "R$5,37", "value": 5.37}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$53,75", "value": 53.75}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$3,75", "value": 3.75}}, {"hasInterestFee": true, "installmentTerm": 11, "termAmount": {"currency": "BRL", "formatedAmount": "R$4,90", "value": 4.9}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$53,99", "value": 53.99}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$3,99", "value": 3.99}}, {"hasInterestFee": true, "installmentTerm": 12, "termAmount": {"currency": "BRL", "formatedAmount": "R$4,54", "value": 4.54}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$54,50", "value": 54.5}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$4,50", "value": 4.5}}], "priceTagText": "2x R$25,25"}}
PS D:\programming\projects\actives\python\aliexpress-api-wrapper> & C:/Users/Usuario/AppData/Local/Programs/Python/Python311/python.exe d:/programming/projects/actives/python/aliexpress-api-wrapper/main.py
{"displayMultipleFreight": false, "hideFreight": true, "installmentInfo": {"defaultInstallmentForm": {"hasInterestFee": true, "installmentTerm": 2, "termAmount": {"currency": "BRL", "formatedAmount": "R$50,50", "value": 50.5}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$101,00", "value": 101.0}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$1,00", "value": 1.0}}, "installmentFormList": [{"hasInterestFee": true, "installmentTerm": 2, "termAmount": {"currency": "BRL", "formatedAmount": "R$50,50", "value": 50.5}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$101,00", "value": 101.0}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$1,00", "value": 1.0}}, {"hasInterestFee": true, "installmentTerm": 3, "termAmount": {"currency": "BRL", "formatedAmount": "R$33,83", "value": 33.83}, "termTotalAmount": {"currency": 
"BRL", "formatedAmount": "R$101,50", "value": 101.5}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$1,50", "value": 1.5}}, {"hasInterestFee": true, "installmentTerm": 4, "termAmount": {"currency": "BRL", "formatedAmount": "R$25,62", "value": 25.62}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$102,50", "value": 102.5}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$2,50", "value": 2.5}}, {"hasInterestFee": true, "installmentTerm": 5, "termAmount": {"currency": "BRL", "formatedAmount": "R$21,00", "value": 21.0}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$105,00", "value": 105.0}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$5,00", "value": 5.0}}, {"hasInterestFee": true, "installmentTerm": 6, "termAmount": {"currency": "BRL", "formatedAmount": "R$17,53", "value": 17.53}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$105,20", "value": 105.2}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$5,20", "value": 5.2}}, {"hasInterestFee": true, "installmentTerm": 7, "termAmount": 
{"currency": "BRL", "formatedAmount": "R$15,14", "value": 15.14}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$105,99", "value": 105.99}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$5,99", "value": 5.99}}, {"hasInterestFee": true, "installmentTerm": 8, "termAmount": {"currency": "BRL", "formatedAmount": "R$13,31", "value": 13.31}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$106,50", "value": 106.5}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$6,50", "value": 6.5}}, {"hasInterestFee": true, "installmentTerm": 9, "termAmount": {"currency": "BRL", "formatedAmount": "R$11,88", "value": 11.88}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$107,00", "value": 107.0}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$7,00", "value": 7.0}}, {"hasInterestFee": true, "installmentTerm": 10, "termAmount": {"currency": "BRL", "formatedAmount": "R$10,75", "value": 10.75}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$107,50", "value": 107.5}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$7,50", "value": 7.5}}, {"hasInterestFee": true, "installmentTerm": 11, "termAmount": {"currency": "BRL", "formatedAmount": "R$9,81", "value": 9.81}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$107,99", "value": 107.99}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$7,99", "value": 7.99}}, {"hasInterestFee": true, "installmentTerm": 12, "termAmount": {"currency": "BRL", "formatedAmount": "R$9,08", "value": 9.08}, "termTotalAmount": {"currency": "BRL", "formatedAmount": "R$109,00", "value": 109.0}, "termTotalInterestFee": {"currency": "BRL", "formatedAmount": "R$9,00", "value": 9.0}}], "priceTagText": "2x R$50,50"}}

Repositódio do código Python: https://github.com/1Marcuth/aliexpress-api-wrapper

Carregando publicação patrocinada...