Getting Started Guide

Learn how to use the Web Scraper API Builder or run scraping tools programmatically. It should be noted that when you want to use our Web Scraper API, you need to have a Thordata account. If you don’t have one, please complete the registration first. If you already know how to use it, you can go to the dashboard to log in and use it.

1. Choose Your First Scraper After logging in to the Thordata dashboard, navigate to the Web Scraper Store. To get your first scraping file, you must select an existing scraper from the Web Scraper Store.

   We will choose "Amazon - Collect Product Information via URL" as an example in this article. After selecting a scraper, you can launch it in multiple ways.

1. Understand Your First Scraper After selecting your first scraper, you will be directed to a new page. On this page, you can view several details about the scraper, including its introduction, sample JSON/CSV outputs, explanations of the fields in the sample outputs, as well as descriptions of the input parameters required for the scraper to run.

   In addition, if the scraper does not include a field you need, you can provide feedback through the feedback entry at the missing data point or at the bottom of the page. We will contact you within 24 hours to help resolve the issue.

1. Use Your First Scraper ① Scrape via the API Builder: Replace the "URLs" in the API Builder with the Amazon product detail links you want to scrape, then click "Start Scraping".

② Scrape Programmatically: Call the scraper by sending a POST request using the code on the right side.

curl -X POST "https://scraperapi.thordata.com/builder" \
  -H "Authorization: Bearer Token-ID" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "spider_name=amazon.com" \
  -d "spider_id=amazon_product_by-url" \
  -d "spider_parameters=[{\"url\": \"https://www.amazon.com/HISDERN-Checkered-Handkerchief-Classic-Necktie/dp/B0BRXPR726\",\"zip_code\": \"94107\"}]" \
  -d "spider_errors=true" \
  -d "file_name={{TasksID}}"

curl -X POST "https://scraperapi.thordata.com/builder" ^
  -H "Authorization: Bearer Token-ID" ^
  -H "Content-Type: application/x-www-form-urlencoded" ^
  -d "spider_name=amazon.com" ^
  -d "spider_id=amazon_product_by-url" ^
  -d "spider_parameters=[{\"url\": \"https://www.amazon.com/HISDERN-Checkered-Handkerchief-Classic-Necktie/dp/B0BRXPR726\",\"zip_code\": \"94107\"}]" ^
  -d "spider_errors=true" ^
  -d "file_name={{TasksID}}"

1. Retrieving Results and Historical Scraping Records The Web Scraper may take some time to complete its operation, so please wait patiently.

   You will soon see the success of your first created task. You can export data in multiple formats via the download button in the "Tasks" list. Of course, if you closed the scraping page directly after the scraping was completed but still want to view the historical scraping content, you can check all historical scraping records through the "Tasks" list of the scraper.

If you need further assistance, please contact us via email at [email protected].