Getting Started Guide

Learn how to use the Web Scraper API Builder or run scraping tools programmatically. Please note that to use our Web Scraper API, you need to have a Thordata account. If you don’t have one yet, please complete the register process first. Once you're familiar with how to use it, you can login to the dashboard and start using the service.

1. Choose Your First Scraper After logging in to the Thordata dashboard, navigate to the Web Scraper Store. To obtain your first scraped dataset, you must choose an existing scraper from the Web Scraper Store. In this example, we’ll use “Amazon - Collect Product Information via URL” as a demonstration. Once selected, you can launch the scraper in several different ways.

1. Understand Your First Scraper After selecting your first scraper, you will be taken to a new page where you can view an introduction to the scraper, sample outputs in JSON/CSV formats, explanations of output fields, and details about the input parameters required for the scraper to run. Additionally, if the field you need is not included in the current scraper, you can submit feedback either at the missing data point or through the feedback section at the bottom of the page. We will contact you within 24 hours and assist you in resolving the issue.

1. Use Your First Scraper ① Scrape via the API Builder: Replace the “URLs” field in the API Builder with the Amazon product detail link you want to scrape, then click “Start Scraping” to begin.

② Scrape Programmatically: You can use the code on the right to send a POST request to call the scraper. For example:

curl --request POST --url https://scraperapi.thordata.com/builder?product_id=amazon_product_by-url --header "Authorization: Bearer Token-ID" --header "content-type: application/json" --data "{\"spider_info\":{\"spider_parameters\":[{\"url\":\"https://www.amazon.com/LAURA-GELLER-NEW-YORK-Retractable/dp/B086H4VCBJ\"}],\"spider_id\":\"amazon_product_by-url\",\"spider_errors\":true},\"spider_name\":\"amazon.com\"}"

1. Retrieving Results and Historical Scraping Records The Web Scraper tool may take some time to complete execution. Please wait patiently. That's it! You'll soon receive the successful results of your first created task. You can export the data in multiple formats via the download button in the "Tasks" list. Of course, if you closed the scraping page directly after completion but still wish to view the historical scraped content, you can access all historical scraping records through the "Tasks" list in the scraping tool.

Need more help? Contact us at [email protected].