The hardest part about web scraping can be getting to the data you want to scrape.

For example, you might want to scrape data from a search results page for a number of keywords.

You mighty setup separate scraping projects for each keyword.

However, there are powerful web scrapers that can automate the searching process and scrape the data you want.

Today, we will set up a web scraper to search through a list of keywords and scrape data for each one.

A Free and Powerful Web Scraper

For this project, we will use ParseHub. A free and powerful web scraper that can scrape data from any website.

Make sure to download and install ParseHub for free before we get started.

We will also scrape data from Amazon’s search result page for a short list of keywords.

Video Tutorial on how to scrape keyword results

Searching and Scraping Data from a List of Keywords

Now it’s time to setup our project and start scraping data.

  1. Install and Open ParseHub. Click on “New Project” and enter the URL of the website you will be scraping from. In this case, we will scrape data from The page will then render inside the app and allow you to start extracting data.
  2. Now, we need to give ParseHub our list of keywords we will be searching through to extract data. To do this, click on the settings icon at the top left and click on “settings”.
Opening ParseHub Settings
  1. Under the “Starting Value” section you can enter your list of keywords either as a CSV file or in JSON format right in the text box below it.
ParseHub Settings
  1. If you’re using a CSV file to upload your keywords, make sure you have a header cell. In this case, it will be the word “keywords”.
Excel of keywords
  1. Once you’ve submitted your list of keywords, click on “Back to Commands” to go back to your project. Click on the PLUS (+) sign next to your “page” selection, click on Advanced and click on the “Loop” command.
Selecting the loop command
  1. By default, your list of keywords will be selected as the list of items to loop through. If not, make sure to select “keywords” from the dropdown.
Select "keywords" from dropdown menu
  1. Click on the PLUS (+) sign next to your “For each item in keywords” selection and choose the “Begin New Entry” command. This command will be named “list1” by default.
Selecting the "Begin New Entry" command
  1. Click on the PLUS (+) sign next to your “list1” tool and choose the “Select” command.
Using the Select Command
  1. With the select command, click directly on the Amazon search bar to select it.
Select the Amazon search bar
  1. This will create an input command, under it, choose “expression” from the dropdown and enter the word “item” on the text box.
Input type is Expression and the word item in the text box
  1. Now we will make it so ParseHub adds the keyword for each result next to it. To do this, click on the PLUS (+) sign next to the “list1” command and choose the “extract” command.
select the extract command from list command
  1. Under the extract command, enter the word “item” into the first text box.
  1. Now, let’s tell ParseHub to perform the search for the keywords in the list. Click on the PLUS (+) sign next to your “list1” selection and choose the select command.
Use the select command for "list1" command
  1. Click on the Search Button to select it and rename it to “search_bar”
rename selection to "search_bar"
  1. Click on the PLUS (+) sign next to your “search_bar” selection and choose the “Click” command
Under your search_bar command, choose the click command
  1. A pop-up will appear asking you if this is a “next page” button. Click on “No” and rename your new template to “search_results”
Pop up window click command setup
  1. Now, let’s navigate to the search results page of the first keyword on the list and extract some data.
  2. Start by switching over to browse mode on the top left and search for the first keyword on the list.
Enter Browse mode and enter the first keyword
  1. Once the page renders, make sure you are still working on your new “search_results” template by selecting it with the tabs on the left.
Search results page
  1. Now, turn off Browse Mode and click on the name of the first result on the page to select it. It will be highlighted in green to indicate that it has been selected.
Turn off browse mode, select first name
  1. The rest of the products on the page will be highlighted in yellow. Click on the second one on the list to select them all.
Selecting all of the product names
  1. ParseHub is now extracting the product name and URL for each product on the first page of results for each keyword.

Do you want to extract more data from each product? Check out our guide on how to scrape product data from Amazon including prices, ASIN codes and more.

Do you want to extract more pages worth of data? Check out our guide on how to add pagination to your project and extract data from more than one page of results.

Running your Scrape

It is now time to run your project and export it as an Excel file.

To do this, click on the green “Get Data” button on left sidebar.

Here you will be able to test, run or schedule your scrape job. In this case, we will run it right away.

Closing Thoughts

Exported data in an excel sheet

ParseHub is now off to extract all the data you’ve selected. Once your scrape is complete, you will be able to download it as a CSV or JSON file.

You now know how to search through a list of keywords and extract data from each result.

What website will you scrape first?

Download and install ParseHub for free