In this guide, you will learn how to scrape Google Trends’ headlines and outlets for free, with ParseHub, in just a few easy steps!
Google is the largest search engine in the world and has access to billions of searches a day. With this data, Google is able to provide a good glimpse into trending news and headlines every single day, in real-time! According to Google, its Trends data is anonymized, categorized and aggregated; which means they’re grouped together. For example, you can find trending topics within a particular geographic area or category. This is great for businesses that require trending topics, for their content, or for their research. Aside from real-time viral news, web scraping can be combined with Google Trends for enterprise market research. You can compare topics and see the difference in growth, allowing you to make a better business decisions.
Ready to start scraping Google Trends? Begin by downloading our free web scraping tool!
Step 1: Preloading Trending Topics
- Begin by opening ParseHub and clicking the “New Project” button.
- Enter the Google Trends URL you wish to scrape, we will be scraping real-time search trends in Canada, with this URL: https://trends.google.com/trends/trendingsearches/realtime?geo=CA&category=all
- Let’s scrape more than a single page by preloading more topics, scroll all the way down until you see the “Load More” button and click it.
- Rename this selection on the left to “loadMore”, expand it and delete the extraction.
- Click the PLUS(+) button next to your loadMore selection and choose “Click”.
- Choose “Yes” on the popup, and now you can choose the additional amount of pages you wish to scrape.
- We chose 2, to scrape 3 pages of data in total.
Step 2: Extracting Trending Topics
- Now that we have preloaded data, let's extract trending topics. Begin by clicking the PLUS(+) button on the “page” selection on the left pane.
- Choose “Select” and hover over the first trending topic.
- Press and hold CTRL/CMD+2, and you should see a div with a blue rectangle. Click to extract the first headline.
- Repeat this on the following headline and click to train the algorithm.
- All topics should now be extracted, rename this selection on the left to “topics”.
Step 3: Scraping Additional Data
To get additional data from each topic, such as its news outlet and the day it was published, we need to use ParseHub’s Relative Select tool.
- Firstly, click the PLUS(+) button next to your “topics” selection and choose “Relative Select”.
- Click the first topic’s headline, then its news outlet to close the relative selection.
- All news outlets should now be extracted, rename this selection to “outlet” on the left.
- Redo the “Relative Select” tool on the “topics” selection, and this time after clicking the headline, click the publish date.
- Rename this selection on the left to “date”.
- Finally, you can repeat these steps for other relative data as required.
Step 4: Start Google Trends Scraping
To start your scraping project on ParseHub’s servers, click the green “Get Data” button on the left pane. You can choose to Test, Run or Schedule your scrape. We chose Run, to scrape the trending topics a single time.
Here is what our data export looked like:
If you require assistance with web scraping, contact our live support!
Happy Scraping! 💻