Using Web Scraping to Compare Sneaker Prices

While I was in university, I was introduced to sneaker collecting by one of my floormates in the dorm I was living in. I wouldn’t call myself a “sneakerhead”, but I definitely love rocking a fresh pair of kicks. But one of the worst things about sneaker collecting (as a broke college student) are the prices. $750 for a pair of Levi’s 4s? No thanks. I started comparing sneaker prices from popular online sneaker retailers because of this large barrier to entry. I tried doing this by maintaining a spreadsheet by hand, but it was way too time consuming. A web scraper, such as ParseHub, made this process of comparing sneaker prices much easier. A couple of simple projects and some excel manipulation makes finding the best price for a particular colorway of sneaker is a breeze. I’ll show you how to do it using 2 of the most popular online sneaker retailers.

Website 1: FlightClub.com

Flight Club was the first website I was introduced to. It’s a good source for both deadstock and used sneakers. For this example, I’m using the Jordan 3s, since it’s my favorite Jordan sneaker.

The Flight Club website has a navigation bar, which makes it easy to see deals for the sneakers that you are interested in.

flightclub1

I started out by selecting all of the elements in this bar and nesting a condition under this selection to look for the Jordan category. I did this so that you can easily change this command to look for any category that you are interested in. After setting it to hover over this element and waiting for the menu to appear, I set another condition to look for the Jordan 3 link and click on it once found.

flightclub2

The landing page for the listings has a starting price under each sneaker, but I wanted to find more details. So, I set ParseHub to click on each listing page, and then set it to go to the next page once it’s done scraping the listings from this first page. This webpage has a problem with an infinite loop, so I set a condition looking for an “inactive” class on the next button. If the button is inactive, ParseHub will not click on the next button.

flightclub3

Once on the page of the actual listing, I set ParseHub to select and extract the colorway of the sneaker, and then trained it onto the sizes section.

flightclub4

I set it to click on my size using a condition (10.5) and the scrape the resulting price of a new pair. I followed the same process for a used pair as well.

flightclub5

Website 2: StadiumGoods.com

I find that Stadium Goods has a wider selection of merchandise, so it’s usually my go-to website for looking up sneaker prices. Again, I’m just going to search for Jordan 3s, but you can find pretty much any sneaker you want on their website.

The majority of my set up for this project was the same as my Flight Club project, but there were a few differences.

The listing pages have both colorway and nickname fields on them. Since I know most of these colorways by their nicknames, I set ParseHub to extract both the nicknames and colorways (just in case a colorway didn’t have a nickname).

stadiumgoods1

In addition, the sizes on Stadium Goods are in a dropdown menu. I set ParseHub to select and click on the dropdown menu, then did the same search for my size that I did on Flight Club.

stadiumgoods2

Now that all of the data is gathered, it’s time for some Excel. First, I transferred both data sets to a clean Excel spreadsheet. The colorways from Flight Club were a little messy, so I did some simple find + replace to isolate the colorways.

excel1

I then set up a simple formula to look for certain colorway (Katrina, in this example). If found, it will return the price of the shoe.

excel2

After that, it’s just a matter of looking up the listing page that you are interested in. You can also set ParseHub to extract the URLs of the listing pages to make finding them even easier! This project should be easy to recreate using this article, but if you have any questions about your own project, feel free to contact us at hello@parsehub.com