Building Automated Geotargeted Web Scraping Tool With a Proxy Scraper and IP GeoLocation API - Part 2
Location-based web scraping is getting more and more popular. There are many reasons why businesses and developers prefer location-based web scraping. The first of these is to increase the user experience. Providing users with geotargeted data automatically with a web scraping tool has recently been the easiest way to improve user experience and analyze business processes with location-based data. There are many web scraping tools available today that offer location-based web scraping.
Developing geotargeted applications with web scraping tools has become quite easy nowadays. With the IP geolocation API, it is just a few steps to obtain the location data of the visitors, extract the data at this location with a web scraping tool, and parse this data with almost any programming language. In this article, we will continue our development processes in Part 1. But before that, let's take a look at how to do HTML data parsing with JavaScript and why we chose the Zenscrape API for web scraping.
How to Parse HTML Data in JavaScript?
There are several methods for extracting and analyzing HTML data using JavaScript. To be mentioned firstly, data scraping is often used to extract data from websites. This process involves pulling content called web data. This is data in HTML format. This data can be found on more than one page, so it may be necessary to navigate and pull the data on multiple pages.
To parse HTML data, JavaScript's Document Object Model (DOM) API is often preferred. This API represents web pages as a tree-like structure and provides the ability to navigate and modify this structure. For example, it is possible to easily access certain HTML elements using the querySelector or querySelectorAll methods. In this way, developers can find and extract the data they want in scraped data HTML.
Why Did We Choose Zenscrape API as a Scraping Tool for Data Extraction?
Zenscrape API is a powerful web scraper API that is booming in the market and has more than 10,000 happy customers. It has many features that stand out from other best web scraping tools on the market.
Learn what is web scraping in Python!
One of the primary reasons we prefer the Zenscrape API in this application is that it has a unique proxy pool. Its pool has millions of IP addresses. Due to its proxy pool, it provides fast real-time web data extraction from many websites, including those that are difficult to scrape today. One of the biggest benefits of the Zenscrape API, thanks to the proxy, is that it provides location-based web scraping. Thanks to this feature, we can access data from the official Nike website at the user's location in our application.
The Zenscrape API also provides automatic proxy rotation. This feature is one of the most important web scraping techniques to implement in a web scraping software. Zenscrape API provides this service automatically, while many web scraper tools today charge extra configuration or fees for this feature.
These and many more reasons keep the Zenscrape API one step ahead of its competitors. Also, the fact that we can easily integrate it into any programming language has become one of the reasons why we prefer the Zenscrape API for this application.
Visualize Extracted Data in the Web Application
In this step, we will talk about the visualization part of the application we developed in Part 1. To summarize briefly, there is an 'Extract Data for My Location' button in the application we have developed up to this stage. When the user presses this button after entering the application, we detect the location of the user with the ipstack API, then extract the official Nike website in the user's location and press the console screen of the application.
Discover the best practices for using location services in your web app!
In this step, we will print the products in the 'Popular Right Now' field from the data we have extracted from the Nike website, on the screen of the application. For this, let's put the following codes in the file where our application is located:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Nike Products</title>
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css">
<style>
.container {
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
height: 100vh;
}
.hidden {
display: none;
}
.product-card {
border: 1px solid #ccc;
padding: 10px;
margin: 10px;
text-align: center;
width: 250px;
}
.product-list {
display: flex;
}
.product-image {
max-width: 100%;
height: auto;
}
</style>
</head>
<body>
<div class="container">
<button class="btn btn-primary btn-lg" id="extractDataButton">Extract Data for My Location</button>
<div class="product-list" id="product-list">
</div>
<div class="d-flex justify-content-center mt-3" id="pagination">
<button class="btn btn-secondary hidden" id="prevButton" disabled>Previous</button>
<button class="btn btn-secondary ml-2 hidden" id="nextButton">Next</button>
</div>
</div>
<script>
const extractDataButton = document.getElementById('extractDataButton');
const productList = document.getElementById('product-list');
const prevButton = document.getElementById('prevButton');
const nextButton = document.getElementById('nextButton');
const pagination = document.getElementById('pagination');
let currentPage = 0;
let productsPerPage = 3;
let productData = [];
extractDataButton.addEventListener('click', async () => {
fetch('https://api.ipstack.com/check?access_key=YOUR-API-KEY')
.then(response => response.json())
.then(data => {
const countryCode = data.country_code;
const apiUrl = `https://app.zenscrape.com/api/v1/get?url=https://www.nike.com/&apikey=YOUR-API-KEY&premium=true&country=${countryCode}`;
fetch(apiUrl)
.then(response => response.text())
.then(htmlContent => {
const parser = new DOMParser();
const doc = parser.parseFromString(htmlContent, 'text/html');
extractDataButton.classList.add('hidden');
prevButton.classList.remove('hidden');
nextButton.classList.remove('hidden');
const productSlidesName = doc.querySelectorAll('.body-2._1S3--jXk');
const productSlidesSubTitle = doc.querySelectorAll('.body-2.text-color-secondary._2LjqmQB3')
const productSlidesImage = doc.querySelectorAll('._363pyMxS._1J1k7Sv3.media-container.css-znecc2')
productData = [];
for (let i = 0; i < productSlidesName.length; i++) {
const productName = productSlidesName[i].textContent;
const productSubTitle = productSlidesSubTitle[i].textContent;
const productImageDiv = productSlidesImage[i].querySelector('._3jm9Bm_E');
const productImageURL = productImageDiv.getAttribute('data-landscape-url');
productData.push({
name: productName,
subTitle: productSubTitle,
imageURL: productImageURL
});
}
renderProductCards();
})
.catch(error => {
console.error('Hata:', error);
});
});
});
function renderProductCards() {
productList.innerHTML = "";
const startIndex = currentPage * productsPerPage;
const endIndex = startIndex + productsPerPage;
for (let i = startIndex; i < endIndex && i < productData.length; i++) {
const productCard = document.createElement('div');
productCard.className = "product-card";
const productImage = document.createElement('img');
productImage.className = "product-image";
productImage.src = productData[i].imageURL;
productImage.alt = productData[i].name;
const productNameElem = document.createElement('h4');
productNameElem.textContent = productData[i].name;
const productSubtitle = document.createElement('p');
productSubtitle.textContent = productData[i].subTitle;
productCard.appendChild(productImage);
productCard.appendChild(productNameElem);
productCard.appendChild(productSubtitle);
productList.appendChild(productCard);
}
if (currentPage === 0) {
prevButton.disabled = true;
} else {
prevButton.disabled = false
}
if (endIndex >= productData.length) {
nextButton.disabled = true;
} else {
nextButton.disabled = false;
}
}
prevButton.addEventListener('click', () => {
if (currentPage > 0) {
currentPage--;
renderProductCards();
}
});
nextButton.addEventListener('click', () => {
if (currentPage < Math.ceil(productData.length / productsPerPage) - 1) {
currentPage++;
renderProductCards();
}
});
</script>
</body>
</html>
With the piece of JavaScript code, we added to the script part, we obtain the product name, subtitle, and image in the 'Popular Right Now' field from the data we obtained from the Nike website and insert it into a card. We use the 'DOMParser' in JavaScript to do this. Before running the application, let's put our own API keys in the 'YOUR-API-KEY' fields and run the application.
After running the application, the following button on the screen will welcome us:
When we click the 'Extract Data for My Location' button, we will see the data we obtained from the official Nike website in our location. Let's press the button for that.
The product card and products we will see after pressing the button are as follows:
We easily visualized the 'Popular Right Now' field from the data we obtained from the Nike website. When we click the 'Next' button, we display other popular products:
Conclusion
In summary, combining an IP geolocation API and a web scraping API offers many advantages and benefits to businesses and developers. In particular, APIs such as ipstack API and Zenscrape API, which provide the best experience in the services they provide, provide great convenience to their users with the comprehensive data they provide. They even allow them to create new and unique use cases. In this article series, we presented geotargeted content to users as location-based using the ipstack API and the Zenscrape API. It took only a few steps to parse and visualize the location-based data we scraped with the Zenscrape API in JavaScript.
Sign up for the web scraping API that offers location-based scraping, and scrapes data from every targeted website in real-time.
FAQs
Q: What are the Some Popular Web Scraping Tools?
A: There are many web scraping tools in the market nowadays. Some of the popular ones are as follows:
Zenscrape API
Zenserp API
The scrapestack API
ScraperAPI
ScrapingBee API
Q: What is the Main Advantage of Using IP Geolocation API and Web Scraping API Together?
A: The main advantage of using IP geolocation API and web scraping API together is the ability to deliver geotargeted content to users.
Q: Is Zenscrape a Free Web Scraping Tool?
A: Yes, it is. Zenscrape API offers free plan users a usage limit of up to 1,000 API requests per month.
Q: Does the Zenscrape API Integrate with All Programming Languages?
A: Zenscrape API integrates easily into all programming languages. In addition, the documentation provided by Zenscrape includes sample integration codes for programming languages such as Python and PHP.