Skip to content

Latest commit

 

History

History

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 
 
 

readme.md

Web Scraping Using Scrapy

Group Members:

Name Matric
Muhammad Imran Hakimi Bin Mohd Shukri A20EC0213
Afif Hazmie Arsyad Bin Agus A20EC0176
Rasmin Kaur Sandhu A19ET0216

Introduction

Scrapy is a Python web scraping framework. It provides a pre-defined set of methods and classes for crawling websites and extracting structured data, such as data for items like products, prices, reviews, etc. Scrapy is built on top of the Twisted asynchronous networking library, which means that it can handle large amounts of data and high concurrency without blocking the execution of the program.

In this assignment, we were ask to do some web scraping with the tool that we assigned on. We will try do do web scraping using scrapy on CompAsia.com. The data that we will be scraping are product's name, price, discount and URL. After that, data will be put in a dataframe and export to a .CSV file. The csv file can be found with the name IphoneProduct.csv file.

In conclusion, using scrapy for web scrapping is very benificial for extracting important information from the web. By doing so, only necessary details can be achieved and imported to an external file.