Uso de web scraping para mineração de produtos e preços em e-commerce

Abstract

The development of information and communication technologies has led to the emergence of collaborative systems, such as crowdsourcing, which allows the mutual sharing of data, as diverse as possible, generated by smartphones of a community of users. One of these most popular systems is Waze which allows you to identify congestion-free transit routes using geolocation information from mobile devices. LudiiPrice is a crowdsourcing application that is being developed at UFS under the coordination of the supervisors of this monograph and consists of a price finder for personal consumables and groceries obtained by QR code photographed by smartphones. Invoice data is stored in a database that is used to respond to user queries. The objective of this work is to automatically populate this database using a web crawler to extract price data from products published in e-commerce portals. The data obtained is processed using Natural Language Processing techniques and aims to suggest each item entered through the invoice. The crawler and the suggestions are requested through an API, which was built using the Django framework that uses Python as its language. As a result of the work, crawlers are performed in three e-commerce that populate a base and suggest items extracted from crawler for each product present in the invoice inserted in the LudiiPrice application.