Project duration:
2019 - 2021
webinsight
optimised web crawling with page-freshness metrics
Project summary:
The project WebInsight will deliver high-value analysis of World-Wide-Web (WWW) content by crawling the entire web in a way that enforces high freshness of any page, extracting and analyzing the updated data, and proposing real-time automatic webservices (via a SaaS platform) that will be usable by decision makers.
The key feature is to be able to compute metrics on webpages related to their position in the Web graph, and to use this, in addition with a semantic analysis, to predict the likelihood of them having changed. Combining dynamic computation of these metrics with machine learning and importance weighting will allow us to provide a fresh vision of the web at a minimal cost.
Project Leader:
Technical Assistance:
Funding:
Eurostars-2 joint programme with co-funding from the European Union Horizon 2020 research and innovation programme
