Subject: [SWIPL] Web-whacking feasibility

From: Paul Singleton <[email-masked]>
Date: Thu, 02 Aug 2001 11:21:06 +0100

I need to build a "web whacker" to take a snapshot of a (finite) dynamic
web site I'm working on.  Are most of the parts available to build this
in SWI-Prolog?  I need to make HTTP GET calls, parse the returned HTML,
extract all the linked-to URLs, identify those within the same site,
avoid going around cycles, save the pages as HTML files with all local
links replaced by 'file:' URLs.

(I also seek reassurance that I haven't seriously underestimated the
complexity of the problem :-)

Paul Singleton