adamdehaven/fetchurls

A bash script to spider a site, follow links, and fetch urls (with built-in filtering) into a generated text file.

语言：Shell · GitHub 星标：130 · Fork：43

话题：bash-scripting · crawl · shell-script · spider · urls · website · wget

在 GitHub 上查看原仓库