🌱 get urls of site

Source

wget --spider --force-html -r -l1 http://tickettoride.org 2>&1 \
  | grep '^--' | awk '{ print $3 }' \
  | grep -v '\.\(css\|js\|png\|gif\|jpg\|ico\|txt\)$' \
  | sort | uniq > urllist.txt

Download a list of files

wget -r -i list_of_urls

cli