#include # entries are one url per line. comment lines beginning with '#' are allowed # urls are passed to urllib.urlopen, so the format must be supported there http://www.ubuntu.com/robots.txt http://www.w3schools.com/html/lastpage.htm
distrib
>
Fedora
>
18
>
x86_64
>
media
>
updates
>
by-pkgid
>
e81917d6bf142e2e723f45d0988c88c4
>
files
>
355
