Commit 30bdc98db1e597e29e367185d8a843556bd037de

  • avatar
  • arvind
  • Fri Mar 28 22:31:23 IST 2014
First commit.  The crawler, crawls a page specified and gets allowed
pages from that domain.  Further there is parsing to extract specific
portion of the page, using XPath selectors.  The parsed content is then
posted to a social networking site.

Run `$python setup.py install` to get the required dependencies.
Start a crawl by `$scrapy crawl swara`.

This is the initial commit in this repository, browse the initial tree state.