GitHub · Where software is built

Labels Milestones New issue

Error raised when there's a valid <base> tag in <head>

#61

· svenaas opened

on Nov 5, 2020

Feature request: Stop crawl at time

#54

· samnissen opened

on Apr 19, 2017

Falling into Crawl Traps

#53

· fuzzygroup opened

on Apr 8, 2017

undefined method `banner' for main:Object (NoMethodError) on calling from command line

#52

· gushonorato opened

on Jul 21, 2016

Should depend on slop 3.6.0 or update cobweb bin

#47

· mhmaguire opened

on Jan 7, 2016

How can I start stop crawling website

#46

· qiun opened

on Jan 4, 2016

Redirect Limit causing crawl to stop

#36

· stewartmckee opened

on Jan 23, 2015

Code organization

#34

· andrejj opened

on Nov 18, 2014

Should it be possible to add "depth" in the data hash ?

#28

· ABrisset opened

on May 13, 2014

Should be possible to use Ruby's URI implementation instead of Addressable::URI

#27

· gh2k opened

on May 13, 2014

Improve connection handling

#24

· stewartmckee opened

on Nov 22, 2013

external_urls not treated as external

#16

· stewartmckee opened

on Feb 18, 2013