can not run crawler within login and captcha part

[davisbra]

hi ...
I added the commit that you'd added about login and captcha detection in https://github.com/GoSecure/freshonions-torscraper/commit/def5f96cfc2dfd8a58e5a8a8204292010180dcc1
Now, when I am pushing a domain, I face with this:

Do you know any idea what is my fault!?

[obilodeau]

That work is currently a work-in-progress. You'll have to debug it yourself.

CaptchgaMiddleware's process_response seems to return a bool instead of a Response or Request object.

[davisbra]

@obilodeau Hi, did you finished the captcha solving? Do you have any idea about adding the automatic login and captcha resolving part to freshonion? I mean,is it possible that we register manually in the sites, which require registration. Then we pass the user, password, captcha to the script that it can login and crawl the site?

[obilodeau]

did you finished the captcha solving?

No, and we are not working on it at the moment.

Do you have any idea about adding the automatic login and captcha resolving part to freshonion?

Not really but I'm sure it's doable.

I mean,is it possible that we register manually in the sites, which require registration. Then we pass the user, password, captcha to the script that it can login and crawl the site?

I would go in that direction. User / password associated with a domain in a configuration file or something similar and captcha solved by the code or a captcha solving service.

We are aware of paid-for services that do that already. So at this point we are not sure if we should purchase their service or continue spending development time on this project. Right now nothing is planned regarding further development on freshonion.