Unable to crawl /admin URLs

[mpasternak]

Hi,

I run django-crawler as:

python manage.py crawl --auth=login:sysdba,password:XXX /admin/bpp

The password should be good, I double-checked it.

The results I get:

crawler [INFO] base: Log in with login: sysdba, password: ********************
crawler [WARNING] base: /admin/bpp links to /static/admin/css/base.css, which returned HTTP status 404
crawler [WARNING] base: /admin/bpp links to /static/admin/css/login.css, which returned HTTP status 404
crawler [WARNING] base: /admin/bpp links to /static/admin_tools/css/theming.css, which returned HTTP status 404
crawler [INFO] time_plugin: http://testserver/admin/bpp/ took 0.156000
crawler [INFO] time_plugin: /static/admin/css/base.css took 0.016000
crawler [INFO] time_plugin: /static/admin_tools/css/theming.css took 0.015000
crawler [INFO] time_plugin: /static/admin/css/login.css took 0.000000
crawler [INFO] time_plugin: /admin/bpp took 0.000000
make: *** [crawl] Error 1

My INSTALLED_APPS looks like this:

INSTALLED_APPS = (
    'south',
    'admin_tools',
    'admin_tools.theming',
    'admin_tools.menu',
    'admin_tools.dashboard',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django.contrib.admin',
    'django_jenkins',
    'bpp',
    'debug_toolbar',
    'werkzeug_debugger_runserver',
    'crawler'
)

[chrisv2]

bug in documentation, correct usage of auth switch would be

--auth username:foo,password:bar

The parameters are directly passed to testclient.login, see https://github.com/ericholscher/django-crawler/blob/master/crawler/base.py#L82