Skip to content
This repository was archived by the owner on Nov 13, 2024. It is now read-only.
This repository was archived by the owner on Nov 13, 2024. It is now read-only.

elasticsearch bulk indexing (via invoke) sometimes loops over and over due to re-indexing #929

@naknomum

Description

@naknomum

manual indexing sometimes becomes stuck in loop and indexing fails

  • sometimes happens (especially on large databases) when invoke app.elasticsearch.now --model CLASS is run
  • unsure if problem also exists for other bulk indexing (e.g. via celery)
  • manifests as progress-bar being interrupted and Serializing (Bulk) Individual [retry=N]: X% message appearing, and progress bar starts over -- usually with a much smaller set of object
  • this behavior continues usually indefinitely and has to be interrupted
  • when this happens, all indexing progress is usually not persisted to ES and therefore lost
  • interrupting and restarting sometimes just works and indexing is done properly
  • is a pain on larger production codex when new deployments happen -- makes re-indexing a hassle

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions