Skip to content

Add functionality to ignore certain url paths or files #86

@XThomasBU

Description

@XThomasBU

All the child urls are read, processed and stored in the vectorestore, build functionality to ignore certain url paths or files, for example: defined in storage/data/to_ignore.txt

https://github.com/DL4DS/dl4ds_tutor/blob/e934b90b1d5e9771ed279e085200b30c83c29d60/code/modules/vectorstore/store_manager.py#L49

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions