download from http://polyglot.cs.stonybrook.edu/~polyglot is not working

[zhongzhenzhen]

from polyglot.downloader import downloader
downloader.download("morph2.en")

[polyglot_data] Error loading morph2.en: HTTP Error 404: Not Found

[DiegoPino]

Same here. The Downloader() collections() method is also failing with a 404

[idasilva]

Hi there, i am having the same problem here but when using Dockerfile:

RUN polyglot download sentiment2.en
RUN polyglot download embeddings2.en
RUN polyglot download pos2.en

[polyglot_data] Error loading sentiment2.en: HTTP Error 404: Not Found

[MuhAssar]

is there any mirror?

[julesUlrich]

has anyone found a solution yet? I've run into the same problem

[MuhAssar]

we replaced polyglot with stanza

[lokesh12345678910]

Same error here, both on colab and locally

[pulciblight]

it seems that it's still not working((

[hexindent]

I've contacted the web admin at cs.stonybrook.edu and they are not hosting the files anymore. One of the leading professors said the project is quite old and his students are long gone. It's unlikely they can share the model files on a third-party platform like GitHub.

Either we can "crowd source" the files, coping and sharing from others who have downloaded before, OR we switch to other projects like suggested previously.

[ndandanov]

Hi,
I uploaded the polyglot models I have cached (mainly English and Bulgarian) to this HuggingFace dataset repo:
https://huggingface.co/datasets/ndandanov/polyglot-models

To make the polyglot downloader work with this repo, you shall set the URL which points to the files from this repo: https://huggingface.co/datasets/ndandanov/polyglot-models/resolve/main

For example, to download all resources for a language, you can use the CLI interface:

polyglot download LANG:en --url https://huggingface.co/datasets/ndandanov/polyglot-models/resolve/main

To download only a specific model, you can specify it:

polyglot download embeddings2.en --url https://huggingface.co/datasets/ndandanov/polyglot-models/resolve/main

In case you have some polyglot models, feel free to open a PR to the HF repo and I will include them.

[DiegoPino]

Hi @ndandanov that is great!

I have local cached copies of German, English, Hindi, Italian and Portuguese.

polyglot_data/counts2-de-en-es-hi-it-pt.tar.gz
polyglot_data/embeddings2-de-en-es-hi-it-pt.tar.gz
polyglot_data/morph2-de-en-es-hi-it-pt.tar.gz
polyglot_data/ner2-de-en-es-hi-it-pt.tar.gz
polyglot_data/pos2-de-en-es-hi-it-pt.tar.gz
polyglot_data/sentiment2-de-en-es-hi-it-pt.tar.gz
polyglot_data/sgns2-de-en-es-hi-it-pt.tar.gz
polyglot_data/transliteration2-de-en-es-hi-it-pt.tar.gz
polyglot_data/tsne2-de-en-es-hi-it-pt.tar.gz
polyglot_data/uniemb-de-en-es-hi-it-pt.tar.gz
polyglot_data/unipos-de-en-es-hi-it-pt.tar.gz

Would you like those there too? And if so, just complete tars? Does huggingface accept some type of git/pull against your repo? Let me know

[ndandanov]

Hi @DiegoPino,
Thank you, your contribution is welcome!

One way to contribute is to directly upload the files:

PRs shall also be readily available as described here:
https://huggingface.co/docs/hub/repositories-pull-requests-discussions

This may be also of help:
https://github.com/huggingface/datasets/blob/main/CONTRIBUTING.md#how-to-create-a-pull-request

Complete tar archives would be best, as they were originally downloaded by polyglot.

I checked now and in the repo there are already the following directories for German:

• ./pos2/de
• ./embeddings2/de

And for Portuguese:

• ./pos2/pt
• ./embeddings2/pt

Feel free to override them if yours are newer.

The files you mention suggest also availability of Spanish, which is great!

[DiegoPino]

@ndandanov cool. Will try to find time during the weekend. Thanks!