Error while computing local transformation matrix

[kilbu]

Hello, and thank you for this great package!

I followed your vignette to estimate local GloVe embeddings. This worked so far, but when I try to estimate the transformation matrix with compute_transform(x = toks_fcm, pre_trained = local_glove, weighting = 'log'), I get the error:
.subscript.2ary(x, i, , drop = TRUE) : subscript out of bounds

This does not happen when I do not set weighting = log.

Can someone point me to why this might happen, since weighting = log is the recommended option here.

Best,
k

[ArthurSpirling]

I'm very sorry @kilbu, but I can't replicate the error as of this morning with

• quanteda version 3.3.1
• conText version 2.0.0 (also tried with 1.4.3 from CRAN and it also runs ok)
• RVersion version 4.3.1

Can you tell us what you are using?

[kilbu]

Thank you for trying. My session looks like this:

_> sessionInfo()
R version 4.3.0 (2023-04-21 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)

Matrix products: default

locale:
[1] LC_COLLATE=German_Germany.utf8 LC_CTYPE=German_Germany.utf8 LC_MONETARY=German_Germany.utf8 LC_NUMERIC=C LC_TIME=German_Germany.utf8

time zone: Europe/Berlin
tzcode source: internal

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] ggplot2_3.4.4 lubridate_1.9.3 data.table_1.14.10 text2vec_0.6.4 dplyr_1.1.4 quanteda_3.3.1 conText_1.4.3

loaded via a namespace (and not attached):
[1] fastmatch_1.1-4 gtable_0.3.4 rstatix_0.7.2 lattice_0.21-8 vctrs_0.6.5 tools_4.3.0 generics_0.1.3 parallel_4.3.0
[9] tibble_3.2.1 fansi_1.0.6 pkgconfig_2.0.3 Matrix_1.6-5 RcppParallel_5.1.7 lifecycle_1.0.4 compiler_4.3.0 stringr_1.5.1
[17] munsell_0.5.0 RhpcBLASctl_0.23-42 carData_3.0-5 pillar_1.9.0 car_3.1-2 ggpubr_0.6.0 nloptr_2.0.3 crayon_1.5.2
[25] tidyr_1.3.0 MASS_7.3-58.4 boot_1.3-28.1 abind_1.4-5 nlme_3.1-162 rsparse_0.5.1 stopwords_2.3 tidyselect_1.2.0
[33] digest_0.6.34 stringi_1.8.3 reshape2_1.4.4 purrr_1.0.2 forcats_1.0.0 splines_4.3.0 grid_4.3.0 colorspace_2.1-0
[41] cli_3.6.2 magrittr_2.0.3 fastDummies_1.7.3 utf8_1.2.4 broom_1.0.5 withr_2.5.2 scales_1.3.0 backports_1.4.1
[49] float_0.3-2 timechange_0.2.0 lme4_1.1-35.1 mlapi_0.1.1 ggsignif_0.6.4 pbapply_1.7-2 rlang_1.1.3 Rcpp_1.0.12
[57] glue_1.7.0 rstudioapi_0.15.0 minqa_1.2.6 lgr_0.4.4 R6_2.5.1 plyr_1.8.9_

[ArthurSpirling]

sorry @kilbu, I can't identify an obvious problem.

@prodriguezsosa @bstewart - any ideas? Looks like a problem with the Matrix package, as it interacts with conText, right?

[prodriguezsosa]

@kilbu mind downloading the latest version from GitHub, see if the error persists?

You can use: devtools::install_github("prodriguezsosa/conText")

[yucluo]

Hi! Having the exact same issue here and I have downloaded the latest version from Github. Similarly if I don't set the weighting to log, there is no error. I was using FastText's Chinese pretrained embedding and my own fcm

[ArthurSpirling]

Not sure, @yucluo -- @ElisaWirsching may know more for non-English embeddings.

[ElisaWirsching]

Unfortunately, I cannot replicate the error when training transformation matrices locally for Italian or French (see appendix here or example here).

I use the latest conText version from Git and:
toks_fcm <- fcm(toks_nostop, context = "window", window = 5, count = "frequency")
locallocalA <- compute_transform(x = toks_fcm, pre_trained = local_embeddings, weighting = 'log')