Odd issue using Gemini model - chat enters loop

[iainwallacebms]

Very exciting package, thanks

I was able to install it (via remotes rather than pak) and run it with our internal Azure endpoint

        querychat_config <- querychat_init(mtcars,
                                           create_chat_func = purrr::partial(ellmer::chat_azure, 
                                                                       endpoint = Sys.getenv('endpoint'),
                                                                      api_key = Sys.getenv('key'),
                                                                      api_version = "2024-08-01-preview",
                                                                     deployment_id = "gpt-4o-global")

However when I used a gemini model, the chat just goes in a loop and eventually hits a rate limit erroring out.

         querychat_config <- querychat_init(mtcars,
                                       create_chat_func = purrr::partial(ellmer::chat_gemini, 
                                                                      api_key =Sys.getenv('GEMINI_KEY'),
                                                                     model = 'gemini-2.0-pro-exp-02-05'))

[chendaniely]

Hello. Just realized I referenced the wrong issue when I initially tagged #23.

I think the hanging issue was fixed when we updated querychat to match how shinychat handles namespaces in #27 and #10 can you confirm that things are locking up after you install the latest dev versions?

[orderlyquant]

I'm experiencing the repeated loop issue when using ollama:

querychat_handle <- querychat_init(
  tips,
  # This is the greeting that should initially appear in the sidebar when the app
  # loads.
  greeting = readLines(here("greeting.md"), warn = FALSE),
  # create_chat_func = purrr::partial(ellmer::chat_ollama, model = "gpt-oss:20b")
  create_chat_func = purrr::partial(
    ellmer::chat_ollama,
    model = "qwen3:8b"
  )
)

Doesn't seem to matter what "model" is being used.

[chendaniely]

Hi @orderlyquant : I believe we fixed this issue. But i do want to confirm a few things with you, just in case we missed something.

Uninstall + reinstall querychat package. I think I forgot to do a version bump from 0.0.0.9000 so you may still have an older version, even though the version numbers are the same.

pak::pkg_remove('querychat')
pak::pak("posit-dev/querychat/pkg-r")

Can you send us the info from sessionInfo()? this will let us know what version of packages you are using.

library(querychat)
library(shinychat)

sessionInfo()

Test your ollama model (for me this took a while to get a response back)

chat <- ellmer::chat_ollama(model = "qwen3:0.6b")
chat$chat("Tell me three jokes about statisticians")

the gpt-oss models take a very long time to load up. so that may be an issue (will look into this)

Also, can you see if you run into the same issue with this example app?
Btw the qwen3:0.6b is a very small (and terrible model for this. Just trying to get a small reprex example)

library(shiny)
library(bslib)
library(querychat)

mtcars_source <- querychat_data_source(mtcars)

querychat_config <- querychat_init(
  data_source = mtcars_source,
  create_chat_func = purrr::partial(ellmer::chat_ollama, model = "qwen3:0.6b")
)


ui <- page_sidebar(
  sidebar = querychat_sidebar("chat"),
  DT::DTOutput("dt")
)

server <- function(input, output, session) {

  querychat <- querychat_server("chat", querychat_config)

  output$dt <- DT::renderDT({
    DT::datatable(querychat$df())
  })
}

shinyApp(ui, server)

[orderlyquant]

> library(querychat)
> library(shinychat)
> 
> sessionInfo()
R version 4.5.1 (2025-06-13)
Platform: aarch64-apple-darwin20
Running under: macOS Sequoia 15.5

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.5-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.1

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: America/New_York
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] shinychat_0.2.0.9000 querychat_0.0.0.9000 devtools_2.4.5       usethis_3.1.0       

loaded via a namespace (and not attached):
 [1] vctrs_0.6.5       cli_3.6.5         rlang_1.1.6       purrr_1.1.0       pkgload_1.4.0     promises_1.3.3    coro_1.1.0        S7_0.2.0         
 [9] shiny_1.11.1      xtable_1.8-4      glue_1.8.0        htmltools_0.5.8.1 httpuv_1.6.16     pkgbuild_1.4.8    pak_0.9.0         ellipsis_0.3.2   
[17] fastmap_1.2.0     lifecycle_1.0.4   memoise_2.0.1     compiler_4.5.1    miniUI_0.1.2      sessioninfo_1.2.3 fs_1.6.6          htmlwidgets_1.6.4
[25] Rcpp_1.1.0        urlchecker_1.0.1  rstudioapi_0.17.1 later_1.4.2       digest_0.6.37     R6_2.6.1          magrittr_2.0.3    tools_4.5.1      
[33] mime_0.13         profvis_0.4.0     cachem_1.1.0      remotes_2.5.0    

Jokes with [qwen3:0.6b, qwen3:8b] worked.

Simple app with qwen3:0.6 worked. With qwen3:8b, the app went back into a loop, ending on the 7th loop.

Thanks for your effort.

[chendaniely]

Thank you for confirming!

i'm going to close this issue now since the original bug has been fixed.

The hang up that looks like it's looping seems to be from local models that have a very long load time. I'm not sure how to address this since this part of loading a model for use, but maybe it is something we can put in the docs so people know that's what is happening.
This is just for the initial model load time. Subsequent messages are much faster.

If this is for a demo/test before deployment, i would suggest using Github as the model provider, this way you still get some (rate-limited) access to the latest models.

[orderlyquant]

Should I open a new issue? The chat assistant works now, but after a "random" number of loops, potentially tied to the complexity of the question. qwen3:0.6b loads quickly and produces a response, but the dashboards don't ever update. qwen3:8b eventually stops looping, produces the correct SQL and updates the dashboard. I'm not sure what you mean by "local models that have a very long load time", as the model starts producing a response quickly, but repeats the work...

[chendaniely]

Yes. let's track this in a separate issue. Thank you!