init_window/3 hangs and doesn't open a window on MacOS

[japhib]

Hi,

Awesome project! Would love to make some games using Elixir.

However, when I try to run the example (or even just run Rayex.Core.init_window/3 in iex) it hangs indefinitely and doesn't open a window or do anything else as far as I can tell. By using a local version of raylib and adding a bunch of log statements, I'm able to pin down the problem to this line in GLFW:

    if (![[NSRunningApplication currentApplication] isFinishedLaunching])
        [NSApp run];

https://github.com/raysan5/raylib/blob/master/src/external/glfw/src/cocoa_init.m#L555

(Note this is on MacOS, hence the objective-C.) It's the [NSApp run] call that seems to just hang forever. I tried removing that line but I get a completely different error that makes it seem like that line probably shouldn't be removed. And I'm guessing that there's a better solution than fiddling around with the GLFW sources, since it wouldn't be ideal to have to use a fork of GLFW just for this one project.

Any ideas what could be going wrong here? Has anyone else been able to get this running on mac?

System details:

macOS Big Sur 11.6.1
elixir 1.12
erlang 24.1.7

Note -- the problem isn't Raylib or GLFW, since I'm able to compile and run the basic Raylib example with C.

[shiryel]

Hello,
Glad that you liked the project!

About your problem, I don't have a MacOS to test it myself, but I can help trying to find the problem.

First you need to make sure that you are running the project with the Raylib 4.0.0, you can find all dependencies on the shell.nix file, its original intention is to make the project easy to setup and reproductive with the Nix package manager, if you want you can even install it on your MacOS, but its not necessary.

When you start the project, Unifex and Bundlex will generate and compile the NIFs, you can verify that looking at c_src/rayex/nif/, it should have rayex.c, rayex.cpp and rayex.h generated files and at _build/dev/lib/rayex/priv/bundlex/nif/, it should have the rayex.so compiled file.

After starting the project, with iex -S mix you can run Rayex.Core.init_window 200, 200, "window name", it should log something like:

INFO: Initializing raylib 4.0
INFO: DISPLAY: Device initialized successfully
INFO:     > Display size: 4920 x 2560
INFO:     > Screen size:  200 x 200
INFO:     > Render size:  200 x 200
INFO:     > Viewport offsets: 0, 0
INFO: GLAD: OpenGL extensions loaded successfully
INFO: GL: Supported extensions count: 228
INFO: GL: OpenGL device information:
INFO:     > Vendor:   AMD
INFO:     > Renderer: Radeon RX 590 Series (POLARIS10, DRM 3.40.0, 5.10.79, LLVM 11.1.0)
INFO:     > Version:  4.6 (Core Profile) Mesa 21.1.4
INFO:     > GLSL:     4.60
INFO: GL: DXT compressed textures supported
INFO: GL: ETC2/EAC compressed textures supported
INFO: TEXTURE: [ID 1] Texture loaded successfully (1x1 | R8G8B8A8 | 1 mipmaps)
INFO: TEXTURE: [ID 1] Default texture loaded successfully
INFO: SHADER: [ID 1] Vertex shader compiled successfully
INFO: SHADER: [ID 2] Fragment shader compiled successfully
INFO: SHADER: [ID 3] Program shader loaded successfully
INFO: SHADER: [ID 3] Default shader loaded successfully
INFO: RLGL: Render batch vertex buffers loaded successfully in RAM (CPU)
INFO: RLGL: Render batch vertex buffers loaded successfully in VRAM (GPU)
INFO: RLGL: Default OpenGL state initialized successfully
INFO: TEXTURE: [ID 2] Texture loaded successfully (128x128 | GRAY_ALPHA | 1 mipmaps)
INFO: FONT: Default font loaded successfully (224 glyphs)
:ok

I know that this last bit is not working for you right now, then I recommend taking a look at the generated and compiled files, maybe the problem is somewhere there.

Let me know if you see any strange log while compiling or running the project, I'll try my best to help

[japhib]

Yeah, it does generate rayex.c, rayex.cpp and rayex.h as expected. _build/dev/lib/rayex/priv/bundlex/nif/rayex.so is present as well.

All I see is the first log, INFO: Initializing raylib 4.0. Since that log comes from raylib, I believe that it's loading and calling the NIF just fine. It's just that it hangs indefinitely after that.

[shiryel]

That is very strange, it should work if the raylib examples are working, as its only calling the raylib code...

I searched a bit to see if other people had a similar issue with raylib and yes, seems like a GLFW problem, maybe correlated with the build flags, here: raysan5/raylib#1817 and here: glfw/glfw#1915

I wonder if the Bundlex is getting the right versions of the packages on your system, its configured on bundlex.exs, it uses the pkg-config --libs raylib command to link the lib to rayex

Also, as a test (that will probably don't work), try to use the same values as the examples on raylib, like init_window 800, 450, "raylib [core] example - basic window"

[japhib]

I'm able to reproduce the problem with just regular Erlang and GLFW, so I opened an issue with GLFW ^

[japhib]

Interesting, when I make the same test initializing the window with SDL instead of GLFW, I get this error:

Error creating window: NSWindow drag regions should only be invalidated on the Main Thread!

How do I make it so code runs on the main thread? That could be the issue with GLFW as well. I've tried running erlang single-threaded by erl +S 1, but that gives me the same issue ...

[shiryel]

Interesting indeed, I asked a friend to try to reproduce your problem on their MacOS M1 and indeed, seems reproductive, with almost the same error (just slight different), and about your question, I don't know if you can run it as a main thread using NIFs, but you can try to run using a CNode, like this:

iex> require Unifex.CNode
Unifex.CNode

iex> {:ok, cnode} = Unifex.CNode.start_link(:rayex)  
18:47:50.646 [info]  Protocol 'inet_tcp': register/listen error: econnrefused
18:47:50.655 [info]  Trying to start epmd...
{:ok,
 %Unifex.CNode{
   bundlex_cnode: %Bundlex.CNode{
     node: :"bundlex_cnode_0_468f47b0-eb73-4ded-895c-32140772ba67@shiryel",
     server: #PID<0.291.0>
   },
   node: :"bundlex_cnode_0_468f47b0-eb73-4ded-895c-32140772ba67@shiryel",
   server: #PID<0.291.0>
 }}

iex> Unifex.CNode.call(cnode, :init_window, [850, 400, "name"])
:ok

It would probably work with erlang ports, but unfortunately Unifex does not support that, and probably would be a complete new project to do so...

Let me know if with CNode it works, if so, I can focus on implementing an easier way to call it on rayex :)

[japhib]

Yeah, that works! I was able to run the most basic example with code like this:

defmodule OpenWindowExample do
  import Unifex.CNode
  require Unifex.CNode

  @color_white %{r: 245, g: 245, b: 245, a: 255}
  @color_gray %{r: 130, g: 130, b: 130, a: 255}

  def run() do
    {:ok, cnode} = Unifex.CNode.start_link(:rayex)

    call(cnode, :init_window, [850, 400, "name"])
    call(cnode, :set_target_fps, [60])

    main_loop(cnode)

    call(cnode, :close_window, [])
  end

  defp main_loop(cnode) do
    call(cnode, :begin_drawing, [])
    call(cnode, :clear_background, [@color_white])
    call(cnode, :draw_text, ["First window", 190, 200, 20, @color_gray])
    call(cnode, :end_drawing, [])

    unless call(cnode, :window_should_close, []) do
      main_loop(cnode)
    end
  end
end

OpenWindowExample.run()

An easier way to make cnode function calls would be awesome :)

I may continue looking into a solution using erlang ports though, mainly since that seems like it might have slightly better performance than cnode.

[shiryel]

Nice!

I'll see what I can do to make CNode's code easier, I want something like an option that can change the same code-base between NIF and CNode, this would be the best solution, and would provide the benefices of both worlds :)

Also, I'm not sure, but I think NIF and CNode are faster than ports, would be a good benchmark to do

[harrisi]

Hey, just wanted to share some insight from getting Metal rendering working in Elixir.

Cocoa (or AppKit? I don't know) requires some things to happen in the special "main" thread. I think the initial window creation has to be done on the main thread. I'm not sure how much else needs to be done on the main thread, as I've seen conflicting information. Messing with the view layer is one of them, apparently, as well.

Erlang/OTP handles this with an undocumented function, erl_drv_steal_main_thread, to ensure wxWidgets runs on the main thread on macOS. It's available for use, but I imagine there's no guarantee it will work ever for any particular version for any reason. Watch out for dragons :)

How I fixed my issue was to use dispatch_sync when I make the Metal layer and bind it to the window handle from wx. I'm not sure how you would do something like that with Unifex. Maybe with load and unload functions or something? I don't know.

You can see how esdl2 does it (via nif_helpers) here.

Hope that helps!

[shiryel]

Thanks for letting me know @harrisi, it will for sure help!

I have some ideas of how I could try to integrate it on Unifex, the most promising one would be by using this callback.

I will put some work in the next days to see if I can get it working, thanks.