Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to change dlopen flag #14

Closed
olehlong opened this issue Apr 10, 2016 · 15 comments
Closed

Ability to change dlopen flag #14

olehlong opened this issue Apr 10, 2016 · 15 comments

Comments

@olehlong
Copy link

Is it possible to add an ability to change dlopen flag? Currently, it's always RTLD_NOW, but I need to change it. I think it can be done through SharedLibLoader constructor or just a parameter.
I can do it, but maybe @aldacron has a better idea.

@mdparker
Copy link
Member

I don't like the idea of adding anything platform-specific to the API. What's your use-case for this?

@olehlong
Copy link
Author

I'm trying to make dynamic bindings for a libraries that bgfx uses (nanovg,imgui). They use bgfx functions to render, so they need to have an access to bgfx symbols. With RTLD_NOW they don't see bgfx functions, that was loaded before. So i would need to use RTLD_LAZY or do static linking.

@mdparker
Copy link
Member

Actually, it appears what you want here is for RTLD_GLOBAL to be passed along. I don't see how RTLD_LAZY can help. That's going to completely change the nature of how DerelictUtil operates. Before I go and make such a change, I need to make sure that it isn't going to break anything.

My knowledge of shared libraries on Posix systems is rather shallow. So I've a few questions.

Why this is a problem in the first place? Both GLFW and SDL, for example, use OpenGL internally, yet there are no issues that arise when using those libraries in conjunction with DerelictGL3. What is it about the libraries you are binding that causes this issue?

In all the years I've maintained Derelict (12 and counting!) this issue has never arisen. Why now with these libraries in particular?

Does using RTLD_GLOBAL actually solve the problem? And if so, what are the consequences elsewhere?

Until these questions are answered such that I have a much clearer understanding, there's no way I'm going to make such a change in a fundamental part of Derelict.

@olehlong
Copy link
Author

With SDL - you need to link GL, GLU, X11 (through linker) or else it will say "undefined symbol: glXGetFBConfigAttrib" for example. So, I can also link bgfx library and it will work fine. It's another solution, thanks for pointing out.

But does it mean that library (GL for example) loads two times? Once because you linked it and second time with dlopen.

I think it can be useful and can be done without breaking other bindings, if RTLD_NOW will be set by default. Like in this example http://www.codeproject.com/Articles/13501/Platform-Independent-Coding-DLLs-and-SOs

@mdparker
Copy link
Member

With SDL - you need to link GL, GLU, X11 (through linker)

You need to link X11, anyway, but my understanding is that it loads OpenGL dynamically just as Derelict does.

But does it mean that library (GL for example) loads two times? Once because you linked it and second time with dlopen.

No. A shared library is only loaded into the process space once, no matter if it is linked dynamically at compile time or how many times dlopen is called. Derelict loads its own pointers to the symbols via dlsym, but the library itself is not loaded more than once.

I think it can be useful and can be done without breaking other bindings, if RTLD_NOW will be set by default.

Well, RTLD_NOW is set by default. I suppose you mean if I allow the caller to change the mode? I don't think so. Besides, RTLD_NOW and. RTLD_LAZY have no bearing on the issue you were referring to earlier. It's loading symbols locally (the default) vs. globally (RTLD_GLOBAL) that impact that situation. The latter flag can be logically OR'ed with RTLD_NOW and RTLD_LAZY, but I don't see that Derelict needs to expose that.

@olehlong
Copy link
Author

Yes, change the mode.

RTLD_LAZY have no bearing on the issue you were referring to earlier

Well, I changed the mode to RTLD_LAZY (in nanovg binding) and it worked. Maybe if I'll add RTLD_GLOBAL in bgfx binding, it will also work, I didn't test it.

Ok, thanks. It's up to you, add it or not.
I'll try to do it without Derelict.

@mdparker
Copy link
Member

Are you linking to nanovg statically? I'm trying to fully understand your problem.

What you can do, if you're morivated to, is to checkout DerelictUtil locally and change the calk to dlopen so that it uses RTLD_NOW | RTLD_GLOBAL and see if it solves your problem. Follow the instructions in the Derelict docs to set up local dependencies in dub. You'll need a local copy of derelict-bgfx that depends on your modified derelict-util.

If the change solves your problem, I'll add a free function to DerelictUtil to allow setting the global option.

@mdparker
Copy link
Member

The relevant info is at the bottom od thia page http://derelictorg.github.io/compiling.html

@olehlong
Copy link
Author

It's dynamically. Like any other Derelict binding. So I changed mode for nanovg.
RTLD_LAZY - works
RTLD_NOW - undefined symbol: bgfx_destroy_uniform
RTLD_NOW | RTLD_GLOBAL - works

@mdparker
Copy link
Member

RTLD_LAZY - works
RTLD_NOW - undefined symbol: bgfx_destroy_uniform

OK, now I understand why you kept mentioning RTLD_LAZY. Looking at DerelictBGFX, there are many other symbols loaded before this one, so this raises the question, what causes this to fail loading in these circumstances?

I'm not 100% adverse to changing the RTLD_NOW to RTLD_LAZY, but what I worry about is whether it will cause issues elsewhere. This is the first instance I've ever heard of a problem with RTLD_NOW being used in Derelict. If I blindly change it, a similar problem may crop up elsewhere. I'd really like to know why this is happening.

@p0nce Any ideas?

@olehlong Is it possible for you to post an archive of your test project and the relevant bindings somewhere? I'd like to try to reproduce this on my Linux box and see what I can do with it.

@mdparker
Copy link
Member

Alright, I've decided it's not going to kill me to add the ability to modify the flags to dlopen. Here's what you can do:

import derelict.util.sharedlib;
version(linux) derelictLDFlags = LDFlags.rtldLazy;
DerelictFoo.load();
version(linux) derelictLDFlags = LDFlags.rtldNow | LDFlags.rtldGlobal;
DerelictBar.load();

The flags are available for version(Posix) and are not implemented on Windows. Use version="~>2.0.5" for your dub dependency.

@p0nce
Copy link

p0nce commented Apr 12, 2016

@aldacron No idea at all what it could be unfortunately. I'm happy with the default flag not changing :)
On Mac something else than RTLD_NOW weren't necessary.

@mdparker
Copy link
Member

Thanks for the input. I suppose this is one of those issues for which we'll stumble across a cause when no one is looking for one and we've forgotten all about it. derelictLDFlags is painless, though, and actually not a bad idea. Glad the OP brought it up.

@p0nce
Copy link

p0nce commented Apr 12, 2016

Is it a global? Shouldn't it be a field of SharedLib ?

@mdparker
Copy link
Member

That would necessitate bumping up the minor version at least. For something that will be used as rarely as this, and only on specific platforms, I don't think it's worth it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants