Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

marisa 0.2.6 librime 1.12.0 (new formula) #196833

Merged
merged 6 commits into from
Dec 12, 2024
Merged

Conversation

Freed-Wu
Copy link
Contributor

@Freed-Wu Freed-Wu commented Nov 6, 2024

  • Have you followed the guidelines for contributing?
  • Have you ensured that your commits follow the commit style guide?
  • Have you checked that there aren't other open pull requests for the same formula update/change?
  • Have you built your formula locally with HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source <formula>, where <formula> is the name of the formula you're submitting?
  • Is your test running fine brew test <formula>, where <formula> is the name of the formula you're submitting?
  • Does your build pass brew audit --strict <formula> (after doing HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source <formula>)? If this is a new formula, does it pass brew audit --new <formula>?

@github-actions github-actions bot added new formula PR adds a new formula to Homebrew/homebrew-core boost Boost use is a significant feature of the PR or issue icu4c ICU use is a significant feature of the PR or issue labels Nov 6, 2024
@chenrui333 chenrui333 added the CI-no-fail-fast Continue CI tests despite failing GitHub Actions matrix builds. label Nov 7, 2024
Formula/lib/librime.rb Outdated Show resolved Hide resolved
Formula/lib/librime.rb Outdated Show resolved Hide resolved
@tshu-w
Copy link
Contributor

tshu-w commented Nov 14, 2024

Hi, I wander if we can download the assets from releases rather than build in this formula.

@Freed-Wu
Copy link
Contributor Author

althought haven't tested, I think it should be right (and a little troublesome)

@SMillerDev
Copy link
Member

Hi, I wander if we can download the assets from releases rather than build in this formula.

No, because all formula must build from source.

@daeho-ro daeho-ro force-pushed the librime branch 2 times, most recently from fd01f52 to 5557f1b Compare November 14, 2024 10:54
@daeho-ro
Copy link
Member

daeho-ro commented Nov 14, 2024

I have updated the formula, @Freed-Wu @tshu-w please check that the library built correctly?
Maybe add more test case is good.

@tshu-w
Copy link
Contributor

tshu-w commented Nov 14, 2024

I have updated the formula, @Freed-Wu @tshu-w please check that the library built correctly? Maybe add more test case is good.

The library built correctly on my side (M1 15.1 (24B83)).

Suggestion: The version of librime should be updated to the latest 1.12.0.

@github-actions github-actions bot added the automerge-skip `brew pr-automerge` will skip this pull request label Nov 14, 2024
@daeho-ro daeho-ro changed the title librime 1.11.2 (new formula) librime 1.12.0 (new formula) Nov 14, 2024
@daeho-ro daeho-ro added the ready to merge PR can be merged once CI is green label Nov 14, 2024
@tshu-w
Copy link
Contributor

tshu-w commented Nov 14, 2024

Sorry, but I got crash when using emacs-rime with librime build from this formula.
There is no problem when using librime downloaded from GitHub releases.

λ emacs
2024-11-14 21:03:07.339 Emacs[17341:1793670]  [IMKClient subclass]: chose IMKClient_Modern
src/tcmalloc.cc:309] Attempt to free invalid pointer 0x600000a11f40
Fatal error 5: Trace/breakpoint trap
[1]    17341 abort      emacs

I have asked emacs-rime author @DogLooksGood for help.

@daeho-ro daeho-ro removed the ready to merge PR can be merged once CI is green label Nov 14, 2024
default.custom.yaml Outdated Show resolved Hide resolved
@tshu-w
Copy link
Contributor

tshu-w commented Nov 14, 2024

Sorry, but I got crash when using emacs-rime with librime build from this formula. There is no problem when using librime downloaded from GitHub releases.

λ emacs
2024-11-14 21:03:07.339 Emacs[17341:1793670]  [IMKClient subclass]: chose IMKClient_Modern
src/tcmalloc.cc:309] Attempt to free invalid pointer 0x600000a11f40
Fatal error 5: Trace/breakpoint trap
[1]    17341 abort      emacs

I have asked emacs-rime author @DogLooksGood for help.

It seems the build librime.1.12.0.dylib is different from released librime.1.12.0.dylib

/opt/homebrew/Cellar/librime/1.12.0 stable
λ ll lib/librime.1.12.0.dylib
-r--r--r-- 1 wangtianshu admin 2.4M Nov 14 21:18 lib/librime.1.12.0.dylib

/opt/homebrew/Cellar/librime/1.12.0 stable
λ ll /Users/wangtianshu/Downloads/librime/dist/lib/librime.1.12.0.dylib
-rwxr-xr-x 1 wangtianshu staff 7.2M Nov 11 22:57 /Users/wangtianshu/Downloads/librime/dist/lib/librime.1.12.0.dylib

@github-actions github-actions bot removed the automerge-skip `brew pr-automerge` will skip this pull request label Nov 14, 2024
@daeho-ro
Copy link
Member

Indeed, the file size differ 3 times but I have no idea. 🥲

@Freed-Wu
Copy link
Contributor Author

downloaded file is a universe arch (support amd64 and arm64), so it should be larger reasonably.

@tshu-w
Copy link
Contributor

tshu-w commented Nov 15, 2024

downloaded file is a universe arch (support amd64 and arm64), so it should be larger reasonably.

This is reasonable. I might try manually build librime to see if emacs-rime works properly, but I can't guarantee when I'll have the time to do so.

@github-actions github-actions bot removed the autosquash Automatically squash pull request commits according to Homebrew style. label Dec 11, 2024
@daeho-ro
Copy link
Member

daeho-ro commented Dec 11, 2024

As @carlocab mentioned, library itself is ok and it does not have a problem.
I think, emacs-rime is not built correctly with librime.dylib.

Maybe I miss something 🤔
Also tried to debug emacs but I really don't understand how to attach the debugger.

@carlocab
Copy link
Member

I understand, but it should be possible to mimic what emacs-rime does in the background to replicate the crash, no?

Copy link
Member

@carlocab carlocab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with merging as-is for now and fixing emacs-rime integration in a follow-up.

@daeho-ro
Copy link
Member

Agree, I will try this after changing my pc but also need help 😆

@daeho-ro daeho-ro removed the in progress Stale bot should stay away label Dec 11, 2024
Copy link
Contributor

@github-actions github-actions bot added the CI-published-bottle-commits The commits for the built bottles have been pushed to the PR branch. label Dec 12, 2024
@BrewTestBot BrewTestBot added this pull request to the merge queue Dec 12, 2024
Merged via the queue into Homebrew:master with commit adb52ee Dec 12, 2024
15 checks passed
@chenrui333 chenrui333 changed the title librime 1.12.0 (new formula) marisa 0.2.6 librime 1.12.0 (new formula) Dec 12, 2024
@tshu-w
Copy link
Contributor

tshu-w commented Dec 19, 2024

I'm ok with merging as-is for now and fixing emacs-rime integration in a follow-up.

Should we open an issue for emacs-rime integration, and I wonder how can I help to fix this error.

@tshu-w
Copy link
Contributor

tshu-w commented Dec 20, 2024

@daeho-ro @carlocab

All crash stacktraces look like this

  * frame #0: 0x0000000134943f9d libtcmalloc.4.dylib`tcmalloc::Log(tcmalloc::LogMode, char const*, int, tcmalloc::LogItem, tcmalloc::LogItem, tcmalloc::LogItem, tcmalloc::LogItem)   541
    frame #1: 0x000000013493fa9d libtcmalloc.4.dylib`(anonymous namespace)::InvalidFree(void*)   106
    frame #2: 0x00007ff817a96f9e CoreFoundation`_CFRelease   1243
    frame #3: 0x00007ff81751ed42 libobjc.A.dylib`AutoreleasePoolPage::releaseUntil(objc_object**)   186
    frame #4: 0x00007ff81751c05a libobjc.A.dylib`objc_autoreleasePoolPop   235
    frame #5: 0x00007ff81798e8a5 CoreFoundation`_CFAutoreleasePoolPop   22
    frame #6: 0x00007ff8189b5a87 Foundation`-[NSAutoreleasePool release]   131
    frame #7: 0x00000001002415f9 emacs`ns_read_socket_1(terminal=<unavailable>, hold_quit=0x00007ff7bfefc8f0, no_release=NO) at nsterm.m:4834:4
    frame #8: 0x00000001000f2310 emacs`gobble_input at keyboard.c:7919:17
    frame #9: 0x00000001000f26a5 emacs`process_pending_signals at keyboard.c:8158:19
    frame #10: 0x00000001000f26a0 emacs`process_pending_signals at keyboard.c:8172:3
    frame #11: 0x000000010017c614 emacs`probably_quit at eval.c:1794:5
    frame #12: 0x00000001001797a9 emacs`Ffuncall [inlined] maybe_quit at lisp.h:3946:5
    frame #13: 0x0000000100179787 emacs`Ffuncall(nargs=3, args=(struct Lisp_Symbol *) $8 = 0x00007ff8c0782730) at eval.c:3076:3
    frame #14: 0x00000001001b44f8 emacs`module_funcall(env=0x00007ff7bfefdbb8, func=<unavailable>, nargs=2, args=0x00007ff7bfefcaf8) at emacs-module.c:682:44
    frame #15: 0x00000001290d76f8 librime-emacs.dylib`emacs_defun(env=0x00007ff7bfefdbb8, rime=0x00000001357345a0, cfunc=0x00000001290d6c90, func_name="rime-lib-version", doc="Version", min=0, max=0) at lib.c:450:3

Only leveldb is linked to tcmalloc.

❯ otool -L /usr/local/opt/leveldb/lib/libleveldb.1.23.0.dylib
/usr/local/opt/leveldb/lib/libleveldb.1.23.0.dylib:
	/usr/local/opt/leveldb/lib/libleveldb.1.dylib (compatibility version 1.0.0, current version 1.23.0)
	/usr/local/opt/snappy/lib/libsnappy.1.dylib (compatibility version 1.0.0, current version 1.1.10)
	/usr/local/opt/gperftools/lib/libtcmalloc.4.dylib (compatibility version 10.0.0, current version 10.16.0)
	/usr/lib/libc  .1.dylib (compatibility version 1.0.0, current version 1700.255.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1345.100.2)

Break on tc_free:

* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 2.1
  * frame #0: 0x000000013490e980 libtcmalloc.4.dylib`tc_free
    frame #1: 0x000000013490e937 libtcmalloc.4.dylib`TCMallocGuard::TCMallocGuard()   553
    frame #2: 0x0000000134922bfc libtcmalloc.4.dylib`RAW_VLOG(int, char const*, ...) (.cold.1)   44
    frame #3: 0x00007ff81756f959 dyld`invocation function for block in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const   241
    frame #4: 0x00007ff8175a8b69 dyld`invocation function for block in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const   241
    frame #5: 0x00007ff81759d1e3 dyld`invocation function for block in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const   543
    frame #6: 0x00007ff81755707b dyld`dyld3::MachOFile::forEachLoadCommand(Diagnostics&, void (load_command const*, bool&) block_pointer) const   249
    frame #7: 0x00007ff81759c258 dyld`dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const   176
    frame #8: 0x00007ff8175a8744 dyld`dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const   470
    frame #9: 0x00007ff81756f7f2 dyld`dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const   150
    frame #10: 0x00007ff817575ed7 dyld`dyld4::JustInTimeLoader::runInitializers(dyld4::RuntimeState&) const   21
    frame #11: 0x00007ff81756fb58 dyld`dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&, dyld3::Array<dyld4::Loader const*>&) const   276
    frame #12: 0x00007ff81756fb03 dyld`dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&, dyld3::Array<dyld4::Loader const*>&) const   191
    frame #13: 0x00007ff81756fb03 dyld`dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&, dyld3::Array<dyld4::Loader const*>&) const   191
    frame #14: 0x00007ff81756fb03 dyld`dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&, dyld3::Array<dyld4::Loader const*>&) const   191
    frame #15: 0x00007ff817573371 dyld`dyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const::$_0::operator()() const   147
    frame #16: 0x00007ff81756fbec dyld`dyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const   90
    frame #17: 0x00007ff817586753 dyld`dyld4::APIs::dlopen_from(char const*, int, void*)   1859
    frame #18: 0x00000001001b26f3 emacs`Fmodule_load(file=(struct Lisp_String *) $4 = 0x00007fb9b281fdc0) at emacs-module.c:1201:12

Debug prints from dyld:

dyld[18688]: <9231638B-67DD-3C16-953C-0BA0D1730AA7> /Users/user/.emacs.d/elpa/rime-20241211.808/librime-emacs.dylib
dyld[18688]: <2604B5F8-98DE-3B4E-9DB9-AF645B863E4D> /usr/local/Cellar/librime/1.12.0/lib/librime.1.12.0.dylib
dyld[18688]: <22DFE32D-2078-3D42-9998-664FE0ED3008> /usr/local/Cellar/glog/0.6.0/lib/libglog.0.6.0.dylib
dyld[18688]: <A37BED3C-CB1D-399F-AFCD-AF82058E1493> /usr/local/Cellar/yaml-cpp/0.8.0/lib/libyaml-cpp.0.8.0.dylib
dyld[18688]: <A9545E5C-58C2-3967-A1F8-FB71F293FA9F> /usr/local/Cellar/gflags/2.2.2/lib/libgflags.2.2.2.dylib
dyld[18688]: <D3C695FB-1AC5-3AE8-831B-4F15759C3EF8> /usr/local/Cellar/leveldb/1.23_1/lib/libleveldb.1.23.0.dylib
dyld[18688]: <3F5254E1-7995-377A-A410-D33A60BF3625> /usr/local/Cellar/marisa/0.2.6/lib/libmarisa.0.dylib
dyld[18688]: <9D1C9FDC-44C4-312A-A6FF-DCA4D42A62E0> /usr/local/Cellar/opencc/1.1.9/lib/libopencc.1.1.9.dylib
dyld[18688]: <626398A4-88F6-3B18-B581-07632736C792> /usr/local/Cellar/snappy/1.2.1/lib/libsnappy.1.2.1.dylib
dyld[18688]: <33F156EB-956D-3EE8-A928-49A329973A42> /usr/local/Cellar/gperftools/2.16/lib/libtcmalloc.4.dylib
dyld[18688]: libtcmalloc.4.dylib has weak-def (or flat lookup) symbol used by libgflags.2.2.2.dylib, so cannot be delayed
dyld[18688]: libgflags.2.2.2.dylib has weak-def (or flat lookup) symbol used by libglog.0.6.0.dylib, so cannot be delayed
dyld[18688]: libtcmalloc.4.dylib has weak-def (or flat lookup) symbol used by libglog.0.6.0.dylib, so cannot be delayed
dyld[18688]: libtcmalloc.4.dylib has weak-def (or flat lookup) symbol used by libyaml-cpp.0.8.0.dylib, so cannot be delayed
dyld[18688]: libtcmalloc.4.dylib has weak-def (or flat lookup) symbol used by libsnappy.1.2.1.dylib, so cannot be delayed
dyld[18688]: libtcmalloc.4.dylib has weak-def (or flat lookup) symbol used by libleveldb.1.23.0.dylib, so cannot be delayed
dyld[18688]: librime.1.12.0.dylib has weak-def (or flat lookup) symbol used by libmarisa.0.dylib, so cannot be delayed
dyld[18688]: libtcmalloc.4.dylib has weak-def (or flat lookup) symbol used by libmarisa.0.dylib, so cannot be delayed
dyld[18688]: librime.1.12.0.dylib has weak-def (or flat lookup) symbol used by libopencc.1.1.9.dylib, so cannot be delayed
dyld[18688]: libtcmalloc.4.dylib has weak-def (or flat lookup) symbol used by libopencc.1.1.9.dylib, so cannot be delayed
dyld[18688]: libtcmalloc.4.dylib has weak-def (or flat lookup) symbol used by librime.1.12.0.dylib, so cannot be delayed
src/tcmalloc.cc:309] Attempt to free invalid pointer 0x6000022b0ee0
Fatal error 4: Illegal instruction
src/tcmalloc.cc:309] Attempt to free invalid pointer 0x6000022b0f50
src/tcmalloc.cc:309] Attempt to free invalid pointer 0x60000092cb00

It's likely due to the leveldb depending on tcmalloc.

edit: For reference, Debian's libleveldb does not depend on libtcmalloc so I consider this a packaging error by Homebrew. Introduced here. Please consider reporting this to Homebrew.

Originally posted by @ksqsf in rime/librime#963 (comment)

@carlocab
Copy link
Member

Does removing the gperftools dependency, e.g.,

diff --git a/Formula/l/leveldb.rb b/Formula/l/leveldb.rb
index e7560b92577..3bad433fd35 100644
--- a/Formula/l/leveldb.rb
    b/Formula/l/leveldb.rb
@@ -18,7  18,6 @@ class Leveldb < Formula
   end
 
   depends_on "cmake" => :build
-  depends_on "gperftools"
   depends_on "snappy"
 
   def install

and doing brew reinstall -s leveldb fix this problem?

You may need to do

export HOMEBREW_NO_INSTALL_FROM_API=1
brew tap homebrew/core

in order to do this.

@tshu-w
Copy link
Contributor

tshu-w commented Dec 21, 2024

and doing brew reinstall -s leveldb fix this problem?

Yes, removing the gperftools dependency fix this problem.

Maybe related: https://trac.macports.org/ticket/68775#no1

@daeho-ro
Copy link
Member

It seems that the bug was reported long ago.

@tshu-w
Copy link
Contributor

tshu-w commented Dec 21, 2024

Is there anything Homebrew can do to fix this problem temporarily?

@carlocab
Copy link
Member

Looks like we had another PR that tried to disable leveldb's use of tcmalloc in the past: #88758

Back then it wasn't clear that was a good idea, but discussion here has persuaded me otherwise. Feel free to create a new version of that PR; I am 👍 on that change now (assuming CI passes).

@daeho-ro daeho-ro mentioned this pull request Dec 21, 2024
6 tasks
@carlocab
Copy link
Member

I think this should be fixed now with

brew update
brew upgrade leveldb

@tshu-w
Copy link
Contributor

tshu-w commented Dec 21, 2024

I think this should be fixed now with

Confirmed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
boost Boost use is a significant feature of the PR or issue CI-no-fail-fast Continue CI tests despite failing GitHub Actions matrix builds. CI-published-bottle-commits The commits for the built bottles have been pushed to the PR branch. icu4c ICU use is a significant feature of the PR or issue lua Lua use is a significant feature of the PR or issue new formula PR adds a new formula to Homebrew/homebrew-core
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants