Fix line wrap for CJK characters #11296

huacnlee · 2024-05-02T11:18:10Z

Release Notes:

Fixed line wrap for CJK characters.

Demo

input1.mov

Render case

## fr

Bien démarrer avec la documentation GitHub Découvrez comment commencer à créer, à livrer et à gérer des logiciels avec GitHub. Explorez nos produits, inscrivez-vous pour obtenir un compte et connectez-vous à la plus grande communauté de développement du monde.

## zh

GitHub 入门文档 了解如何开始构建、运输和维护具有 GitHub 的软件。 了解我们的产品，注册一个帐户，与世界上最大的发展社区建立联系。

## es

Documentación sobre la introducción a GitHub Aprende cómo comenzar a crear, enviar y mantener software con GitHub. Explora nuestros productos, regístrate para una cuenta y conéctate con la comunidad de desarrollo más grande del mundo.

## kr

GitHub 설명서 시작 GitHub를 사용하여 소프트웨어 빌드, 납품 및 유지 관리를 시작하는 방법을 알아봅니다. 제품을 탐색하고, 계정에 등록하고, 세계 최대의 개발 커뮤니티와 연결합니다.

## ja

GitHub の概要に関するドキュメント GitHub を使用してソフトウェアの構築、出荷、および保守を始める方法を学びます。 当社の製品を探索し、アカウントにサインアップして、世界最大の開発コミュニティと繋がりましょう。

## pt

Documentação de introdução ao GitHub Aprenda a começar a criar, enviar e manter um software com a GitHub. Explore nossos produtos, inscreva-se em uma conta e conecte-se com a maior comunidade de desenvolvimento do mundo.

## ru

Начало работы с документацией по GitHub Узнайте, как начать создание, доставку и обслуживание программного обеспечения с помощью GitHub. Изучите наши продукты, зарегистрируйте учетную запись и присоединитесь к крупнейшему в мире сообществу разработчиков.

huacnlee · 2024-05-02T13:56:12Z

But it will crash some time, I don't why:

Thread "main" panicked with "point Point { row: 0, column: 2 } is inside of character '等'" at crates/rope/src/rope.rs:794:21
   0: backtrace::backtrace::libunwind::trace
             at /Users/jason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/backtrace-0.3.69/src/backtrace/libunwind.rs:93:5
      backtrace::backtrace::trace_unsynchronized::<<backtrace::capture::Backtrace>::create::{closure#0}>
             at /Users/jason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/backtrace-0.3.69/src/backtrace/mod.rs:66:5
   1: backtrace::backtrace::trace::<<backtrace::capture::Backtrace>::create::{closure#0}>
             at /Users/jason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/backtrace-0.3.69/src/backtrace/mod.rs:53:14
   2: <backtrace::capture::Backtrace>::create
             at /Users/jason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/backtrace-0.3.69/src/capture.rs:176:9
   3: <backtrace::capture::Backtrace>::new
             at /Users/jason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/backtrace-0.3.69/src/capture.rs:140:22
   4: Zed::reliability::init_panic_hook::{closure#0}
             at crates/zed/src/reliability.rs:83:29
   5: <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/alloc/src/boxed.rs:2029:9
      std::panicking::rust_panic_with_hook
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panicking.rs:785:13
   6: std::panicking::begin_panic_handler::{{closure}}
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panicking.rs:659:13
   7: std::sys_common::backtrace::__rust_end_short_backtrace
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/sys_common/backtrace.rs:171:18
   8: rust_begin_unwind
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panicking.rs:647:5
   9: core::panicking::panic_fmt
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/panicking.rs:72:14
  10: <rope::Chunk>::point_to_offset
             at crates/rope/src/rope.rs:794:21
  11: <rope::Rope>::point_to_offset::{closure#0}
             at crates/rope/src/rope.rs:319:36
  12: <core::option::Option<&rope::Chunk>>::map_or::<usize, <rope::Rope>::point_to_offset::{closure#0}>
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/option.rs:1126:24
  13: <rope::Rope>::point_to_offset
             at crates/rope/src/rope.rs:317:15
  14: <text::BufferSnapshot>::point_to_offset
             at crates/text/src/text.rs:1778:9
  15: <multi_buffer::MultiBufferSnapshot>::point_to_offset
             at crates/multi_buffer/src/multi_buffer.rs:2391:20
  16: <rope::point::Point as multi_buffer::ToOffset>::to_offset
             at crates/multi_buffer/src/multi_buffer.rs:4366:9
  17: <multi_buffer::MultiBufferSnapshot>::anchor_at::<rope::point::Point>
             at crates/multi_buffer/src/multi_buffer.rs:2874:22
  18: <editor::element::EditorElement>::update_visible_cursor
             at crates/editor/src/element.rs:667:21
  19: <editor::element::EditorElement>::mouse_moved
             at crates/editor/src/element.rs:638:17
  20: <editor::element::EditorElement>::paint_mouse_listeners::{closure#2}::{closure#0}
             at crates/editor/src/element.rs:3178:25
  21: <gpui::window::WindowContext as gpui::VisualContext>::update_view::<editor::Editor, (), <editor::element::EditorElement>::paint_mouse_listeners::{closure#2}::{closure#0}>
             at crates/gpui/src/window.rs:3579:22
  22: <gpui::view::View<editor::Editor>>::update::<gpui::window::WindowContext, (), <editor::element::EditorElement>::paint_mouse_listeners::{closure#2}::{closure#0}>
             at crates/gpui/src/view.rs:76:9
  23: <editor::element::EditorElement>::paint_mouse_listeners::{closure#2}
             at crates/editor/src/element.rs:3167:21
  24: <gpui::window::WindowContext>::on_mouse_event::<gpui::interactive::MouseMoveEvent, <editor::element::EditorElement>::paint_mouse_listeners::{closure#2}>::{closure#0}
             at crates/gpui/src/window.rs:2703:21
  25: <alloc::boxed::Box<dyn for<'a, 'b, 'c> core::ops::function::FnMut<(&'a dyn core::any::Any, gpui::window::DispatchPhase, &'b mut gpui::window::WindowContext<'c>), Output = ()>> as core::ops::function::FnMut<(&dyn core::any::Any, gpui::window::DispatchPhase, &mut gpui::window::WindowContext)>>::call_mut
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/alloc/src/boxed.rs:2022:9
  26: <gpui::window::WindowContext>::dispatch_mouse_event
             at crates/gpui/src/window.rs:2936:17
  27: <gpui::window::WindowContext>::dispatch_event
             at crates/gpui/src/window.rs:2902:13
  28: <gpui::window::Window>::new::{closure#7}::{closure#0}
             at crates/gpui/src/window.rs:734:46
  29: <gpui::app::AppContext as gpui::Context>::update_window::<gpui::window::DispatchEventResult, <gpui::window::Window>::new::{closure#7}::{closure#0}>::{closure#0}
             at crates/gpui/src/app.rs:1322:26
  30: <gpui::app::AppContext>::update::<core::result::Result<gpui::window::DispatchEventResult, anyhow::Error>, <gpui::app::AppContext as gpui::Context>::update_window<gpui::window::DispatchEventResult, <gpui::window::Window>::new::{closure#7}::{closure#0}>::{closure#0}>
             at crates/gpui/src/app.rs:363:22
  31: <gpui::app::AppContext as gpui::Context>::update_window::<gpui::window::DispatchEventResult, <gpui::window::Window>::new::{closure#7}::{closure#0}>
             at crates/gpui/src/app.rs:1313:9
  32: <gpui::app::async_context::AsyncAppContext as gpui::Context>::update_window::<gpui::window::DispatchEventResult, <gpui::window::Window>::new::{closure#7}::{closure#0}>
             at crates/gpui/src/app/async_context.rs:94:9
  33: <gpui::window::AnyWindowHandle>::update::<gpui::app::async_context::AsyncAppContext, gpui::window::DispatchEventResult, <gpui::window::Window>::new::{closure#7}::{closure#0}>
             at crates/gpui/src/window.rs:4411:9
  34: <gpui::window::Window>::new::{closure#7}
             at crates/gpui/src/window.rs:733:17
  35: <alloc::boxed::Box<dyn core::ops::function::FnMut<(gpui::interactive::PlatformInput,), Output = gpui::window::DispatchEventResult>> as core::ops::function::FnMut<(gpui::interactive::PlatformInput,)>>::call_mut
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/alloc/src/boxed.rs:2022:9
  36: gpui::platform::mac::window::handle_view_event
             at crates/gpui/src/platform/mac/window.rs:1459:13
  37: <unknown>
  38: <unknown>
  39: <unknown>
  40: <unknown>
  41: <unknown>
  42: <(*mut objc::runtime::Object,) as objc::message::MessageArguments>::invoke::<()>
             at /Users/jason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/objc-0.2.7/src/message/mod.rs:128:17
  43: objc::message::platform::send_super_unverified::<objc::runtime::Object, (*mut objc::runtime::Object,), ()>
             at /Users/jason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/objc-0.2.7/src/message/apple/mod.rs:38:9
  44: objc::message::send_super_message::<objc::runtime::Object, (*mut objc::runtime::Object,), ()>
             at /Users/jason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/objc-0.2.7/src/message/mod.rs:205:5
      gpui::platform::mac::platform::send_event
             at crates/gpui/src/platform/mac/platform.rs:1073:9
  45: <unknown>
  46: <unknown>
  47: <() as objc::message::MessageArguments>::invoke::<()>
             at /Users/jason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/objc-0.2.7/src/message/mod.rs:128:17
  48: objc::message::platform::send_unverified::<objc::runtime::Object, (), ()>
             at /Users/jason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/objc-0.2.7/src/message/apple/mod.rs:27:9
  49: objc::message::send_message::<objc::runtime::Object, (), ()>
             at /Users/jason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/objc-0.2.7/src/message/mod.rs:178:5
      <*mut objc::runtime::Object as cocoa::appkit::NSApplication>::run
             at /Users/jason/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cocoa-0.25.0/src/appkit.rs:603:9
  50: <gpui::platform::mac::platform::MacPlatform as gpui::platform::Platform>::run
             at crates/gpui/src/platform/mac/platform.rs:389:13
  51: <gpui::app::App>::run::<Zed::init_ui::{closure#3}>
             at crates/gpui/src/app.rs:142:9
  52: Zed::init_ui
             at crates/zed/src/main.rs:233:5
  53: Zed::main
             at crates/zed/src/main.rs:442:9
  54: <fn() as core::ops::function::FnOnce<()>>::call_once
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/ops/function.rs:250:5
  55: std::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/sys_common/backtrace.rs:155:18
  56: std::rt::lang_start::<()>::{closure#0}
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/rt.rs:166:18
  57: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/core/src/ops/function.rs:284:13
      std::panicking::try::do_call
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panicking.rs:554:40
      std::panicking::try
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panicking.rs:518:19
      std::panic::catch_unwind
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panic.rs:142:14
      std::rt::lang_start_internal::{{closure}}
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/rt.rs:148:48
      std::panicking::try::do_call
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panicking.rs:554:40
      std::panicking::try
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panicking.rs:518:19
      std::panic::catch_unwind
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/panic.rs:142:14
      std::rt::lang_start_internal
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/rt.rs:148:20
  58: std::rt::lang_start::<()>
             at /rustc/aedd173a2c086e558c2b66d3743b344f977621a7/library/std/src/rt.rs:165:17
  59: _main

Thread "main" panicked with "point Point { row: 0, column: 2 } is inside of character '等'" at crates/rope/src/rope.rs:794:21

This crash not by this PR changes, I have checkout the previous commit, the crash is still happened.

😂 For this I spend 2 hours.

aeff468

huacnlee · 2024-05-02T15:04:44Z

The second commit is updated to match the exist test cases.

https://github.com/zed-industries/zed/blob/v0.134.1-pre/crates/editor/src/editor_tests.rs#L1272

But in this way, the URL cannot be forcibly truncated, as shown below. Compared with my previous video, the new changes have a little issue compared to the previous video.

However, at least most of the previous details will be retained (because I did not modify the previous test cases).

huacnlee · 2024-05-03T08:50:24Z

The third commit update to add special chars for word check.

Now text wrap is better, especially the URL.

For JA

aohanhongzhi · 2024-05-06T16:32:37Z

good job!

crates/gpui/src/text_system/line_wrapper.rs

mikayla-maki · 2024-05-18T01:04:06Z

Thanks for this PR! Really appreciate the attention to detail on other languages and I'd like to get it merged.

…ases.

huacnlee · 2024-05-18T02:25:47Z

Added more support:

Latin-1 Supplement, Extended-A,B,C, Cyrillic

Ref:

https://en.wikipedia.org/wiki/List_of_Unicode_characters

mikayla-maki · 2024-05-18T02:41:34Z

Given the increase in ergonomics, and the coverage for the languages we're capable of supporting, I think it's good to try this one out. Thank you!

mikayla-maki · 2024-05-18T02:42:05Z

Ahh, seems there's an issue with the macOS tests :)

crates/gpui/src/text_system/line_wrapper.rs

Incorrect.

huacnlee · 2024-05-18T03:23:10Z

Test now passed

huacnlee · 2024-05-18T03:36:08Z

Final test

2024-05-18.11.33.58.mov

huacnlee · 2024-05-29T02:07:33Z

Hi @mikayla-maki, how about this PR

0x2CA · 2024-06-27T09:25:51Z

I've observed that the wrapping behavior is not quite the same as vscode, and I'm not sure if that's correct (by intentional design)

The last character in vscode will not be [ and the first character will not be ,.

mikayla-maki

We looked at this PR again and immediately ran into a bunch of crashes from the rope code splitting characters. This needs to be fixed before we can merge this!

huacnlee · 2024-07-10T16:06:14Z

Yes, I know that rope crash: I have logged this issue: #12011

By my test:

That crash has no relative with this PR changes, it has always existed. I never encountered it before because I rarely write Chinese in Zed, and I only discovered it when implementing this CJK wrap support.

huacnlee · 2024-07-10T16:15:04Z

I am going to test again.

huacnlee · 2024-07-10T16:58:03Z

Hi @mikayla-maki I have make a record for show this crash, it was existed in main branch.

This version by tag: v0.144.0-pre

CleanShot.2024-07-11.at.00.56.04.mp4

Here is a test text:

此方案最早于 [2013 年](https://github.com/huacnlee/auto-correct/commit/47d7b7836f3e5f97dd31b5dc477eb0dbf8176e6b) 出现于 Ruby China 的项目，并逐步完善规则细节，当前准确率较高（极少数异常情况），你可以放心用来辅助你完成自动纠正动作。

And it's interest, that looks like not happen in the release version.

huacnlee · 2024-07-10T17:26:28Z

And I build a release version of this branch to confirm my guess.

Yes, It is no crash now:

CleanShot.2024-07-11.at.01.24.00.mp4

ConradIrwin · 2024-07-10T19:10:12Z

I have a fix for the panic and will merge that in a few

Fixes: #12011 When hovering over a multibyte character in a debug build, Zed would panic. Follow up to #11296 Release Notes: - N/A

cla-bot bot added the cla-signed The user has signed the Contributor License Agreement label May 2, 2024

huacnlee force-pushed the fix-cjk-wrap branch 3 times, most recently from b95e7c3 to 9113c99 Compare May 2, 2024 13:50

huacnlee force-pushed the fix-cjk-wrap branch from 9113c99 to 60b09d7 Compare May 2, 2024 14:29

huacnlee marked this pull request as ready for review May 2, 2024 14:30

huacnlee force-pushed the fix-cjk-wrap branch 2 times, most recently from 49720a1 to 311c858 Compare May 3, 2024 08:47

mikayla-maki assigned as-cii May 6, 2024

mikayla-maki requested changes May 18, 2024

View reviewed changes

crates/gpui/src/text_system/line_wrapper.rs Show resolved Hide resolved

mikayla-maki self-assigned this May 18, 2024

huacnlee added 4 commits May 18, 2024 09:16

Fix line wrap for CJK characters.

22e457d

Update wrap check to include ascii_punctuation for match exist test c…

425168f

…ases.

Update to special chars for word check for text wrap.

2bbfdea

Add ' to is_word_char is a command char in english words.

6b4ecfb

huacnlee force-pushed the fix-cjk-wrap branch from 311c858 to 7616ce0 Compare May 18, 2024 02:23

Add to support Latin-1 Supplement, Extended-A,B,C, Cyrillic

54ad96d

huacnlee force-pushed the fix-cjk-wrap branch from 7616ce0 to 54ad96d Compare May 18, 2024 02:25

huacnlee requested a review from mikayla-maki May 18, 2024 02:26

Update comment

c65fea8

.

6aeec4a

mikayla-maki approved these changes May 18, 2024

View reviewed changes

mikayla-maki reviewed May 18, 2024

View reviewed changes

crates/gpui/src/text_system/line_wrapper.rs Outdated Show resolved Hide resolved

.

a25b1a5

huacnlee commented May 18, 2024

View reviewed changes

crates/gpui/src/text_system/line_wrapper.rs Outdated Show resolved Hide resolved

mikayla-maki previously requested changes May 18, 2024

View reviewed changes

crates/gpui/src/text_system/line_wrapper.rs Show resolved Hide resolved

huacnlee added 2 commits May 18, 2024 11:19

Fix test

51fe26a

.

7edcc96

Keep / for url and path for better wrap style.

10211a8

huacnlee mentioned this pull request May 18, 2024

Crash when scrolling #12011

Closed

1 task

mikayla-maki requested changes Jul 10, 2024

View reviewed changes

ConradIrwin merged commit b693cbf into zed-industries:main Jul 10, 2024
8 checks passed

ConradIrwin mentioned this pull request Jul 10, 2024

Fix panic clicking on multibyte chars #14086

Merged

ConradIrwin added a commit that referenced this pull request Jul 10, 2024

Fix panic clicking on multibyte chars (#14086)

3ff738f

Fixes: #12011 When hovering over a multibyte character in a debug build, Zed would panic. Follow up to #11296 Release Notes: - N/A

huacnlee deleted the fix-cjk-wrap branch July 22, 2024 14:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix line wrap for CJK characters #11296

Fix line wrap for CJK characters #11296

huacnlee commented May 2, 2024 •

edited

Loading

huacnlee commented May 2, 2024 •

edited

Loading

huacnlee commented May 2, 2024 •

edited

Loading

huacnlee commented May 3, 2024 •

edited

Loading

aohanhongzhi commented May 6, 2024

mikayla-maki commented May 18, 2024 •

edited

Loading

huacnlee commented May 18, 2024 •

edited

Loading

mikayla-maki commented May 18, 2024

mikayla-maki commented May 18, 2024

huacnlee commented May 18, 2024

huacnlee commented May 18, 2024 •

edited

Loading

huacnlee commented May 29, 2024

0x2CA commented Jun 27, 2024 •

edited

Loading

mikayla-maki left a comment

huacnlee commented Jul 10, 2024

huacnlee commented Jul 10, 2024

huacnlee commented Jul 10, 2024 •

edited

Loading

huacnlee commented Jul 10, 2024

ConradIrwin commented Jul 10, 2024

Fix line wrap for CJK characters #11296

Fix line wrap for CJK characters #11296

Conversation

huacnlee commented May 2, 2024 • edited Loading

Demo

Render case

huacnlee commented May 2, 2024 • edited Loading

But it will crash some time, I don't why:

huacnlee commented May 2, 2024 • edited Loading

huacnlee commented May 3, 2024 • edited Loading

For JA

aohanhongzhi commented May 6, 2024

mikayla-maki commented May 18, 2024 • edited Loading

huacnlee commented May 18, 2024 • edited Loading

mikayla-maki commented May 18, 2024

mikayla-maki commented May 18, 2024

huacnlee commented May 18, 2024

huacnlee commented May 18, 2024 • edited Loading

huacnlee commented May 29, 2024

0x2CA commented Jun 27, 2024 • edited Loading

mikayla-maki left a comment

Choose a reason for hiding this comment

huacnlee commented Jul 10, 2024

huacnlee commented Jul 10, 2024

huacnlee commented Jul 10, 2024 • edited Loading

huacnlee commented Jul 10, 2024

ConradIrwin commented Jul 10, 2024

huacnlee commented May 2, 2024 •

edited

Loading

huacnlee commented May 2, 2024 •

edited

Loading

huacnlee commented May 2, 2024 •

edited

Loading

huacnlee commented May 3, 2024 •

edited

Loading

mikayla-maki commented May 18, 2024 •

edited

Loading

huacnlee commented May 18, 2024 •

edited

Loading

huacnlee commented May 18, 2024 •

edited

Loading

0x2CA commented Jun 27, 2024 •

edited

Loading

huacnlee commented Jul 10, 2024 •

edited

Loading