Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

C# debugger has a very low (1024 chars) JSON length limit #206

Open
ErwanFagnou opened this issue Jan 4, 2024 · 13 comments
Open

C# debugger has a very low (1024 chars) JSON length limit #206

ErwanFagnou opened this issue Jan 4, 2024 · 13 comments

Comments

@ErwanFagnou
Copy link

ErwanFagnou commented Jan 4, 2024

Problem description

Hi! I wanted to use this extension with my C# project, and it worked really well.

However when I wanted to plot slightly more complex plots (like 1000 points), the visualization does not work.
I believe this is a MAJOR issue, considering in the most simple plot i cannot plot more than 218 points...

var data = Enumerable.Range(0, 1000).Select(i => i.ToString()); // 1000 points
string s = $$"""
"kind":{"plotly": true}, "data":[{"y": [{{ string.Join(", ", data) }}]}]
""";
Could not parse evaluation result as JSON:
Unexpected non-whitespace character after JSON at position 6
Evaluation result was:
{"kind":{"plotly": true}, "data":[{"y": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218...
Used debug adapter: coreclr

The reason is that the C# debugger truncates the strings in the debugger console to 1024 characters (in the error message above the string is cut at item 218 and ends with three dots. As explained here, the limit does not seem to be controllable (contrary to other languages with the same issues), however when right clicking the text we can copy the full (untruncated) text.

Possible solutions

I thought of a few ways of fixing this:

  • if you manage to get the true value returned by the expression instead of what is printed on the debug console, that's perfect (the value is somewhere because we can copy it, but where?)
  • maybe the extension could support using the text printed in debug (with System.Diagnostics.Debug.WriteLine) to receive the json, so that we could print it on multiple lines and avoid the limit. That would also solve the issue on any other language.
  • or take the json from a file, i can imagine that instead of an expression returning a json string, it writes the json somewhere and returns a string indicating the path (e.g. "{"file": "C:/directory/file.json"}").
  • i noticed that if the expression returns a StringBuilder instead of string, the extension manages to get the whole data, although it is formatted wrongly (it appears as a graph, where one element containts the complete json string). Maybe the extension could support that or a similar approach. But that's kind of hacky

Environment details

  • Windows 11
  • VS Code 1.85.1
    • C# extension 2.14.8
    • C# Dev Kit extension 1.1.16
    • Debug Visualizer extension 2.4.0
@ErwanFagnou ErwanFagnou changed the title C# debugger has a very low (1024) string length limit C# debugger has a very low (1024 chars) JSON length limit Jan 4, 2024
@hediet
Copy link
Owner

hediet commented Jan 4, 2024

Maybe we can use the "Copy value" command somehow

although it is formatted wrongly (it appears as a graph, where one element containts the complete json string). Maybe the extension could support that or a similar approach. But that's kind of hacky

That is fallback implementation

Maybe we should add a csharp specific debug backend and use "watch" instead of repl

@ErwanFagnou
Copy link
Author

Thanks for the quick response!

That is fallback implementation
vscode-debug-visualizer/extension/src/VisualizationBackend/GenericVisualizationSupport.ts

I just meant that there are ways to send longer JSON strings to the visualizer, through a StringBuilder object for instance. If you don't manage to get the clean solutions working, this may be a reasonable alternative. It could be handled also as a fallback implementation, and would only require the user to convert the string s using new StringBuilder(s).

I am not familiar with how the debugger and the extension work, so I have no idea if switching to "watch" would solve the issue. Also maybe you could add this choice in the parameters of the extension somewhere?

@hediet
Copy link
Owner

hediet commented Jan 4, 2024

I think there actually might be a setting for this extension to configure watch/reply. I don't have a csharp env at the moment, can you try?

@ErwanFagnou
Copy link
Author

ErwanFagnou commented Jan 4, 2024

Sure, I can try in about 4 hours I think (time to go back home).

What setting do I have to change? I think I found something from the readme:

debugVisualizer.debugAdapterConfigurations
Allows to set expression templates for specific debug adapter types. Example:

"debugVisualizer.debugAdapterConfigurations": {
    "lldb": {
        "expressionTemplate": "script to_json(\"${expr}\")",
        "context": "repl"
    }
}

Where I could change "lldb" to "coreclr","repl" to "watch" (and remove the expressionTemplate)?

@hediet
Copy link
Owner

hediet commented Jan 4, 2024

Where I could change "lldb" to "coreclr","repl" to "watch" (and remove the expressionTemplate)?

Exactly

@ErwanFagnou
Copy link
Author

Unfortunately that didn't work, the string is still truncated (no change from before)...

I'm sure the settings were well loaded because changing expressionTemplate worked. But I have no way of checking if the backend was set correctly.

@ErwanFagnou
Copy link
Author

ErwanFagnou commented Jan 4, 2024

I found a solution! 🥳

Although it's a hack, I managed to bypass the string length limit of the C# debugger.
It exploits multiple crucial observations:

  • when an expression returns a class object, the visualizer seems to read a string like: {<object string>, <other info like attributes, static class variables>, etc...}. (note the curly braces around the values)
  • the default object string is the class name, but when the object has a ToString method, it is called to set the object name
  • the object string is NOT truncated! (but class attributes are)
  • JSON strings are wrapped around curly braces

So if we create an empty object without any variables, and only a ToString method that returns the JSON string (stored as a static variable in another class) without the surrounding curly braces, the visualizer gets the correct string:

class JsonString {
    public JsonString(string json) { JsonStringContainer.json = json;}
    public override string ToString() { return JsonStringContainer.json[1..^1]; }
}

class JsonStringContainer {
    public static string json = "";
}

And then instead of visualizing a string my_json_string, we can use new JsonString(my_json_string). This can be done automatically by changing the visualizer setting:

"debugVisualizer.debugAdapterConfigurations": {
  "coreclr": {
    "expressionTemplate": "new JsonString(${expr})",
  }
}

I am able to plot 100,000 points without an issue.

For the record there seems to be a limit but much higher (I couldn't plot 1 million points), but it's more than enough like this.

@hediet
Copy link
Owner

hediet commented Jan 5, 2024

Up for a PR? :) The string based approach should still work for backwards compat reasons. Also, make sure that your suggestion does not have escaping issues.

@ErwanFagnou
Copy link
Author

I'm not too sure about how this can be integrated. The JsonString class that is needed cannot be defined at debug time (it seems we can't define classes in the C# debugger). So the only way to have it is to have the user include the class in their code before compiling.

I'll try to think of other ways but I'm out of ideas (I've tried anonymous classes, struct...).

@hediet
Copy link
Owner

hediet commented Jan 8, 2024

<- {"command":"evaluate","arguments":{"expression":"str","frameId":1000,"context":"clipboard"},"type":"request","seq":16}

This seems to work for me for strings up to 5MB (i.e. just set the context to "clipboard", which is probably a good default anyway).

Image

@ErwanFagnou
Copy link
Author

{"command":"evaluate","arguments":{"expression":"str","frameId":1000,"context":"clipboard"},"type":"request","seq":16}

I am unsure how I can use this. Could you give me a bit more explanations? 😅

@hediet
Copy link
Owner

hediet commented Jan 11, 2024

Just use clipboard as context!

"debugVisualizer.debugAdapterConfigurations": {
    "coreclr": {
        "context": "clipboard"
    }
}

@arjunarjun07
Copy link

arjunarjun07 commented Jul 23, 2024

@ErwanFagnou Looks like I am facing similar issue of json string limitation with cpp debugger

#228
image

Shed some light here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants