Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Interacting with boolean data causes queries to become unresponsive #4033

Open
mattrankin83 opened this issue Jul 23, 2024 · 2 comments
Open
Assignees
Labels
☢️ Bug Something isn't working

Comments

@mattrankin83
Copy link

Which OpenObserve functionalities are the source of the bug?

log search, dashboards

Is this a regression?

No

Description

We are ingesting data from an application (via fluentd) and in the message there is potentially a number of boolean data fields. Whenever I try to use those fields as a filter, or use them in any dashboard panel, it locks the system up (it seems like the query is maybe having an issue?).

Eventually I'm returned an error. On the below screenshot I am filtering on the logs screen by one other field (a text match) and the boolean field.
image

If I run that same query without the boolean field it completes totally fine, and very fast. But if I try to find the 'True' values within that result set (which was 11 records) I am stuck with the loading icon for approximately 10-12 minutes and it results in the log screen displaying "Error: Search SQL execute error."

During those 10 minutes I'm unable to load any other data, whether it be a different log stream, or a dashboard/graph. If I try to view a Dashboard I just get the "No Data" message:

image

The same thing happens when I try to incorporate that field into a dashboard, the entire system fails to load data until that first action errors out. But on the graphs I do get a different, and more informative, error message:
type_coercion caused by Error during planning: Can not find compatible types to compare Boolean with [Utf8]

image

On the "Stream Detail" page, these fields have been classified, correctly, as Boolean. Although this field is not incorporated in every message that is logged (not sure if that's relevant).

image

The most minimal reproduction I can think of is just trying to display a count (Metric Text panel) of messages where a particular boolean field is true, the SQL is quite simple (this is using a different boolean field from the one above):

image

This returns the same error as above:

image

This deployment is running in Kubernetes. I have tried watching the logs on various pods but I didn't see anything that looked particularly relevant but I'm more than happy to retrieve any log entries that may help.

Please provide a link to a minimal reproduction of the bug

No response

Please provide the exception or error you saw

"Error: Search SQL execute error."
type_coercion caused by Error during planning: Can not find compatible types to compare Boolean with [Utf8]

Please provide the version you discovered this bug in (check about page for version information)

Version v0.10.7
Commit Hash 611f2e1c29df33a2df8e3be8a726889b07485284
Build Date 2024-06-12T14:11:18Z

Anything else?

To mitigate this I have written a VRL function that applies on ingestion to create a new field set to a 1 or 0 and that works, but the issue can pop its head up when you click on the wrong field and it prevents any users from loading dashboards until the problem resolves itself, and that only works for data going forward, not the old data.

Let me know if there is any additional information I can provide! Absolutely loving using o2 otherwise!

@mattrankin83 mattrankin83 added the ☢️ Bug Something isn't working label Jul 23, 2024
@hengfeiyang
Copy link
Contributor

You should use dev_build is true if the field is boolean, it is not string, you can't use dev_build='true' for it.

@hengfeiyang hengfeiyang self-assigned this Jul 29, 2024
@mattrankin83
Copy link
Author

@hengfeiyang Thank you! That does seem to do the trick.

I guess where I got confused is that when I click the filters from the list on the left of the log screen, for example in the image below if I click that equals sign (=):

image

The query in the editor that fills in above the log becomes "dev_build='true' " rather than " dev_build is true "

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
☢️ Bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants