Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SparkSQL: Incompatiblity between : (colon operator) and L028 #4346

Closed
3 tasks done
julianopiovezan opened this issue Feb 3, 2023 · 1 comment
Closed
3 tasks done

SparkSQL: Incompatiblity between : (colon operator) and L028 #4346

julianopiovezan opened this issue Feb 3, 2023 · 1 comment
Labels
bug Something isn't working sparksql Related to the SparkSQL (aka Spark3) dialect

Comments

@julianopiovezan
Copy link
Contributor

Search before asking

  • I searched the issues and found no similar issues.

What Happened

Support for Databricks colon operator was added in PR #3956. Parse is working fine, but rule L028 is handling the column reference as a qualifier.

Expected Behaviour

No violation.

❯ sqlfluff lint teste2.sql
All Finished 📜 🎉!

Observed Behaviour

❯ sqlfluff lint teste2.sql
== [teste2.sql] FAIL                                                                                                                                                                  
L:   3 | P:   5 | L028 | Qualified reference 'a:id' found in single table select.
All Finished 📜 🎉!

How to reproduce

SQL

SELECT
    a,
    a:id
FROM source_table

Parse tree

[L:  1, P:  1]      |file:
[L:  1, P:  1]      |    statement:
[L:  1, P:  1]      |        select_statement:
[L:  1, P:  1]      |            select_clause:
[L:  1, P:  1]      |                keyword:                                      'SELECT'
[L:  1, P:  7]      |                newline:                                      '\n'
[L:  2, P:  1]      |                whitespace:                                   '    '
[L:  2, P:  5]      |                [META] indent:
[L:  2, P:  5]      |                select_clause_element:
[L:  2, P:  5]      |                    column_reference:
[L:  2, P:  5]      |                        naked_identifier:                     'a'
[L:  2, P:  6]      |                comma:                                        ','
[L:  2, P:  7]      |                newline:                                      '\n'
[L:  3, P:  1]      |                whitespace:                                   '    '
[L:  3, P:  5]      |                select_clause_element:
[L:  3, P:  5]      |                    column_reference:
[L:  3, P:  5]      |                        naked_identifier:                     'a'
[L:  3, P:  6]      |                        colon:                                ':'
[L:  3, P:  7]      |                        naked_identifier:                     'id'
[L:  3, P:  9]      |            newline:                                          '\n'
[L:  4, P:  1]      |            [META] dedent:
[L:  4, P:  1]      |            from_clause:
[L:  4, P:  1]      |                keyword:                                      'FROM'
[L:  4, P:  5]      |                whitespace:                                   ' '
[L:  4, P:  6]      |                from_expression:
[L:  4, P:  6]      |                    [META] indent:
[L:  4, P:  6]      |                    from_expression_element:
[L:  4, P:  6]      |                        table_expression:
[L:  4, P:  6]      |                            table_reference:
[L:  4, P:  6]      |                                naked_identifier:             'source_table'
[L:  4, P: 18]      |                    [META] dedent:
[L:  4, P: 18]      |    newline:                                                  '\n'
[L:  5, P:  1]      |    [META] end_of_file:

Dialect

dialect = databricks
templater = jinja

Version

sqlfluff, version 1.4.5
Python 3.7.13

Configuration

[sqlfluff]
verbose = 0
nocolor = False
dialect = databricks
templater = jinja
rules = all
exclude_rules = L031,L056
recurse = 0
output_line_length = 120
runaway_limit = 10
ignore = None
warnings = None
ignore_templated_areas = True
encoding = autodetect
disable_noqa = False
sql_file_exts = .sql
fix_even_unparsable = False
large_file_skip_char_limit = 0
large_file_skip_byte_limit = 20000
processes = 0

[sqlfluff:indentation]
indented_joins = False
indented_ctes = False
indented_using_on = True
indented_on_contents = True
template_blocks_indent = True

[sqlfluff:layout:type:comma]
spacing_before = touch
line_position = trailing

[sqlfluff:layout:type:binary_operator]
spacing_within = touch
line_position = leading

[sqlfluff:layout:type:statement_terminator]
spacing_before = touch
line_position = trailing

[sqlfluff:layout:type:end_of_file]
spacing_before = touch

[sqlfluff:layout:type:set_operator]
line_position = alone:strict

[sqlfluff:layout:type:start_bracket]
spacing_after = touch

[sqlfluff:layout:type:end_bracket]
spacing_before = touch

[sqlfluff:layout:type:start_square_bracket]
spacing_after = touch

[sqlfluff:layout:type:end_square_bracket]
spacing_before = touch

[sqlfluff:layout:type:casting_operator]
spacing_before = touch
spacing_after = touch

[sqlfluff:layout:type:comparison_operator]
spacing_within = touch
line_position = leading

[sqlfluff:layout:type:assignment_operator]
spacing_within = touch
line_position = leading

[sqlfluff:layout:type:object_reference]
spacing_within = inline

[sqlfluff:layout:type:comment]
spacing_before = any
spacing_after = any

[sqlfluff:layout:type:template_loop]
spacing_before = any
spacing_after = any

[sqlfluff:templater]
unwrap_wrapped_queries = True

[sqlfluff:templater:jinja]
apply_dbt_builtins = True

[sqlfluff:rules]
tab_space_size = 4
max_line_length = 120
indent_unit = space
allow_scalar = True
single_table_references = unqualified
unquoted_identifiers_policy = all

[sqlfluff:rules:L003]
hanging_indents = False

[sqlfluff:rules:L010]
capitalisation_policy = upper
ignore_words = None
ignore_words_regex = None

[sqlfluff:rules:L011]
aliasing = explicit

[sqlfluff:rules:L012]
aliasing = explicit

[sqlfluff:rules:L014]
extended_capitalisation_policy = lower
ignore_words = None
ignore_words_regex = None

[sqlfluff:rules:L016]
ignore_comment_lines = False
ignore_comment_clauses = False

[sqlfluff:rules:L026]
force_enable = False

[sqlfluff:rules:L027]
ignore_words = None
ignore_words_regex = None

[sqlfluff:rules:L028]
force_enable = False

[sqlfluff:rules:L029]
unquoted_identifiers_policy = all
quoted_identifiers_policy = none
ignore_words = None
ignore_words_regex = None

[sqlfluff:rules:L030]
extended_capitalisation_policy = upper
ignore_words = None
ignore_words_regex = None

[sqlfluff:rules:L031]
force_enable = False

[sqlfluff:rules:L036]
wildcard_policy = single

[sqlfluff:rules:L038]
select_clause_trailing_comma = forbid

[sqlfluff:rules:L040]
capitalisation_policy = upper
ignore_words = None
ignore_words_regex = None

[sqlfluff:rules:L042]
forbid_subquery_in = both

[sqlfluff:rules:L047]
prefer_count_1 = False
prefer_count_0 = False

[sqlfluff:rules:L051]
fully_qualify_join_types = both

[sqlfluff:rules:L052]
multiline_newline = False
require_final_semicolon = False

[sqlfluff:rules:L054]
group_by_and_order_by_style = implicit

[sqlfluff:rules:L057]
unquoted_identifiers_policy = all
quoted_identifiers_policy = all
allow_space_in_identifier = False
additional_allowed_characters = None
ignore_words = None
ignore_words_regex = None

[sqlfluff:rules:L059]
prefer_quoted_identifiers = False
ignore_words = None
ignore_words_regex = None
force_enable = False

[sqlfluff:rules:L062]
blocked_words = None
blocked_regex = None

[sqlfluff:rules:L063]
extended_capitalisation_policy = upper
ignore_words = None
ignore_words_regex = None

[sqlfluff:rules:L064]
preferred_quoted_literal_style = single_quotes
force_enable = False

[sqlfluff:rules:L066]
min_alias_length = 4
max_alias_length = None

[sqlfluff:rules:L067]
preferred_type_casting_style = shorthand

Are you willing to work on and submit a PR to address the issue?

  • Yes I am willing to submit a PR!

Code of Conduct
@julianopiovezan julianopiovezan added the bug Something isn't working label Feb 3, 2023
@github-actions github-actions bot added the sparksql Related to the SparkSQL (aka Spark3) dialect label Feb 3, 2023
@keraion
Copy link
Contributor

keraion commented Aug 7, 2024

Fixed by #5226

@keraion keraion closed this as completed Aug 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working sparksql Related to the SparkSQL (aka Spark3) dialect
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@julianopiovezan @keraion