Skip to content

support CREATE/DROP STAGE for Snowflake#833

Merged
alamb merged 1 commit into
apache:mainfrom
pawel-big-lebowski:snowflake/create-drop-stage
Mar 26, 2023
Merged

support CREATE/DROP STAGE for Snowflake#833
alamb merged 1 commit into
apache:mainfrom
pawel-big-lebowski:snowflake/create-drop-stage

Conversation

@pawel-big-lebowski

Copy link
Copy Markdown
Contributor

Support snowflake CREATE STAGE syntax from here.

PR supports whole syntax from Snowflake docs except for two things:

  • [ [ WITH ] TAG ( <tag_name> = '<tag_value>' [ , <tag_name> = '<tag_value>' , ... ] ),
  • options with NULL_IF = ( '<string>' [ , '<string>' ... ] ).
    which can be added later on if necessary.

I would like later on to add support for other Snowflake loading/unloading syntaxes like COPY INTO.

I've created a separate file src/ast/helpers/stmt_data_loading.rs to contain newly created structs. These are required as syntax of data loading in Snowflake is pretty complex and I did not want to put them into existing mod.rs which is already quite massive.

@coveralls

coveralls commented Mar 10, 2023

Copy link
Copy Markdown

Pull Request Test Coverage Report for Build 4466156302

  • 274 of 290 (94.48%) changed or added relevant lines in 5 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.2%) to 86.281%

Changes Missing Coverage Covered Lines Changed/Added Lines %
src/parser.rs 2 3 66.67%
tests/sqlparser_snowflake.rs 117 118 99.15%
src/ast/helpers/stmt_data_loading.rs 28 31 90.32%
src/ast/mod.rs 23 28 82.14%
src/dialect/snowflake.rs 104 110 94.55%
Totals Coverage Status
Change from base Build 4456045946: 0.2%
Covered Lines: 13849
Relevant Lines: 16051

💛 - Coveralls

@pawel-big-lebowski pawel-big-lebowski marked this pull request as draft March 10, 2023 14:41
@pawel-big-lebowski pawel-big-lebowski force-pushed the snowflake/create-drop-stage branch 3 times, most recently from dcf9fd1 to e1c420a Compare March 13, 2023 07:47
@pawel-big-lebowski pawel-big-lebowski marked this pull request as ready for review March 13, 2023 08:10

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me -- thank you @pawel-big-lebowski

Any concerns @ankrgyl ?

@alamb

alamb commented Mar 18, 2023

Copy link
Copy Markdown
Contributor

@pawel-big-lebowski I think if you merge up from main the CI will pass (it was fixed in another PR)

Comment thread src/ast/mod.rs
},
/// ```sql
/// CREATE STAGE
/// ```

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread src/dialect/snowflake.rs Outdated
parser.expect_token(&Token::Eq)?;
url = Some(match parser.next_token().token {
Token::SingleQuotedString(word) => Ok(word),
_ => parser.expected("an URL statement", parser.peek_token()),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you say "a URL"?

Comment thread src/dialect/snowflake.rs Outdated
})
}

/// Parses options provided within parenthesis like:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be parentheses (not paranthesis) in the commend and function name below

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, will change it.

};
assert_eq!(snowflake().verified_stmt(sql).to_string(), sql);

let extended_sql = concat!(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add at least one test case that roundtrips CREATE [OR REPLACE] [TEMPORARY] STAGE (via one_statement_parses_to)?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am testing roundtrip with:

assert_eq!(snowflake().verified_stmt(sql).to_string(), sql);

In case it's not sufficient, please provide more details (or example) to help me understand what did you mean. one_statement_parses_to helps to verify if similar SQLs are semantically the same and I couldn't see how to fit this here.

@ankrgyl

ankrgyl commented Mar 18, 2023

Copy link
Copy Markdown
Contributor

left a few nit comments!

Signed-off-by: Pawel Leszczynski <leszczynski.pawel@gmail.com>
@pawel-big-lebowski pawel-big-lebowski force-pushed the snowflake/create-drop-stage branch from e1c420a to fe6cac7 Compare March 20, 2023 08:29

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me -- what do you think @ankrgyl ?

@alamb alamb merged commit 79c7ac7 into apache:main Mar 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants