How Does pyspark deal with Spaces in Queries

[cpatte7372]

Hello community,

Can someone let me know how pyspark.sql deals with spaces?

For example the following query won't work:

myresults = spark.sql("""SELECT  Name AS [Product Name] FROM Production_vProductAndDescription""")

As you can see above the I have tried to concantenate Product Name with square brackets, but I get error:

Error:
== SQL ==
SELECT  Name AS [Product Name] FROM Production_vProductAndDescription
----------------^^^

However, with no spaces between Product and Name (without the square brackets as follows it works:

myresults = spark.sql("""SELECT  Name AS Product_Name FROM Production_vProductAndDescription""")

Any thoughts welcomed

Cheers

Carlton

[micseydel]

I believe this is the same issue as I said in your other thread. It seems like you're used to MSSQL or something and are now using MySQL, Postgres, or something else with a different SQL dialect than you're used to. I don't think this is PySpark-specific.

[cpatte7372]

hi micseydel,

The answer is to change [Product Name] to Product Name

Thanks anyway for reaching out.

Cheers

[micseydel]

I understand that. That's consistent with what I'm saying. It's the SQL dialect, not PySpark. I've dabbled with MSSQL, I've never seen the syntax you're trying to use. You need to identify the dialect of the database you're using and familiarize yourself with it, or you're going to hit a great deal of blockers like this.