Home / Spark / Query compilation
Spark: [UNRESOLVED_COLUMN] A column, variable, or function parameter cannot be resolved
org.apache.spark.sql.AnalysisException[UNRESOLVED_COLUMN.WITH_SUGGESTION] A column, variable, or function parameter with name `usr_id` cannot be resolved. Did you mean one of the following? [`user_id`, `amount`, `dt`].Spark could not find a column with that name in the tables of the query. The full message usually adds "Did you mean one of the following?" with the closest names, which is the quickest way to spot a typo or a case problem.
Common causes
- A typo, or a column that was renamed or dropped upstream.How to confirm: Compare with the suggested names in the message, or run DESCRIBE on the table.
- A column alias from the SELECT list is used in WHERE or GROUP BY of the same query level.How to confirm: The name only exists as an alias in the same SELECT.
- Column names with special characters or dots are not quoted.How to confirm: The name contains a dot, space or dash; quote it with backticks.
How to fix it
Use the name from the suggestion list, quote unusual names with backticks, and qualify ambiguous columns with their table alias.
Where the message comes from
error-conditions.jsoninapache/sparkatv4.1.3- Message checked in Spark 4.1.3 (error condition UNRESOLVED_COLUMN); Spark 3.3 and earlier said "cannot resolve"
Message template in the source: A column, variable, or function parameter with name <objectName> cannot be resolved.