spark sql 数据类型转换,通过类型的大小写匹配根据SparkSQL中的类型转换值

Is it possible to match a parametric type in Scala? Lets say I have a function that receives two parameters: a value and a type. I would like to use pattern matching to do a type conversion.

Something like this:

datatype match {

case IntegerType => return value.toInt

case FloatType => return value.toFloat

case StringType => return value

case DecimalType(_,_) => return BigDecimal(value) // this is not working

case _ => return strrepr

}

Here DecimalType accepts two parameters to specify precision the required precision. It can be for example:

org.apache.spark.sql.types.DecimalType = DecimalType(10,2)

I have tried several options and nothing seems to be working:

For case DecimalType => return BigDecimal(value) I get:

error: pattern type is incompatible with expected type;

found : org.apache.spark.sql.types.DecimalType.type

required: org.apache.spark.sql.types.DataType

Note: if you intended to match against the class, try `case DecimalType(_,_)`

For case DecimalType(_,_) => return BigDecimal(value) I get:

error: result type Boolean of unapply defined in method unapply in object DecimalType does not conform to Option[_] or Boolean

For case DecimalType[_,_] => return BigDecimal(value) I get:

error: org.apache.spark.sql.types.DecimalType does not take type parameters

解决方案

The problem is the use of the return in your code. You said you use this code snippet in a function somewhere. What is the return type of that function? Obviously, you intend that sometimes it is Integer, sometimes String, sometimes BigDecimal; but if you use return, it will look to the type of the returned object to determine the return type of the function. In general, you should strongly avoid using return in Scala code. The last evaluated value in the function body is returned. The only case for using a return is when you want to force returning a value somewhere else in the function body. But still, a better way would be to save the return object in a variable and just evaluate that variable in the last line of your function body. And never use return!

Without return it works

scala> val datatype = DecimalType(10, 2)

datatype: org.apache.spark.sql.types.DecimalType = DecimalType(10,2)

scala> val value = BigDecimal(10)

value: scala.math.BigDecimal = 10

scala> datatype match {case DecimalType(_,_) => value}

res150: scala.math.BigDecimal = 10

** Problems with return **

scala> def test = {datatype match {case DecimalType(_,_) => return value}}

:138: error: method test has return statement; needs result type

def test = {datatype match {case DecimalType(_,_) => return value}}

scala> def test:BigDecimal = {datatype match {case DecimalType(_,_) => return value}}

test: BigDecimal

scala> def test:DataType = {datatype match {case DecimalType(_,_) => return value}}

:138: error: type mismatch;

found : scala.math.BigDecimal

required: org.apache.spark.sql.types.DataType

def test:DataType = {datatype match {case DecimalType(_,_) => return value}}

scala> def test3 = {datatype match {case DecimalType(_,_) => value}}

test3: scala.math.BigDecimal