Donate. I desperately need donations to survive due to my health

Get paid by answering surveys Click here

Click here to donate

Remote/Work from Home jobs

Scala Convert Array[String] to List[Map[String,Double]]

I have a text file and data is in string format like below

{"a":1,"b":2678,"c":40,"d":2}
{"a":4,"b":267,"c":26,"d":2}
{"a":1,"b":2678,"c":90,"d":17}
{"a":4,"b":2,"c":7,"d":15}

I read the file and converted into Array[String] using below code

val data = Source.fromFile(key+".txt").mkString.split("\n")

and o/p is like

[{"a":1,"b":2678,"c":40,"d":2},{"a":4,"b":267,"c":26,"d":2},{"a":1,"b":2678,"c":90,"d":17},{"a":4,"b":2,"c":7,"d":15}]

Till now, everything is fine and very fast,

then I wrote a function to convert every element into map

 def convertToMap(msg:String): Map[String,Double] ={

  var parseMsg = JSON.parseFull(msg)

    var parseMsgToMap:Map[String,Any] = parseMsg match {
      case Some(data: Map[String, Any]) => data
    }

    return parseMsgToMap.mapValues(_.toString.toDouble)

}

Then I called above function and converted result to list

var accumulatedData = data.map(x => convertToMap(x)).toList

But because I am using map and calling convertToMap function for each element, and converting the result to list.. It is consuming hell lot of time... approx 0.5ms for each element, I have around 200k records and it is taking 45 secs to convert to map

I am figuring out the most efficient way of converting it into List[Map[String,Double]] It should be very very fast

I need to convert this in list of Map

Comments