As I am new to spark, can anyone give an example of spark structured streaming and spark SQL integration example.
I want to take some CSV files on streaming basis from a directory using spark streaming and I need to join the input data with one of the tables from hive using spark SQL and want to write the result in hive table.
I have searched for a similar example but couldn't find any. Only word count and Twitter streaming examples are available. So can anyone please provide sample code for solving my problem
Note: I am using Spark version 2.3
Comments
Post a Comment