I am performing queries on a dataframe of 15gb data. After complete execution it shows time taken is 16min but if I perform the same operations and at end write the result on datalake store in csv file. Even the data is reduced and becomes 1gb it takes double time that is 38min. Any Idea?
Donate. I desperately need donations to survive due to my health
Get paid by answering surveys Click here
Click here to donate
Remote/Work from Home jobs
Comments
Post a Comment