[FEA] Add support for escape characters in CSV #11984
Labels
0 - Backlog
In queue waiting for assignment
cuIO
cuIO issue
feature request
New feature or request
libcudf
Affects libcudf (C++/CUDA) code.
Spark
Functionality that helps Spark RAPIDS
Milestone
Is your feature request related to a problem? Please describe.
By default Spark uses an escape character
\
to escape things like double quotes within other double quotes i.e."\""
. Pandas by default uses two double quotes right next to each other. Spark does not support this method of escaping.Describe the solution you'd like
We would like an option in the csv reader setting to allow setting an escape character. There should be an option to either disable the double double quote escaping, or have it automatically disabled if an escape character is provided.
Describe alternatives you've considered
write our own CSV parser.
The text was updated successfully, but these errors were encountered: