Mapping JSON values to enum fields is broken #205
Comments
|
This is unfortunate; thanks for the report. I don't think downgrading is the right solution, especially since I'd like to support some of the 1.8 features like the logical timestamp types. Instead I'd like to see us work around the issue (with test cases). Some things the test-cases need to check:
For invalid values I think it should be treated as an empty value instead of throwing an exception. |
|
It turns out the problem is not related to the Avro version; the linked report describes a different problem. Downgrading will not resolve the problem. I've uncovered a few problems with the enum support as things stand:
With this in mind I'm going to update this issue to address the former. Extending the mapping DSL so that string-based values can be applied to enum fields is an enhancement and I've created issue #208 to deal with that. |
asnare
changed the title
|
Hello, First of all, thank you for having a look at this! I would like to respectfully disagree about your conclusion that downgrading Avro will not help in this specific situation. Note that I'm not claiming that doing that is a good idea otherwise (I agree with your argument about supporting logical timestamp values). While stock Divolte distribution (0.7.0 and also 0.8.0) throws "org.apache.avro.AvroTypeException", there is no such exception thrown (and Avro file is sucessfully written) if I manually replace avro-1.8.2.jar with avro-1.7.7.jar. My mapping is included bellow: BTEventRecord.avsc.txt The exception I'm seeing is: |
|
You're right. Downgrading to pre-Avro-1.8.0 does resolve the problem; I missed that AVRO-997 introduced a breaking-change with 1.8.0 that triggers this issue. |
|
This issue has been resolved by #211. |
It seems that enums are not working in the latest (1.8.2) version of Avro which is bundled with divolte-collector-0.7.0. When collecting data defined as enum, Divolte Collector throws the following exception: org.apache.avro.AvroTypeException: Not an enum: OW
Additional information is available at https://issues.apache.org/jira/browse/AVRO-1810. One possible solution is to downgrade Avro to 1.7.7.
For the reference, I'm using the following Avro field definition:
{ "name": "search", "type": [ "null", { "name": "SearchRecord", "type": "record", "fields": [ { "name": "screenName", "type": "string", "doc": "Screen name (e.g. fb_avail)" }, { "name": "language", "type": "string", "doc": "Language code in ISO 639-1 (lv, et)" }, { "name": "pointOfSale", "type": "string", "doc": "Point of Sale, for example, LV" }, { "name": "origin", "type": "string", "doc": "Origin, e.g. RIX" }, { "name": "destination", "type": "string", "doc": "Destination, e.g. TLL" }, { "name": "tripType", "type": {"name": "TripType", "type": "enum", "symbols" : ["OW", "RT"]}, "doc": "RT for round-trip, OW for one-way" }, { "name": "outboundDate", "type": "string", "doc": "Outbound date in format 2018-04-29" }, { "name": "inboundDate", "type": ["null", "string"], "default": null, "doc": "Inbound date in format 2018-04-29" }, { "name": "adults", "type": "int", "doc": "Number of adults" }, { "name": "children", "type": ["null", "int"], "default": null, "doc": "Number of children" }, { "name": "infants", "type": ["null", "int"], "default": null, "doc": "Number of infants" } ] } ], "default": null }The text was updated successfully, but these errors were encountered: