Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

quarkus-rest-jackson doesn't support @JsonManagedReference/@JsonBackReference #45753

Closed
nicolasduminil opened this issue Jan 21, 2025 · 19 comments
Labels
area/jackson Issues related to Jackson (JSON library) kind/bug Something isn't working

Comments

@nicolasduminil
Copy link

nicolasduminil commented Jan 21, 2025

Describe the bug

Hello,

I'm facing a strange behavior with Quarkus 3.17.6 and quarkus-rest-jackson extension. A REST service CRUDs a couple of JPA entities. These entities maintain a bi-directional one-to-many relationship. In order to avoid infinite recursion, the Jackson @JsonManagedReference/@JsonBackReference annotation are used.

Trying to serialize payloads containing these entities, the following exception is raised:

2025-01-21 15:41:44,261 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (executor-thread-1) HTTP Request to /customers failed, error id:  3cb2f243-8560-4e34-865b-6ffa1b214a3b-1: java.lang.IllegalArgumentException: Cannot handle managed/back reference 'defaultReference':  type: value deserializer of type com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializerNR does not support them
    at com.fasterxml.jackson.databind.JsonDeserializer.findBackReference(JsonDeserializer.java:433)
    at com.fasterxml.jackson.databind.deser.std.ContainerDeserializerBase.findBackReference(ContainerDeserializerBase.java:104)
    at com.fasterxml.jackson.databind.deser.BeanDeserializerBase._resolveManagedReferenceProperty(BeanDeserializerBase.java:968)
    at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.resolve(BeanDeserializerBase.java:566)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCache2(DeserializerCache.java:347)
    at com.fasterxml.jackson.databind.deser.DeserializerCache._createAndCacheValueDeserializer(DeserializerCache.java:284)
    at com.fasterxml.jackson.databind.deser.DeserializerCache.findValueDeserializer(DeserializerCache.java:174)
    at com.fasterxml.jackson.databind.DeserializationContext.findRootValueDeserializer(DeserializationContext.java:669)
    at com.fasterxml.jackson.databind.ObjectReader._prefetchRootDeserializer(ObjectReader.java:2450)
    at com.fasterxml.jackson.databind.ObjectReader.forType(ObjectReader.java:771)
...

Expected behavior

I expect that quarkus-rest-jackson be able to use @JsonManagedReference/@JsonBackReference and to successfuly serialize the entities.

In order to make sure that this should work, I implemented exactly the same test case using Jakarta EE 10 and it works as expected.

Actual behavior

The Quarkus implementation doesn't work and raises the mentioned exception, as opposed to the Jakarta EE implemntation, whichworks as expected.

How to Reproduce?

Reproducer: https://github.com/nicolasduminil/orms.git

The repository has two projects: quarkus-orm which reproduces the issue and jakarta-orm which works as expected.

The README.md file in the repository contains instructions of how to reproduce.

Output of uname -a or ver

Linux nicolas-XPS-15-9570 6.8.0-51-generic #52-Ubuntu SMP PREEMPT_DYNAMIC Thu Dec 5 13:09:44 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Output of java -version

java version "21.0.3" 2024-04-16 LTS Java(TM) SE Runtime Environment (build 21.0.3+7-LTS-152) Java HotSpot(TM) 64-Bit Server VM (build 21.0.3+7-LTS-152, mixed mode, sharing)

Quarkus version or git rev

3.17.6

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.9.5 (57804ffe001d7215b5e7bcb531cf83df38f93546) Maven home: /opt/apache-maven-3.9.5 Java version: 21.0.3, vendor: Oracle Corporation, runtime: /usr/lib/jvm/jdk-21-oracle-x64 Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "6.8.0-51-generic", arch: "amd64", family: "unix"

Additional information

N/A

@nicolasduminil nicolasduminil added the kind/bug Something isn't working label Jan 21, 2025
@quarkus-bot quarkus-bot bot added the area/jackson Issues related to Jackson (JSON library) label Jan 21, 2025
Copy link

quarkus-bot bot commented Jan 21, 2025

/cc @geoand (jackson), @gsmet (jackson), @mariofusco (jackson)

@nicolasduminil
Copy link
Author

Have tried to replace @JsonManagedReference/@JsonBackReference by @JsonIgnoreProperties but, in this case, I got:

 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (executor-thread-1) HTTP Request to /customers failed, error id: 66abbcbc-0bff-43a0-a815-96b50bb4717b-1: org.hibernate.property.access.spi.PropertyAccessException: Error accessing field [java.lang.Long fr.simplex_software.workshop.quarkus.orm.data.Order.id] by reflection for persistent property [fr.simplex_software.workshop.quarkus.orm.data.Order#id] : {id=null, item=myItem01, price=150}
    at org.hibernate.property.access.spi.GetterFieldImpl.get(GetterFieldImpl.java:52)
    at org.hibernate.metamodel.mapping.internal.BasicEntityIdentifierMappingImpl.getIdentifier(BasicEntityIdentifierMappingImpl.java:158)
...
Caused by: java.lang.IllegalArgumentException: Can not get java.lang.Long field fr.simplex_software.workshop.quarkus.orm.data.Order.id on java.util.LinkedHashMap

Not sure where this LinkedMap comes from as there is none in my case.

@geoand
Copy link
Contributor

geoand commented Jan 22, 2025

Thanks a lot for the reproducer!

I could indeed reproduce the quarkus issue (which is also present if you replace quarkus-rest-jackson with quarkus-resteasy-jackson), but I was not able to run the Widlfly tests as I got:

com.github.dockerjava.api.exception.NotFoundException: Status 404: {"message":"pull access denied for wildfly-bootable/jakarta-orm, repository does not exist or may require 'docker login': denied: requested access to the resource is denied"}

That said, I have zero clue what is causing the original problem...

@nicolasduminil
Copy link
Author

nicolasduminil commented Jan 22, 2025

... I was not able to run Wildfly tests ...

The Docker image built during the package phase of maven hasn't been found locally. Are you sure to have followed the instructions in the README.md ? You probably tried to run the test before creating the Docker image. So you need to run:

$ mvn -DskipTests clean package  # creates the Docker image
$ mvn test   # runs the Docker image created previously which contains Wildfly Bootable + the app

But anyway, the Wildfly tests are only meant to show that that's probably not RESTeasy responsible of the issue but its Quarkus extension.

... I have zero clue what is causing the original problem ....

IMHO what's causing the issue is that, as the exception is saying, the Jackson extension doesn't support @JsonManagedReference/@JsonBackReference. And since @JsonIgnoreProperties doesn't seem to work neither, I need to know what's the Quarkus recommended way to avoid infinite recursion while serializing bidirectional one-to-many relationships ?

@geoand
Copy link
Contributor

geoand commented Jan 22, 2025

IMHO what's causing the issue is that, as the exception is showing, the Jackson extension doesn't support @JsonManagedReference/@JsonBackReference. And since @JsonIgnoreProperties doesn't seem to work neither,

There is nothing specific that Quarkus does to support any Jackson features - we just configure and use ObjectMapper.

I need to know what's the Quarkus recommended way to avoid infinite recursion wile serializing bidirectional one-to-many relationships

Quarkus doesn't provide anything OOTB for supporting serialization / deserialization of ORM entities with Jackson, but I know this topic has come up before. Perhaps @yrodiere remember where.

@yrodiere
Copy link
Member

I need to know what's the Quarkus recommended way to avoid infinite recursion wile serializing bidirectional one-to-many relationships

Quarkus doesn't provide anything OOTB for supporting serialization / deserialization of ORM entities with Jackson, but I know this topic has come up before. Perhaps @yrodiere remember where.

The main recommendation is not to do it at all, because of this infinite recursion problem, but also because that's unsafe (implies database access outside of transactions). Instead, you should map your entities to DTOs before returning them from your REST endpoint, either explicitly (e.g. with mapstruct) or by leveraging "projection" features (e.g. entityManager.createQuery("from MyEntity", MyEntityDTO.class)) to have queries return DTO directly. See #44918 for details, or to join the argument :)

@nicolasduminil
Copy link
Author

nicolasduminil commented Jan 22, 2025

What I'm trying to say is that the same test case that fails when run with Quarkus, succeeds when run with Jakarta EE and Wildfly. Meaning that used with io.quarkus:quarkus-rest-jakson:3.17.6 the test fails, while used with org.jboss.reseasy:resteasy-jackson2-provider:6.2.9.Final, it succeeds.

@nicolasduminil
Copy link
Author

nicolasduminil commented Jan 22, 2025

@yrodiere

The main recommendation is not to do it at all

Sure, if you don't do it, you don't have the issue. :-)

... because that's unsafe ...

That's not the issue here.

... implies database access outside of transactions ...

Not sure where din you see something like this in the reproducer but this isn't at all the case.

Instead, you should map your entities to DTOs ...

Doing things differently and working around an issue doesn't fix the issue. While using DTOs is recommended, this isn't an option in this case, for reasons that I won't develop here. And anyway, using JPA entities is perfectly legal, DTOs aren't mandatory.

@yrodiere
Copy link
Member

yrodiere commented Jan 22, 2025

... implies database access outside of transactions ...

Not sure where din you see something like this in the reproducer but this isn't at all the case.

Indeed it isn't, I jumped to conclusions because using entities as parameters / return type on REST endpoint methods is the only use case that's ever been presented to me.
What you're doing is indeed not affected by this particular problem.

Instead, you should map your entities to DTOs ...

Doing things differently and working around an issue doesn't fix the issue. While using DTOs is recommended, this isn't an option in this case, for reasons that I won't develop here. And anyway, using JPA entities is perfectly legal, DTOs aren't mandatory.

Indeed. In that case I'll just say I don't know how to address this use case, as I've never seen it. Not a single bit helpful either, but maybe less annoying.

That being said, looking at your reproducer, the Jakarta one is cheating, as it serializes the customer of an order manually: https://github.com/nicolasduminil/orms/blob/647584734eaed170abdc4b34220bf9529f681457/jakarta-orm/src/main/java/fr/simplex_software/workshop/jakarta/orm/data/serializers/OrderListSerializer.java
Any reason not to use the same workaround in Quarkus?

EDIT: Though that class doesn't seem referenced anywhere... Some Jakarta EE magic? 🤔

@nicolasduminil
Copy link
Author

nicolasduminil commented Jan 22, 2025

@yrodiere

...the Jakarta one is cheating, as it serializes the customer of an order manually...

No, it doesn't. You might be thinking that because you've seen this serailzers package that I forgot there while preparing the simplified reproducer. Now, I removed this package which seems to confuse you.

And you may be sure that I wouldn't provide a cheating reproducer.

@yrodiere
Copy link
Member

I wasn't accusing you 🙄

Anyway, I'll get of your way, since it doesn't look like these concerns are really Hibernate-related, and I know next to nothing about Jackson. All the best to you.

@nicolasduminil
Copy link
Author

nicolasduminil commented Jan 22, 2025

Just a short update to let you know that the same Quarkus test, using XML instead of JSON serialization, i.e. using io.quarkus:quarkus-rest-jaxb:3.17.6 instead of io.quarkus:quarkus-rest-jackson:3.17.6, works as expected. Of course, there aren't @JsonManagedReference/@JsonBackReference annotations in this case but the approximative "equivalent" for XML, which is @XmlTransient.

I updated the reproducer as well.

@nicolasduminil
Copy link
Author

nicolasduminil commented Jan 23, 2025

@yrodiere

...it doesn't look like these concerns are really Hibernate-related...

There is a 2nd concern mentioned above as I replaced, in the Quarkus test, @JsonManagedReference/@JsonBackReference by @JsonIgnoreProperties and got the following:

ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (executor-thread-1) HTTP Request to /customers failed, error id: 66abbcbc-0bff-43a0-a815-96b50bb4717b-1: org.hibernate.property.access.spi.PropertyAccessException: Error accessing field [java.lang.Long fr.simplex_software.workshop.quarkus.orm.data.Order.id] by reflection for persistent property [fr.simplex_software.workshop.quarkus.orm.data.Order#id] : {id=null, item=myItem01, price=150}
at org.hibernate.property.access.spi.GetterFieldImpl.get(GetterFieldImpl.java:52)
at org.hibernate.metamodel.mapping.internal.BasicEntityIdentifierMappingImpl.getIdentifier(BasicEntityIdentifierMappingImpl.java:158)
...
Caused by: java.lang.IllegalArgumentException: Can not get java.lang.Long field fr.simplex_software.workshop.quarkus.orm.data.Order.id on java.util.LinkedHashMap

IMHO, this one seems to be related to Hibernate, right ? I'm not saying it's a bug but I think that we should try to understand how come that most of Jackson specific annotations don't seem to be supported by the Quarkus Jackson RESTeasy extension, while they are supported by the Wildfly RESTeasy Jackson binder ?

@yrodiere
Copy link
Member

yrodiere commented Jan 23, 2025

IMHO, this one seems to be related to Hibernate, right ? I'm not saying it's a bug but I think that we should try to understand how come that most of Jackson specific annotations don't seem to be supported by the Quarkus Jackson RESTeasy extension, while they are supported by the Wildfly RESTeasy Jackson binder ?

Would be interesting to see the rest of the stack trace. It looks like some Hibernate ORM internals are used to retrieve an entity identifier, but they get passed a Map instead of the entity instance that they would need to work with. Something up the stack mixed up their data. I don't know of anything in Hibernate ORM that would turn entity data to a Map, but I can imagine Jackson doing that, and I can imagine a Jackson "Hibernate" module (or something in the Quarkus Jackson extension) somehow calling Hibernate to serialize entities...

@gsmet
Copy link
Member

gsmet commented Jan 23, 2025

If I had one thing to recommend to understand the initial issue, it would be to decompile the bytecode for the entities in both cases (Quarkus and WildFly) and compare them.

I wouldn't be surprised if Quarkus was applying some magic that makes Jackson unhappy, either via the Hibernate bytecode enhancement or something Quarkus-specific.

@geoand
Copy link
Contributor

geoand commented Jan 23, 2025

My suggestion would be to debug the working Wldfly tests vs the non working Quarkus one and see why Jackson is behaving differently.
Note however that the comparison is likely between different Hibernate and Jackson versions

@gsmet
Copy link
Member

gsmet commented Jan 23, 2025

FWIW, I applied this change:

diff --git a/quarkus-orm/src/main/java/fr/simplex_software/workshop/quarkus/orm/data/Customer.java b/quarkus-orm/src/main/java/fr/simplex_software/workshop/quarkus/orm/data/Customer.java
index 4bfebea..cac6bad 100644
--- a/quarkus-orm/src/main/java/fr/simplex_software/workshop/quarkus/orm/data/Customer.java
+++ b/quarkus-orm/src/main/java/fr/simplex_software/workshop/quarkus/orm/data/Customer.java
@@ -52,7 +52,7 @@ public class Customer
   @JsonManagedReference
   @XmlTransient
   @JsonProperty
-  public List<Order> orders = new ArrayList<>();
+  private List<Order> orders = new ArrayList<>();
 
   public Customer() {}
 

and I have a different error:

	Caused by: [CIRCULAR REFERENCE: org.hibernate.exception.ConstraintViolationException: could not execute statement [Unique index or primary key violation: "PUBLIC.CONSTRAINT_INDEX_6 ON PUBLIC.CUSTOMERS(EMAIL NULLS FIRST) VALUES ( /* 1 */ '[email protected]' )"; SQL statement:
insert into CUSTOMERS (EMAIL,FIRST_NAME,LAST_NAME,PHONE,ID) values (?,?,?,?,?) [23505-230]] [insert into CUSTOMERS (EMAIL,FIRST_NAME,LAST_NAME,PHONE,ID) values (?,?,?,?,?)]]

@nicolasduminil
Copy link
Author

@gsmet : this is already much clearer and I need to check as it seems to be just an unique field duplicate error. However, I'm surprised that by simply changing the visibility of that property, the behavior changes dramatically as well.

@nicolasduminil
Copy link
Author

@gsmet: yeah, this was it, changing to private the visibility of List<Order> orders and addressing this unique value duplicates error fixed the issue, many thanks.

Is it expected that such an insignificant modification affects the whole behavior of the test case ?

In any case, I'm closing the issue as fixed.

Many thanks again for your help and support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/jackson Issues related to Jackson (JSON library) kind/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants