Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pljava.ddr This script file related issues #491

Open
BruceBaiz opened this issue Jun 20, 2024 · 13 comments
Open

pljava.ddr This script file related issues #491

BruceBaiz opened this issue Jun 20, 2024 · 13 comments
Labels

Comments

@BruceBaiz
Copy link

When I import the Pljava-API dependency into java, at which point I can use some of the features of pljava in java, I can then type my project into a jar package with maven and install the jar package into the postgresql database using the install_jar method.
My first problem is that some of the annotations in the pljava-api dependencies, such as @function, @MappedUDT, @BaseUDT, etc., I can't figure out how to use them.
My second problem is that I don't understand what the pljava.ddr file is for. Will the contents of the.ddr script file be automatically executed when I install the jar package into postgresql? If so, some of the methods I defined in java or some of the entity types I defined in java such as Complex need to be used in postgresql, then how do I properly generate the.ddr file, How to correctly map a Complex class and its methods in java to a postgresql database. If not, how am I supposed to install some of the entity types, methods, operators, and cast I defined in java properly into postgresql for use?

I'm very new to pljava. Any help would be really appreciated.

java version - 11
postgresql version -14
apache maven - 3.9.6

@jcflack
Copy link
Contributor

jcflack commented Jun 20, 2024

My first problem is that some of the annotations in the pljava-api dependencies, such as @Function, @MappedUDT, @BaseUDT, etc., I can't figure out how to use them. My second problem is that I don't understand what the pljava.ddr file is for. Will the contents of the.ddr script file be automatically executed when I install the jar package into postgresql?

That's exactly what it's for. For that to happen, the pljava.ddr file needs to be included in the jar, and the jar manifest needs to have a stanza saying

Name: pljava.ddr
SQLJDeploymentDescriptor: true

and the deploy argument to install_jar needs to be true.

If you have not yet worked through the Hello, world example in PL/Java's user guide, I recommend starting there. The basic usage of the annotations and deployment of a jar with a pljava.ddr file all get explained there.

Going beyond the Hello, world, there are plenty of examples of the annotations in use to be found in PL/Java's own example classes.

The reference material for the annotations, you will find in the javadocs.

If so, some of the methods I defined in java or some of the entity types I defined in java such as Complex need to be used in postgresql, then how do I properly generate the.ddr file, How to correctly map a Complex class and its methods in java to a postgresql database.

If the above references do not get you started, you might want to look at the very thorough example constructed by Bear Giles in which he creates a complex and a rational data type, along with operators and aggregates supporting them. There is a pull request applied to his original example (which did not use annotations) that shows step-by-step how to convert it to the same example done with annotations.

@BruceBaiz
Copy link
Author

Hi,
I tried TO execute the SET client_min_messages TO debug5; But I still didn't get any valuable information.
Here's what I got after doing it in pgAdmin:

DEBUG: StartTransaction(1) name: unnamed; blockState: DEFAULT; state: INPROGRESS, xid/subid/cid: 0/1/0 Error: java.sql.SQLException LINE 1: INSERT into udt_test VALUES (1, '(1, 2)'); ^ ERROR: java.sql.SQLException SQL status: XX000 characters: 33

Here is the error stack when I inserted the udt using java's jdbc:

org.postgresql.util.PSQLException: ERROR: java.sql.SQLException
位置:29
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2725)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2412)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:371)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:502)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:419)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:341)
at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:326)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:302)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:297)
at com.alibaba.druid.pool.DruidPooledStatement.execute(DruidPooledStatement.java:635)
at com.example.PointDataBaseTest.test_01(PointDataBaseTest.java:22)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728)
at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:218)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:214)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:139)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1541)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141)
at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85)
at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47)
at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:63)
at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:57)
at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)

I have no idea how to solve this problem, and I would be grateful if you could give me a little help.

@jcflack
Copy link
Contributor

jcflack commented Jun 23, 2024

PL/Java code runs in the database server, not the client. So an exception stacktrace that matters will be one from PL/Java in the server, not one from JDBC in the client.

If you do

SET log_min_messages TO debug1;

on a new connection, before any first use of a PL/Java function, then PL/Java exceptions will include stacktraces. The stacktraces, however, only go to the database server's standard error channel. That will be included in the server log if logging_collector is on.

(In my earlier message I suggested to set client_min_messages rather than log_min_messages. Either one works to generate PL/Java stacktraces, but the stacktraces only go to the server's standard error either way, so perhaps it is clearer when I say log_min_messages.)

@BruceBaiz
Copy link
Author

Yes, I have tried
SET log_min_messages TO debug1;
and I have run the PL/Java function in my ubuntu server.And it still hint these:

Error: java.sql.SQLException
LINE 1: INSERT into udt_test VALUES (1, '(1, 2)');
^

ERROR: java.sql.SQLException
SQL status: XX000
Character: 33

If what I have done is still wrong, I would appreciate it if you could tell me in more detail what I should do.

@BruceBaiz
Copy link
Author

Yes, I have tried

SET log_min_messages TO debug1;

before any first use of a PL/Java function and I have run the PL/Java function in my ubuntu server.And it still hint these:

Error: java.sql.SQLException
LINE 1: INSERT into udt_test VALUES (1, '(1, 2)');
^

ERROR: java.sql.SQLException
SQL status: XX000
Character: 33

If what I have done is still wrong, I would appreciate it if you could tell me in more detail what I should do.

@jcflack
Copy link
Contributor

jcflack commented Jun 24, 2024

It will be difficult or impossible to help without the server-side (not client-side) stacktrace mentioned earlier.

@BruceBaiz
Copy link
Author

Hi,
That's the details of my stacktrace, After some analysis, I still don't know what the problem is with my parse function.

java.sql.SQLException
at [email protected]/org.postgresql.pljava.internal.EntryPoints.doPrivilegedAndUnwrap(EntryPoints.java:341)
at [email protected]/org.postgresql.pljava.internal.EntryPoints.udtParseInvoke(EntryPoints.java:298)
Caused by: java.lang.NullPointerException
at java.base/jdk.internal.math.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:1838)
at java.base/jdk.internal.math.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
at java.base/java.lang.Double.parseDouble(Double.java:543)
at schema:public//com.example.udt.Point.parse(Point.java:64)
at schema:public//com.example.udt.sql.PointUDT.parse(PointUDT.java:68)
at [email protected]/org.postgresql.pljava.internal.EntryPoints.lambda$udtParseInvoke$4(EntryPoints.java:286)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at [email protected]/org.postgresql.pljava.internal.EntryPoints.doPrivilegedAndUnwrap(EntryPoints.java:312)
... 1 more
2024-06-27 14:24:35.694 UTC [5827] postgres@postgres ERROR: java.sql.SQLException at character 33
2024-06-27 14:24:35.694 UTC [5827] postgres@postgres STATEMENT: INSERT into udt_test VALUES (1, '(1, 2)');

I would appreciate it if you could tell me in more detail what I should do.

@jcflack
Copy link
Contributor

jcflack commented Jun 27, 2024

Can you include the code of your PointUDT.parse method?

Also, I see that you are using Java 11. A feature in Java 14 and later, helpful NullPointerExceptions, can provide additional helpful information in a stacktrace like this one.

It is easy to change the Java version PL/Java will use. No rebuilding is required. It is enough to install another version of Java and change the setting of pljava.libjvm_location in PostgreSQL to point to the chosen version's libjvm shared object.

@BruceBaiz
Copy link
Author

This is my parse method:


public static Point parse(String input) {
        Pattern pattern =
                Pattern.compile("\\((-?[0-9]+(\\.[0-9]+)?), *(-?[0-9]+(\\.[0-9]+)?)\\)");
        Matcher matcher = pattern.matcher(input);
        if (!matcher.matches()) {
            throw new IllegalArgumentException(
                    "Unable to parse point from string \"" + input + '"');
        }
        if (matcher.groupCount() == 4) {
            return new Point(Double.parseDouble(matcher.group(1)),
                    Double.parseDouble(matcher.group(3)));
        }
        throw new IllegalArgumentException("invalid format: \"" + input + '"');
    }

 @Function(effects=IMMUTABLE, onNullInput=RETURNS_NULL)
    public static PointUDT parse(String input, String typeName) throws SQLException {
        Point point;
        try {
            point = Point.parse(input);
        } catch (IllegalArgumentException e) {
            throw new SQLException(e);
        }
        return new PointUDT(point);
    }

Sorry, I don't understand what you mean. Do you mean that I need to use a newer version of java or an older version of java? I would appreciate it if you could give me a more specific description.

Also, I see that you are using Java 11. A feature in Java 14 and later, helpful NullPointerExceptions, can provide additional helpful information in a stacktrace like this one.

It is easy to change the Java version PL/Java will use. No rebuilding is required. It is enough to install another version of Java and change the setting of pljava.libjvm_location in PostgreSQL to point to the chosen version's libjvm shared object.

@jcflack
Copy link
Contributor

jcflack commented Jun 28, 2024

Sorry, I don't understand what you mean. Do you mean that I need to use a newer version of java or an older version of java?

You don't need to do anything. Java 11 is ok. I merely mentioned it is rather old, and newer versions have added useful features. If you use Java 14 or later, you get extra information with NullPointerException that can help with finding the problem.

But that's not critical for this case: the exception is coming from the Java library. In Java 11 you just get NullPointerException and in Java 14 or later it would say

NullPointerException: Cannot invoke "String.trim()" because "s" is null

In any case, it's clear that a null argument has been passed to Double.parseDouble.

Can you attach the jar file you have installed into PostgreSQL for this test?

@BruceBaiz
Copy link
Author

This is the jar package I installed into PostgreSQL.
udt_test.zip

You don't need to do anything. Java 11 is ok. I merely mentioned it is rather old, and newer versions have added useful features. If you use Java 14 or later, you get extra information with NullPointerException that can help with finding the problem.

I used to use jdk version 17, but when working with a database in PostgreSQL, I was told that the version did not match.

But that's not critical for this case: the exception is coming from the Java library. In Java 11 you just get NullPointerException and in Java 14 or later it would say

Thank you very much for providing me with the above information. But I am only a beginner and it may be difficult for me to understand this official document. From the parse function I can't really see why an empty argument was passed to Double.parseDouble. I would appreciate it if you could give me a more specific description.

@jcflack
Copy link
Contributor

jcflack commented Jun 29, 2024

I used to use jdk version 17, but when working with a database in PostgreSQL, I was told that the version did not match.

Who, or what, told you that? If it was a message you were shown, can you include it?

@jcflack
Copy link
Contributor

jcflack commented Jun 29, 2024

In the source code you attached in an earlier comment, Point.parse contains this line:

            return new Point(Double.parseDouble(matcher.group(1)),
                    Double.parseDouble(matcher.group(3)));

That looks correct as far as I can see. The group(1) is the whole subpattern matching the first numeric value, and the group(3) is the whole subpattern matching the second numeric value.

However, when I inspect the jar file you attached:

javap -classpath udt_test/UDT.jar -c com.example.udt.Point

I think that this jar file does not correspond to the source code you showed earlier. It looks as if you made a later edit to the source, but didn't build a new jar file with the recompiled classes to load into PostgreSQL.

The key difference is in how the parameters to the Point constructor are being computed. The first one is computed this way:

      45: aload_2
      46: iconst_1
      47: invokevirtual #22                 // Method java/util/regex/Matcher.group:(I)Ljava/lang/String;
      50: invokestatic  #23                 // Method java/lang/Double.parseDouble:(Ljava/lang/String;)D

(The aload_2 loads the Matcher reference held in local slot 2, and iconst_1 is of course the integer constant 1.) So the invokevirtual calls matcher.group with the argument 1, and that is the same as in the source you showed.

However, the second constructor parameter is computed this way:

      56: aload_2
      57: iconst_4
      58: invokevirtual #22                 // Method java/util/regex/Matcher.group:(I)Ljava/lang/String;
      61: invokestatic  #23                 // Method java/lang/Double.parseDouble:(Ljava/lang/String;)D

This code is calling matcher.group(4), which is different from the source you showed that calls matcher.group(3). The group(4) is the subpattern that matches only the optional fractional part of the y coordinate, and because in INSERT into udt_test VALUES (1, '(1, 2)') the y coordinate has no fractional part, group(4) returns null, and that null being passed to parseDouble is the cause of the exception.

I think if you make sure to compile your current source code (as you showed it earlier), and build a jar file from the new compiled classes, and use sqlj.replace_jar in PostgreSQL to put the new jar file in place, this NullPointerException problem should be fixed.

There are some other changes you might want to consider. (I understand that Point is probably just an example for learning, but these comments may still be useful.)

  • You could make pattern a static final field and initialize it once. There is no need to spend time on every call of Point.parse compiling a regular expression that never changes.
  • The regular expression you are using will not match the valid double values Infinity, -Infinity, or NaN. This may be what you want (perhaps you intend every Point to have only finite coordinates), but sometimes programmers just forget to write patterns that will match all of the forms of valid input. It can be better sometimes (and may also be faster at runtime) to use a simpler pattern (say, match whatever is inside the parentheses, on either side of the comma) and pass that to parseDouble, which can be relied on to accept all valid input and throw an exception otherwise.
  • The call to groupCount() does not tell you what you perhaps think it does. It simply tells you how many capturing groups the pattern contains. For this pattern (which contains 4 capturing groups), groupCount() simply always returns 4. It does not give information on which groups matched anything in the current match.
  • You have a choice for the x and y fields of Point to have type Double or double. If you do not need them ever to be null (which is possible for Double but not possible for the primitive double), then you could make the code more efficient by using double.

If you are a beginner, you may be unfamiliar with the javap tool or with reading the disassembled bytecode of your classes. Most of the time you never have to, of course, and can simply concern yourself with high-level Java coding. But when you face a puzzle like this one, where the answer may lie in whether the compiled class came from the same source code you are looking at, it is invaluable to know how to do this. The instructions (like aload and invokevirtual) that you see when disassembling a class file are explained in the Java Virtual Machine Specification.

But that's not critical for this case: the exception is coming from the Java library.

Thank you very much for providing me with the above information. But I am only a beginner and it may be difficult for me to understand this official document.

The link I gave in that case was simply to the GitHub repository for the Java library source code itself, showing the line 1838 where the stacktrace you supplied showed the NullPointerException being thrown. It is, again, a useful skill to know how to follow a stacktrace that way: when you see that the variable s in that code was the argument passed by the caller, and one level up in the stacktrace was line 110 in the same file and it was the single argument to parseDouble, then you know that what happened was a null got passed to parseDouble, and you can focus your attention on finding out how your code could have done that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants