CI for forward-porting GC3 patches to GC4 #147

ddeclerck · 2024-05-29T22:26:35Z

Follow-up of #146.

GitMensch · 2024-05-31T18:56:18Z

build_aux/bootstrap

@@ -96,12 +96,15 @@ autoreconf $AC_OPTS $MAINPATH > $msgs 2>&1; ret=$?
 # Filter aminclude_static as those are only used _within_ another
 #        check so reporting as portability problem is only noise.
 # This has the effect of redirecting some error messages to stdout. 
+# to be moved to the Makefile - currently only usable for bootstrap,
+# but should be done on autogen, too


A, that old TODO...

GitMensch

I suggest to wrap the commits again. From what I've inspected we need one refactor for integrating 4.x logic (you've spotted that well) nicely.

GitMensch · 2024-05-31T19:02:49Z

libcob/fileio.c

-	snprintf (file_open_env, (size_t)COB_FILE_MAX, "%s%s", "IO_", s);
-	if ((file_open_io_env = cob_get_env (file_open_env, NULL)) == NULL) {
-		snprintf (file_open_env, (size_t)COB_FILE_MAX, "%s%s", "io_", s);
+	if (f != NULL) {


When/why should f be null here?

Because some functions (open_cbl_file, cob_sys_delete_file, ...) were "improved" to perform file mapping in GC3 (rev 3944), by calling the cob_chk_file_mapping function, which does not take a cob_file argument in GC3 but does in GC4, and that function in turn calls cob_chk_file_env. Since these functions (open_cbl_file, cob_sys_delete_file, ...) do not use a cob_file object, I resorted to passing NULL and coping with that...

Is this okay for you @GitMensch ?

libcob/fileio.c

GitMensch · 2024-05-31T19:48:51Z

oops, hope I haven't broken the gitignore in f36dcda - if not then we likely should apply that to the gcos3x branch as well.

ddeclerck · 2024-05-31T21:21:13Z

I suggest to wrap the commits again. From what I've inspected we need one refactor for integrating 4.x logic (you've spotted that well) nicely.

Saw your message a bit late, added another commit in the meantime 😅

By wrapping up you mean, committing to SVN ? (after doing the requested modifications of course)

GitMensch · 2024-06-01T07:50:07Z

This approach is reasonable in general. ... So you did already check that this piece of code changed in a later commit? If you see a difference in the current code, then you can use "svn annotate" to check which commit did the change in this party of fileio.c to know what to merge before the refactoring. :-) Am 1. Juni 2024 00:07:13 MESZ schrieb OCP David Declerck ***@***.***>:

…

@ddeclerck commented on this pull request. > + /* apply COB_FILE_PATH if set (similar to ACUCOBOL's FILE-PREFIX) */ + if (file_paths) { + for(k=0; file_paths[k] != NULL; k++) { + snprintf (file_open_buff, (size_t)COB_FILE_MAX, "%s%c%s", + file_paths[k], SLASH_CHAR, file_open_name); + file_open_buff[COB_FILE_MAX] = 0; + if (access (file_open_buff, F_OK) == 0) { + break; + } +#if defined(WITH_CISAM) || defined(WITH_DISAM) || defined(WITH_VBISAM) || defined(WITH_VISAM) + /* ISAM may append '.dat' to file name */ + snprintf (file_open_buff, (size_t)COB_FILE_MAX, "%s%c%s.dat", + file_paths[k], SLASH_CHAR, file_open_name); + file_open_buff[COB_FILE_MAX] = 0; + if (access (file_open_buff, F_OK) == 0) { + snprintf (file_open_buff, (size_t)COB_FILE_MAX, "%s%c%s", + file_paths[k], SLASH_CHAR, file_open_name); + file_open_buff[COB_FILE_MAX] = 0; + break; + } +#endif + } + if (file_paths[k] == NULL) { + snprintf (file_open_buff, (size_t)COB_FILE_MAX, "%s%c%s", + file_paths[0], SLASH_CHAR, file_open_name); + file_open_buff[COB_FILE_MAX] = 0; + } + strncpy (file_open_name, file_open_buff, (size_t)COB_FILE_MAX); + } I remember why I haven't refactored that straight away : in case subsequent commits modify the same code, conflicts will be much easier to handle. I was thinking about keeping the refactoring for after all the patches are merged... -- Reply to this email directly or view it on GitHub: #147 (comment) You are receiving this because you commented. Message ID: ***@***.***>

ddeclerck · 2024-06-01T09:03:33Z

I tend to be overly "conservative". Indeed this piece of code is barely modified afterwards, so I'll do the refactoring.

ddeclerck · 2024-06-05T08:53:45Z

Is this okay to merge (@GitMensch) ?

GitMensch · 2024-06-05T14:37:29Z

I'll try to review this (late) evening.

GitMensch · 2024-06-06T05:38:21Z

looks_absolute should use "src", not file_open_name directly (merge issue?)

"apply_file_paths" should get that via argument as well and have a function comment that it writes to the global buffer. Then add a Changelog "extracted from xyz and also used in abc" to finish that last commit.

We either have to remember for later that we need to add a testcase for the new use or (potentially easier) also include it in the last commit as well.

ddeclerck · 2024-06-06T07:02:09Z

looks_absolute should use "src", not file_open_name directly (merge issue?)

This change is introduced in a later commit (3993).

"apply_file_paths" should get that via argument as well and have a function comment that it writes to the global buffer. Then add a Changelog "extracted from xyz and also used in abc" to finish that last commit.

Alright ; as for its output, should it write it through its argument or directly to file_open_name ?

We either have to remember for later that we need to add a testcase for the new use or (potentially easier) also include it in the last commit as well.

By "new use", de you mean the fact that we apply file paths to the complex case ?

GitMensch · 2024-06-06T12:48:20Z

By "new use", de you mean the fact that we apply file paths to the complex case ?

yes

This change is introduced in a later commit (3993).

good catch - then it is fine to leave as is; if you don't expect any big problem it would be nice to merge that in this bunch to commit that together, but a later bunch is fine as well

Alright ; as for its output, should it write it through its argument or directly to file_open_name ?

Depends on how other functions do it - it is best for now to mimic that (once the merge is completed we may revisit that part, but there are "some" commits left until we get there).

ddeclerck · 2024-06-08T10:01:57Z

I made the necessary changes.

This change is introduced in a later commit (3993).

good catch - then it is fine to leave as is; if you don't expect any big problem it would be nice to merge that in this bunch to commit that together, but a later bunch is fine as well

I find it more convenient to merge consecutive commits. If that's okay for you I could add to the current batch the next eligible commits until 3993 (that would be 6 commits: 3973, 3979, 3988, 3989, 3992 and 3993).

GitMensch · 2024-06-08T10:52:34Z

That batch is good to go :-)

ddeclerck · 2024-06-08T11:14:22Z

That batch is good to go :-)

Merged in SVN ;)

I see the next commits deal with translation files. Checking the history, it seems those files are usually just copied "as-is" from the GC3 branch to the trunk; is this correct ?

GitMensch · 2024-06-08T11:38:48Z

I see the next commits deal with translation files. Checking the history, it seems those files are usually just copied "as-is" from the GC3 branch to the trunk; is this correct ?

No, only new files are copied, the others left as-is; before a release I regenerate the files but the files are nearly completely maintained by the translation project.
So just record the merge, then copy over new files and svn add them, if there are any.

And of course

ddeclerck · 2024-06-08T13:54:13Z

Alright.

And of course

Hope this unfinished sentence did not have any vital info 😅

GitMensch · 2024-06-08T14:01:22Z

I can't remember any important info missing there.

codecov-commenter · 2024-06-08T14:17:10Z

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 88.88889% with 1 line in your changes missing coverage. Please review.

Please upload report for BASE (gc4@49792f4). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
cobc/codegen.c	88.88%	0 Missing and 1 partial ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@          Coverage Diff           @@
##             gc4     #147   +/-   ##
======================================
  Coverage       ?   65.39%           
======================================
  Files          ?       37           
  Lines          ?    67100           
  Branches       ?    18780           
======================================
  Hits           ?    43882           
  Misses         ?    16084           
  Partials       ?     7134

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

ddeclerck · 2024-06-08T22:54:46Z

Quick question: I sometimes see alternative code for GC4 in #if 0 blocks; I guess I should implement those and drop the other branch, correct ?

I'm talking about those:

#if 0 /* TODO for 4.0: set the attributes from the field given outside on the stack */
		output ("cob_field *cob_fret, const int cob_pam");
#else
		output ("cob_field **cob_fret, const int cob_pam");
#endif

GitMensch · 2024-06-12T20:21:25Z

That's quite a bunch - any reason to not merge upstream?
Open issues you are aware of or special adjustments needed?

[we really need to get to commits that have someone else in the ChangeLogs...]

ddeclerck · 2024-06-12T21:35:57Z

That's quite a bunch - any reason to not merge upstream? Open issues you are aware of or special adjustments needed?

No good reason. It may be many commits, but the first batch had way more lines (this one is only +1,162 −904).
Anyways, I'll merge upstream if that's okay with you.

[we really need to get to commits that have someone else in the ChangeLogs...]

I'm looking forward to reaching commit 4614 - our first contribution to GC3 ;)

GitMensch · 2025-01-08T19:26:06Z

Hm, seems 8:0:0 is the way to go (because 3.3 will be 7:0:3) if 3.2 was 6:0:2.
We could also arbitrarily increase the number to keep space for possible further additions to 3.x... In any case we don't need the age at other places so far, so no new extra define needed.

GitMensch · 2025-01-08T19:27:43Z

And yes, merging the revision (early) that has the MacOS fix [= within this bunch] would be preferable in this bunch. Should likely have the revision number that broke it in its svn log message...

ddeclerck · 2025-01-09T15:55:27Z

And yes, merging the revision (early) that has the MacOS fix [= within this bunch] would be preferable in this bunch. Should likely have the revision number that broke it in its svn log message...

Fixed by 5092.

Now this batch is ready for review.

GitMensch · 2025-01-09T16:05:53Z

I take it the commit will have correct svn-mergeinfo (without the merged, then reverted parts) and you'll merge the #210 commit from 3.x "clean" as soon as this is upstream, right?

ddeclerck · 2025-01-09T16:10:21Z

I take it the commit will have correct svn-mergeinfo (without the merged, then reverted parts) and you'll merge the #210 commit from 3.x "clean" as soon as this is upstream, right?

Yeah, I updated the mergeinfo appropriately, and I'll indeed merge the 3.x commit for PR 210 as soon as it is upstreamed ;)

GitMensch · 2025-01-09T16:25:18Z

The MSVC checks have a bunch of new failures with this bunch - I guess you need to merge r5120 to fix that (but that creates conflicts later if 5105+5107+5119 are not merged before).
As an alternative you could manually do the necessary "update to current atlocal.in" as a minor adjustment (the changelog entry will get in with a later merge, then).

ddeclerck · 2025-01-09T17:40:34Z

The MSVC checks have a bunch of new failures with this bunch - I guess you need to merge r5120 to fix that (but that creates conflicts later if 5105+5107+5119 are not merged before). As an alternative you could manually do the necessary "update to current atlocal.in" as a minor adjustment (the changelog entry will get in with a later merge, then).

Indeed. Well, just adding -fdiagnostics-plain-output seems sufficient for now.

GitMensch · 2025-01-09T19:03:30Z

LGTM 4 upstream

ddeclerck · 2025-01-10T11:26:27Z

I think that's enough for this batch.
Mostly straightforward and automatically merged.

I just have a question regarding the pack_to_bin tables in numeric.c and move.c: there are two versions of that table, guarded by an #if .... 3.x used the second version, while trunk already had a similar table, but equivalent to the first version. Which one should actually be used in trunk ? (testsuite is fine with either versions)

GitMensch · 2025-01-10T12:43:10Z

Take the first version to be enabled by default (as you have), because that is what trunk had so far; the issue for 3.x was that this was a behavior change, which we did not want.

... reading the Changelog diff

numeric.c (cob_decimal_set_packed): backport pack_to_bin change while adjusting its translation table for invalid BCD "digits" to yield the same result as before

:-)

Possibly add am comment on the #if 1 about that as well.
Note: there would be also the possibility to just ignore the upper nibble - in this case A=2, B=3, ...

As this is worked on: how do the invalid packed fields convert to numeric for GCOS?
I think on IBM it will always abend (if not "numeric fixup" is enabled), on MF not sure - but I could test both of those.

Also, it would be nice to fix this in gcos-strict.conf, when working on those commits:

init-justify: yes # TODO: verify

ddeclerck · 2025-01-10T12:46:56Z

Take the first version to be enabled by default (as you have), because that is what trunk had so far; the issue for 3.x was that this was a behavior change, which we did not want.

Alright.

... reading the Changelog diff

numeric.c (cob_decimal_set_packed): backport pack_to_bin change while adjusting its translation table for invalid BCD "digits" to yield the same result as before

:-)

Yeah, that was a trunk patch ported back to 3.x with adjustments :p

As this is worked on: how do the invalid packed fields convert to numeric for GCOS? I think on IBM it will always abend (if not "numeric fixup" is enabled), on MF not sure - but I could test both of those.

I'll check, but I think I remember this causes an "illegal decimal data" exception on GCOS.

GitMensch · 2025-01-10T13:04:05Z

LGTM, so can be merged upstream - we'd like to want to know about the invalid data and the init-justify in any case...

ddeclerck · 2025-01-10T14:02:48Z

LGTM, so can be merged upstream - we'd like to want to know about the invalid data and the init-justify in any case...

So, I can confirm the ILLDEC exception on GCOS when trying to read invalid packed decimal data.

I also checked whether initial data complies with JUSTIFIED RIGHT, and in fact, it does not. So, I'll fix gcos-strict.conf in 3.x and add that commit to this batch, if that's okay with you.

GitMensch · 2025-01-10T15:19:27Z

So, I can confirm the ILLDEC exception on GCOS when trying to read invalid packed decimal data.

Just to do the same - would you please post your test code here - then I'll recheck with the other compilers.

GitMensch · 2025-01-10T15:41:58Z

bunch is good to check in

ddeclerck · 2025-01-10T15:47:27Z

So, I can confirm the ILLDEC exception on GCOS when trying to read invalid packed decimal data.

Just to do the same - would you please post your test code here - then I'll recheck with the other compilers.

Here is it, but note that:

I'm using the GCOS's COMP-3 type, which is basically PACKED-DECIMAL without sign (don't know if there's such variant on IBM ?)
I'm assuming EBCDIC encoding

       IDENTIFICATION DIVISION.
       PROGRAM-ID. prog.
       DATA DIVISION.
       WORKING-STORAGE SECTION.
       01 SRC-NUMPACK   PIC 9(6) COMP-3.
       01 SRC-NUMPACK-X REDEFINES SRC-NUMPACK PIC X(3).
       01 DST-NUMDISP   PIC 9(6).

       PROCEDURE DIVISION.
       MAIN.

      * akr = 0x819299 in EBCDIC => valid PACKED-DECIMAL
           MOVE "akr" TO SRC-NUMPACK-X.

           DISPLAY "SRC-NUMPACK-X    : '" SRC-NUMPACK-X "'".
           DISPLAY "SRC-NUMPACK      : '" SRC-NUMPACK "'".
           MOVE SRC-NUMPACK TO DST-NUMDISP.
           DISPLAY "DST-NUMDISP      : '" DST-NUMDISP "'".

      * aks = 0x8192A2 in EBCDIC => invalid PACKED-DECIMAL
           MOVE "aks" TO SRC-NUMPACK-X.

           DISPLAY "SRC-NUMPACK-X    : '" SRC-NUMPACK-X "'".
           DISPLAY "SRC-NUMPACK      : '" SRC-NUMPACK "'".
           MOVE SRC-NUMPACK TO DST-NUMDISP.
           DISPLAY "DST-NUMDISP      : '" DST-NUMDISP "'".

           STOP RUN.

ddeclerck added the DO NOT MERGE label May 29, 2024

ddeclerck changed the title ~~Dummy commit~~ CI for forward-porting GC3 patches to GC4 May 29, 2024

ddeclerck force-pushed the gc4 branch from dc9dda1 to b9778c3 Compare May 30, 2024 17:29

ddeclerck force-pushed the gc3_to_gc4 branch 3 times, most recently from 6900d8d to 1c95bd0 Compare May 30, 2024 21:21

GitMensch reviewed May 31, 2024

View reviewed changes

GitMensch requested changes May 31, 2024

View reviewed changes

ddeclerck force-pushed the gc3_to_gc4 branch from c2829b1 to 18626fb Compare May 31, 2024 20:45

ddeclerck force-pushed the gc3_to_gc4 branch from 05442df to 5174bb3 Compare June 8, 2024 09:39

ddeclerck force-pushed the gc3_to_gc4 branch from 5174bb3 to e31913b Compare June 8, 2024 14:14

ddeclerck force-pushed the gc3_to_gc4 branch from 3dfb385 to 7ef9f53 Compare January 9, 2025 17:09

ddeclerck closed this Jan 10, 2025

ddeclerck force-pushed the gc3_to_gc4 branch from a041efe to 49792f4 Compare January 10, 2025 08:55

Merge SVN 5084

c7351a8

ddeclerck reopened this Jan 10, 2025

ddeclerck added 8 commits January 10, 2025 10:10

Merge SVN 5085

929d53f

Merge SVN 5086

379da13

Merge SVN 5089

2870a39

Merge SVN 5088

3687dfa

Merge SVN 5090, 5091

b8a3783

Merge SVN 5093

a0d4a56

Merge SVN 5094

4af6c20

Merge SVN 5095, 5096

8051dd3

Merge SVN 5420

79f53d4

CI for forward-porting GC3 patches to GC4 #147

Are you sure you want to change the base?

CI for forward-porting GC3 patches to GC4 #147

Conversation

ddeclerck commented May 29, 2024

GitMensch May 31, 2024

Choose a reason for hiding this comment

GitMensch left a comment

Choose a reason for hiding this comment

GitMensch May 31, 2024

Choose a reason for hiding this comment

ddeclerck May 31, 2024

Choose a reason for hiding this comment

ddeclerck Jun 3, 2024

Choose a reason for hiding this comment

GitMensch commented May 31, 2024

ddeclerck commented May 31, 2024 • edited Loading

GitMensch commented Jun 1, 2024 via email

ddeclerck commented Jun 1, 2024

ddeclerck commented Jun 5, 2024

GitMensch commented Jun 5, 2024

GitMensch commented Jun 6, 2024

ddeclerck commented Jun 6, 2024

GitMensch commented Jun 6, 2024

ddeclerck commented Jun 8, 2024

GitMensch commented Jun 8, 2024

ddeclerck commented Jun 8, 2024

GitMensch commented Jun 8, 2024

ddeclerck commented Jun 8, 2024 • edited Loading

GitMensch commented Jun 8, 2024

codecov-commenter commented Jun 8, 2024 • edited Loading

Codecov Report

ddeclerck commented Jun 8, 2024

GitMensch commented Jun 12, 2024

ddeclerck commented Jun 12, 2024 • edited by GitMensch Loading

GitMensch commented Jan 8, 2025

GitMensch commented Jan 8, 2025 • edited Loading

ddeclerck commented Jan 9, 2025

GitMensch commented Jan 9, 2025

ddeclerck commented Jan 9, 2025

GitMensch commented Jan 9, 2025

ddeclerck commented Jan 9, 2025

GitMensch commented Jan 9, 2025

ddeclerck commented Jan 10, 2025

GitMensch commented Jan 10, 2025 • edited Loading

ddeclerck commented Jan 10, 2025

GitMensch commented Jan 10, 2025

ddeclerck commented Jan 10, 2025 • edited Loading

GitMensch commented Jan 10, 2025

GitMensch commented Jan 10, 2025

ddeclerck commented Jan 10, 2025

ddeclerck commented May 31, 2024 •

edited

Loading

ddeclerck commented Jun 8, 2024 •

edited

Loading

codecov-commenter commented Jun 8, 2024 •

edited

Loading

ddeclerck commented Jun 12, 2024 •

edited by GitMensch

Loading

GitMensch commented Jan 8, 2025 •

edited

Loading

GitMensch commented Jan 10, 2025 •

edited

Loading

ddeclerck commented Jan 10, 2025 •

edited

Loading