chore: Upgrade build deps to ensure compatible with Scala Native #178

lqhuang · 2024-12-30T04:00:32Z

Hi, Haoyi,

This PR will be part of my work to introduce native supports to requests-scala [#156].

I try to update current build dependencies to make sure that current tool chains are compatible with Scala Native and current implementations before introducing other breaking changes.

Here are some notable changes:

Scala 2.11 must be deprecated because geny has deprecated Scala 2.11 since 1.1.0
Add Java 21 LTS to GitHub CI for testing job
All test cases passed in my local env.
But mima binary checks failed. I'm not very sure if it's fine or not.

More details attached here

Local testing results after changes

$ ./mill -i __.publishArtifacts + __.test
[build.mill-57/61] compile
[build.mill-57] [info] compiling 1 Scala source to /home/lqhuang/Git/requests-scala/out/mill-build/compile.dest/classes ...
[build.mill-57] [info] done compiling
[418/420] requests[2.13.15].test.test
[418]
[416/420] requests[3.6.2].test.test
[416]
[420/420] requests[2.12.20].test.test
[420]
[420/420] =============================================================== __.publishArtifacts + __.test ================================================================== 6s

report binary issues

$ ./mill -i __.mimaReportBinaryIssues
[183/185] requests[2.13.15].mimaReportBinaryIssues
[184/185] requests[2.12.20].mimaReportBinaryIssues
...
[183] Found 52 issue when checking against com.lihaoyi:requests:0.7.0
[184] Found 52 issue when checking against com.lihaoyi:requests:0.7.0
...
[185/185] ================================================================= __.mimaReportBinaryIssues ==================================================================== 2s
3 tasks failed
requests[2.12.20].mimaReportBinaryIssues Failed binary compatibility check! Found 206 potential problems
requests[2.13.15].mimaReportBinaryIssues Failed binary compatibility check! Found 206 potential problems
requests[3.6.2].resolvedMimaPreviousArtifacts scala.MatchError: Failure(
Resolution failed for 1 modules:
--------------------------------------------
  com.lihaoyi:requests_3:0.6.7
        not found: /home/lqhuang/.ivy2/local/com.lihaoyi/requests_3/0.6.7/ivys/ivy.xml
        not found: https://repo1.maven.org/maven2/com/lihaoyi/requests_3/0.6.7/requests_3-0.6.7.pom

--------------------------------------------

Any feedback and suggestion is appreciated!

Regards,
Lanqing

Updates:

After refactoring build.mill, now binary check passed in my local env, too.

$ ./mill clean && ./mill -i __.mimaReportBinaryIssues
[build.mill-64/68] compile
[build.mill-64] [info] compiling 1 Scala source to /home/lqhuang/Git/requests-scala/out/mill-build/compile.dest/classes ...
[build.mill-64] [info] done compiling
[1/1] clean
[1/1] ============================== clean ==============================
[197/212] mill.scalalib.ZincWorkerModule.worker
[197] Compiling compiler interface...
[200/212] requests.jvm[3.3.4].compile
[199/212] requests.jvm[2.13.15].compile
[199] [info] compiling 7 Scala sources to /home/lqhuang/Git/requests-scala/out/requests/jvm/2.13.15/compile.dest/classes ...
[200] [info] compiling 7 Scala sources to /home/lqhuang/Git/requests-scala/out/requests/jvm/3.3.4/compile.dest/classes ...
[199] [warn] 4 deprecations (since 0.9.0)
[199] [warn] 4 deprecations (since 2.13.0)
[199] [warn] 8 deprecations in total; re-run with -deprecation for details
[199] [warn] 3 feature warnings; re-run with -feature for details
[199] [warn] four warnings found
[199] [info] done compiling
[200] [warn] there were 3 feature warnings; re-run with -feature for details
[200] [warn] there were 6 deprecation warnings; re-run with -deprecation for details
[200] [warn] two warnings found
[200] [info] done compiling
[204/212] requests.jvm[2.13.15].mimaReportBinaryIssues
[208/212] requests.jvm[3.3.4].mimaReportBinaryIssues
[204] Scanning binary compatibility in /home/lqhuang/Git/requests-scala/out/requests/jvm/2.13.15/compile.dest/classes ...
[208] Scanning binary compatibility in /home/lqhuang/Git/requests-scala/out/requests/jvm/3.3.4/compile.dest/classes ...
[204] Binary compatibility check passed
[198/212] requests.jvm[2.12.20].compile
[198] [info] compiling 7 Scala sources to /home/lqhuang/Git/requests-scala/out/requests/jvm/2.12.20/compile.dest/classes ...
[208] Binary compatibility check passed
[198] [warn] 6 deprecations (since 0.9.0); re-run with -deprecation for details
[198] [warn] three feature warnings; re-run with -feature for details
[198] [warn] two warnings found
[198] [info] done compiling
[212/212] requests.jvm[2.12.20].mimaReportBinaryIssues
[212] Scanning binary compatibility in /home/lqhuang/Git/requests-scala/out/requests/jvm/2.12.20/compile.dest/classes ...
[212] Binary compatibility check passed
[212/212] ============================== __.mimaReportBinaryIssues ============================== 13s

And now if you access https://self-signed.badssl.com/, it literally returns a "404 Not Found". Moreover, it looks like the whole site badssl.com returns 404 (???? Shutdown or server error?)

I update corresponding unit tests.

$ curl --insecure -vvv https://self-signed.badssl.com/
* Uses proxy env variable no_proxy == '.local,.internal,.arpa,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16'
* Host self-signed.badssl.com:443 was resolved.
* IPv6: (none)
* IPv4: 104.154.89.105
*   Trying 104.154.89.105:443...
* Connected to self-signed.badssl.com (104.154.89.105) port 443
* ALPN: curl offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 / prime256v1 / rsaEncryption
* ALPN: server accepted http/1.1
* Server certificate:
*  subject: C=US; ST=California; L=San Francisco; O=BadSSL; CN=*.badssl.com
*  start date: Dec 19 21:03:33 2024 GMT
*  expire date: Dec 19 21:03:33 2026 GMT
*  issuer: C=US; ST=California; L=San Francisco; O=BadSSL; CN=*.badssl.com
*  SSL certificate verify result: self-signed certificate (18), continuing anyway.
*   Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption
* using HTTP/1.x
> GET / HTTP/1.1
> Host: self-signed.badssl.com
> User-Agent: curl/8.5.0
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Server: nginx/1.10.3 (Ubuntu)
< Date: Tue, 07 Jan 2025 10:41:04 GMT
< Content-Type: text/html
< Content-Length: 178
< Connection: keep-alive
<
<html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.10.3 (Ubuntu)</center>
</body>
</html>
* Connection #0 to host self-signed.badssl.com left intact

Current notable changes:

Scala 2.11 must be deprecated because geny has deprecated Scala 2.11 since 1.1.0
Option.when(VcsVersion.vcsState().commitsSinceLastTag != 0)(VcsVersion.vcsState().lastTag).flatten only appends 0.6.7, so I replaced it with concrete release versions.
Add Java 21 LTS to GitHub CI for testing job
Add Scala 3.6.2 to GitHub CI to run testing job for Scala Next
test cases passed in my local env.
binary check passed in my local env

lqhuang · 2024-12-30T04:07:11Z

And did you mind that I introduce scalafmt for project? It's probably will reformat all codes. If you allow that, which kinds of style you prefer? Basically, give me a reference project, then I will copy their .scalafmt into repo.

lihaoyi · 2024-12-30T04:25:32Z

scalafmt would be great. Just pick an arbitrary style, we can tweak it later as necessary.

lqhuang · 2024-12-30T05:45:09Z

Other closable PRs (by scala-steward / dependabot) if merged

Close Update scala-library to 2.13.15 #176
Close Update scala-library to 2.12.20 #171
Close Update mill-main to 0.11.12 #170
Close Update mill-main to 0.11.11 #168
Close Update mill-main to 0.11.9 #165
Close Bump actions/setup-java from 3 to 4 #147
Close Update mill-main to 0.10.13 #145
Close Bump actions/checkout from 3 to 4 #142
Close Update mill-mima to 0.0.24 #139
Close Update de.tobiasroeser.mill.vcs.version to 0.3.1 #131
Close Update scala3-library to 3.1.3 #121
Close Update scala-library to 2.12.20 #171
Close Update ujson to 1.3.15 #116
Close Update mill-main to 0.12.5 #181

The above PRs will be closed automatically after resolving.

build.mill

Co-authored-by: Ondra Pelech <[email protected]>

build.mill

lihaoyi · 2025-01-07T03:03:52Z

TBH I'm not sure why bincompat is failing, you might need to bisect your changes and see what change is at fault

lqhuang · 2025-01-07T03:50:33Z

Thanks!

I found there is no Scala 3 release before version 0.6.9. So I removed auto discovered versions by VCS for binary check with manually specified versions.

Seq("0.6.9", "0.7.0", "0.7.1", "0.8.0","0.8.2", "0.9.0")

Now error hints changed. But It still failed. I fold some outputs, it almost show problems for every classes.

» ./mill -i __.mimaReportBinaryIssues
[195/197] requests[2.12.20].mimaReportBinaryIssues
[196/197] requests[2.13.15].mimaReportBinaryIssues
[197/197] requests[3.3.4].mimaReportBinaryIssues
[195] Scanning binary compatibility in /home/lqhuang/Git/requests-scala/out/requests/2.12.20/mimaReportBinaryIssues.dest/emptyClasses ...
[196] Scanning binary compatibility in /home/lqhuang/Git/requests-scala/out/requests/2.13.15/mimaReportBinaryIssues.dest/emptyClasses ...
[197] Scanning binary compatibility in /home/lqhuang/Git/requests-scala/out/requests/3.3.4/mimaReportBinaryIssues.dest/emptyClasses ...
[195] Found 50 issue when checking against com.lihaoyi:requests:0.6.9
[195]  * interface requests.BaseSession does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.BaseSession")
[195]  * object requests.BaseSession does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.BaseSession$")
[195]  * interface requests.Cert does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.Cert")
[195]  * object requests.Cert does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.Cert$")
[195]  * class requests.Cert#P12 does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.Cert$P12")
[195]  * object requests.Cert#P12 does not have a correspondent in current version

.....

[197]  * class requests.Response does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.Response")
[197]  * object requests.Response does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.Response$")
[197]  * class requests.ResponseBlob does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.ResponseBlob")
[197]  * class requests.Session does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.Session")
[197]  * object requests.Session does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.Session$")
[197]  * class requests.StatusMessages does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.StatusMessages")
[197]  * object requests.StatusMessages does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.StatusMessages$")
[197]  * class requests.StreamHeaders does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.StreamHeaders")
[197]  * object requests.StreamHeaders does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.StreamHeaders$")
[197]  * class requests.TimeoutException does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.TimeoutException")
[197]  * class requests.UnknownHostException does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.UnknownHostException")
[197]  * class requests.Util does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.Util")
[197]  * object requests.Util does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.Util$")
[197]  * class requests.package does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.package")
[197]  * object requests.package does not have a correspondent in current version
   filter with: ProblemFilter.exclude[MissingClassProblem]("requests.package$")
[197/197] ============================== __.mimaReportBinaryIssues ============================== 1s
3 tasks failed
requests[2.12.20].mimaReportBinaryIssues Failed binary compatibility check! Found 312 potential problems
requests[2.13.15].mimaReportBinaryIssues Failed binary compatibility check! Found 312 potential problems
requests[3.3.4].mimaReportBinaryIssues Failed binary compatibility check! Found 312 potential problems

I'm trying to downgrade some core deps to figure out problems.

lqhuang · 2025-01-07T04:07:01Z

I'm also learning how to write build.mill for CrossScalaModule from geny https://github.com/com-lihaoyi/geny/blob/main/build.mill

…the first argument`

lqhuang · 2025-01-07T09:43:26Z

@lihaoyi I solve it :)

It's majorly because I configured PlatformScalaModule mistakenly.

Everything is working normally now. Please check the updated PR description and review again.

Thanks for your hints.

lqhuang · 2025-01-08T02:08:37Z

@lihaoyi It looks like self-signed.badssl.com is not very stable 😅. Could you help me to trigger CI again. Sorry for that

lihaoyi · 2025-01-08T03:09:37Z

@lqhuang looks green, if you're happy with this I'll merge it

lqhuang · 2025-01-08T03:12:21Z

@lqhuang looks green, if you're happy with this I'll merge it

No problem, definitely ok!

Appreciate your kind helps :)

lqhuang · 2025-01-08T03:31:53Z

Other closable PRs (by scala-steward / dependabot) if merged

* Close [Update scala-library to 2.13.15 #176](https://github.com/com-lihaoyi/requests-scala/pull/176)

* Close [Update scala-library to 2.12.20 #171](https://github.com/com-lihaoyi/requests-scala/pull/171)

* Close [Update mill-main to 0.11.12 #170](https://github.com/com-lihaoyi/requests-scala/pull/170)

* Close [Update mill-main to 0.11.11 #168](https://github.com/com-lihaoyi/requests-scala/pull/168)

* Close [Update mill-main to 0.11.9 #165](https://github.com/com-lihaoyi/requests-scala/pull/165)

* Close [Bump actions/setup-java from 3 to 4 #147](https://github.com/com-lihaoyi/requests-scala/pull/147)

* Close [Update mill-main to 0.10.13 #145](https://github.com/com-lihaoyi/requests-scala/pull/145)

* Close [Bump actions/checkout from 3 to 4 #142](https://github.com/com-lihaoyi/requests-scala/pull/142)

* Close [Update mill-mima to 0.0.24 #139](https://github.com/com-lihaoyi/requests-scala/pull/139)

* Close [Update de.tobiasroeser.mill.vcs.version to 0.3.1 #131](https://github.com/com-lihaoyi/requests-scala/pull/131)

* Close [Update scala3-library to 3.1.3 #121](https://github.com/com-lihaoyi/requests-scala/pull/121)

* Close [Update scala-library to 2.12.20 #171](https://github.com/com-lihaoyi/requests-scala/pull/171)

* Close [Update ujson to 1.3.15 #116](https://github.com/com-lihaoyi/requests-scala/pull/116)

* Close [Update mill-main to 0.12.5 #181](https://github.com/com-lihaoyi/requests-scala/pull/181)

The above PRs will be closed automatically after resolving.

Why these PRs don't close automatically as described in Merging the referencing pull request also closes the referenced pull request.

Confusing 🤔 Never mind, just a compulsion comment

sideeffffect · 2025-01-08T10:25:32Z

@lqhuang the close keyword needs to be in the PR description. If it's in a further comment, then it's not going to work.

lqhuang · 2025-01-08T12:52:14Z

@sideeffffect I see. Thanks. Got it ✌️

chore: Upgrade build deps to ensure compatible with Scala Native

5c589f9

lqhuang mentioned this pull request Dec 30, 2024

Scala-Native Support (1000USD Bounty) #156

Open

chore: Update versions of GitHub Actions

68d2580

sideeffffect reviewed Dec 30, 2024

View reviewed changes

build.mill Outdated Show resolved Hide resolved

Update build.mill

b5ef43a

Co-authored-by: Ondra Pelech <[email protected]>

sideeffffect reviewed Dec 31, 2024

View reviewed changes

build.mill Outdated Show resolved Hide resolved

chore: Update mill version to 0.12.5 (Close com-lihaoyi#181)

cd919cf

lqhuang mentioned this pull request Jan 4, 2025

Add scalafmt #179

Merged

chore: Depreate versions that donot have scala 3 release in binary check

a094614

lqhuang added 2 commits January 7, 2025 04:42

chore: Add scalaNextVersion and add it into GitHub CI Actions

bd4f194

bugfix: Fix `-i/--interactive/--no-server/--bsp must be passed in as …

e085b0d

…the first argument`

bugfix: Make mima check passed

419aac2

lqhuang requested review from sideeffffect and lihaoyi January 7, 2025 09:46

lqhuang added 2 commits January 7, 2025 11:10

bugfix: Fix statusCode assertation in test case "selfSignedCertificate"

b7455ab

Reverit return status code to 200

a6f4740

lihaoyi merged commit 6f566e6 into com-lihaoyi:master Jan 8, 2025
5 checks passed

lqhuang deleted the update-build-deps branch January 8, 2025 03:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: Upgrade build deps to ensure compatible with Scala Native #178

chore: Upgrade build deps to ensure compatible with Scala Native #178

lqhuang commented Dec 30, 2024 •

edited

Loading

lqhuang commented Dec 30, 2024 •

edited

Loading

lihaoyi commented Dec 30, 2024

lqhuang commented Dec 30, 2024 •

edited

Loading

lihaoyi commented Jan 7, 2025

lqhuang commented Jan 7, 2025 •

edited

Loading

lqhuang commented Jan 7, 2025

lqhuang commented Jan 7, 2025 •

edited

Loading

lqhuang commented Jan 8, 2025

lihaoyi commented Jan 8, 2025

lqhuang commented Jan 8, 2025 •

edited

Loading

lqhuang commented Jan 8, 2025 •

edited

Loading

sideeffffect commented Jan 8, 2025

lqhuang commented Jan 8, 2025

chore: Upgrade build deps to ensure compatible with Scala Native #178

chore: Upgrade build deps to ensure compatible with Scala Native #178

Conversation

lqhuang commented Dec 30, 2024 • edited Loading

lqhuang commented Dec 30, 2024 • edited Loading

lihaoyi commented Dec 30, 2024

lqhuang commented Dec 30, 2024 • edited Loading

lihaoyi commented Jan 7, 2025

lqhuang commented Jan 7, 2025 • edited Loading

lqhuang commented Jan 7, 2025

lqhuang commented Jan 7, 2025 • edited Loading

lqhuang commented Jan 8, 2025

lihaoyi commented Jan 8, 2025

lqhuang commented Jan 8, 2025 • edited Loading

lqhuang commented Jan 8, 2025 • edited Loading

sideeffffect commented Jan 8, 2025

lqhuang commented Jan 8, 2025

lqhuang commented Dec 30, 2024 •

edited

Loading

lqhuang commented Dec 30, 2024 •

edited

Loading

lqhuang commented Dec 30, 2024 •

edited

Loading

lqhuang commented Jan 7, 2025 •

edited

Loading

lqhuang commented Jan 7, 2025 •

edited

Loading

lqhuang commented Jan 8, 2025 •

edited

Loading

lqhuang commented Jan 8, 2025 •

edited

Loading