Should we In-source mill-jacoco into com-lihaoyi/mill as a contrib/ module, include as an example/ doc test

[lihaoyi]

Currently we have an example/scalalib/module/13-contrib/scoverage example doc test, but we do not have anything equivalent for Java. https://github.com/lefou/mill-jacoco exists, but because it is in a third party repo we cannot use it in our example tests.

We should in-source the mill-jacoco Plugin under contrib/, similar to what we have for scoverage, and include it in the example/ folder and through that in the doc-site

[lefou]

TBH, I don't think in-sourcing all plugins is the right way. The question is, why do you want that? Maybe, we can try to address the most pressing issues and explore alternatives first.

The actual issue i see with in-sourcing into the Mill repo, as it is currently organized, is that plugins loose their own release and maintenance lifecycle. Development of Mill and contrib-plugins is tightly coupled. It's rather hard to fix issues for older Mill versions. But given how Mill is designed (via traits), we will get binary breaking changes now and then. And not all users can migrate at a schedule Mill dictates. But the same user might need to update some plugins to address vulnerabilities or newer features.

As the author of mill-jacoco and many other Mill plugins, this is also the main reason why these plugins live in their own projects rather than in contrib. It's definitely more work that way, but as in-repo contrib plugins there is no way to support multiple Mill versions. This might change in future, e.g. we lately improved out integration test API. But from a maintenance aspect, I think it will produce a bottleneck.

From experience as Mill maintainer form many years I know contrib plugins don't see much contributions but can make the development of Mill much harder. E.g. the scoverage plugin still tries to support older Scala versions and therefore need to work with Scoverage 1.x. This is really hard, as we need to use really old Scala versions and libraries and what not, just to have some tests. It would be definitely much easier for both, Mill and the scoverage plugin, if it would be maintained outside the Mill tree.

Although I see the benefit of "dogfooding" Mill and being able to bootstrap from the main branch, I also think we should use a stable Mill release to build Mill and therefore external maintained plugins should not hold back Mill development.

[lihaoyi]

I think one solution to solve the problem with older Mill versions is long-lived release branches. That's already something we've started to do with 0.11.12, and probably will become more serious in 0.12.0 vs 0.13.0. If we decide to take 0.11.x and 0.12.x seriously, we can continue to backport contrib plugin improvements to those branches so projects on old Mill versions can benefit without having to update to latest master.

I think from a user perspective I would get value from some of these plugins being bundled. It provides some discoverability, legitimacy, and consistency with the other bundled plugins that may not exist on a random repo you find on the internet. One solution to this is to move stuff to a dedicated Github org that can provide this, so then the question comes down to one big repo vs a bunch of small repos and I'm generally on the one-big-repo side of things.

One of the big initial decisions in Mill was bundling a lot of things that other build tools require third-party plugins: assembly, publishing, autoformatting, etc. I don't think we need to in-source every plugin, but I do think code coverage is sufficiently "core" to be bundled. But it's a subjective judgement that reasonable people can disagree, and if you as the author prefer that mill-jacoco lives outside I can close this issue

[lefou]

Currently we have an example/scalalib/module/13-contrib/scoverage example doc test, but we do not have anything equivalent for Java. https://github.com/lefou/mill-jacoco exists, but because it is in a third party repo we cannot use it in our example tests.

I don't get that argument. It might hold true, if you specifically mean, that you want be able to run all examples as integration tests against the latest main branch build while in a binary breaking development windows, but for the gros of users and especially the target audience of the documentation, this isn't an issue at all.

[lihaoyi]

I don't get that argument. It might hold true, if you specifically mean, that you want be able to run all examples as integration tests against the latest main branch build while in a binary breaking development windows, but for the gros of users and especially the target audience of the documentation, this isn't an issue at all.

Yeah that's what I mean. I generally don't want to have the CI for a particular project tied to the release cycle for other projects. If the projects are truly independent, then having separate repos is fine. But I don't want to have to disable certain tests every time we release a new major version and re-enable them later.

I know we already do that with the mill-bootstrap.patch and disabling parts of the root build.sc, but I always viewed that as a hack and I do not like it either. IMO separate repos should move separately on their own schedule, and once you have to start doing coordinated changing-publishing-changing-again like that they should be in the same repo. That's just a single monolith with more steps

[lefou]

I think one solution to solve the problem with older Mill versions is long-lived release branches. That's already something we've started to do with 0.11.12, and probably will become more serious in 0.12.0 vs 0.13.0. If we decide to take 0.11.x and 0.12.x seriously, we can continue to backport contrib plugin improvements to those branches so projects on old Mill versions can benefit without having to update to latest master.

I already maintained release branches previously for Mill 0.9.x line and 0.10.x line, because I took these versions seriously. While this works for Mill itself, that doesn't scale for contrib plugins. IMHO, the only way of doing plzgins right is to have one code base to address multiple Mill versions via cross building.

[lefou]

I don't get that argument. It might hold true, if you specifically mean, that you want be able to run all examples as integration tests against the latest main branch build while in a binary breaking development windows, but for the gros of users and especially the target audience of the documentation, this isn't an issue at all.

Yeah that's what I mean. I generally don't want to have the CI for a particular project tied to the release cycle for other projects. If the projects are truly independent, then having separate repos is fine. But I don't want to have to disable certain tests every time we release a new major version and re-enable them later.

I know we already do that with the mill-bootstrap.patch and disabling parts of the root build.sc, but I always viewed that as a hack and I do not like it either. IMO separate repos should move separately on their own schedule, and once you have to start doing coordinated changing-publishing-changing-again like that they should be in the same repo. That's just a single monolith with more steps

Maybe, the extended documentation needs to be an external project then. Although I love having the docs in-repo, since this allows us the evolve docs alongside features. Maybe we could set up a separate repo just for the integration tests? I'm not sure it works. Another solution could be to optionally build external plugins against the in-repo Mill API and use these in tests, for the sake of early testing breaking APIs. This needs some more efforts but could be doable.