Add odo watch + quarkus:dev live reload support as one mode of the S2I

[emmanuelbernard]

In Quarkus, we have an integrated live reload model that listens to resources (like config files) and .class changes.
When someone hits refresh in the browser, Quarkus checks if any file has changed and restart the app
(not the JVM).
It’s fast enough (less than .5 to 1s for a reasonable size app).
So for Quarkus we do need of odo watch to rsync specific directories so that Quarkus’s live reload that we call quarkus:dev today kicks in accordingly.

Note that it's different from a classical S2I Java process which AFAIK is like this:

1. push code
2. s2i starts maven build
3. start java app off the maven build

In the Live reload mode, the model is more like this

1. enable odo watch
2. deploy the container which will run mvn package quarkus:dev and be ready to listen to requests
3. on .class or resource changes in a specific set of directories, rsync said files into the container in the specific location /some/determined/path/target
4. on resource changes, Quarkus does its magic from within the container

[cescoffier]

Isn't odo tied to OpenShift. What about something working with "bare" Kubernetes too?

[geoand]

Currently odo is tied to Openshift yes

[emmanuelbernard]

I don't know if there is an accepted watch model in Kubernetes atm. CC @kadel

[kadel]

I don't know if there is an accepted watch model in Kubernetes atm. CC @kadel

@emmanuelbernard
There is skaffold. It can watch over files similar to what odo is doing, but last time I checked it was rebuilding the image every time and there was no way to do hot reload.

Isn't odo tied to OpenShift. What about something working with "bare" Kubernetes too?

@cescoffier
Right now it is OpenShift only. But we have plans to add "bare" Kubernetes support, it is on our roadmap.

[emmanuelbernard]

Thanks kadel, anyways let's see what we can do for odo first.

[geoand]

I can take up the work on the odo side - provided that we figure out what builder image should be used @kadel

[gsmet]

@emmanuelbernard hmm, we were talking about exposing "dev only resources" when dev mode is enabled (such as the config) in another issue.

Are you seeing the feature described in this issue as something people would use in production?

If so, we can't expose the config unconditionally in dev mode.

[vorburger]

I'm interested in this topic as well, and have given it some thought...

Not actually convinced we even need "real S2I" in this scenario (as in my #1358 for native, non-dev) - in this use case, we technically don't want (need) to be "building" a new container image (which is what S2I is for), do we? What we really want is just to run quarkus:dev in-container, don't we? I'm exploring that in master...vorburger:issue-1390_dev-container ... currently stuck with #1449.

on .class or resource changes in a specific set of directories, rsync

I was thinking of simply proposing to use oc rsync, for starters.... Full disclosure: I'm not very familiar with Odo yet, but I sort of don't see yet what you would need it for, given how easy this could actually be with this approach.

BTW Skaffold is quite interesting IMHO; I'm hoping to learn more about it soon-ish. And if we were to integrate Skaffold with Knative Build, then things could start to get interesting...

[tgrall]

@gsmet it isn’t for production.odo Is used for development, but the idea is to be able to use the container/OpenShift during development and let the user doing it with iteration (without reloading the container)

So if you have a specific flag we should be able to pass it to the S2I/supervisord to enable the dev mode in the application

[geoand]

Using odo instead of plain rsync would allow users to easily handle a Quarkus app as part of their entire architecture, meaning that the app can be linked to some service via the Service Catalog, it provides a lot of conveniences such as creating a Route and much more.

Not to mention that there is a VSCode plugin that uses odo under the covers.

[kadel]

I was thinking of simply proposing to use oc rsync, for starters.... Full disclosure: I'm not very familiar with Odo yet, but I sort of don't see yet what you would need it for, given how easy this could actually be with this approach.

ou could use oc rsync and odo watch actually does something that is similar to it, but also handles all the hassle of setting the container up, it shields users from all the complexities.

odo gives this to users in a nice user-friendly way without a need to know anything about OpenShift.
That is the main odo promise

With odo it looks like this:

$ odo component create java
$ odo watch

Now you just edit files in your ide. Odo automatically detects any changes and syncs your files into the running container in OpenShift cluster (plus handles restarts, if required). You are editing files locally, but the application is running on OpenShift.

[emmanuelbernard]

@emmanuelbernard hmm, we were talking about exposing "dev only resources" when dev mode is enabled (such as the config) in another issue.

Are you seeing the feature described in this issue as something people would use in production?

If so, we can't expose the config unconditionally in dev mode.

odo watch is for dev mode. not to deploy in production.

[emmanuelbernard]

Not actually convinced we even need "real S2I" in this scenario (as in my #1358 for native, non-dev) - in this use case, we technically don't want (need) to be "building" a new container image (which is what S2I is for), do we? What we really want is just to run quarkus:dev in-container, don't we? I'm exploring that in master...vorburger:issue-1390_dev-container ... currently stuck with #1449.

Yes I think it would be a fake S2I that just do nothing on the S2 part aside from copying the source files and have the run parts start mvn compile quarkus:dev
Unless the S2 part can prefetch some maven dependencies or something. But again maybe we need a base image with prefetched quarkus dependencies?

[vorburger]

a fake S2I that just do nothing on the S2 part aside from copying the source files

I'm just not sure why we would use S2I at all in that case. Source files could just be copied into image with oc rsync (or odo watch I guess, but I'm not familiar enough with ODO, yet).

base image with prefetched quarkus dependencies?

master...vorburger:issue-1390_dev-container does exactly that! It's currently just NOK due to some file system permission / changing user IDs on OpenShift business (which also causes #1449, I suspect)... I'll try to find time to look more into sorting that out (or anyone else is more than welcome to try as well).

[emmanuelbernard]

@vorburger I was told odo only speaks S2I and the team asked for our S2I to integrate with Quarkus. @kadel ?

[geoand]

@emmanuelbernard that is true for the current state of odo. But I think @kadel has some plans to make change that

[emmanuelbernard]

OK, We can't wait plans I think at this stage. Burr is yelling at me for not having this work yesterday already.

[geoand]

Got it :) @kadel in that case we need to discuss how we can sneak this into in odo ASAP :P

[akram]

Note that it's different from a classical S2I Java process which AFAIK is like this:

1. push code
2. s2i starts maven build
3. start java app off the maven build

Hi @emmanuelbernard
S2I last step is to build a docker image and to push it to OpenShift internal docker registry.
Then a Deployment is created that will run this docker image by starting the java app of the maven build.

So, indeed, that would not really fit well in the Quarkus reload mode. Also especially, because s2i intend to produce container images, not to run code. And these container images are required to be immutable.

I think it would be worth exploring a different workflow than s2i to integrate such a thing.
What about creating your own CRD (Custom Resource Definition) with its own Operator, and then plug this into the S2I process once a certain checkpoint is reached during the development phase?

[kadel]

So, indeed, that would not really fit well in the Quarkus reload mode. Also especially, because s2i intend to produce container images, not to run code. And these container images are required to be immutable.

Hi @akram. We are leveraging s2i to run the code in odo project. It is not ideal, as s2i was never designed for this, but It can be done.

s2i is starting to be limiting factor for us, so I'm currently exploring different ways how we can create images with better development mode (inner loop) support especially for languages with dev mode with hot reload (Quarkus, NodeJS/nodemon).

I'm collecting my thoughts and ideas in this document https://docs.google.com/document/d/1Cj4ZXmsaQ-S9NDvrPaAb90i5I4yQg8gYQ8KQ41gIawc
It mentions odo a lot, but the main idea is that it should be a generic interface that any other tool can use.

[stuartwdouglas]

I have implemented direct remote sync in the dev mode itself: #1666 . This allows you to run mvn quarkus:remote-dev locally, and connect to a remote container. This will then use the same per request model that the dev mode already uses over a websocket, so there is no chance of seeing stale content.

[stuartwdouglas]

I think this can be closed, the new remote dev mode should support this.