Skip to content

Commit

Permalink
deploy: e480c7f
Browse files Browse the repository at this point in the history
  • Loading branch information
jdries committed Nov 4, 2023
1 parent 4c04a11 commit d9683c2
Show file tree
Hide file tree
Showing 40 changed files with 6,145 additions and 366 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
26 changes: 26 additions & 0 deletions _sources/part2/advanced_workflows.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,21 @@
- What are openEO platform 'credits'
- Learn about metadata generation in openEO

## Algorithm types

When writing an algorithm in openEO or Pangeo, the first thing to do is to determine the type of algorithm.
Once you know the type, you can fall back to well-known examples as a template to implement your algorithm.

Using the right method for your type of algorithm also ensures that it can be parallellized and executed
in the most efficient way. An algorithm that only requires a single observation for instance, does not need
to bother with retrieving a full timeseries of observations.

In part 2 of this tutorial, we'll cover a few common types:
- single observation, single pixel
- temporal analysis over a single pixel

In part 3, we have an example of an analysis that requires multiple values in both space and time.


## Case 1 Rank composites

Expand All @@ -20,6 +35,10 @@ Key takeaways:
- Use array_apply to 'loop' over data
- Create masks to selectively load data

Algorithm type:
- Spatial neighborhood for cloud proximity score/masking
- Per pixel temporal analysis for scoring

https://github.com/Open-EO/openeo-community-examples/blob/main/python/RankComposites/rank_composites.ipynb

## Case 2 Machine learning training and inference
Expand All @@ -35,6 +54,8 @@ Key takeaways:
- Transform a timeseries datacube into features for learning
- Use ML processes directly from openEO

Algorithm type: per-pixel timeseries analysis

Dynamic landcover: https://docs.openeo.cloud/usecases/landcover/

## Case 3 ESA WorldWater
Expand All @@ -45,6 +66,11 @@ by DHI-GRAS.
Key takeaways:
- Use of complex, rule-based classification

Algorithm type:

- temporal compositing into ARD
- single observation, single pixel, analysis

Note: the most recent code of this example is still under review in a pull request, so we include the relevant bits inline.

https://github.com/DHI-GRAS/worldwater-toolbox
Expand Down
11 changes: 10 additions & 1 deletion _sources/part3/advanced_udf.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,12 @@
"\n",
"\n",
"The openEO user defined functions (UDF's), provide a way to run Pangeo code as part of a larger openEO workflow.\n",
"This allows you to get the best of both worlds.\n",
"This allows you to get the best of both worlds:\n",
"- You can use openEO to get access to a wide variety and full archives of EO data.\n",
"- Preprocessing to ARD such as Sentinel-1 backscatter is easily done in openEO\n",
"- For very fast interactive debugging of the core of your algorithm, local testing based on Pangeo is faster than basically any cloud based service\n",
"- Once your algorithm is ready, you can run it in the cloud without worries thanks to openEO\n",
"\n",
"\n",
"To understand user defined functions, it is recommended to start here:\n",
"\n",
Expand Down Expand Up @@ -43,6 +48,8 @@
"\n",
"## UDF with basic XArray code: peak-valley detection\n",
"\n",
"Algorithm type: single pixel timeseries analysis\n",
"\n",
"This example may require a `pip install fusets`\n",
"\n",
"The actual peak-valley algorithm can be found here:\n",
Expand All @@ -59,6 +66,8 @@
"\n",
"## UDF with machine learning\n",
"\n",
"Algorithm type: spatiotemporal data chunk\n",
"\n",
"Applying deep learning models is one of the most common reason to use UDF's.\n",
"\n",
"This example relies on understanding 'apply_neighborhood', which is explained here:\n",
Expand Down
Loading

0 comments on commit d9683c2

Please sign in to comment.