Skip to content

Commit

Permalink
Merge pull request #394 from OpenFn/roles
Browse files Browse the repository at this point in the history
docs sprint 2
  • Loading branch information
aleksa-krolls authored Feb 10, 2024
2 parents 454009d + 5dc67db commit 7bcad24
Show file tree
Hide file tree
Showing 8 changed files with 229 additions and 31 deletions.
27 changes: 22 additions & 5 deletions docs/design/design-workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,28 @@ Next, consider visually outlining the structure and flow of a workflow to ensure
The output of this exercise is clear documentation on how a business process will be executed by automation, humans, and often a combination of both.

## Diagram using global standards
When diagramming, consider using global standards like BPMN (business process model and notation) so that diagrams are consistent and can be understood by outside parties.
When diagramming, consider using global standards like BPMN (business process model and notation) so that diagrams are consistent and can be understood by outside parties. BPMN (learn more about standard
[BPMN 2.0](https://www.omg.org/spec/BPMN/2.0/)) has flowchart-like symbols and
precise notation that can be translated to software process components.

1. There are lots of online resources to learn basic concepts like this post.
2. The OpenFn team has compiled some learning resources and BPMN diagramming tools in this document.
Check out these resources for learning & building your own BPMN diagrams:

Your organization may already be using a different standard notation (e.g., UML). What’s important is consistency across your documentation to ensure that your solutions can be easily understood by other stakeholders.
- `BPMN.io` open-source modeler: https://bpmn.io/
- `Camunda BPMN Tool` includes a free tool and tutorial:
https://camunda.com/bpmn/
- `LucidChart` provides a very user-friendly diagramming interface:
https://www.lucidchart.com/pages/bpmn

![Workflow](/img/BPMN_example.png)
Looking for a crash course? This video provides a quick overview of BPMN and how
to use it: https://www.youtube.com/watch?v=BwkNceoybvA

### OpenFn Examples of BPMN Diagrams

See the below example BPMN diagram for the user story:

> As a program manager, I want to extract beneficiary details ("tracked entity
> instances") from my country's DHIS2 system, so that I can enroll them as
> contacts in my SMS campaign configured on RapidPro to send them automated
> alerts and program updates.
<img src="/img/sample-bpmn.png" url />
51 changes: 25 additions & 26 deletions docs/manage-projects/user-roles-permissions.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,35 +3,34 @@ title: User Roles & Permissions
sidebar_label: User Roles
---


There are four project roles on Lightning: Owner (**only 1 per project**), Admin, Editor & Viewer.
Check out the table below for the permissions available to each role.
When OpenFn Users are invited to work on your Project as `Collaborators`, they are assigned a `Role` that determines their permissions. The four available Roles are: Owner (**only 1 per project**), Admin, Editor & Viewer. Check out the table below for the permissions available to each role.

|Context | Action | Owner | Admin | Editor | Viewer |
| :--------| :-------- | :--: | :------------: | :----------: | :----------: |
| Workflows| Create a workflow | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |"
| Workflows| Edit a job in a workflow | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |"
| Workflows| Add/remove webhook authentication method for workflow | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |"
| Workflows| Delete a workflow | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |"
| Workflows| Run from the inspector | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |"
| Workflows| Select the 5 latest inputs for a job in a workflow | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |"
| History| View/search/filter on the history page | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |"
| History| View a run from the work order history | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |"
| History| View an input from a work order history | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |"
| History| Run from the work order history | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |"
| Settings| View project name | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |"
| Settings| Edit project name | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |"
| Settings| View project description | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |"
| Settings| Edit project description | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |"
| Settings| Export project | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |"
| Settings| Delete a project | :heavy_check_mark: | :x: | :x: | :x: |"
| Settings| View project credentials, type, and owner | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |"
| Settings| Add/remove webhook authentication method for project | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |"
| Settings| Change MFA requirement for project | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |"
| Settings| Add project collaborator | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |"
| Settings| View project collaborators (project_users, role, digest and alerts) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |"
| Workflows| Create a Workflow | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |"
| Workflows| Edit a Job in a Workflow | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |"
| Workflows| Add/remove webhook authentication method for Workflow | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |"
| Workflows| Delete a Workflow | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |"
| Workflows| Run from the Inspector | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |"
| Workflows| Select the 5 latest Inputs for a Job in a Workflow | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |"
| History| View/search/filter on the History page | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |"
| History| View a Run from the Work Order history | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |"
| History| View a Input from a Work Order history | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |"
| History| Run from the Work Order history | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |"
| Settings| View Project name | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |"
| Settings| Edit Project name | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |"
| Settings| View Project description | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |"
| Settings| Edit Project description | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |"
| Settings| Export Project | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |"
| Settings| Delete a Project | :heavy_check_mark: | :x: | :x: | :x: |"
| Settings| View Project Credentials, type, and owner | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |"
| Settings| Add/remove webhook authentication method for Project | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |"
| Settings| Change MFA requirement for Project | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |"
| Settings| Add Project Collaborator | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |"
| Settings| View Project Collaborators (project_users, role, digest and alerts) | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |"
| Settings| Edit digest and alerts for themselves | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |"
| Settings| Change input/output dataclip storage policy | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |"
| Settings| Change history retention period | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |"
| Settings| Edit digest and alerts for others | :x: | :x: | :x: | :x: |"
| Settings| Change Input/Output Dataclip storage policy | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |"
| Settings| Change History retention period | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |"
| Settings| Update GitHub project/repo connection | :heavy_check_mark: | :heavy_check_mark: | :x: | :x: |"
| Settings| Initiate GitHub sync | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: | :x: |"
11 changes: 11 additions & 0 deletions docs/monitor-history/failure-notifications-digests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
title: Failure notifications and Digests
sidebar_label: Failure notifications and Digests
---



On the `Collaboration` page of your project settings, you can turn on notifications to receive email alerts when a job fails. You can also subscribe to digests that summarize project activity including the number of `Work Orders` that were processed correctly and the number that failed.

<img src="/img/notifications.png" url />

8 changes: 8 additions & 0 deletions docs/monitor-history/rerunning-workflow.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
title: Re-running your workflows
sidebar_label: Re-running your workflows
---

Did a step in your workflow fail? Looking to resync historical data? Whatever the reason, you can re-run your OpenFn workflow from the History page or the Inspector view. Check out the video below for a how-to.

Video tutorial: [Re-running your workflows](https://www.loom.com/share/281a1f8bbc8547078d52e5c03df65a96?sid=c212a374-1343-4d0f-8ed8-e30642a12695).
149 changes: 149 additions & 0 deletions docs/monitor-history/troubleshooting.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
---
title: Troubleshooting
sidebar_label: Troubleshooting
---



## Runs

One of the most helpful pages for troubleshooting on OpenFn is the
[History](/documentation/next/monitor-history/activity-history) page.
This page provides a list of all of the runs executed for a Work Order and their status. Project administrators can troubleshoot errors by clicking into the run to review the run details. Learn more about runs [here](/documentation/next/monitor-history/viewing-workflow-runs) here.

### Status codes

Every run will have a status code. The status code is a way for OpenFn to classify
the run status and can help you troubleshoot errors. Learn more about OpenFn
status codes and what each one means [here](/documentation/next/monitor-history/status-codes).

### The time it took for the workflow to fail

The run will also record how long it took before the workflow failed. This
information helps users understand if the workflow is taking longer than it should
and is especially helpful with errors that involve timeouts. You can use the run
to determine at which operation the workflow is timing out and determine if the workflow
performance can be optimized.

### Run logs

As workflows are developed it is important to log details which will make testing and
troubleshooting much easier in the future. Keep reading for the two most
important parts of a run log!

#### Mappings

If possible, the logs should be written so that you can see exactly what was mapped between
the source system and the destination system. In summary,
the log can have a **"Data received from source system"** section and a **"Data
to be uploaded to destination system"** section.

These logs can help admins verify that the source data and the data being
uploaded to the destination system is correct. For example, seeing in the logs
that a unique identifier is being mapped to `undefined` in the destination
system can help you understand this Salesforce error message:

`METHOD_NOT_ALLOWED: HTTP Method 'PATCH' not allowed. Allowed are GET,HEAD,POST at HttpApi.getError`.

#### Error messages

The run log should also tell us if an error has been thrown, and depending on
the destination system, what the error message is. Sometimes the error message
is very specific like:

`NOT_FOUND: Provided external ID field does not exist or is not accessible`

This error from Salesforce usually indicates that `External ID` has not been
checked in the field settings in Salesforce.

Other error messages are not as clear and can take some time to debug:

`TypeError [Error]: Cannot read property 'split' of undefined`

**`TypeErrors`** usually indicate that the job received a part of the input
that it wasn't expecting, or there is a syntax error in your job code. It means
that the job needs to be updated to know how to handle the input. In this
case, the job received an old version of the Commcare form which was missing a
field on which the job called the `split` function. You can determine this by
reviewing the job for which fields the split function is being called on and
checking that they are all present in the message.

The more you test and troubleshoot with a particular system, the more familiar
with its error messages you become.

:::tip

OpenFn has outlined several of the more common error messages specific to some
of the systems that we have integrated in the past. Explore these systems and
their error messages [here](/adaptors#connect-anything).

:::

## Leveraging search and filtering in OpenFn

Leverage the various search functionalities in OpenFn to find the right runs to support your troubleshooting. You can search on the History page across OpenFn IDs, Inputs, and/or Logs. Check out this [video](https://www.loom.com/share/083b371a26de43cfbbb72321249a694b?sid=3e2a897b-81d5-487e-a7a4-f8a7b88c7b63) for how to use Search.



## Sign up for email alerts

You can turn on notifications to receive [email alerts ](/documentation/next/monitor-history/failure-notifications-digests)when a workflow fails and subscribe to digests that summarize project activity.

## More

> What happens if my survey data from ODK needs to link to existing records in
> my Salesforce system but a respondent enters or selects an invalid
> `external ID`?
Great question, and don't worry, it happens all the time. Assuming you've
already taken all possible measures to either pre-load external IDs in your ODK
form or use more human-proof IDs (like barcodes and fingerprints) here's the
flow of work:

1. Read the email, and inspect the reason for failure.

2. 99% of failed runs on OpenFn are due to `value mismatches`. The _collected_
`id` in ODK doesn't match the _expected_ `id` in Salesforce. You must now
choose to either:

A. Edit the source `id` in your `receipt` & retry the attempt.

B. Edit the related `id` in your destination system & retry the attempt.

C. Ignore the attempt—this source data will never reach your destination
system. (There have been reports of ODK Aggregate's JSON publisher sending
duplicate values. If that happens and your run fails due to "duplicate
values" on a particular unique field you can safely ignore the run in
OpenFn.)

Editing data in your destination system can be done through that system's
interface. Many tools that act as `sources` (like ODK) do not allow for easy
editing and re-submission of data. You can use OpenFn to edit the source data
before retrying the attempt.

### Common Error Messages

The most common error messages with explanations are:

```sh
DUPLICATE_VALUE: duplicate value found: ODK_uuid__c duplicates value on record with id: a0524000005wNw0
The insert is blocked because you are attempting to create a new record with a
unique field with the same value as an existing record.
```

```sh
Required value missing
```

```sh
ExternalId not found
```

```sh
{ INVALID_FIELD_FOR_INSERT_UPDATE: Unable to create/update fields: Contact__c.
Please check the security settings of this field and verify that it is
read/write for your profile or permission set. }
```
This last one may arise if a master-detail relationship in Salesforce is not set
as reparentable and the user attempts to run an upsert.
10 changes: 10 additions & 0 deletions docs/monitor-history/viewing-workflow-runs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
title: Viewing Workflow Runs
sidebar_label: Viewing Workflow Runs
---

A Run is each time OpenFn attempts to excute a Work Order. All runs are recorded on the History page. Check out the video below for more details!

Video tutorial: [Viewing Workflow Runs](https://www.loom.com/share/0ebfeb5671274d739c55f08b6e218353?sid=b835d96a-b667-470b-b5be-675e2d699f03)


4 changes: 4 additions & 0 deletions sidebars-main.js
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,10 @@ module.exports = {
label: 'Monitor History',
items: [
'monitor-history/activity-history',
'monitor-history/failure-notifications-digests',
'monitor-history/viewing-workflow-runs',
'monitor-history/rerunning-workflow',
'monitor-history/troubleshooting',
'monitor-history/status-codes',
],
},
Expand Down
Binary file added static/img/notifications.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 7bcad24

Please sign in to comment.