Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model Registry docs #3778

Merged
merged 55 commits into from
Jul 26, 2022
Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
2804073
nav: clean up and rename files + remove UG index
jorgeorpinel Jun 4, 2022
f2632bc
studio: misc. edits
jorgeorpinel Jun 4, 2022
983beaa
studio: redirects per sidebar changes
jorgeorpinel Jun 5, 2022
b38a94f
nav: viz exps (Studio UG)
jorgeorpinel Jun 5, 2022
c997215
links: changes studio URLs
jorgeorpinel Jun 5, 2022
6523ebf
studio: remove how-it-works info from overview
jorgeorpinel Jun 5, 2022
f7cb138
Merge branch 'master' into nav/studio
jorgeorpinel Jun 7, 2022
87cb4c5
Merge branch 'main' into nav/studio and
jorgeorpinel Jul 18, 2022
8eea458
Update content/docs/studio/user-guide/prepare-your-repositories.md
jorgeorpinel Jul 18, 2022
ac16eea
Update content/docs/studio/get-started.md
jorgeorpinel Jul 18, 2022
6da2aaf
Update content/docs/studio/get-started.md
jorgeorpinel Jul 18, 2022
e58b61d
guide: better md ref link
jorgeorpinel Jul 18, 2022
184fe6a
Restyled by prettier (#3775)
restyled-io[bot] Jul 18, 2022
7ce6eca
guide: better titles for Change Team Plan and Size
jorgeorpinel Jul 18, 2022
16d2db9
Merge branch 'nav/studio' of github.com:iterative/dvc.org into nav/st…
jorgeorpinel Jul 18, 2022
fad68e8
guide: GH App title
jorgeorpinel Jul 18, 2022
2a60ad1
guide: Connect GL Server title
jorgeorpinel Jul 18, 2022
3d08d3e
Update content/docs/studio/user-guide/billing-and-payment.md
jorgeorpinel Jul 18, 2022
a6c78b6
Restyled by prettier (#3777)
restyled-io[bot] Jul 18, 2022
1cd6935
Draft of Model Registry docs
tapadipti Jul 19, 2022
9c0fc68
Deleted package-lock.json which was committed by mistake
tapadipti Jul 19, 2022
f64f209
Further work on MR docs
tapadipti Jul 19, 2022
42ed930
Removed unnecessary gifs
tapadipti Jul 19, 2022
6bbf5d9
Further work on MR docs
tapadipti Jul 19, 2022
f29f216
Further work on MR docs
tapadipti Jul 19, 2022
3eff852
Further work on MR docs
tapadipti Jul 19, 2022
4e38681
Studio MR docs continued
tapadipti Jul 21, 2022
ba1cd92
Apply suggestions from code review
tapadipti Jul 21, 2022
59486aa
Applied PR review suggestions
tapadipti Jul 21, 2022
ea0b594
Applied PR review comments
tapadipti Jul 21, 2022
42cfbfe
Applied PR review comments
tapadipti Jul 21, 2022
29f70c2
Applied PR review comments
tapadipti Jul 21, 2022
5c04beb
Applied PR review comments
tapadipti Jul 21, 2022
e4b99bb
Applied PR review comments
tapadipti Jul 21, 2022
56818c2
Applied PR review comments
tapadipti Jul 21, 2022
e704469
Apply PR review comments
tapadipti Jul 22, 2022
f5d130f
Change promote to assign stage
tapadipti Jul 22, 2022
d2ca588
Small fixes to MR docs
tapadipti Jul 22, 2022
404f549
Small fixes to MR docs
tapadipti Jul 22, 2022
2fd5568
Small fixes to MR docs
tapadipti Jul 22, 2022
4e8bf6c
Update urls of tutorial videos
tapadipti Jul 25, 2022
0f0afbf
Replaced the alert in the header to announce model registry release
tapadipti Jul 25, 2022
9f43584
Improved troubleshooting section
tapadipti Jul 25, 2022
1bffb56
Update content/docs/studio/user-guide/model-registry/add-a-model.md
tapadipti Jul 25, 2022
03a1e0c
Lint fix
tapadipti Jul 25, 2022
5b31d73
Removed TODOs
tapadipti Jul 25, 2022
e770af8
Merge branch 'main' into mr-docs
tapadipti Jul 25, 2022
8f8298d
Revert unrelated changes (wrong merge)
jorgeorpinel Jul 26, 2022
1b398c1
Other small fixes
jorgeorpinel Jul 26, 2022
3b70aad
Update content/docs/studio/user-guide/prepare-your-repositories.md
jorgeorpinel Jul 26, 2022
19f9577
Update content/docs/sidebar.json
jorgeorpinel Jul 26, 2022
4e7e5da
Restyled by prettier (#3807)
restyled-io[bot] Jul 26, 2022
0091ac8
Apply suggestions from code review
jorgeorpinel Jul 26, 2022
a3ffb84
Restyled by prettier (#3808)
restyled-io[bot] Jul 26, 2022
6e1eb21
Correct MR video links
tapadipti Jul 26, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 20 additions & 28 deletions content/docs/sidebar.json
Original file line number Diff line number Diff line change
Expand Up @@ -521,46 +521,38 @@
"icon": "vscode"
},
{
"label": "Studio",
"slug": "studio",
"source": "studio/index.md",
"icon": "studio",
"children": [
{
"label": "How It Works",
"slug": "overview"
},
"how-it-works",
"get-started",
{
"label": "User Guide",
"slug": "user-guide",
"source": "user-guide/index.md",
"source": false,
"children": [
"prepare-your-repositories",
{
"label": "Prepare Your Repositories",
"slug": "prepare-repositories"
},
{
"label": "Projects and Experiments",
"slug": "projects",
"slug": "projects-and-experiments",
"source": false,
"children": [
"what-is-a-project",
"create-project",
{
"label": "Configure Project",
"slug": "project-settings"
},
{
"label": "Explore ML Experiments",
"slug": "explore-experiments"
},
{
"label": "Visualize and Compare",
"slug": "visualize-experiments"
},
"create-a-project",
"configure-a-project",
"explore-ml-experiments",
"visualize-and-compare",
"run-experiments",
"share-project"
"share-a-project"
]
},
{
"slug": "model-registry",
"source": false,
"children": [
"what-is-a-model-registry",
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
"add-a-model",
"view-models",
"version-models",
"promote-models"
]
},
"teams",
Expand Down
95 changes: 67 additions & 28 deletions content/docs/studio/get-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ Studio.

https://www.youtube.com/watch?v=hKf4twg832g

## Sign up / sign in

1. Sign in to your [Iterative Studio](https://studio.iterative.ai/) dashboard
using your GitHub.com, GitLab.com or Bitbucket.org account, or your email
address.
Expand All @@ -25,20 +27,24 @@ different plans, check out our

</admon>

2. If this is the first time you are signing in to Iterative Studio, you will
## Explore demo project

1. If this is the first time you are signing in to Iterative Studio, you will
see that there already exists a `Demo` project that connects to an example
DVC repository. Use this project to explore the features that Iterative
Studio has to offer.
repository. Use this project to explore the features that Iterative Studio
has to offer.

![](https://static.iterative.ai/img/studio/login_home_v3.png) _When you first
login, an example project is already created for you to explore, and you can add
more projects._

3. Each project on this dashboard displays the metrics that Iterative Studio
2. Each project on this dashboard displays the metrics that Iterative Studio
identified in your Git repository. In the figure above, you can see that
`avg_prec` and `roc_auc` metrics are displayed.

4. To connect to a new repository, click on `Add a Project`. All the
## Add your projects

1. To connect to a new repository, click on `Add a Project`. All the
organizations that you have access to will be listed.

<admon type="info">
Expand All @@ -59,47 +65,80 @@ more projects._

</admon>

5. Open the organization whose repository you want to connect to. You can also
2. Open the organization whose repository you want to connect to. You can also
use the search bar to directly look for a repository.

![](https://static.iterative.ai/img/studio/select_repo_v3.png)

6. Specify additional connection settings if required.
3. Specify additional connection settings if required.

![](https://static.iterative.ai/img/studio/project_settings.png)

> Project settings must be configured if you are connecting to a non-DVC
> repository, if your metrics are in some custom files, if you are connecting to
> a monorepo, or if your metrics are in cloud or other remote storage. However,
> you can configure the project settings after the project has been created. So,
> you can `Skip and Continue` now. Refer to the
> [Project Settings](/doc/studio/user-guide/projects/project-settings) section
> for more details.
> you can configure the project settings after the project has been created. So
> you can `Skip and Continue` now. Refer to the [Project Settings] section for
> more details.

You should now see that a project has been added in your dashboard.

7. If your project requires any of the additional settings, then remember to
configure them by opening the
[project settings](/doc/studio/user-guide/projects/project-settings).
Otherwise, your project may not work as expected. To go to project settings,
click on the
4. If your project requires any of the additional settings, then remember to
configure them by opening the [project settings]. Otherwise, your project may
not work as expected. To go to project settings, click on the
![](https://static.iterative.ai/img/studio/view_open_settings_icon.png) icon
in the project. In the menu that opens up, click on `Settings`.

![](https://static.iterative.ai/img/studio/project_open_settings.png)

8. You can also
[create multiple projects from a single Git repository](/doc/studio/user-guide/projects/create-project#create-multiple-projects-from-a-single-git-repository).
5. You can also [create multiple projects from a single Git repository].

## Explore and share your projects

1. After creating a project, you can use it to [explore the ML experiments] as
well as [submit new experiments] from Iterative Studio.

2. You can also make the project public by [sharing it on the web].

## Use model registry

1. Click on the `Models` tab to open the central
[Models dashboard](/doc/studio/user-guide/model-registry/view-models#models-dashboard).
If any of your projects contain [GTO-annotated models], they will be listed
here automatically.

[gto-annotated models]: /doc/studio/user-guide/model-registry/add-a-model
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved

2. You can [add new models](/doc/studio/user-guide/model-registry/add-a-model)
to the model registry by clicking on the `Add a model` button and following
on-screen instructions.

3. To
[view the details](/doc/studio/user-guide/model-registry/view-models#model-details-page)
of a model, click on the name of the model.

4. You can
[register new versions](/doc/studio/user-guide/model-registry/version-models)
of the models and
[promote them to different stages](/doc/studio/user-guide/model-registry/promote-models)
(e.g., development, staging, production).

9. After creating a project, you can use it to
[explore the ML experiments](/doc/studio/user-guide/explore-experiments) as
well as
[submit new experiments from Iterative Studio](/doc/studio/user-guide/run-experiments).
## Create and manage teams

10. You can also make the project public by
[sharing it on the web](/doc/studio/user-guide/projects/share-project).
1. You can [create a team] and invite collaborators. Each team will have its own
projects dashboard. To create teams with more than 2 team members, [sign up
for the **Team** plan or **Enterprise** plan].

11. You can [create a team](/doc/studio/user-guide/teams) and invite
collaborators. Each team will have its own projects dashboard. To create
teams with more than 2 team members,
[sign up for the **Team** plan or **Enterprise** plan](/doc/studio/user-guide/change-team-plan-and-size).
[project settings]:
/doc/studio/user-guide/projects-and-experiments/configure-a-project
[create multiple projects from a single git repository]:
/doc/studio/user-guide/projects-and-experiments/create-a-project#create-multiple-projects-from-a-single-git-repository
[explore the ml experiments]:
/doc/studio/user-guide/projects-and-experiments/explore-ml-experiments
[submit new experiments]:
/doc/studio/user-guide/projects-and-experiments/run-experiments
[sharing it on the web]:
/doc/studio/user-guide/projects-and-experiments/share-a-project
[create a team]: /doc/studio/user-guide/teams
[sign up for the **team** plan or **enterprise** plan]:
/doc/studio/user-guide/change-team-plan-and-size
66 changes: 66 additions & 0 deletions content/docs/studio/how-it-works.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# How Iterative Studio Works

[Iterative Studio](https://studio.iterative.ai/) works with the data, metrics,
hyperparameters and model metadata that you add to your ML project Git
repositories. It works very closely with your Git ecosystem.

This video illustrates how Iterative Studio works closely with your Git
ecosystem.

> Note that we have renamed DVC Studio to Iterative Studio and Views to
> Projects.

https://www.youtube.com/watch?v=5xM5az78Lrg

## How your project data is saved

- Using [DVC] and [Git], you will push all your ML experiments to your GitHub,
GitLab or Bitbucket repositories as Git commits.
- Using Iterative Studio, or using the command line interface (CLI) of [GTO],
and possibly [MLEM], you will push all your ML model details to the Git
repositories as Git commits and Git tags.

## How Iterative Studio extracts your project data

- When you connect to these Git repositories from Iterative Studio, the
project's `dvc.yaml` is used to identify all the data, metrics and
hyperparameters in your experiments.
- If you are not using DVC, you can
[add the metrics and hyperparameters to your Git repositories manually](/doc/studio/user-guide/projects-and-experiments/configure-a-project#custom-metrics-and-parameters).
- Details of your ML models, including versions and stage assignments, are
extracted from the Git commits and tags.

## How Iterative Studio presents your project data

- Iterative Studio creates an
[interactive, tabular representation](/doc/studio/user-guide/projects-and-experiments/explore-ml-experiments#components-of-a-project)
of all the identified values.
- All the projects that you have created are presented in a central projects
dashboard.
- All identified models are included in an interactive
[models dashboard](/doc/studio/user-guide/model-registry/view-models#models-dashboard)
and individual
[model details pages](/doc/studio/user-guide/model-registry/view-models#model-details-page).

## How Iterative Studio saves updates to your ML projects

- When you
[run new experiments](/doc/studio/user-guide/projects-and-experiments/run-experiments)
or
[add models to your model registry](/doc/studio/user-guide/model-registry/add-a-model)
in Iterative Studio, it creates Git commits and pull requests with the
changes.
- You can set up your CI/CD actions (e.g. GitHub Actions) to run model training
upon the creation of Git commits, tags or pull requests. You can use [CML] in
your CI/CD actions for continuous machine learning.
- When you
[register new versions](/doc/studio/user-guide/model-registry/version-models)
of your ML models or
[assign stages](/doc/studio/user-guide/model-registry/promote-models) to them,
Iterative Studio creates annotated Git tags representing the actions.

[dvc]: https://dvc.org/
[cml]: https://cml.dev
[mlem]: https://mlem.ai/
[gto]: https://github.com/iterative/gto
[git]: https://git-scm.com/
18 changes: 11 additions & 7 deletions content/docs/studio/index.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
# Iterative Studio

[`Iterative Studio`](https://studio.iterative.ai/) is a web application that you
can [access online](https://studio.iterative.ai/) or even host on-prem. It works
with the data, metrics and hyperparameters that you add to your ML project
repositories. Using the power of leading open-source tools DVC,
[CML](https://cml.dev) and Git, it enables you to seamlessly manage data and
models, run and track experiments, and visualize and share results.
[Iterative Studio](https://studio.iterative.ai/) is a web application that you
can access online or even host on-prem. Using the power of leading open-source
tools [DVC](https://dvc.org/), [CML](https://cml.dev), [MLEM](https://mlem.ai/),
[GTO](https://github.com/iterative/gto) and [Git](https://git-scm.com/), it
Comment on lines +4 to +6
Copy link
Contributor

@jorgeorpinel jorgeorpinel Jul 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's an example of the lack of serial comma causing confusion. Is Git one of the tools? Or is the combination between GTO and Git the tool in question?

Suggested change
can access online or even host on-prem. Using the power of leading open-source
tools [DVC](https://dvc.org/), [CML](https://cml.dev), [MLEM](https://mlem.ai/),
[GTO](https://github.com/iterative/gto) and [Git](https://git-scm.com/), it
can access online or even host on-prem. Using the power of leading open-source
tools [DVC](https://dvc.org/), [CML](https://cml.dev), [MLEM](https://mlem.ai/),
[GTO](https://github.com/iterative/gto), and [Git](https://git-scm.com/), it

Copy link
Contributor

@jorgeorpinel jorgeorpinel Jul 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, I wouldn't even mention Git here. It's "included" in some of the other tools (mainly DVC and GTO). Up to you.

Suggested change
can access online or even host on-prem. Using the power of leading open-source
tools [DVC](https://dvc.org/), [CML](https://cml.dev), [MLEM](https://mlem.ai/),
[GTO](https://github.com/iterative/gto) and [Git](https://git-scm.com/), it
can access online or even host on-prem. Using the power of leading open-source
tools [DVC](https://dvc.org/), [CML](https://cml.dev), [MLEM](https://mlem.ai/),
and [GTO](https://github.com/iterative/gto), it

enables you to seamlessly manage data and machine learning models, run and track
experiments, and visualize and share results.

<cards>

<card href="/doc/studio/overview" heading="How Iterative Studio works">
<card href="/doc/studio/how-it-works" heading="How Iterative Studio works">
Learn how Iterative Studio works closely with the Git ecosystem
</card>

Expand All @@ -35,3 +35,7 @@ models, run and track experiments, and visualize and share results.
- Apply your existing software engineering stack for ML teams.
- Automate your ML process by transitioning to a no-code interface for running
experiments on cloud resources of your choice.
- Build an ML model registry for transparency and discovery across all your
projects.
- Because your model registry is built on top of Git, you can use GitOps for
model deployment.
24 changes: 0 additions & 24 deletions content/docs/studio/overview.md

This file was deleted.

Loading