Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More docs for v1.2 #923

Merged
merged 23 commits into from
Jan 20, 2025
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 19 additions & 15 deletions guides/components_lifecycle.md
Original file line number Diff line number Diff line change
@@ -1,45 +1,46 @@
# Lifecycle of Membrane Components

The lifecycle of Membrane Components is closely related to the execution of Membrane callbacks within these components. While there are some differences among the lifecycles of Membrane Pipelines, Bins, and Elements, they share many similarities. Let's explore the component lifecycle and identify differences depending on the type of component being addressed.
The lifecycle of Membrane Components is closely related to the execution of Membrane callbacks within these components. While some differences exist among the lifecycles of Membrane Pipelines, Bins, and Elements, they share many similarities. Let's explore the component lifecycle and identify differences depending on the type of component being addressed.

## Initialization
The first callback executed in every Membrane Component is `handle_init/2`. This function is executed synchronously and blocks the parent component, except in the case of a Membrane Pipeline, as it does not have a parent. It is advisable to avoid heavy computations within this function. `handle_init/2` is ideally used for spawning children in a pipeline or bin through the `:spec` action.
The first callback executed in every Membrane Component is `handle_init/2`. This function is executed synchronously and blocks the parent process (the process that spawns a pipeline or the parent component) until it is done and actions returned from it are handled. It is advisable to avoid heavy computations within this function. `handle_init/2` is ideally used for spawning children in a Pipeline or Bin through the `:spec` action or parsing some options.

## Setup
Following `handle_init/2` is `handle_setup/2`, which is executed asynchronously. This is an optimal time to set up resources or perform other intensive operations required for the element to function properly.
Following `handle_init/2` is `handle_setup/2`, which is executed asynchronously (the parent process does not wait for its completion). This is an optimal time to set up resources or perform other intensive operations required for the component to function properly.

Note: By default, after `handle_setup/2`, a component's setup is considered complete. This behavior can be modified by returning `setup: :incomplete` from `handle_setup/2`. The component must then mark its setup as completed by returning `setup: :complete` from another callback, like `handle_info/3`, to enter `:playing` playback and go further in lifecycle.

## Linking pads
For components with pads having `availability: :on_request`, the corresponding `handle_pad_added/3` callbacks are called between `handle_setup/2` and `handle_playing/2` if they are linked in the same spec where the component was spawned. Linking the pad in a different spec from the one spawning the element may lead to `handle_pad_added/3` being invoked after `handle_playing/2`.
For components with pads having `availability: :on_request`, the corresponding `handle_pad_added/3` callbacks are called just after setup is completed, but only if they are linked in the same spec where the component was spawned. Linking a pad in a different spec from the one spawning the component may lead to `handle_pad_added/3` being invoked after `handle_playing/2`.

## Playing
Once setup is completed, a component can enter the `:playing` state by invoking the `handle_playing/2` callback. Note that:
- Components spawned within the same `:spec` always enter the `:playing` state simultaneously. If the setup process for one component takes longer, the others will wait.
Once the setup is completed and appropriate `handle_pad_added/3` are invoked, a component can enter the `:playing` state by invoking the `handle_playing/2` callback. Note that:
- Components spawned within the same `:spec` always enter the `:playing` state simultaneously. If the setup of the one component lasts longer, the others will wait.
- Elements and Bins wait for their parent to enter the `playing` state before executing `handle_playing/2`.
- By default, after `handle_setup/2`, a component's setup is considered complete. This behavior can be modified by returning `setup: :incomplete` from `handle_setup/2`. The component must then mark its setup as completed by returning `setup: :complete` from another callback, like `handle_info/3`, to enter `:playing` playback.

## Processing the data (applies only to Elements)
After `handle_playing/2`, Elements are prepared to process data flowing through their pads.
After `handle_playing/2`, Elements are prepared to process the data flowing through their pads.

### Events
Events are one type of item that can be sent via an Element's pads and are managed in `handle_event/4`. Events can travel both upstream and downstream relative to the pad’s direction.
Events are one type of item that can be sent via an Element's pads and are managed in `handle_event/4`. Events are the only things that can travel both upstream and downstream relative to the pad’s direction - all other types of data sent through pads have to follow the direction of the link.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Events are one type of item that can be sent via an Element's pads and are managed in `handle_event/4`. Events are the only things that can travel both upstream and downstream relative to the pad’s direction - all other types of data sent through pads have to follow the direction of the link.
Events are one type of data that can be sent via an Element's pads and are managed in `handle_event/4`. Events are the only items that can travel both upstream and downstream - all other types of data sent through pads have to follow the direction of the link.


### Stream Formats
The stream format, which defines the type of data carried by `Membrane.Buffer`s, must be declared before the first data buffer and is managed in `handle_stream_format/4`.

### Start of Stream
This callback (`handle_start_of_stream`) is activated just before processing the first `Membrane.Buffer` from a specific input pad.
This callback `handle_start_of_stream/3` is invoked just before processing the first `Membrane.Buffer` from a specific input pad.

### Buffers
The core of multimedia processing involves handling `Membrane.Buffer`s, which contain multimedia payload and may also include metadata or timestamps, all managed within the `handle_buffer/4` callback.
The core of multimedia processing is handling `Membrane.Buffer`s, which contain multimedia payload and may also include metadata or timestamps. Buffers are managed within the `handle_buffer/4` callback.

### Demands
If the Element has pads with `flow_control: :manual`, entering `:playing` playback allows it to send demand using `:demand` action or to receive it in `handle_demand/5` callback.
If the Element has pads with `flow_control: :manual`, entering `:playing` playback allows it to send demand on input pads using `:demand` action or to receive the demand via output pads in `handle_demand/5` callback.

## After processing the data
When an element determines that it will no longer send buffers from a specific pad, it can return `:end_of_stream` action to that pad. The linked element receives this in `handle_end_of_stream/3`. The parent component (either a Bin or Pipeline) is notified via `handle_element_end_of_stream/4`.
When an Element determines that it will no longer send buffers from a specific pad, it can return `:end_of_stream` action to that pad. The linked element receives it in `handle_end_of_stream/3`. The parent component (either a Bin or Pipeline) is notified via `handle_element_end_of_stream/4`.

## Terminating
Typically, the last callback executed within a Membrane Component is `handle_terminate_request`. By default, it returns a `terminate: :normal` action, concluding the component's lifespan with the reason `:normal`. This behavior can be modified by overriding the default implementation, but ensure to return a `terminate: reason` elsewhere to avoid termination issues in your Pipeline.
Typically, the last callback executed within a Membrane Component is `handle_terminate_request`. By default, it returns a `terminate: :normal` action, terminating the component process with the reason `:normal`. This behavior can be modified by overriding the default implementation, but ensure to return a `terminate: reason` elsewhere to avoid termination issues in your Pipeline.

## Callbacks not strictly related to the lifecycle
Some callbacks are not confined to specific stages of the Membrane Component lifecycle.
Expand All @@ -48,4 +49,7 @@ Some callbacks are not confined to specific stages of the Membrane Component lif
`handle_parent_notification/3` and `handle_child_notification/4` can occur at any point during the component's lifecycle and are tasked with managing notifications from a parent or child component, respectively.

### Handling messages from non-Membrane Erlang Processes
The `handle_info/3` callback is present in all Membrane Components and `handle_call/3` in Membrane Pipelines. These can be triggered at any time while the component is alive, functioning similarly to their substituted in `GenServer`.
The `handle_info/3` callback is present in all Membrane Components and `handle_call/3` in Membrane Pipelines. These can be triggered at any time while the component is alive, functioning similarly to their counterparts in `GenServer`.

## Default implementations
If you are new to Membrane and you just read about some new callbacks that are not implemented in your Element or Pipeline, that was supposed to work fine - don't worry! We decided to provide default implementations for the majority of the callbacks (some of the exceptions are `handle_buffer/4` or `handle_demand/5`), which in general is to do nothing or to do the most expected operation. For more details, visit the documentation of `Membrane.Pipeline`, `Membrane.Bin` and `Membrane.Element`.
46 changes: 19 additions & 27 deletions guides/timer.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
# Timer usage examples
Exampls below illustrate how to use `:start_timer`, `:timer_interval` and `:stop_timer` actions on the example of `Membrane.Source`, but the API looks the same for all kinds of the Membrane Components
Examples below illustrate how to use `:start_timer`, `:timer_interval` and `:stop_timer` actions on the example of `Membrane.Source`, but the API looks the same for all kinds of the Membrane Components

### Simple example
The example below emits an empty buffer every 100 milliseconds.

### Emit empty buffer every 100 milliseconds
```elixir
defmodule MySource do
use Membrane.Source
Expand All @@ -13,35 +15,30 @@ defmodule MySource do

@impl true
def handle_playing(_ctx, state) do
# let's start a timer named :my_timer that will tick every 100 milliseconds
# let's start a timer named :my_timer that will tick every 100 milliseconds ...

actions = [
start_timer_action = [
start_timer: {:my_timer, Membrane.Time.milliseconds(100)}
]


# ... and send a stream format
actions = start_timer_action ++ [stream_format: {:output, %SomeFormat{}}]
{actions, state}
end

@impl true
def handle_tick(:my_timer, ctx, state) do
# in this callback we handle ticks of :my_timer
# we send a stream format if it hasn't been sent yet and a buffer

maybe_stream_format =
if ctx.pads.output.stream_format == nil,
do: [stream_format: %SomeFormat{}],
else: []

buffer = [buffer: {:output, %Membrane.Buffer{payload: ""}}]

{maybe_stream_format ++ buffer, state}
actions = [buffer: {:output, %Membrane.Buffer{payload: ""}}]
{actions, state}
end
end
```

### Emit empty buffer every 100 millisecond if parent hasn't stopped you
The source below accepts following notifications from the parent:
- `:pause` - after receiving it the source will pause sending buffers. The paused soure can be resumed again.
### Advanced example
The example below emits an empty buffer every 100 milliseconds if it hasn't been stopped or paused by the parent.

The source accepts the following notifications from the parent:
- `:pause` - after receiving it the source will pause sending buffers. The paused source can be resumed again.
- `:resume` - resumes sending buffers from the paused source.
- `:stop` - the stopped source won't send any buffer again.

Expand All @@ -53,7 +50,7 @@ defmodule MyComplexSource

@impl true
def handle_init(_ctx, _opts) do
# after starting a timer, status will always be either :resumed, :paused
# after starting a timer, the status will always be either :resumed, :paused
# or :pause_on_next_handle_tick
{[], %{status: nil}}
end
Expand All @@ -66,20 +63,15 @@ defmodule MyComplexSource
]

# ... and send a stream format
actions = start_timer_action ++ [stream_format: %SomeFormat{}]
actions = start_timer_action ++ [stream_format: {:output, %SomeFormat{}}]
{actions, %{state | status: :resumed}}
end

@impl true
def handle_parent_notification(notification, ctx, _state) when ctx.playback == :stopped do
raise "Cannot handle parent notification: #{inspect(notification)} before handle_playing"
end

@impl true
def handle_parent_notification(notification, _ctx, state) when notification in [:pause, :resume, :stop] do
case notification do
:pause when state.status == :resumed ->
# let's postopne pausing :my_timer to the upcomping handle_tick
# let's postpone pausing :my_timer to the upcoming handle_tick
{[], %{state | status: :pause_on_next_handle_tick}}

:resume when state.status == :paused ->
Expand Down
6 changes: 3 additions & 3 deletions lib/membrane/bin/action.ex
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please apply the same changes as in case of pipeline/action.ex ;)

Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ defmodule Membrane.Bin.Action do
|> child(:demuxer, My.Demuxer)
|> via_out(:video)
|> child(:decoder, My.Decoder)
|> child(:ai_filter, My.AI.Filter{mode: :picasso_effect)
|> child(:ai_filter, %My.AI.Filter{mode: :picasso_effect})
|> child(:encoder, My.Encoder)
|> via_in(:video)
|> child(:webrtc_sink, My.WebRTC.Sink)
Expand All @@ -54,7 +54,7 @@ defmodule Membrane.Bin.Action do

![](assets/images/spec_without_audio.svg)

Returning another spec
Returning another spec (on top of the previous one)
```elixir
get_child(:demuxer)
|> via_out(:audio)
Expand All @@ -63,7 +63,7 @@ defmodule Membrane.Bin.Action do
|> get_child(:webrtc_sink)
```

will result in having following children topology:
will result in having the following children's topology:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
will result in having the following children's topology:
will result the following children's topology:


![](assets/images/spec_with_audio.svg)
"""
Expand Down
2 changes: 1 addition & 1 deletion lib/membrane/element/action.ex
Original file line number Diff line number Diff line change
Expand Up @@ -224,7 +224,7 @@ defmodule Membrane.Element.Action do

This action is permitted only in callback `c:Membrane.Element.Base.handle_init/2`.

The example of usage of this actions is [there](../../../guides/timer.md)
The example of usage of these actions is [there](../../../guides/timer.md)
"""
@type latency :: {:latency, latency :: Membrane.Time.non_neg()}

Expand Down
6 changes: 3 additions & 3 deletions lib/membrane/pipeline/action.ex
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ defmodule Membrane.Pipeline.Action do
|> child(:demuxer, My.Demuxer)
|> via_out(:video)
|> child(:decoder, My.Decoder)
|> child(:ai_filter, My.AI.Filter{mode: :picasso_effect)
|> child(:ai_filter, %My.AI.Filter{mode: :picasso_effect})
|> child(:encoder, My.Encoder)
|> via_in(:video)
|> child(:webrtc_sink, My.WebRTC.Sink)
Expand All @@ -47,7 +47,7 @@ defmodule Membrane.Pipeline.Action do

![](assets/images/spec_without_audio.svg)

Returning another spec
Returning another spec (on top of the previous one)
```elixir
get_child(:demuxer)
|> via_out(:audio)
Expand All @@ -56,7 +56,7 @@ defmodule Membrane.Pipeline.Action do
|> get_child(:webrtc_sink)
```

will result in having following children topology:
will result in having the following children's topology:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
will result in having the following children's topology:
will result in the following childrens' topology:

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

childrens' - are you sure? children is already plural 😛

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, you are right, children is already plural, so it should be will result in the following children's topology:


![](assets/images/spec_with_audio.svg)
"""
Expand Down
2 changes: 1 addition & 1 deletion mix.exs
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ defmodule Membrane.Mixfile do
{:bunch, "~> 1.6"},
{:ratio, "~> 3.0 or ~> 4.0"},
# Development
{:ex_doc, "0.34.2", only: :dev, runtime: false},
{:ex_doc, "~> 0.28", only: :dev, runtime: false},
{:makeup_diff, "~> 0.1", only: :dev, runtime: false},
{:dialyxir, "~> 1.1", only: :dev, runtime: false},
{:credo, "~> 1.7", only: :dev, runtime: false},
Expand Down
2 changes: 1 addition & 1 deletion mix.lock
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"dialyxir": {:hex, :dialyxir, "1.4.5", "ca1571ac18e0f88d4ab245f0b60fa31ff1b12cbae2b11bd25d207f865e8ae78a", [:mix], [{:erlex, ">= 0.2.7", [hex: :erlex, repo: "hexpm", optional: false]}], "hexpm", "b0fb08bb8107c750db5c0b324fa2df5ceaa0f9307690ee3c1f6ba5b9eb5d35c3"},
"earmark_parser": {:hex, :earmark_parser, "1.4.42", "f23d856f41919f17cd06a493923a722d87a2d684f143a1e663c04a2b93100682", [:mix], [], "hexpm", "6915b6ca369b5f7346636a2f41c6a6d78b5af419d61a611079189233358b8b8b"},
"erlex": {:hex, :erlex, "0.2.7", "810e8725f96ab74d17aac676e748627a07bc87eb950d2b83acd29dc047a30595", [:mix], [], "hexpm", "3ed95f79d1a844c3f6bf0cea61e0d5612a42ce56da9c03f01df538685365efb0"},
"ex_doc": {:hex, :ex_doc, "0.34.2", "13eedf3844ccdce25cfd837b99bea9ad92c4e511233199440488d217c92571e8", [:mix], [{:earmark_parser, "~> 1.4.39", [hex: :earmark_parser, repo: "hexpm", optional: false]}, {:makeup_c, ">= 0.1.0", [hex: :makeup_c, repo: "hexpm", optional: true]}, {:makeup_elixir, "~> 0.14 or ~> 1.0", [hex: :makeup_elixir, repo: "hexpm", optional: false]}, {:makeup_erlang, "~> 0.1 or ~> 1.0", [hex: :makeup_erlang, repo: "hexpm", optional: false]}, {:makeup_html, ">= 0.1.0", [hex: :makeup_html, repo: "hexpm", optional: true]}], "hexpm", "5ce5f16b41208a50106afed3de6a2ed34f4acfd65715b82a0b84b49d995f95c1"},
"ex_doc": {:hex, :ex_doc, "0.36.1", "4197d034f93e0b89ec79fac56e226107824adcce8d2dd0a26f5ed3a95efc36b1", [:mix], [{:earmark_parser, "~> 1.4.42", [hex: :earmark_parser, repo: "hexpm", optional: false]}, {:makeup_c, ">= 0.1.0", [hex: :makeup_c, repo: "hexpm", optional: true]}, {:makeup_elixir, "~> 0.14 or ~> 1.0", [hex: :makeup_elixir, repo: "hexpm", optional: false]}, {:makeup_erlang, "~> 0.1 or ~> 1.0", [hex: :makeup_erlang, repo: "hexpm", optional: false]}, {:makeup_html, ">= 0.1.0", [hex: :makeup_html, repo: "hexpm", optional: true]}], "hexpm", "d7d26a7cf965dacadcd48f9fa7b5953d7d0cfa3b44fa7a65514427da44eafd89"},
"excoveralls": {:hex, :excoveralls, "0.18.3", "bca47a24d69a3179951f51f1db6d3ed63bca9017f476fe520eb78602d45f7756", [:mix], [{:castore, "~> 1.0", [hex: :castore, repo: "hexpm", optional: true]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm", "746f404fcd09d5029f1b211739afb8fb8575d775b21f6a3908e7ce3e640724c6"},
"file_system": {:hex, :file_system, "1.0.1", "79e8ceaddb0416f8b8cd02a0127bdbababe7bf4a23d2a395b983c1f8b3f73edd", [:mix], [], "hexpm", "4414d1f38863ddf9120720cd976fce5bdde8e91d8283353f0e31850fa89feb9e"},
"jason": {:hex, :jason, "1.4.4", "b9226785a9aa77b6857ca22832cffa5d5011a667207eb2a0ad56adb5db443b8a", [:mix], [{:decimal, "~> 1.0 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm", "c5eb0cab91f094599f94d55bc63409236a8ec69a21a67814529e8d5f6cc90b3b"},
Expand Down