Adding video with a code-first approach

[Venthe]

Hey,
I am unable to add video with the code-first approach. What I am trying to do is to load and use an arbitrary file during a runtime.

Please note, there are several issues with my code; all described below. Also, this is the first couple of hours with Stride; I am a dev by trade - just not in a game dev, and not in c#. The closest I've got is:

var videoAsset = new VideoAsset();
videoAsset.Source = @"<AbsolutePathToTheNewRoot>/Movies/Trailer.bik";
videoAsset.Width = 100;
videoAsset.Height = 100;
var location = new UFile("Movies/Trailer");
var assetItem = new AssetItem(location, videoAsset);
assetItem.SourceFolder = @"<AbsolutePathToTheNewRoot>";
var logger = new LoggerResult();
var indexName = VirtualFileSystem.ApplicationDatabaseIndexName;
var builder = new Stride.Core.BuildEngine.Builder(logger, BuildPath, indexName) { BuilderName = "TestBuilder" };

var assetCompilerContext = new AssetCompilerContext();

var result = new VideoAssetCompiler().Prepare(assetCompilerContext, assetItem);
builder.Root.Add(result.BuildSteps);
builder.Run(Stride.Core.BuildEngine.Builder.Mode.Build);
var video = game.Content.Load<Video>(location);

VideoComponent component = new VideoComponent();
component.Source = video;
component.LoopVideo = true;
component.PlayAudio = true;


var entity = new Entity("Video");
entity.Add(new VideoManager());
entity.Add(component);

// ---

public class VideoManager : SyncScript
{

    public override void Update()
    {
    }

    public override void Start()
    {
            var videoComponent = Entity.Get<VideoComponent>();
            videoComponent.Instance.Play();
    }
}

My reasoning:

1. VideoComponent requires VideoInstance
2. VideoInstance is created from Video attached to the VideoComponent by the VideoProcessor
   1. VideoProcessor requires the VideoComponent to be present
   2. VideoInstance requires VideoComponent.Source.FileProvider to contain the DataUrl from a Video on a FileProvider -> "Video files needs to be stored on the virtual file system in a non-compressed form."
   3. Hopefully, by using STRIDE_GRAPHICS_API_DIRECT3D11 and https://github.com/stride3d/stride/blob/master/sources/engine/Stride.Video/VideoInstance.Direct3D.cs I can avoid FFMpeg requirement on the target computer.
3. Video is created by the VideoAssetCompiler.
   1. The only way I've found to use compiler is to set up a new Builder instance alongside VideoAssetCompiler. It seems like neither are present within the Game.Services. Builder was created according to Stride's own deprecated tests.
   2. ❗VideoAssetCompiler uses new ContentManager(MicrothreadLocalDatabases.ProviderService); which does not seem to share the space with Game.Content; so I have no access to the result with var video = game.Content.Load<Video>(location); in my code, as in - "file not found". Manual inspection of the Game.Content's does also confirm this.
      1. VideoAssetCompiler's ContentManager is hard-coded and can't be injected as a strategy.
   3. ❗VideoAssetCompiler on windows has a hard limitation - FFMpeg must be present either in the directory of the executable; or the ../../Content; even when my system has FFMpeg in the PATH; it will not be cosidered due to ToolLocator.LocateTool if(!OperatingSystem.IsWindows()).
      1. ToolWindow behaviour is hard-coded and can't be injected as a strategy.
      2. FFMpeg as-a-converter is hard-coded and can't be injected as a strategy.
      3. ❗FFMpeg is also required even if the conversion is unnecessary
   4. With manually-copied FFMpeg I've managed to run the compilation; but even then the file is not in the same file provider.

What I am trying to achieve:

1. Load a video file in *.bik format to play as a intro video.
2. Video file is loaded at runtime from a user-given location.

What are my "wishes"

The best situation for me would be to stream the video from the original format, using e.g. similar approach to https://github.com/nickdu088/Bink-Unity-Player/blob/main/Assets/Scripts/BinkPlayer.cs ; reusing VideoInstance logic to modify the VideoTexture.

I can accept copying the video to the VirtualDisk, but that adds unnecessary latency if I understand it correctly.

I can also understand the need to convert to an uncompressed format, but then I should not have to rely on the VideoConverter that is intrinsically tied to the Asset's. and FFMpeg

e: So ideally,

Video video = Video.LoadStreaming(new BinkVideoStreamer(@"Path"));
VideoComponent component = new VideoComponent();
component.Source = video;

var entity = new Entity("Video");
entity.Add(new VideoManager());
entity.Add(component);

[VaclavElias]

@Doprez , do we have any example on this? I recall others trying it but not sure where it finished.

Also some other references here:

• Content Loading at runtime #176
• Assets Loading #8

[Venthe]

@VaclavElias am I correct in the assumption that Stride is not usable at the moment with Code First? I've tried another thing, with scene management. The issue boiled down to same; and the solution seemed possible; by swapping ContentManager within the Game class. Unfortunately; while I can override the PrepareContext; I don't have the access to the Content variable as it is a private set

I do realize that my case is a bit against the grain, but a lot of elements of the engine seems intrinsically tied to one or the other implementation without the possibility to swap them :(

[VaclavElias]

I don't have a definite answer for you at the moment because I haven't gotten that far yet, and I mostly do white-boxing without any assets in Code First.

When I implemented a skybox for Code First, it took me a while to figure out how to do it, but I managed.

So, the answer might be:

• Many scenarios are possible, but we don’t yet know how to accomplish them in Code First.
• Some scenarios require only minor tweaks in the engine.
• Some scenarios need major updates in the engine to fully support Code First.

The maintainers would consider any PRs that improve the Code First experience. Actually, Code First itself, including using F# and Visual Basic required some tiny engine updates.

I believe Stride is usable with Code First for certain scenarios.

[Doprez]

Hey sorry, I missed the @ somehow. Unfortunately there arent any resources/examples for videos that I know of.

If you are already at the point of trying to modify the Game class you might have better lock inheriting from the GameBase class instead. I am currently looking into modifying the startup as there are a lot of internal systems that can make things like that hard.

I will also add this to the other checklist for code first examples here