Support for falcon models

[AntoineCharton]

Hello :)
Really nice repo!

https://docs.gpt4all.io/
Falcon transformer architecture seems to be supported by gpt4all is it possible to add it to the library?

[Macoron]

This repo uses relatively old version of gpt4all, which sadly doesn't support Falcon and some newer architectures. This could be fixed by updating compiled libraries and C# bindings.

I'll probably do this in the future, but if anyone want to do PR with update - I'll glad to merge it.

[AntoineCharton]

Ok will try this week-end. Anything you did in particular for the build that's not on the instructions?

[Macoron]

Not really. Check build_cpp.sh and build_cpp.bat for Mac and Windows builds.

Right now gpt4all.unity compiled using this commit:
https://github.com/nomic-ai/gpt4all/tree/f4a1f7340c76e7d925035fda89315aaeb69f09f0

After that they changed C-API. You would need to rewrite C# bindings for that:
https://github.com/nomic-ai/gpt4all/blob/main/gpt4all-backend/llmodel_c.h

[AntoineCharton]

To make sure I understand. Is LlmNative.cs a gpt4all script or your own binding? Looks like they did a lot of refactoring around that.
The equivalent now in the samples seem to be Nativemethods.cs

[Macoron]

To make sure I understand. Is LlmNative.cs a gpt4all script or your own binding? Looks like they did a lot of refactoring around that. The equivalent now in the samples seem to be Nativemethods.cs

All .cs files are written by me for Unity. LlmNative contains bindings to C functions from llmmodel_c. As you said there a lot of changes, so current bindings won't work.

I believe gpt4all has its own C# bindings, however I didn't use them yet.

[AntoineCharton]

Yeah the bindings projects don't seems to work. Looks like someone did a whole new build pipeline for them but it's not documented anywhere.
https://app.circleci.com/pipelines/github/mvenditto/gpt4all
Gonna ask how is it supposed to be used.

[Elevons]

Would love to have this also support the new Orca interface as well.

[chadb101]

can we expect an update version of this at any point? tried to update myself w/ current gpt4 to bit quant models working but cant do it

[AntoineCharton]

It's actually a big job. Not just about recompiling everything. I have only a few hours a week to spend on this if you need it right now you will have to do it yourself. Maybe ask the gpt 4 all team about your issue?

[Elevons]

I am offering a $400 bounty to anyone who can update this plugin to support the models that Gpt4all supports.

[dreemur99]

It's actually a big job. Not just about recompiling everything. I have only a few hours a week to spend on this if you need it right now you will have to do it yourself. Maybe ask the gpt 4 all team about your issue?

I'm pretty sure they were talking to Macoron.

In either case, would love to see this updated to modern GPT4All as well. Growing demand for this integration. Dude should start a Patreon.

[baroquedub]

Adding my support for the update request. And all in on the idea of a patreon too.

I don't know much about these things but I find Orca and Falcon much quicker and less resource intensive (running in the windows installer UI) whereas this project hits my CPU hard and is slow to respond. What a great project though. Thank you @Macoron

[Bl4ckh34d]

Same here, demand is definitely there. Good opportunity to earn some good bucks.