Oh yes. Ok, that's probably on bash, but you look at the script and it's like 200 lines of code. Then you read the alternate install instructions and it goes like "download binary, make executable, add to $PATH, run" - ???
Text is misleading too. 5-7 tok/sec is not reading speed, it's a tad slower. For me, at least, and I am an experienced reader, not especially schooled in quick-reading though.
I happened to "live" on 7.0-7.5 tok/sec output speed for a while, and it is an annoying experience. It is the equivalent of walking behind someone slightly slower on a footwalk. I dealt with this by deliberately looking away for a minute until output was "buffered" and only then started reading.
For any local setup I'd try to reach for 10 tok/sec. Sacrifice some kv cache and shove a few more layers on your GPU, it's worth it.
An example for this is the Blender addon ecosystem. Blender moves very fast, breaking API changes every few versions. Now I am not an addon developer myself, but from github issues I follow, changes are fairly often trivial to do.
Yet, someone has to do them. Ideally it is the creator of the addon, sometimes it's the users who do it, when the addon is not maintained anymore (in case of trivial changes).
It kinda works that way, but it also is some kind of gamble for the user. When you see a new addon (and a new addon developer), you can't know if they gonna stick to it or not.
If you have to pay for the addon, it's more likely they maintain it, of course. But also not a guarantee.
LM Studio has an option on model load that I believe does what you describing here: "K Cache Quantization Type" (and similar for "V"). It's marked as experimental and says the effect is basically hard to predict. Never tried myself, though.
These 200 LOC install scripts turn me heavily off as well. But at least in this case, you can also just download the correct zip, extract the binary and do "./llmfit".
I just tried exactly the same (at https://loops.video ), but I was able to watch without account, and registering afterwards also worked. Guess it's something on your side.
I can too, now. I wonder if any changes were made or if it was just a problem on my side.
Checking loops.video now, these were the first 5 videos I saw, in order:
1. Left-Wing American Politics
2. Promotion of the Fediverse and Loops
3. Left-Wing American Politics
4. A Non-English Play
5. Left-Wing American Politics
6. Stop Motion Flipbook Thing
7. Advocation for Loops Itself and Decentralization
8. Loops Promo
9. Left-Wing American Politics
So out of the first 9 videos, 4 centre around American politics, 1 I couldn't understand, 3 were promotion for the service I was currently using and only one was interesting and understandable.
I don't have a Loops account, but check multiple sites for news and information, landing on the loops homepage several times. I haven't needed a login to see videos appear for some time.
If it's anything like the rest of the Fediverse applications, it's meant to give you a full chronological feed of people you subscribe to. While several of these sites seem to have a simple trending page, one of the themes of the Fediverse seems to be getting away from overly predatory algorithms and leaning into letting people curate their own feeds and interactions again.
It sounds a lot like a "be the change" situation. If you want to see other stuff, follow people you like instead of drinking from the hose. It's still a small site, so if you don't see the content you want, then make it or build the community there.
These sites can also have basic interoperability. I don't know if the Loops UI supports subscribing to people in other Fedi networks yet, but I've seen people say Loops videos have started trickling onto Mastodon.
The output looks pretty useful. It got a bit weird when I wanted to explore alternative branchens and nodes started to overlap each other (I tried free/unregistered, if that helps).
What is to note here, this is without export templates, these are ~800MB extra (200 per platform, but it seems like you can download only all at once nowadays).
Engines like Unity and UE include those in the primary download already.
reply