- cross-posted to:
- technology@lemmy.ml
- cross-posted to:
- technology@lemmy.ml
You must log in or # to comment.
Most of us here think streaming is useful, despite the fact that streaming services are based on the idea that they own and control all access to media with you renting it from them, raising prices, shoving ads in your face, and removing media from the collection you’re renting from them. Our solution in this community is investing in the open source community around Jellyfin, the Arr stack, etc. so we can still enjoy the benefits of streaming services while owning our own content and not having to financially support companies we don’t agree with. I’d say a lot of us here who are happily using Jellyfin might otherwise have a streaming account if it didn’t exist. Admittedly, I used Netflix until I realized I had better options.
I think the same is true with local LLMs. Not all of us agree that LLMs are useful, but most of us here agree that a few tech giants tightly controlling LLMs and renting them to everyone is not going to be a good thing. Without self-hosted LLMs, many people who do find value using them will go ahead and financially support the rent-seekers who are hell bent on destroying the world for their own financial gain, as well as support them by sharing data that can be used to train their models. Even when you use the free tier of Chat GPT, for example, you’re supporting OpenAI by giving them your prompts that they can use to make their models better.
The ecosystem around running open weight models is rapidly evolving. I’m already running the Qwen 3.6 MoE model with the desktop beta of OpenCode on my gaming laptop and it’s pretty decent. Personally, I’ve found ways to use LLMs where they are actually useful and not just slop generators, though I initially thought they were useless before I spent a lot of time working with them. I’m all for supporting and contributing to this ecosystem so that people can use LLMs without giving their money and data to shithead psychopaths.
👏 This is the way to do it!
Mmm… better that all AI fail.
Perhaps in Candyland, or maybe even the land of chocolate. But I think in the real world it’s here to stay.
Good luck!
I just wish we could invest the time/money/resources into compressing AI and making it smaller and more efficient. I’d so much rather have a somewhat capable AI that can be run locally and offline, to outsource menial tasks to like alphabetizing spreadsheets and so basic image modification, than to have to upgrade my hardware constantly or use cloud based SaaS and/or have newer models that are more accurate in their predictions.
Of course that assumes a lot of things, like the intent to help people and not make money. Maybe someone in the Linux-sphere will make something.
Look into Bonsai Ternary models. They’re “1.5” bit models that have to be trained that way (so no taking a full model and quantizing it down) but they are so efficient and they can run on CPU only, though it’s a bit alpha at the moment. Really cool company and projects.
You have to create a specific environment for them though, using Bonsai’s GGUF version which enables them to run properly. So unfortunately, no use in LM Studio yet.
I would like to see one integrated into a gnu os like linux where its only capability is to understand the os and guide you through it. No generation and no expertise outside the os exosystem. Maybe allow for it to be given the privelege to search the web. I would have it have capability to use other ais to perform other tasks so modules or whatnot could be added to give it more capability as a general computer butler type. Basically an os that acted like a start trek computer.
Using Pi agent with qwen 3.6 35b a3b running with llamacpp on my GPU feels a lot like that. I have a script that watches my downloads folder and keeps it organized, and it used to just get the file extensions and move things based on type, now with a local llm in the loop, it moves things based on what it is, and what it is for. If I download a PDF file from work, it automatically reads the first page, figures out what its about, and moves it to my work documents.
“Whichever port that docker container is on, make it this one”
There are efforts there. The new Deepseek 4 compresses a lot of its knowledge using something they call engrams. But it’s unfortunately still too big for a consumer GPU.
Gemma 4 is small enough to run on your cellphone.
If your GPU has at least 8GB there are a lot of options for self hosting your own local models
If I understand correctly, if we actually said “this model is great, let’s put a pin in it”, then it could be turned into a dedicated chip that would be much more efficient and perhaps even something that could get embedded in consumer hardware - but then you are just stuck with that model instead of “the next shiny new model” that they keep making.
I feel like there is a future of more targeted AI. At the moment something that does spreadsheets has to carry knowledge of programming and chemistry and lots of languages and this seems very heavy for what ultimately we need. A programming language focussed AT dedicated to Rust or Go or Java could potentially be quite a bit smaller especially if they focussed on algorithm snippet and auto complete smarts. There is definitely a market for smaller more targeted uses than these all encompassing chat bots where the goal is to move the state of the art on for existing algorithms.
It’s like people are starting to realize what the “luddites” were saying from the beginning.
This tech is not from the people to the people like the web, it’s from big corpos to fuck you.
You won’t have that tech, you will rent a highly modified one at best, built with the purpose to manipulate you.
Remember: there is a club out there, and you are not part of it.
You must be new to the IT then.
Clouds are only US, and they hold all the data and traffic for most of the world.
Before them, operating systems - everyone and their mother using Windows.
Phones? Google or Apple.
Communication? Teams. Shared documents? Exchange. Countries are balls and hair deep in the US since the 90s and nobody really cared to do anything about it. SAP, Sellsforce, Palantir, Jira, Confluence, Github, many others I’m not even aware of, all US companies holding everything that companies and countries need in the US way before any AI. Which doesn’t even fucking work.
None of this is new.
Alternatives are there, but they need the money everyone sends to the US.
Except SAP is German.
My bad.
They are still on the same level though.
Yeah, that’s what I really don’t get.
Why would any serious company think it’s a great idea to outsource all your intelligence work to a handfull of US companies, making yourself wholely dependant on their goodwill and the goodwill of the US government?
It’s the same delusion and corruption from a neoliberal corporate-whore political class that led to every country growing dependent on US tech companies, and Chinese manufacturing. They represent corporations; not their constituents.
If they’d all committed to open source and domestic companies for support and infra, the compounding effect of tens of thousands of engineers across governments working on linux and other FLOSS products would have made everything significantly cheaper and more efficient for all of them in a matter of years, compared to paying a foreign tech company for everything in perpetuity… and that’s before you consider the multitude of other risks and vulnerabilities to national security.
As a good example look at the national police in france. They ditched windows for a custom linux distro and are now saving like 10 million or more euros a year. That’s money that used to come from taxpayers to a foreign company and now that money can stay local and help improve the taxpayers lives instead of buying someone a 3rd yacht.
I wish all these European Linux projects would pool resources and create one good solution instead of each little country or even city DIYing their own solution.
So much this.
If we take a look at how the current AI behemoths got there, there’s a trail that goes straight from stolen data to proprietary models. They are charging their users for the privilege of using better aggregated public data. I hope that, when they raise prices once again and more and more users are cut off their larger models, people would understand where their place is, according to corpos.
Did anybody click the link to read the paragraph-long website text? Not only is it packed with assumptions about AI, it’s simply unhinged.
The ability to… run intelligence systems without asking permission is of existential importance.
Existential?! No it’s not.
This mirrors the delusion Sam Altman demonstrated when he insisted nobody could raise a child without AI.
I don’t think they mean that having AI is vital. I think they mean it’s vital that the rich not have a monopoly on AI.
Regardless whether AI is useful, its production would never have become this harmful if a rich person didn’t think they could own it and make money with it. Making a thing publicly accessible makes it less attractive to capitalists.
“Of existential importance” literally means they think it’s vital to existence.
Yes it is. If the rich and governments have it, then it’s existential. Since it’s here, it’s now existential that the people can run these intelligent systems as well.
You all trying to curtail this tech by making it illegal in various capacities for the people is fighting for the rich, the mega corps, and the empires of this world.
If the rich and governments have it, then it’s existential.
The rich have private planes too. They are not existential. They are frivolous.
It is now. If you want it not to be, go ahead and stop all this modern bullshit happening around me.
You can’t yell at people on the ocean that they don’t really need boats, and then attack boats. I mean you can, but it’s foolish and unproductive.
AI is not saving you from anything existential.
If you believe you’re drowning without its salvation, you’re the foolish one.
I think giving language a fixed agency in human society 4,000 years ago by permanently structuring it, binding it to physicality, and giving it violent control over people was a bad idea. But gurl, unless you are going back to pre written language, all this living in the modern world and yelling about “certain parts” of technology, is one of the most ignorant stances a thinking being can take.
Unless you are taking us back back, you are a posturing, uneducated, little baby who has drank so much of the Kool aid she can’t see the forest she is in at all.
unless you are going back to pre written language, all this living in the modern world and yelling about “certain parts” of technology, is one of the most ignorant stances a thinking being can take.
Ahh I get it, thank you. The machine is hurting you specifically now and so you’re lashing out.
“The other stuff, that was cool, but daddy said we were good and everything was chill while we watched the degenerates and the environment burn! but that was a lie! We’re not chill! And now Daddy wants my flower too!”
They’re going to do to your AC what they did to the buffalo🤣
What
Datals took mer jerb! Der turk mur jerb!
That’s not what they said.
The Chinese are doing a good job with open source ai models, especially Z. Ai, qwen and minimax, Google also got us something decent with gemma4, but yeah we need commercial AI to fail so we can get affordable access to vram…
The word “intelligence” is doing a lot of heavy lifting here. LLMs lack any mechanism for true logical reasoning, and they always will by nature. This is why they fail at simple questions like “the car wash test”. It’s also why agents are expensive; They just flail around in token hungry “reasoning loops” until they happen to come across a correct solution. And it’s why Claude Opus 4.8 (High) only scores 1.5% on the ARC-AGI-3 benchmark at a cost of $10,000.
This kind of panic is just part of the hype. Wake me up when real intelligence arrives.
Computers are not actually intelligent. What are they but big calculators? They are removing jobs and barely doing anything better! They take up a whole room! No one needs that!
The internet is hardly worth anything. It’s nothing but scams and porn! You can’t believe anything you read on it! It’s overhyped nonsense! No one is going to use that!
Meanwhile in 2026: LLMs have no logical reasoning and that will never change! They can’t answer simple questions! They just flail around and produce slop!
You guys don’t see it? Really?
It doesn’t need to be our new cyber god for it to matter. All the software developers I know use AI as a tool. One person I know who heads a department at a publicly traded company you’ve probably heard of said they’re not hiring more junior developers, and is worried about his long term career outlook.
That’s concerning right? Capitalists who have a choice between hiring ten people or hiring one person who supervises an AI doing the work of nine are going to choose the cheaper option, and pocket the difference.
To say that LLMs are not useful to me indicates a lack of familiarity. I’m not taking about bullshit “write this email for me” type stuff, I mean like “write this web app for me” “find this type of document in this massive trove of documents” “troubleshoot this technical issue”. And it can do it.
LLMs are not replacing humans, but they are reducing labor required for many tasks. Who receives the benefit from that reduced labor? Right now, owners. That’s why I support open source AI or preferably democratic-entity-only open source AI.
OpensourceAI MustWinLoseAll AI should stop
So, say it won’t. What’s your plan B? You’ll probably need it.
It’s not going to. Saying all AI should stop is like saying “all gunpowder should stop”. Would everyone be better off if no one was using it? Maybe. But that’s not going to happen at this point, and will give you a strategic disadvantage if you unilaterally disarm.
And to be clear I’m not talking about using AI to write a screenplay or something creative, I mean like using it to write software, to optimize industrial processes etc.
Gunpowder has some very good uses.
AI has all the worst use-cases while frying the planet.
AI will kill us not because it becomes sentient, but because we have turned our brains off and disregarded the coming extinction event.
We don’t need software for everything. We need to scale down our evergy usage, and we can do that ourselves.
You’re right that global warming is an existential threat. And right now the power grid is contributing to it.
At the same time, we’re seeing constant innovations in solar power and battery technology, while the cost of both are going down. Solar is now the cheapest source of energy per unit. There is enough energy from the sun to cover multiples of all current energy use by humans.
AI models started out as extremely energy intensive, but are becoming increasingly more efficient. You can now run models on your phone with abilities which previously required a server running dedicated graphics cards.
You’re also right that we shouldn’t outsource critical thinking. Still to say AI has “all the worst use cases” is wrong. Yes, it has plenty of bad use cases, by many great ones. I use it in my day to day work to save time.
Something like a law banning the use of AI will never pass, and if it did, other countries wouldn’t also pass it. A country which uses AI will outperform one that does not (economically, although probably militarily as well). In that scenario, leaving AI capability in the hands of private enterprise makes us subject to the whims of rich elites. For an analog see the case of Elon Musk turning Starlink on/off in Ukraine. Which again is why I do think there’s value in open source AI, and more specifically AI only available to democratically controlled entities - nation states but also things like worker cooperatives to compete with corporations.
What are we doing in life that we need ridiculous autocomplete to save time at work?
If we wanna talk about absolutes, we don’t need commute at all, and cars, and we can just build trains and live happy lives without consuming so much shit.
AI is not needed anywhere at all. Unlike food (vegan), water, heat, and shelter.
Which many people don’t have.
You’re right, we don’t “need” most modern conveniences. But they save us time and provide an improved quality of life. Why do we need power tools? Why motors? Why electricity, running water, artificial light? I joke, but AI is one of a long chain of tools designed to save us time, or extend our capabilities and reach. Why are you particularly opposed to this one over others? I return to my argument: AI is concerning because of its potential for change. I don’t think we can prevent it from existing, but we can make it accessible to normal people and not solely the domain of the rich.
I’m talking mostly about ads and random plastic shit polluting the earth. Which ties to planned obsolence.
AI makes sense in many places, but fuck LLMs, we can do better.
“All AI should stop” is a far more reasonable opinion than the one presented by the
websiteblog9-sentence rant linked above.You have far more in common with the person you’re pushing back on than the author of the rant.
AI is a civilizational infrastructure for work, education, science, software, creativity, public services, and national capacity.
I bet me and the person I’m responding to have a ton in common. Likewise me and the person who wrote that post. But I don’t relate to the position of “all AI should stop.”
I think there should be democratically accountable AI development. I’m interested in open source but the problem with it is that private corporations take advantage of it until they get big enough to try to lock out competitors. (See Google with Android) I would love to see an “open source” license which allows use by individuals and democratic entities, but forbids use by private enterprise.
Unless you want to vouch for the rant site’s claim that AI is necessary for civilization and existence, you have almost nothing in common with his fearmongering. And if you do believe that, that is disappointing.
I don’t see it as fear mongering. What do you think of open source software like Linux, Firefox, VLC and so on? Because a similar argument is made there - we need to use these tools day to day, and the only option being to pay a corporation to use them would make us literally poorer, and we also wouldn’t have the features and quality we see in Windows, Mac, Chrome etc if there weren’t open source competitors.
Is lack of open source AI currently a civilizational existential threat? No. Could it be if current trends continue? Potentially. Things could get pretty dystopian if only the mega rich control tools with ultra-intelligence.
There are some open models right now, but they’re mostly ones created by private enterprise that were released to the public. Creating models is more resource intensive than making open source software. So it makes sense to pool resources for the public good.
Ah, the fear of the potential thing as described by the ultra-rich.
Why do you trust them to be the ones who shape your fears?
I don’t think you’re engaging with what I’m saying sincerely, and I’m trying to do that with you.
game theory, my friend. You can’t stop if there’s a chance someone else is building it. The only way to prevent this would be some sort of AI proliferation treaty like we have for nukes but nukes you test by making large explosions which are easy to detect. How are you going to detect some state backed group in a bunker somewhere disconnected from the internet developing a super AI? I’m fully against ai but the cat is already out of the bag.
Monitor fossil fuels burning.
This won’t work on china in the near future.
Solar solar solar
Thats-the-point.jpg
go solar, then nobody gets hurt
Install it and then “ollama run olmo-3:7b” gets you a local AI. If you want to run a smarter AI, then you’re going to need a bigger parameter model, which is going to take more hardware to run.
Not many people can afford to run their own models.
Not the big ones, no. But I think almost everybody can run one with seven billion parameters.
Considering that the vast majority of PCs in the world have 8GB of RAM at most, I still doubt it.
I ran it on my 8GB RAM desktop.
I mean if that’s all that would be loaded in memory, sure.
I got Qwen 3.5:9b running on my 8GB GPU the other day, and it still has some room left over
I was talking about combined system RAM. People often overestimate what the average system specs are.
It depends what the game is, and I don’t think the people that think they know what the game is know what the game is. They believe they are ushering in something that will set them apart from the general population but which in fact will probably not view them as anything of more value than the rest of us.
we don’t need opensource AI, we need to destroy AI
You can run AI without causing environmental problems just like you can drive cars without burning fossil fuels and you can have industrial production without creating pollutants.
All of that just cuts into the profits though.
Then prove it. The whole system starting from creation to use. Otherwise, this is just a thought experiment in the same vein as the debatelords who say “buh murder can be ethical in some cases”.
While real people are suffering in reality thanks to the toy you like, the fantasy deflections just enable that harm.
deleted by creator
Unintentionally funny since murder is the illegal killing of humans so making it legal makes it not murder.
the way the technology is run and built is in itself despicable. It is an abomination.
So is modern food production. Doesn’t mean stop eating.
This is a creepy statement. Human life is not equatable to a billionaire’s toy, and it is dangerously immoral to suggest it is.
The fact that consumers have LLM garbage shoved in their face doesn’t mean that the exponential increase in other ML use cases isn’t driving life saving technology. The COVID vaccine only deployed as fast as it did because of AI ML. 20% of surgeries right now are robot assisted and all of those use AI ML.
On a personal level I’ve been able to move my family to FOSS because of LLMs. LLMs are very good at understanding how open source software works and translating the workflows to human language. I personally can read man pages but others in my family honestly don’t want to have to in order to just watch a show on AppleTV. I can do this with a small local LLM running on a low wattage micro server.
We are seeing LLMs enabling us easily to overwrite the proprietary software in our home devices and take back hardware and improve repairability so we can stop producing so much E Waste. And the more that companies use LLM code to slop up their embedded devices the easier it becomes.
deleted by creator
