What do you think about AI?

I think

Lumeinshin
Just from reading your post, Isn't that what Siri was supposed to be 10 years ago?
Anyway, it's seems a bit overindulgent to me to have my voice command sent accross the planet to have it processed in an whateverflop tier datacenter
just to get a calendar entry synced back to my phone.

A voice alone assistant will never work in that form factor because a lot of people don't want to be shouting at their shirt in public.

Lumeinshin I've seen many people shitting on these, especially the humane AI pin. But honestly? I like the concept.

The execution is absolutely terrible, of course. AI still hallucinates, you can't rely on it for precise or day to day tasks that depend on an external data source, such as current events or news. It's practically an unusable product/service. You talk to it, you wait in an awkward silence, and you get your unreliable response that you might have to double-check. It's completely awful, and I don't see why anyone would use this today. Tomorrow though? I think there's potential for it.

I can see the market for non-intrusive day-to-day devices that can help people do everyday tasks or live the life while answering those small questions that pop right into our heads without having to go out of their way by moving aside in the middle of a walk, unlocking their phones and typing out the question. It's like having a normal person with you, it feels natural! I love it, to be honest.

The projector looks like a nice feature too! Sure, it's pretty bad in its current state, but if I could see my next appointment, read a quick message/email, or check the weather in it, I'm sold! I don't want to keep talking to it out loud in a public place, or even worse, a quiet public place like a train. It'd be cool to just check basic info in it, again, without having to open your shiny addictive brick with 100 ways to keep you hypnotized.

Would I wear one? ...not until it goes open source. I'm not letting them put a camera on me 24/7 (he said cluelessly, while he had a smartphone in his pocket). Jokes aside, it might not be my thing right now, but I absolutely dig it.

...except for the overhyped AI part.

And about the Rabbit R1? No comment

1996FordMondeo
Science fiction is cool until it becomes science reality.

Evernight
I also used to strongly believe that LLMs are merely stochastic parrots, i.e., statistical models that solely follow a probability distribution to predict the next token of a given input. Yet, I am still amazed by the capabilities and recent innovations introduced by transformer based AI models. It is still quite obscure to me how LLMs are able to create content that look seemingly eloquent as if the hidden machinery was genuinely understanding a prompt. In spite of those recent innovations, the familiar argument of the "chinese room" is still used. It goes as follows:

Imagine a person who does not speak Chinese is locked in a room with a set of rules and a large batch of Chinese characters. The person is given a piece of paper with a Chinese sentence written on it, and they follow the rules to produce a response in Chinese. The person does not understand the meaning of the Chinese characters, but they are able to produce a response that is grammatically correct and even clever.
The question is: Does the person in the room truly "understand" the meaning of the Chinese characters? The answer is of course no, because the person does not have any comprehension of the language or the meaning of the characters.

Now LLMs process and generate text by manipulating symbols (words, characters, etc.) according to the rules of the model. They don't truly "understand" the meaning of the text; they're simply rearranging symbols to create a coherent-seeming output. They are simply trained on vast amounts of data, which can lead to overfitting and memorization. This often results in the model producing responses that are statistically likely to be correct but lack true understanding.

It is, of course, clear that machine learning models do little similar to what humans have. Scientific or statistical models, for example, might correctly predict outcomes in some specific domain, but they don't "understand" it. A human might know how orbital mechanics works, the same way an algebraic expression or computer program can predict the positions of satellites, but the human's understanding is different. Still, I will adopt a functionalist view of this question, since the internal experience of models, or lack thereof, isn't especially relevant to their performance.

Still, can a stochastic parrot understand? Is having something that functions like a model enough? Large language models (LLMs) do one thing: predict token likelihoods and output one, probabilistically.

But if an LLM can explain what happens correctly, then somewhere in the model is some set of weights that contain information needed to probabilistically predict answers. A LLM doesn't "understand" that Darwin is the author of On the Origin of Species, but it does correctly answer the question with a probability of X% - meaning, again, that somewhere in the model, it has weights that say that a specific set of tokens should be generated with a high likelihood in response. (Does this, on its own, show that it knows who Darwin was or what evolution is? Of course not. It also doesn't mean that the model knows what the letter D looks like. But then, neither does asking the question to a child who memorized the fact to pass a test.)

https://arxiv.org/abs/2310.02207

For example, this paper argues that language models contain a geographical model of the world (in terms of longitude and latitude), in addition to temporal representations of when events have occurred.

They use linear probes to find these representations and argue that the representations are linear because more complicated probes don't perform any better.

They also look for neurons with similar weights as the probes to show that the neuron actually uses the representations.

mechap
I'm not at all comfortable with the abundant usage of "true human understanding" in AI discussions. It's not like there is any sort of real idea of what that actually is. Or have I missed something?
One angle of the chinese room is that we define the mechanism, the person, as not understanding chinese a priori (did I use that right?). But without doing that, there is no way for us to
know that the mechanism does not have an understanding.
In regards of that paper, if you think of an LLM, or any neural net as a lossy compression of the training data, that would be expected behaviour, right?

Anyway, speaking of languages, how about this: I have pretty much no understanding of my native language, grammar wise. But I still speak with a native like quality.
I would dare to suggest that I'm not the only one. It doesn't seem absurd to imagin a stochastic model powering my expression, maybe even an LLM like one.
That's just on a syntactic level, but even on a semantic level, I feel that most of what I say (in a real time verbal conversation) is LLM-like. Because I don't think about it.
Only when conversations become taxing enough that I have to think about what I say, I feel like I'm entering a distinct mode of operations where I would say I'm doing
more than picking tokens according to previous tokens.

TL;DR I think everyday human operations employ LLM like techniques.

Oh boy... I hope I get to hear your thoughts on this one when enough reviews and details come out

I've played around with Stable Diffusion (Images), LLMs (Chatbots), and a tad but of RVC (Voice Cloning) and had a lot of fun getting them up and working. It all scratches the stockholm syndrome level of enjoyment I get from computer troubleshooting. I would still think that anyone who knows a lot about AI will look at my opinions the same way I look at beginner opinions on topics I'm hyper knowledgeable about, but its a ruthlessly fun topic to speculate about. So I will.

I don't feel AI will ever reach the heights of whats being said by either the extremes. The anti-AI crowd doesn't seem interested in being hyper knowledgeable or realistic, because it detracts from the "possible future" that can scare the public. The vocal pro AI people will do the same for the opposite side of the coin, and seem to be cryptobros more concerned with making cash than understanding technology. Locally having uncensored chatbots that are better than Chat GPT-3.5 is already a reality, but most people aren't tech literate enough to put that to any sort of personal use advantage.

Evernight This is all true. What I hope happens is that a updated daily AI can make the search engine obsolete. I'd much rather have direct responses built from multiple sources than the ad riddled and algorithm driven mess than google searches have become. I really despite a lot of tech standards today (part of the reason I'm on this site) and AI has the potential to burn down the current standards in place of new ones. Clearly, there is a large interest in having that be censored from the beginning, but so is everything else today. You also can't really "jailbreak" a search engine like you can with AI, other than filetype:pdf and the like.

The government will show up late to regulate, companies will go all-in for too good to be true technologies like usual, and the world will shift to adopt. Scammers will get better, AI art will become common for applications that are already phoning it in, and violently shunned in more artistic spaces. As with the giant leaps in technology that were VHS and the internet, the biggest change the majority of people will feel will be pornography. Probably.

Comsat Rex
Can you really hope for LLMs to not become an ad riddled shitfests five seconds after they become a viable replacer for search engines for all the same reasons search engines became unuseable in the first place?
Since they won't even have to declare ads as ads anymore I'd imagine they will end up being a much worse experience. Is there even a player in the LLM space that isn't an ad company?
I think there is this French one behind Mistral, does anyone know of others?

mechap Whaaaat? You're saying that a macro system isn't AI? The rabbit marketing team is so gonna kill you

Melon Yeah, fair point. Any company with the reach to make a widely used LLM search engine are the same culprits of the garbage we have today. Hopefully the trained data of the ads can be jailbroken like the censorship has in the past, but I don't know nearly enough to say if that's possible or not, nor how long it would take for the exploit to be fixed. If it even happens to begin with.

mechap

but the underlying processes are different.

how do we know?