It’s not that the output of an ASI would be incomprehensible but that as humans we’re simply incapable of predicting what it would do/say because we’re not it. We’re incapable of even imagining how convincing of an argument a system like this could make.
The issue isn’t whether we can imagine a smarter entity - obviously we can, as we do in sci-fi. But what we imagine are just results of human intelligence. They’re always bounded by our own cognitive limits. We picture a smarter person, not something categorically beyond us.
The real concept behind Artificial Superintelligence is that it wouldn’t just be smarter in the way Einstein was smarter than average - it would be to us what we are to ants. Or less generously, what we are to bacteria. We can observe bacteria under a microscope, study their behavior, even manipulate them - and they have no concept of what we are, or that we even exist. That’s the kind of intelligence gap we’re talking about.
Imagine trying to argue against a perfect proof. Take something as basic as 1 + 1 = 2. Now imagine an argument for something much more complex - like a definitive answer to climate change, or consciousness, or free will - delivered with the same kind of clarity and irrefutability. That’s the kind of persuasive power we’re dealing with. Not charisma. Not rhetoric. Not “debating skills.” But precision of thought orders of magnitude beyond our own.
The fact that we think we can comprehend what this would be like is part of the limitation. Just like a five-year-old thinks they understand what it means to be an adult - until they grow up and realize they had no idea.
Logic is logic. There is no “advanced” logic that somehow allows you to decipher aspects of reality you otherwise could not. Humanity has yet to encounter anything that cannot be consistently explained in more and more detail, as we investigate it further.
We can and do answer complex questions. That human society is too disorganized to disseminate the answers we do have, and act on them at scale, isn’t going to be changed by explaining the same thing slightly better.
Imagine trying to argue against a perfect proof. Take something as basic as 1 + 1 = 2. Now imagine an argument for something much more complex - like a definitive answer to climate change, or consciousness, or free will - delivered with the same kind of clarity and irrefutability.
Absolutely nothing about humans makes me think we are incapable of finding such answers on our own. And if we are genuinely incapable of developing a definitive answer on something, I’m more inclined to believe there isn’t one, than assume that we are simply too “small-minded” to find an answer that is obvious to the hypothetical superintelligence.
But precision of thought orders of magnitude beyond our own.
This is just the “god doesn’t need to make sense to us, his thoughts are beyond our comprehension” -argument, again.
Just like a five-year-old thinks they understand what it means to be an adult - until they grow up and realize they had no idea.
They don’t know, because we don’t tell them. Children in adverse conditions are perfectly capable of understanding the realities of survival.
You are using the fact that there are things we don’t understand, yet, as if it were proof that there are things we can’t understand, ever. Or eventually figure out on our own.
That non-sentients cannot comprehend sentience (ants and humans) has absolutely no relevance on whether sentients are able to comprehend other sentients (humans and machine intelligences).
I think machine thinking, in contrast to the human mind, will just be a faster processor of logic.
There is absolutely nothing stopping the weakest modern CPU from running the exact same code as the fastest modern CPU. The only difference will be the rate at which the work is completed.
It’s not that the output of an ASI would be incomprehensible but that as humans we’re simply incapable of predicting what it would do/say because we’re not it. We’re incapable of even imagining how convincing of an argument a system like this could make.
Vaguely gestures at all of sci-fi, depicting the full spectrum of artificial sentience, from funny comedic-relief idiot, to literal god.
What exactly do you mean by that?
The issue isn’t whether we can imagine a smarter entity - obviously we can, as we do in sci-fi. But what we imagine are just results of human intelligence. They’re always bounded by our own cognitive limits. We picture a smarter person, not something categorically beyond us.
The real concept behind Artificial Superintelligence is that it wouldn’t just be smarter in the way Einstein was smarter than average - it would be to us what we are to ants. Or less generously, what we are to bacteria. We can observe bacteria under a microscope, study their behavior, even manipulate them - and they have no concept of what we are, or that we even exist. That’s the kind of intelligence gap we’re talking about.
Imagine trying to argue against a perfect proof. Take something as basic as 1 + 1 = 2. Now imagine an argument for something much more complex - like a definitive answer to climate change, or consciousness, or free will - delivered with the same kind of clarity and irrefutability. That’s the kind of persuasive power we’re dealing with. Not charisma. Not rhetoric. Not “debating skills.” But precision of thought orders of magnitude beyond our own.
The fact that we think we can comprehend what this would be like is part of the limitation. Just like a five-year-old thinks they understand what it means to be an adult - until they grow up and realize they had no idea.
Logic is logic. There is no “advanced” logic that somehow allows you to decipher aspects of reality you otherwise could not. Humanity has yet to encounter anything that cannot be consistently explained in more and more detail, as we investigate it further.
We can and do answer complex questions. That human society is too disorganized to disseminate the answers we do have, and act on them at scale, isn’t going to be changed by explaining the same thing slightly better.
Absolutely nothing about humans makes me think we are incapable of finding such answers on our own. And if we are genuinely incapable of developing a definitive answer on something, I’m more inclined to believe there isn’t one, than assume that we are simply too “small-minded” to find an answer that is obvious to the hypothetical superintelligence.
This is just the “god doesn’t need to make sense to us, his thoughts are beyond our comprehension” -argument, again.
They don’t know, because we don’t tell them. Children in adverse conditions are perfectly capable of understanding the realities of survival.
You are using the fact that there are things we don’t understand, yet, as if it were proof that there are things we can’t understand, ever. Or eventually figure out on our own.
That non-sentients cannot comprehend sentience (ants and humans) has absolutely no relevance on whether sentients are able to comprehend other sentients (humans and machine intelligences).
I think machine thinking, in contrast to the human mind, will just be a faster processor of logic.
There is absolutely nothing stopping the weakest modern CPU from running the exact same code as the fastest modern CPU. The only difference will be the rate at which the work is completed.
Beginning by insulting your opponent isn’t exactly the best way to ensure they’ll finish reading your message.
You have a great day.
Fair.
I’ve removed it, and I’m sorry.