Quote from: DanClemmensen on 01/31/2023 06:32 pmQuote from: sanman on 01/31/2023 06:14 pmQuote from: DanClemmensen on 12/28/2022 06:56 pmThere is a large and active AI security academic community that has been studying this for several decades. I don't think we can summarize this work here in a few sentences. Here is an excellent overview book from 2019: https://www.amazon.com/Artificial-Intelligence-Security-Chapman-Robotics/dp/0815369824?fbclid=IwAR1sBo6eLJRhmollCqmDYYd18rEFFCoGvhFn2DB3HgAJ0XQJoZw_QT_gogQApparently, some software engineers from OpenAI have left to create their own company Anthropic AI, and have created their own new rival product called Claude.Among other things, Claude seems to make use of a new concept called a 'Constitution', which as its name suggests, is a construct meant to explicitly embody a set of moral and ethical rules which govern how the AI operates. It also seems to use some sort of AI-supervised reinforcement feedback.The general problem is called "AI alignment". It's hard. https://en.wikipedia.org/wiki/AI_alignmentTo be fair humans are bad at this as well.<quote>AI systems can be challenging to align and misaligned systems can malfunction or cause harm. It can be difficult for AI designers to specify the full range of desired and undesired behaviors. Therefore, they use easy-to-specify proxy goals that omit some desired constraints. However, AI systems exploit the resulting loopholes. As a result, they accomplish their proxy goals efficiently but in unintended, sometimes harmful ways (reward hacking). AI systems can also develop unwanted instrumental behaviors such as seeking power, as this helps them achieve their given goals. Furthermore, they can develop emergent goals that may be hard to detect before the system is deployed, facing new situations and data distributions.</quote>Replace "AI" with political parties, churches, corporations or the local elk lodge. Look at everything from the tribal wasteland of American politics to the war in Ukraine.
Quote from: sanman on 01/31/2023 06:14 pmQuote from: DanClemmensen on 12/28/2022 06:56 pmThere is a large and active AI security academic community that has been studying this for several decades. I don't think we can summarize this work here in a few sentences. Here is an excellent overview book from 2019: https://www.amazon.com/Artificial-Intelligence-Security-Chapman-Robotics/dp/0815369824?fbclid=IwAR1sBo6eLJRhmollCqmDYYd18rEFFCoGvhFn2DB3HgAJ0XQJoZw_QT_gogQApparently, some software engineers from OpenAI have left to create their own company Anthropic AI, and have created their own new rival product called Claude.Among other things, Claude seems to make use of a new concept called a 'Constitution', which as its name suggests, is a construct meant to explicitly embody a set of moral and ethical rules which govern how the AI operates. It also seems to use some sort of AI-supervised reinforcement feedback.The general problem is called "AI alignment". It's hard. https://en.wikipedia.org/wiki/AI_alignment
Quote from: DanClemmensen on 12/28/2022 06:56 pmThere is a large and active AI security academic community that has been studying this for several decades. I don't think we can summarize this work here in a few sentences. Here is an excellent overview book from 2019: https://www.amazon.com/Artificial-Intelligence-Security-Chapman-Robotics/dp/0815369824?fbclid=IwAR1sBo6eLJRhmollCqmDYYd18rEFFCoGvhFn2DB3HgAJ0XQJoZw_QT_gogQApparently, some software engineers from OpenAI have left to create their own company Anthropic AI, and have created their own new rival product called Claude.Among other things, Claude seems to make use of a new concept called a 'Constitution', which as its name suggests, is a construct meant to explicitly embody a set of moral and ethical rules which govern how the AI operates. It also seems to use some sort of AI-supervised reinforcement feedback.
There is a large and active AI security academic community that has been studying this for several decades. I don't think we can summarize this work here in a few sentences. Here is an excellent overview book from 2019: https://www.amazon.com/Artificial-Intelligence-Security-Chapman-Robotics/dp/0815369824?fbclid=IwAR1sBo6eLJRhmollCqmDYYd18rEFFCoGvhFn2DB3HgAJ0XQJoZw_QT_gogQ
Quote from: ppnl on 01/31/2023 07:34 pmTo be fair humans are bad at this as well.<quote>AI systems can be challenging to align and misaligned systems can malfunction or cause harm. It can be difficult for AI designers to specify the full range of desired and undesired behaviors. Therefore, they use easy-to-specify proxy goals that omit some desired constraints. However, AI systems exploit the resulting loopholes. As a result, they accomplish their proxy goals efficiently but in unintended, sometimes harmful ways (reward hacking). AI systems can also develop unwanted instrumental behaviors such as seeking power, as this helps them achieve their given goals. Furthermore, they can develop emergent goals that may be hard to detect before the system is deployed, facing new situations and data distributions.</quote>Replace "AI" with political parties, churches, corporations or the local elk lodge. Look at everything from the tribal wasteland of American politics to the war in Ukraine.Oh yes. If our AI learns ethics by assimilating all the world's writings then we can expect it's ethics to be at best as good as the average. This would be similar to the ChatGPT's level as a writer. Humans cannot agree on the "correct" ethics. Why should an AI be any better?Unfortunately this is NOT irrelevant to the use of AI in space applications. You can try to dismiss the problem that HAL 9000 faced in "2001: A Space Odyssey" as being just science fiction, but the same sort of thing may occur in real life. Space makes the problem harder, because one really major reason to use AI is the communications lag, but the communications lag means the AI must make real-time ethical decisions without the ability to discuss them with humans on Earth.
To be fair humans are bad at this as well.<quote>AI systems can be challenging to align and misaligned systems can malfunction or cause harm. It can be difficult for AI designers to specify the full range of desired and undesired behaviors. Therefore, they use easy-to-specify proxy goals that omit some desired constraints. However, AI systems exploit the resulting loopholes. As a result, they accomplish their proxy goals efficiently but in unintended, sometimes harmful ways (reward hacking). AI systems can also develop unwanted instrumental behaviors such as seeking power, as this helps them achieve their given goals. Furthermore, they can develop emergent goals that may be hard to detect before the system is deployed, facing new situations and data distributions.</quote>Replace "AI" with political parties, churches, corporations or the local elk lodge. Look at everything from the tribal wasteland of American politics to the war in Ukraine.
[quote author=sanman link=topic=57207.msg2454037#msg2454037 Yes, HAL9000 was likewise exactly the first thing I also thought of when reading ppnl's post.And in the absence of communicative feedback from humanity, does the remote AI then develop an independent morality?My analogy would be of the Dog versus the Wolf.The domesticated Dog grows up in the embrace of Man, and is our servant, companion, even family member.The Wolf however grows up separately outside of Man's embrace, having evolved as a predator and potential threat.AI that evolves within the embrace of Man would then likewise be our servant, companion, etc.But AI that evolves remotely in isolation and outside of our embrace or influence of communication, then might have the potential to develop behavior that diverges from what we would consider acceptable.So the remoteness or isolation arising from something like spaceflight could ironically bring about the rogue AI scenario which Elon Musk warns about.Besides Clarke's 2001, I also recall the Bolo series of novels by Keith Laumer, which featured AI-controlled tanks.
I don't think the remoteness of an AI in space is a problem. First it does not "evolve" there. It is developed here. Second, if it is remote then it may not have any reason or opportunity to do harm.The problem is kinda the opposite. Think about a pit bull raised by the wrong kind of person. I've had pit bulls and properly socialized they are the most loyal, loving dogs you can ever hope to see. Raised by flakes they are... flaky. Not as flaky as a chihuahua but much better armed.Now imagine a high level AI created by North Korea, Russia or even the US military. In space or anywhere else they could be flaky. The thing about a dog vs a wolf is that A wolf exhibits morality in a way that a dog does not. A dog simply follows its master. A wolf balances the needs and desires of the individual against the needs of the pack in a very productive way that preserves much of the freedom of the individual. It does allow for conflict but freedom is scary, deal with it.Do you want to send a wolf or a dog into space? And will a high level AI be happy being a dog?
This is why you can't have a hardwired set of rules for morality. They can always be gamed.
Do you want to send a wolf or a dog into space? And will a high level AI be happy being a dog?
An AI isn't a person, and isn't likely to ever be. …
Quote from: Greg Hullender on 03/02/2023 06:37 pmAn AI isn't a person, and isn't likely to ever be. …https://en.wikipedia.org/wiki/Begging_the_question
An AI isn't a person, and isn't likely to ever be. It doesn't "want" anything, and it isn't "content" with anything. It simply does what it's programmed to do. You guys keep imagining that an AI is a human being and that it'll do bad things because a human being would be motivated to, but that's just not how it works.
If it does bad things, it will be for deep, complicated reasons that may be difficult or impossible to fully understand. That's the scary thing about giving an AI too much power. (The paperclip maximizer is a very trivial example of how things could go wrong.) Also, as anyone who's played with Chat-GPT, an AI can simply be wrong.
To give a simple example, I recently tried using the AI with Bing to find out how to merge two Fortnite accounts. Bing confidently told me what the process was called, where in the menus on the EA web site I could find the option, and a step-by-step process to do the merge. It even gave me the URL. I was quite impressed.Except the URL didn't work, and the EA web site didn't have the menu options required. A search of the EA customer support forums shows that customers repeatedly ask how to merge accounts, and the answer is always "you can't do that."AI can be a great tool that helps people navigate large masses of unstructured data, but it won't ever be more than a tool. People who start imagining that their tools are alive and have minds of their own will have problems.
Today, I saw a very relevant article about Dr. Emily Bender's efforts to get people to realize things like ChatGPT aren't human, dated March 1, 2023, so it's definitely up-to-date. :-) I took some classes at UW from her a few years ago, and I have the utmost respect for her. Anyone interested in what AI might really be capable of (and not capable of) ought to give it a read.
Quote from: Greg Hullender on 03/03/2023 11:46 pmToday, I saw a very relevant article about Dr. Emily Bender's efforts to get people to realize things like ChatGPT aren't human, dated March 1, 2023, so it's definitely up-to-date. :-) I took some classes at UW from her a few years ago, and I have the utmost respect for her. Anyone interested in what AI might really be capable of (and not capable of) ought to give it a read.Gee, that was really long - I had to get ChatGPT to synopsize it for me Dear ChatGPT, are you sure Emily M Bender is the author of the article, and not simply its subject?
I actually think you’re right, BUT Emily’s name and perspectives are in its training data, so ChatGPT’s synopsis is actually pretty dang close to the actual article.
Quote from: Robotbeat on 03/06/2023 03:04 pmI actually think you’re right, BUT Emily’s name and perspectives are in its training data, so ChatGPT’s synopsis is actually pretty dang close to the actual article.Agreed. ChatGPT is pretty amazing--even to someone who's worked in the field. The last sentence of the synopsis is debatable, but the rest of it's pretty good.
Well how do we know that humans have “understanding”?