Avoiding Negative Side Effects: Concrete Problems in AI Safety part 1

This highlights an interesting general point - in many situations, there is no simple safe fallback policy. On a highway, an emergency stop is not safe. This is a general problem in AI safety and is covered nicely in this youtube video, as well as the paper referenced there - [VIDEO]

Re: alignment, see the "Where are we" part of this talk: [LINK] See also the AI safety problems for concrete problems that people are currently working on - as in with code and real simulations, not "just" math and thinking: [VIDEO] Re: scaling up. That's called intelligence explosion, there are a lot of write ups about it, but the gist of it is just that by simply increasing working memory size, thinking speed, prediction precision, pattern matching, and so on, you get to unimaginably high levels. Furthermore, if you can make a human level AI, it naturally follows that it's only a matter of time to make it slightly better. At first simply faster, simply more eloquent, a bit more emotional depth, it gets better at speaking, thinking. At first it's a child that asks questions, later it's a clever guy who spots your errors as you code, and in no time, I mean no time, it does whatever it wants, because by the time you try to confront it about something, you lost, because you are bogged down in an argument, while it does whatever it wants. (Unless of course it's contained. For a while at least.)

For those curious about AI safety I heavily recommend Robert Miles channel on YouTube. He has lot of good content describing out the challenges we face when trying to create safe AI. Here is one potential starting point: [VIDEO]