USENIX Security '18-Q: Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?

Wow... this was worth the watch - thank you! awesome lol at [VIDEO] (tldr - feeding gratuitously racist input to ML twitter bot 'TayTweets'

You might be interested in this keynote by James Mickens: [VIDEO] titled “Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?”.

James Mickens is obviously aware of "how this stuff works" and his arguments are very similar: [VIDEO]

James Mickens on this topic: [VIDEO] > Some people enter the technology industry to build newer, more exciting kinds of technology as quickly as possible. My keynote will savage these people and will burn important professional bridges, likely forcing me to join a monastery or another penance-focused organization. In my keynote, I will explain why the proliferation of ubiquitous technology is good in the same sense that ubiquitous Venus weather would be good, i.e., not good at all. > Using case studies involving machine learning and other hastily-executed figments of Silicon Valley’s imagination, I will explain why computer security (and larger notions of ethical computing) are difficult to achieve if developers insist on literally not questioning anything that they do since even brief introspection would reduce the frequency of git commits. At some point, my microphone will be cut off, possibly by hotel management, but possibly by myself, because microphones are technology and we need to reclaim the stark purity that emerges from amplifying our voices using rams’ horns and sheets of papyrus rolled into cone shapes. I will explain why papyrus cones are not vulnerable to buffer overflow attacks, and then I will conclude by observing that my new start-up papyr.us is looking for talented full-stack developers who are comfortable executing computational tasks on an abacus or several nearby sticks.

I think James Mickens (Harvard CS) has it right. ML is inscrutable, black-magic that no one really understands. This is long, but worth watching: [VIDEO]

I've always been a fan of his talks. His keynote at USENIX Security 2018 is by far the best summation of the dangers of ML that I've ever seen. It's hilarious too. [VIDEO]

"What matters now is impact, not intent." James Mickens, using logic and humor, makes this point in a way that even geeks might understand. "Why Do Keynote Speakers Keep Suggesting That Improving Security Is Possible?" [VIDEO]

> the possible MNIST digits are 0-9 Except - and this rather ties into your point - those are not the only possible digits; your network also has to deal with (ie reject) other possible digits such as "P", "E", "3̸̶", or "[Forlorn Sigil of Amon-Gül redacted]"[0], which look like, but are not, decimal digits. 0: [VIDEO]

I generally dislike consuming computer content in video form, but there is plenty of interesting content out there. Lots of conferences video tape sessions. Plenty of noise but some good things too. E.g. one of my faves: [VIDEO]

I believe this talk [0] by James Mickens is very applicable. He touches on trusting neural nets with decisions that have real-world consequences. It is insightful and hilarious but also terrifying. [VIDEO] "Why do keynote speakers keep suggesting that improving security is possible?"

ORT (Only Read Title, sorry) What I'm hearing is "put a non-deterministic black box between you and the man command." Nah. I recommend a stiff dose of Mickens: "Why Do Keynote Speakers..." [VIDEO] ) and a good lie down.

The actual video referenced is a hoot[1] A cautionary tale about AI, Technological Manifest Destiny, IoT, and why you shouldn't trust it, ever. His take down of IoT devices is hilarious. [1] [VIDEO]

"The stuff is what the stuff is, brother." --from a James Mickens talk on machine learning, [VIDEO] Can confirm, the way ML is used in many businesses is like an egg drop: You open up Jupyter, load in some data, and play around with various models until you find one that fits the data, then use it to try to predict future data. If the future results comport with the model, congratulations: your egg is safe, at least for drops of that height.

>> internal people probably won't ask it as awful things or try as hard to break it - I doubt my coworkers would risk asking it super racist things for example. >> 4chan has no such restrictions, and over time what escapes or issues people do find can be mitigated. Hopefully better than previous attempts to put AI on the Internet: [VIDEO]

"Here's the pool of socially well connected people talking to each other, and here's the pool of lonely people that basically talk to nobody. Now, we could create new network edges between the lonely and non-lonely people, so that both groups of people can expand their social network and share their rich sets of deeply personal experiences with each other. But clearly, a much better solution is to direct all that lonely energy to synthetic personalities running on data center VMs. So the people who are happy can continue to be happy, and the people who are lonely can shout into availability zones on the West Coast and Northern Europe. This is clearly the humanist solution." [VIDEO]

> If it doesn't work as I expect it so it should be banned. Well yes that's a very reasonable expectation. In most fields, when you sell something that doesn't work, it's called fraud. But in fields where it actually matter for safety, regulations are most stringent, such as in mechanics or medicine. If a car had 1% chance of blowing up killing everyone inside, would it be legal to sell it? Why is it ok for OpenAI to sell a chatbot that has 1% chance to convince you of utter bullshit when you ask it a serious question? I believe education and communication to be critical to society's wellbeing, so i would hold companies in this field to greater standards. > Also NLP can be used to protect vulnerable groups from online abuse. Automated filters do not understand sarcasm, quotation, criticism... Sure that avoids the scunthorpe problem, but for example, auto-moderation bots trying to prevent racist abuse often censor black people calling one another the n word, which is completely ok. Nothing can replace moderation. > It helps people find related content (which they enjoy). Maybe, but i doubt it. I've yet to see advertisement/recommendations for content i actually enjoy, or to meet people who have such experiences. I've only ever heard that from advertisers, but maybe you have better experience than i do? > Don't blame technology for the bad actions of companies. Their intentions would have corrupted whatever technology is available. Good point! But technology that makes it easie

Reading this article, I kept think about James Micken's 2018 usenix security talk [VIDEO] . He warns quite effectively against widely deploying technology that we don't understand. Kevin Scott, on the other hand, seems to have exactly the opposite take. I suppose there is a reason though why one of them is a successful businessman and the other is an academic.

Nearly all talks by James Mickens are a laugh a minute. The guy is the only computer science standup comedian I have ever encountered. He is also technically excellent. This one is a classic: [VIDEO]

> ML models is most often inscrutable, with the consequence that decisions taken by ML models cannot be fathomed by human decision makers Yup, James Mickens planted this seed 6 years ago, well worth the watch, even if you already agree it's brilliant cathartic humour (the subject isn't really "security"). [VIDEO]

James Micken's talk from almost a decade ago now covers this stuff perfectly... [VIDEO] I re-watch it fairly often, and we just keep getting worse.