E261 - Jesse Kirkpatrick, Co-Director, Mason Autonomy and Robotics Center, George Mason University
[00:00] Debbie Reynolds: The personal views expressed by our podcast guests are their own and are not legal advice or official statements by their organizations.
[00:12] Hello, my name is Debbie Reynolds. They call me The Data Diva. This is the Data Diva Talks Privacy podcast where we discuss data privacy issues with industry leaders around the world with information that businesses need to know.
[00:25] Now I have a very special guest on the show all the way from Washington D.C.
[00:30] Jesse Kirkpatrick. He is the co director of Mason Autonomy and Robotics center at George Mason University.
[00:39] Welcome.
[00:41] Jesse Kirkpatrick: Thanks. Very glad to be here, Debbie.
[00:43] Debbie Reynolds: Well, I am super excited to have you on here. I love to have educators on. I love to have people who are tech geeky folks like me. And so talking about those deep issues would be a lot of fun.
[00:56] But this is, this is a big job that you have. And so especially as there's so much emerging in technology and AI right now.
[01:05] Tell me a bit about your background and how you became the co director of Autonomy and Robotics center at George Mason University.
[01:14] Jesse Kirkpatrick: Yeah, that's a great question. It's one I get asked a lot,
[01:17] particularly because I think my, my path is unorthodox.
[01:20] So the sort of the, the dirty secret or the secret is, is that I,
[01:25] my degree. So I have a PhD. My PhD is actually in political science and public policy.
[01:31] And so it's, it's unusual for someone with a background like mine to be directing,
[01:38] you know, a technical research center at a university.
[01:42] So I got my PhD, I got it in political science,
[01:46] and I had the opportunity to do some consulting as part of this fellowship program with Johns Hopkins Applied Physics Lab right around the time I was getting my PhD, right before.
[01:56] So I did and introduced me to these kind of technical topics and skills that I wouldn't have gotten otherwise.
[02:03] Right.
[02:03] And from there, as they say, kind of the rest is history. But fast forward a few years and I'm at George Mason University.
[02:11] I was co directing or interim director, acting director,
[02:16] assistant director of this other research shop that was pulling me more and more into the direction of Autonomy, Robotics and AI at that time. Also,
[02:25] one of the great advantages of being in Washington D.C. is that you get to experience government warts and all,
[02:32] but provided with opportunities as an academic to engage with government entities.
[02:39] And so in the course of doing so, right. I've, I've consulted with and for the Department of Defense, the State Department,
[02:46] RAND Institute for Defense Analysis, darpa, all these kind of government actors or trusted government advisors,
[02:53] all on kind of the topic of what now we call responsible AI.
[02:58] So sort of full, you know, kind of full summary Here is that I, I get to my position by getting early and deep technical expertise just by, partly by luck, partly by skill combination of the two.
[03:12] Then I land at George Mason University, which is an excellent place to do the type of work that I do. And actually the university has lots of initiatives with respect to responsible AI, which is kind of how I characterize what it is that I do.
[03:26] And I just so happened to be in the right place at the right time. The university was starting up by the mark. The Mason Autonomy and Robotics center hired this all star faculty member, Missy Cummings, who was, you know, consulting in the federal government.
[03:42] We had a series of meetings and she said, why don't you come and co direct this new place with me.
[03:47] And you know, and here I am.
[03:49] I would say one, one additional thing is that, you know, another area of expertise that I bring, I mentioned government,
[03:55] I mentioned academia and scholarship, but also as consulting for the private sector.
[04:00] So kind of pre pandemic and post pandemic and during the pandemic as well.
[04:06] So maybe for the past six or seven years I've been working with some of the major players in the tech industry,
[04:15] one of the largest social media companies in the world, and helping them think about,
[04:24] innovate and then design, build and scale responsible innovation, responsible AI.
[04:30] So I think it's a combination of those skills that you have a guy like me who's gotten to where I am in a really kind of unconventional and super fun way.
[04:40] It's very long answer to a short question, but I'll finish with this is that like,
[04:45] right, one of the reasons I got into academia is because I wanted to have fun and have the autonomy to kind of do what I wanted within certain constraints. Right.
[04:54] We're all constrained in what we do. Academia is actually a pretty conventional place,
[04:59] but the university I'm at has allowed me to kind of chart my own path with some really interesting detours. And where I am right now is at Lamarck and it's a great place to be.
[05:11] Debbie Reynolds: I'm sure you get to do a lot in innovation and get to help people really think through what they're doing in artificial intelligence and robotics and things like that.
[05:22] I want your thoughts about this new era of AI,
[05:28] I call it AI Gaga, where people are just going crazy about it. Right.
[05:32] So my thought about artificial intelligence. So we all know that artificial intelligence is a new.
[05:40] It comes in many different flavors and forms.
[05:44] My thoughts about,
[05:47] you know, people who have worked in this area, a lot of AI before we went into kind of the generative AI, super,
[05:55] super phase of AI.
[05:58] A lot of artificial intelligence was very kind of narrow and purpose built.
[06:03] So generative AI kind of flipped that where it's like, okay, we're multipurpose and you can figure out what you want to do with it. And part of that is that,
[06:12] you know, some of the guardrails that maybe that were naturally there because things were very narrow on how they did things. So having AI now that people can use,
[06:25] they can do many things means that the risk is raised,
[06:30] and in addition to the capabilities, the risk is raised. So I want your thoughts about us in this new era of AI and how people are super excited about it.
[06:40] What are some of those risks that you see or you're thinking about that are emerging here?
[06:45] Jesse Kirkpatrick: Yeah, I think you raise a really good point, right? Is that I like the. I really like the phrase kind of the Gaga phase of AI.
[06:55] And so I'll say this as a, you know, kind of. I haven't thought of this before, but this thought struck me when you're, when you're saying this is that I really like what you're doing and I like the questions that you're asking because there's a lot of gaga, right?
[07:08] It's like AI is the new baby and everyone wants to say it. Everyone's baby's cute,
[07:12] right?
[07:13] Everyone's baby's cute. Everyone's baby is like something to go gaga over.
[07:19] But in this case, this baby might have some issues.
[07:23] You know, this baby might not be all that cute and we just gotta say it sometimes.
[07:27] And so what I want to say here is that,
[07:30] is that sort of what's new in the sense of kind of. If we're thinking about generative AI versus kind of good old fashioned AI might be one way to phrase it.
[07:42] I think there are two important differences.
[07:44] One is that good old. Is that generative AI,
[07:47] you know, as the name indicates, right. It's generative. It creates generative outputs, right, by inputs that we give it.
[07:55] It's multimodal, meaning that it can create different modes of generation,
[08:01] ranging from ChatGPT text outputs to images, to videos, to audio.
[08:07] Okay.
[08:09] The other is, is that kind of what powers that? Right? So we know, and as the title of your podcast suggests, it's. It's data,
[08:17] right?
[08:17] And I think that this is one of the. One of the risks that we see with generative AI is kind of the big picture of where the data comes from,
[08:31] right?
[08:31] How the data is treated,
[08:34] what the data outputs are,
[08:36] and risks associated with those three things. And maybe four if we count kind of some malicious uses of that data.
[08:45] So the first is like data collection and training, data opacity.
[08:49] Okay,
[08:51] so you mentioned kind of good old fashioned AI versus versus new AI and good old fashioned AI used a lot of data.
[09:00] We're told here is that generative AI or Genai is like data usage, data acquisition, data collection on steroids.
[09:10] The term kind of Internet level, Internet scale data has been used. It's not quite that big.
[09:15] That's a bit of marketing, but it's big.
[09:18] You can think of the dust buster sucking up just a little bit of data versus the super huge shop Vac or Hoover vacuum that is sucking everything up.
[09:29] Right?
[09:30] So it relies on massive data sets scraped from the Internet and often sourced from users without transparency.
[09:40] So right then and there that presents maybe risks, I might call them concerns that trend towards risk.
[09:47] Right?
[09:48] And some of those are like how, where does the data come from? Do we know where it comes from? Do we know how it's being used? Is it our data?
[09:55] Is it our children's data? Is it protected data? Is it sensitive data? Is it data that we'd rather not have scraped and retained by private actors and in some cases maybe the government?
[10:06] So there's like this fundamental risk maybe to different elements of privacy in that sense. That's one risk. We can talk about others, but that's one that might be of interest.
[10:15] Right, and those include things like copyrighted works that are, you know, that are scraped and used. And I'm not talking even here about outputs, that's a whole other thing. But this is just data that's taken without permission,
[10:27] retained, and that the models are trained on. Okay, so copyright is another. I mentioned personal data,
[10:33] photos, writings, metadata, identifiers, personal information about you and me and everyone we know. Okay, again, I mentioned that these are often taken and included in data sets without consent.
[10:46] And then again, I, you know, I really want to underscore this is that there's like a lack of disclosure or transparency about where this data comes from.
[10:54] I mean, I assume, but I don't know for certain that my data, my personal data has been sucked up,
[11:01] used in model training.
[11:04] But I don't know.
[11:05] That's because there's a fundamental lack of transparency and clarity around these questions.
[11:10] Debbie Reynolds: And I think one of the big tensions that we have in this big Gaga AI era is that with privacy expectations or regulations and things like that, more, I think expectation is that a lot of the laws or a lot of people,
[11:28] they want more transparency, not less,
[11:31] and they want more control.
[11:33] Not less. And so I think AI has kind of turned the tables on that where it makes it less transparent and it's harder for people to know what's happening with their data.
[11:45] And then I think one of my concerns, and I know that you work a lot with responsible AI,
[11:51] I feel like some of the harms that can happen to people,
[11:55] there may not be adequate redress for those, those things. But I want your thoughts.
[12:01] Jesse Kirkpatrick: Yeah, I think that's right. Okay, so.
[12:03] So the point that you raised, I think, is a good one. And there's a, there's a tension. Well, you raised two points. The first point is what you, what you mentioned is a lot of people want to know, they want transparency.
[12:15] And if there's. That if their data is being used or if other people's data is being used, if data in general is being used and how.
[12:22] Right?
[12:22] So that's one is like,
[12:24] is it mine? Whose is it and how's it being used for these models?
[12:29] And there's a tension here because of course,
[12:31] the companies that are buying the data, bundling the data, brokering the data, don't want people to know,
[12:37] right? Because then maybe they opt out,
[12:40] they have less robust data, less data.
[12:44] The companies that are training the models, the big tech companies, also don't want people to know because for the same reasons.
[12:51] But the other is, is that, that there's this tension, I think, and an interesting one, which is that what has made generative AI so successful,
[13:02] it has problems of bias, it has problems of hallucination, it has problems of acting like a sycophant or hostile, right? There are all these problems, but it is useful.
[13:10] And in many ways what's made it really useful, is that people's data,
[13:16] people's privacy have been taken in some cases trampled upon.
[13:21] What I think the fundamental thing that is missing,
[13:24] and I think you hinted this with your question, is kind of the notion of consent,
[13:29] right?
[13:30] That if someone is going to use my stuff,
[13:34] take my stuff and use it, whether through acquiring my data that I'm using in other platforms or perhaps that I'm using on the Internet, taking it to train a model,
[13:48] I want to have a say in that,
[13:49] right? And if my data is being used to profit off of, then maybe I want to, if I do have a say in consent to it, then maybe I want some profit from it.
[13:56] I want to share in the royalties of my data being used to generate profit for private companies.
[14:04] Now I think the tension is that a lot of the data now that's being.
[14:08] And here we go from kind of training to use.
[14:11] Right. We've been talking about how models are trained,
[14:14] and there is kind of an artificial distinction, because every time that you and I are using a generative AI model, it is going into the training data,
[14:21] right?
[14:22] It's going into the outputs, which is going into the training data. So it is kind of not a neat distinction.
[14:29] But the point that I'm getting at here is this. Is that.
[14:33] Is that the.
[14:37] Is that the engagement with a generative AI system, say, ChatGPT,
[14:42] is going to get better because it is using my data to personalize an experience.
[14:48] Right.
[14:49] We all know this. When it comes to more familiar social media platforms like Facebook or Instagram or Snapchat or YouTube,
[15:00] kind of the best user experiences are ones that are going to be taking our data with something like consent, but it's probably not really actually meaningful informed consent.
[15:11] Oftentimes it's a teenager who's clicking yes. And giving up their data.
[15:17] Kind of the best experiences are going to come when we give up elements of our privacy in our data. So there is a tension here.
[15:23] Now there's a distinction, right. I think an important one, Juan, is that in the first case,
[15:28] when the data is being taken and used to train a model, we're not even asked.
[15:33] There's not even a question of consent, let alone meaningful consent.
[15:37] And the second, there is something approaching consent,
[15:41] whether it's meaningful or not. We could disagree about,
[15:43] right? Whether we're informed, whether we have the autonomy and so on and so forth. But I think this is a really important distinction. Okay, so I'm sort of losing the thread of what your question was, but I wanted to sort of underscore the point moving from what we think of,
[16:00] like, when we think of data that's being taken to train and use a model, the data that's being generated when taken to train and develop a model versus the data that's being taken when we're engaging with a model.
[16:13] Debbie Reynolds: Right, Exactly.
[16:15] I want your thoughts. So we were talking about training. Now I want to talk about what happens in those. These models. So I want you to talk a little bit about the model.
[16:24] Inversion attacks. This is fascinating.
[16:27] Jesse Kirkpatrick: Yeah, yeah. Okay.
[16:29] So let's think of it this way. Is that. All right, so we have the model,
[16:32] it's trained on this data.
[16:34] Then we have this whole. Whole other thing which we may or may not talk about, but is like,
[16:39] is that the model has now been trained.
[16:43] It now has our data.
[16:44] Debbie Reynolds: Right?
[16:45] Jesse Kirkpatrick: It's storing the data. So we have a question of, like, data Retention and control.
[16:49] So it enters the system,
[16:51] right? In this case, the data enters the system. Let's say it's chatgpt, it has our data, it's entered the system.
[16:57] It's kind of unclear how long it's stored,
[16:59] right? How securely and then whether we can reclaim control.
[17:05] This is the setup for like how conversion or poisoning or data leakage can occur. All right,
[17:11] so we have maybe something like data permanence in training sets or logs. Right? Now our data's in the system. Okay. This was evident in it just a little while ago.
[17:20] I'll give you an example of this. Why it's important is that ChatGPT has the function where you can share your chat and make it public,
[17:29] right?
[17:30] Lots of people do it, I think, to create a link to their chat and then send it to someone because they do something funny, right? Like I may create a link to my chat, makes it public.
[17:37] I don't think about it. I send it to Debbie Reynolds because it's hilarious. And I created this output that's very funny, blah, blah, blah.
[17:45] Well,
[17:46] OpenAI created this feature the end of July, I believe the end of July.
[17:52] Within a day they rolled it back. The feature was that these public chats, and I'm putting this in air quotes, these public chats,
[17:59] that the public chats were indexed in Google search.
[18:03] So like I may be asking something.
[18:05] So let's imagine this, this scenario.
[18:08] Let's imagine that I have an illness. I don't luckily, clean bill of health here, full disclosure. But let's imagine that I have an illness and I search on ChatGPT to feel like, get a diagnosis of my symptoms,
[18:19] right?
[18:20] And I send it to my wife. Like, look, I'm worried it could be this. And I click this share public thing, send it to her and then what we have here is like an index search where someone might Google an illness like this and a ChatGPT output comes up.
[18:37] What that output is, it's going to be our chat. Now my name isn't going to be there,
[18:41] right? My personal data isn't going to be there. But we might be able to infer certain things about that chat, about that person. Maybe it indicates it's a professor at George Mason University.
[18:53] Maybe it indicates it's a professor at George Mason University who had an unorthodox past and came to, you know, co directing the Mason Autonomy Robotics center in a way that was different than we might.
[19:03] You know, you get my point, right? This happened with someone actually on a LinkedIn. They were, they said redo My resume,
[19:09] redid their resume,
[19:11] tailored it for a job a reporter at TechCrunch or somewhere else was able to infer for from that,
[19:17] from that chat log that was available on Google who the person was.
[19:23] And look at their LinkedIn account, looked at the job, all of that, right? Spoiler alert. They didn't get the job, apparently, because their LinkedIn status didn't change. But what's the point here of this kind of long anecdote is data retention.
[19:36] You're in the system,
[19:37] your data's in the system, my data's in the system, someone else is. So what someone may want to do is that they may want to engage in a privacy attack,
[19:44] right. Or model exploitation.
[19:47] Okay.
[19:49] I mean, the point here is this, is that even if direct identifiers are stripped, like I said, AI models might again, quotes, leak private information through indirect attacks and their outputs.
[20:01] And there are various ways that we can do this. You mentioned one model inversion.
[20:06] So we can reconstruct training data from outputs.
[20:09] Right. We are able to, in a way that kind of detectives work,
[20:15] look at the outputs of a model and through what we might think of as like digital breadcrumbs,
[20:23] come to an understanding of some of that training data. Now, why is that important?
[20:30] Well, it reduces the security of a model.
[20:32] It could be a bad actor who wants to mess with the model, could be a bad actor who wants to infer or gain access to training data that was once maybe private or sensitive, swept up off of the Internet and used to train the model.
[20:50] Right.
[20:51] And the result could be a malicious actor now has access to sensitive data that they wouldn't otherwise by using this model and inverting it.
[21:01] So not having direct access to the training data, but having direct access to outputs,
[21:06] and through those outputs,
[21:08] through these digital breadcrumbs and inferences,
[21:13] know something about what that training data is,
[21:16] and by extension,
[21:17] perhaps knowing something specific about specific elements of that training data.
[21:23] Debbie Reynolds: This is fascinating. You touched on something I would love your thoughts on. And so I guess one of the things that I'm very concerned about is inference.
[21:34] So inference is a problem for many reasons.
[21:40] One is inference does not necessarily have to be true, right? And so some people can make an inference that is not true,
[21:48] and you don't know, may not know how that inference is being used,
[21:53] especially like in AI models,
[21:55] I tell people an absence of data is data as well,
[22:00] right?
[22:01] Jesse Kirkpatrick: Yeah.
[22:01] Debbie Reynolds: Right. Because it can say, well, hey, there's a gap here. So that must mean something. It may not actually mean something, but just the fact the way that data is used in these systems, it can create outputs or make inferences that can be damaging or harmful.
[22:19] And like you said, like if someone like they're like, let's say they're googling or they're sharing a link about something health related,
[22:26] a lot of these same companies that have these models, they can put data together from other things. Right. So that, so we can't pretend that there aren't other data points about people that can be put together and not all that information is correct and it's not always transparent to a person how it's being used.
[22:47] But I want your thoughts on that inference.
[22:50] Jesse Kirkpatrick: Yeah, I mean, so there are lots of examples on how we can look at not only the, we can look at data. And let's say here these are outputs of a model.
[23:01] Okay, let's say we, yeah, so there are lots of examples where we can infer certain things correct or incor incorrect or somewhere in between about individuals or groups for that matter,
[23:14] based upon the data that's available. In this case, let's say the outputs of generative AI.
[23:21] Right.
[23:22] So there are two fundamental questions here. Right.
[23:25] Some people might think that privacy when it comes to data and generative AI is really about the inputs.
[23:31] Right.
[23:32] The problem of inference, as you strongly describe it,
[23:36] is that privacy and data is also really important when it comes to system outputs and behavior.
[23:44] Right.
[23:46] And so there were some inferences that can be made as a result of these outputs,
[23:52] whether it's individuals or groups that could be incorrect, potentially harmful,
[23:57] or touch on questions when it relates to privacy. So let me give you one example.
[24:02] Lots of inferences are made when it comes to race,
[24:07] class,
[24:08] socioeconomic status and say voting.
[24:11] Okay?
[24:12] So before we started recording, you and I talked about an area where I live in Washington D.C. we're getting to know each other. You had lived in D.C. you told me neighborhoods you lived in.
[24:22] I told you a neighborhood I lived in. We talked about this. Right?
[24:25] Now we could, with lots of data, public records,
[24:29] infer certain things about me based upon where I live, not even knowing me.
[24:34] Right.
[24:35] So what are those data points that we might be able to make inferences about?
[24:39] Could be my zip code,
[24:40] could be the name of my building,
[24:43] could be my street,
[24:45] maybe depending on the data, what type of car I drive,
[24:49] where I went to school,
[24:51] where I got my PhD, where I did my postdoc.
[24:54] You get the point.
[24:56] Lots of this data is available and we can start to paint a picture that may not be accurate about who I am,
[25:06] about my socioeconomic status, about my race without even seeing me make guesses about who I might vote for,
[25:16] when I might vote,
[25:18] what issues I may be motivated to vote on.
[25:21] And that's going to inform perhaps the things that I see when it comes to engagement with AI systems.
[25:28] The type of news feeds and articles that I get,
[25:32] the sources that they come from,
[25:35] when they're targeted, how they're targeted,
[25:38] the time of day,
[25:40] all of these things.
[25:41] Right. And these may be based upon assumptions that,
[25:45] let's say, that are spot on about me. I don't know.
[25:48] Or they could be in cases like the one that you described, Debbie, that in which these inferences could be actually really harmful. Right. This package of this kind of profile of me could be conveyed to AI systems that are trained on, say,
[26:07] HR and hiring.
[26:09] We get a picture of someone like Jesse Kirkpatrick that might indicate how long I might stay at a job,
[26:15] for example.
[26:17] And these things may or may not be true, but they could have kind of radically important outcomes to choices that are beyond our control, that impact our life,
[26:29] whether it's home loans,
[26:31] dating profiles that I've never been. I'm happily married for 20 years. I've never been, never been on dating apps, but.
[26:37] Debbie Reynolds: Right.
[26:37] Jesse Kirkpatrick: I have friends that,
[26:38] that have. And they're like, this system doesn't understand me, man. This is like I'm getting only this type of person, right? So, like all of these things that are important, from the trivial and mundane,
[26:48] you know, to what we see recommended in, in our Amazon shopping carts, right. Whether it's pairing chips with salsa that's mild or spicy, all the way to the types of jobs that we may get to see in our feed.
[27:02] Right. Many of which now are protected by law to try to remedy and redress some of the, you know, some of the risks and harms that have occurred from, you know, past use and development of these systems.
[27:15] Debbie Reynolds: I feel like. And I want your thoughts.
[27:18] I feel like we need to.
[27:22] Maybe this is a matrix reference.
[27:25] I think we need a librarian to curate the data that goes in. But then also I think people need to understand that they're going to be responsible and accountable for what comes out the output and what and how they use those things.
[27:40] But I want your thoughts.
[27:42] Jesse Kirkpatrick: Yeah, Yeah. I haven't thought about the librarian, but I have thought a lot about individual responsibility and choice.
[27:49] Right.
[27:51] You know, I kind of full disclosure that I am.
[27:55] I guess, like people start to kind of silo themselves and put them into label, like, right. Pro safety, pro innovation, pro this, pro that.
[28:05] So I would say that I'm like, kind of like pro market, pro innovation, pro responsible AI. And I think all of these things are actually go together really well. I think that what is going to fuel innovation is going to fuel safety and going to fuel adoption of AI systems is to design,
[28:21] develop, deploy, and use them responsibly.
[28:23] So what's the first step to that? Right?
[28:26] I think one of the first steps to that is,
[28:28] you know, is I think what I'm trying to do, what me and my colleagues are trying to do and what many are trying to do it universities,
[28:35] or K through 12 education,
[28:39] which is something like AI literacy,
[28:42] right?
[28:43] So not telling people what they should be doing with their data, with their privacy, with their engagement with AI, but informing them of their options,
[28:55] the consequences,
[28:57] the benefits, the risks, the potential harms, the potential innovations and increases in productivity and fun that they can have with artificial intelligence or autonomy and robotics. Right. I'm a big believer in these technologies.
[29:14] I think that empowering people with knowledge,
[29:17] skills and abilities to not only use these systems, but critically interrogate them is crucial to adoption.
[29:28] And so it's like,
[29:32] yeah, maybe an AI librarian is like, look, here are the,
[29:36] here are the books that are available for you, right? Here's like the body of knowledge.
[29:41] It's not my job as a librarian to tell you what books to read,
[29:44] but here, what I'm telling you is that there's this body of knowledge.
[29:47] I can point you in the right direction. Do you want to learn about data? Do you want to learn about privacy? Do you want to learn about,
[29:55] you know, how to bootstrap a model and design it? Do you want to learn the technical elements, whatever the case may be,
[30:00] and guiding people,
[30:02] right.
[30:03] So that they're able to make informed decisions about how they use these systems,
[30:09] what the systems mean for them,
[30:11] and what the benefits, risks and opportunities are.
[30:15] Debbie Reynolds: I want your thoughts about this. So I think of an analogy. I think of when I think of AI, when we were talking about innovation, because we always hear people say they think, you know, being responsible or ethical is a barrier to innovation somehow.
[30:34] And I don't agree with that at all. And so the example I give is cars, right? So cars would not have been as popular or as ubiquitous as they are now.
[30:48] And if we didn't have stoplights and stop signs and speed limits and different things like that, right? So part of the success of automobiles in modern life is because they had those things.
[31:05] So, I mean, can you imagine going on a freeway or a highway where everyone was kind of going in different directions? There were no order it would just be complete chaos.
[31:16] And so I feel like we need to have that conversation. So the fact that we have stoplights and stop signs didn't stop innovation in the automobile industry. It actually made the adoption of automobiles possible.
[31:35] So that's kind of the way I think about AI But I want your thoughts.
[31:38] Jesse Kirkpatrick: Yeah, so I've been thinking about this and I thought about it since I've been.
[31:42] For about the past decade that I've been consulting and consulting on tech.
[31:46] I think that's right. I'd like to extend the analogy and I think that there are,
[31:49] there's a model for how to deal with what might be potentially harmful products.
[31:54] Right.
[31:56] And so the first is automobiles. As you mentioned, automobiles are wonderful with all their downsides or despite all of their downsides.
[32:07] We had to innovate to make them safe. As you mentioned,
[32:11] we had to provide seatbelts and airbags and a whole complex array of design features and engineering features to make them safe within reason.
[32:26] And then also as you mentioned, kind of what academics often refer to as kind of the socio technical elements of the, of the system, which is all of the social stuff that goes along with it.
[32:37] So you mentioned kind of policies,
[32:39] the rules of the road,
[32:40] infrastructure, traffic lights, all these things. Right. So that's one model which is to innovate responsibly towards safety and create the,
[32:49] you know, policy,
[32:51] societal,
[32:53] social stuff that is going to make all of that possible.
[32:58] Right. In this case, rules of the road and infrastructure.
[33:01] There's another way to deal with products that, that are harmful and that's regulate them to death.
[33:07] And that's the tobacco industry.
[33:10] Right.
[33:11] And so we have a choice here. I mean we can.
[33:14] And the tobacco industry is like, look,
[33:16] this is a product that people love.
[33:19] And when I was a kid, you could smoke in the mall, like the shopping mall. People would be smoking and the sh. My aunt would send me to, My aunt would send me into the store to, to go buy her a pack of Newport cigarettes.
[33:30] Like, like you could, you could smoke anywhere and people loved it.
[33:34] It's really bad for you. And we knew that. People knew it then.
[33:37] And so what do you do with a, with a product that is bad for you? Well, you improve it and make it better,
[33:42] such as automobiles or you regulate it to death and have people stop using it, which is cigarettes.
[33:49] So I think you're right. Is that.
[33:51] And the automobile industry and making it safer.
[33:54] Right. Was based upon innovation and it was based upon the idea that people typically don't want unsafe products. They typically don't want to involve or have their children involved in unsafe products, putting their kids in the back of a car, that is a death trap.
[34:14] And so what do you do is that you bundle innovation with safety.
[34:18] And I think that's, that's, you know, common sense approach on how to innovate.
[34:23] Now we can, and as they say, the devil is in the details, and we can haggle over price.
[34:28] How safe is safe enough.
[34:30] But I think that even the most kind of pro innovation people will agree that products need to be safe and that innovation needs to be done responsibly. It's a question of what counts as responsible.
[34:45] And then so at that point, we have fundamental agreement. The question is then we're just kind of haggling over price.
[34:51] And I'll say that this is even evident in the Trump administration's AI strategy released a few weeks ago.
[35:01] And the AI strategy has a commitment to innovation, but also, I think, has some really sensible elements that dovetail with safety, security,
[35:12] data, and privacy.
[35:14] Now, it might be that the motivations for those kind of are different than what we saw in the Biden administration's policy,
[35:21] but nevertheless, there is agreement and overlap on kind of some fundamental areas regarding safety and security, and particularly when it comes to privacy and data.
[35:32] Debbie Reynolds: Yeah. And what, since this is your wheelhouse, what are those things that you think are related especially to privacy?
[35:39] Jesse Kirkpatrick: Yeah. Okay, so two, I mean,
[35:42] I've been thinking about this. I thought of two of them.
[35:45] So the first one is the kind of secure by design.
[35:48] This is the phrase that they use to secure by design.
[35:52] So like kind of where the action is is a little bit different.
[35:55] A lot of the innovation and the. On the government side is actually folded under the Department of Defense in,
[36:01] in the,
[36:03] the Trump plan,
[36:04] but. Right. It calls for developing and refining the Department of Defense's frameworks and promoting secure by design AI technologies that resist adversarial attacks.
[36:19] Right.
[36:19] So increases security in all the areas that you and I have talked about as it relates to the government systems,
[36:29] but importantly against adversarial attacks. All right,
[36:33] so strengthening and making more robust American AI systems in the government for the purpose of national security.
[36:43] That's the main motivation there and why much of this is folded under the Department of Defense.
[36:49] But really the intent here is at bottom to increase AI readiness,
[36:56] resilience, and robustness against attack.
[36:59] And that includes data, and it includes sensitive data specific to American citizens and more generally to security.
[37:10] So that's one. Okay. I think that one's like a little less familiar to people. The second One I think is, is, is more familiar and that's the, been the Trump approach to combating deep fakes and kind of malicious synthetic AI.
[37:28] And so that's, that's an error as well. So kind of quote unquote, synthetic media abuse.
[37:34] Now,
[37:35] you know, whether or not the administration's actually serious about holding its own feet to the fire when it comes to synthetic media and posting synthetic media, I mean, we'll see.
[37:47] But,
[37:48] but there is what we've seen, a real commitment to combating deepfakes.
[37:53] I think the one that the administration and this is speculative, but the low hanging fruit here that I think almost anyone except the most repugnant agree about are the deepfakes that exploit often women and girls.
[38:07] Right. The kind of deep fake nudes.
[38:11] And there's, I think there's near unanimity there that among reasonable moral people that, that should not be done.
[38:20] I think the,
[38:21] the sort of sticky wicked as it is is going to be on kind of what counts as like synthetic media abuse,
[38:29] particularly in the political realm.
[38:32] But I think those are two areas that I see as being really important when it comes to privacy, really important when it comes to data, important when it comes to,
[38:40] you know, the values that undergird privacy and data transparency, consent,
[38:47] safety,
[38:48] security,
[38:48] you know, all of these things.
[38:51] I think the ideological motivations are radically different obviously as to why,
[38:56] when,
[38:57] how and to whom we're going to be developing these than the Biden administration, Trump administration.
[39:02] But I think there is some sensible overlap between the two.
[39:05] Debbie Reynolds: Yeah, I think the thing that concerns me is that I think we should focus more on prevention than cure.
[39:15] Jesse Kirkpatrick: Because.
[39:17] Debbie Reynolds: As I said earlier, there will be situations where there will be no adequate redress. Right. For some of the examples where,
[39:24] you know, women and girls are being depicted in deep fakes or whatever, you know, may harm their reputation.
[39:31] Some people may believe it's true,
[39:33] it's like almost impossible to pull that back. And then once it's out there, it's like how, you know,
[39:40] the Internet is written in pen, not a pencil. Right. So it's like it becomes a lot harder. So when I'm talking about responsible AI, for me,
[39:50] I'm more on the prevention.
[39:53] Yeah, let's think about that part.
[39:55] Jesse Kirkpatrick: Yeah, And I agree with you.
[39:57] When I think of responsible AI,
[39:59] I view it across the lifecycle of a system. And so that's from, I'll say it again, but it's the design of the system, it's the development of the system, it's the use and the deployment.
[40:10] It's the whole life cycle from end to end, tail to snout. And so that includes prevention. And I agree with you, and I think you're right. Is that.
[40:21] My thought here is that.
[40:23] Is that often the policy apparatus, particularly of the federal government,
[40:29] is insufficiently nimble to get ahead of this technology.
[40:35] And often, even if there is a right to redress,
[40:39] the redress is impossible. Because as you said, and I think you said it very well, is that the Internet is written in pen and not pencil.
[40:47] And once that stuff is out there, there may not be appropriate redress or satisfactory redress.
[40:53] So prevention is the. Yeah, prevention is the place to start.
[40:57] Debbie Reynolds: Yeah.
[40:58] Very nice. Very nice. This is fantastic.
[41:01] So, Jesse, if it were the world according to you, Jesse, and we did everything you said,
[41:07] what will be your wish for privacy or AI in the world, whether that be technology,
[41:14] human behavior or regulation?
[41:17] Jesse Kirkpatrick: Oh, my gosh. One wish.
[41:19] Debbie Reynolds: You can have more than one.
[41:22] Jesse Kirkpatrick: Right, right. The one wish can't be that I have more than one wish.
[41:28] Debbie Reynolds: Oh, man.
[41:29] Jesse Kirkpatrick: It's always hard for just one. Okay,
[41:32] all right.
[41:33] I would like to see.
[41:36] This is so like squishy, man, this is such a dodge. But I'm going to say it anyway.
[41:42] I like to see the story unfold like this is that the people that are pumping and promoting AI kind of. That are kind of stirring up the AI Gaga become more serious.
[41:56] Stop trying to sell snake oil that we know isn't going to work.
[42:01] There's not a cure all.
[42:03] Get real about the benefits and the risks and the harms of the things that they're creating and get on board with trying to do so. That is innovating in a way that is responsible, full stop.
[42:20] Right.
[42:21] And I think that there are.
[42:25] That the market incentives and the push right now is for kind of a fun house of smoke and mirrors. And I'm mixing my metaphors here, but a fun house of smoke and mirrors, of snake oil salespeople.
[42:39] And I'd like to see the world kind of get real and these people be taken to task and.
[42:45] Yeah, I think that's where I'm at.
[42:47] Debbie Reynolds: I like that. I like that. I agree, I agree.
[42:51] Because,
[42:52] you know, just like with cars, I have to bring that. I love this analogy.
[42:56] A lot of the future of AI will be built on the usefulness of it to humans. And so if we're thinking about that, then safety and responsibility will naturally fold into that and it will make it more robust for the long term as opposed to kind of a short term thing where you just sell a shovel and then run to the hills from there.
[43:21] Right.
[43:22] Jesse Kirkpatrick: Yeah, I think you're right. Is that, like, the safety. The car thing is like,
[43:26] don't tell me that the brakes work on this car when you know they don't.
[43:30] And if you're selling me a lemon, there's a reason there's a lemon law. Because you're going to get taken to task.
[43:36] Right.
[43:37] And that's what I'd like to see.
[43:40] Debbie Reynolds: That's a lot of common sense. Oh, my gosh.
[43:44] I love that uncommon sense, common sense approach.
[43:47] Yeah,
[43:48] exactly.
[43:49] Wow. Thank you so much for being on the show. This was a hard episode for me because I could probably talk to you for hours. So.
[43:56] Jesse Kirkpatrick: Okay, same here. Same here. I. This was so fun. And I told my brother, I said, oh, I'm doing this podcast with. With the. With the Data Diva. And he said, oh, really?
[44:05] And then he said, how long? I said, only an hour. And he said,
[44:09] so you could go for three. Like, you guys. You guys could just go on forever.
[44:14] And he said, you're. I guess you're not going to have time to talk about aliens or super intelligence.
[44:20] So. No, thankfully not. I don't think we're going to go there.
[44:23] But you could have three, and you don't even need to bring it up.
[44:26] Debbie Reynolds: Oh, totally, totally, totally. Well, this is fascinating. Well, thank you so much. I really appreciate you being on the show and for sharing your work, and I'll be following your work as well as I think the listeners will,
[44:40] too, because I think what's happening in academia is very important and very instructive,
[44:46] where you can give people a lot of guidance and things to think about without that kind of market.
[44:53] You know, sometimes getting a message from academia weighs a lot more to people than maybe a peer group or a different group, where they feel like they have more either skin in the game or there's more conflict.
[45:07] Conflict of interest. But this is great.
[45:10] Jesse Kirkpatrick: I really appreciate it. It's been a pleasure, lots of fun, and so far, the highlight of my week. It's been great.
[45:16] Debbie Reynolds: A. That's so sweet. That's so sweet. Yeah. Yeah. We have to talk. I love to find ways we can collaborate together in the future.
[45:23] Jesse Kirkpatrick: That'd be great. I look forward to it, Debbie.
[45:25] Debbie Reynolds: All right. Thank you.
[45:26] Jesse Kirkpatrick: Thank you. Bye. Bye. It.
