Skip to main content
Search form
  • 00:00


  • 00:10

    ADAM PAH: My name is Adam Pah.I am a clinical assistant professorat Management and Organizations at the Kellogg Schoolof Business.I am also the associate director of the Northwestern Instituteon Complex Systems.My research is primarily focused on human behavior,why we do what we do.

  • 00:31

    ADAM PAH [continued]: And I'm largely interested in two completely different areas,one that deals with violence and conflict and the other one thatdeals with adopting new innovations.I happened upon school gun violence by chance.

  • 00:51

    ADAM PAH [continued]: I saw an article that was published,I think, in 2014, sometime around February.And it said that whatever incidenthad occurred that week was now the 43rd or 35th gun violenceincident at a school that year.And I wasn't really sure what that number was related to.

  • 01:11

    ADAM PAH [continued]: I was like, it's February.Is that since January 1?Is that since the beginning of the school year?One seems way too high.The other could be right.It also seems possibly a little low.And I was very confused about where that number came from.So I did my wasting time on the internet.

  • 01:35

    ADAM PAH [continued]: I tried to find out, well, who elsewould corroborate this, which seemedlike it would be pretty basic.And I spent a little bit of time.And I couldn't find any corroboration.I couldn't find any source data.I could find lots of conflicting headlines about the number,and that was about it.

  • 01:56

    ADAM PAH [continued]: And that led me to start thinking, well,who is keeping track of this.Is anyone?How do we know what we think we know about this?This thing happens as you become a researcher.You assume that research continues.

  • 02:21

    ADAM PAH [continued]: I remember that there were theories,or I assume those theories led to research backwhen I was a teenager, about school gun violence.I just kind of assumed it kept going and growingand we knew things.And I came back to it almost 15 years later.And I was like, oh, it doesn't seem like we really know.

  • 02:46

    ADAM PAH [continued]: No one's really in charge.And then I went and looked at the actual research literature.I was like, oh, no.We definitely don't agree about anything at all.I was like, I'm moved to answer this.It seems to me it's a pretty pressing problem

  • 03:08

    ADAM PAH [continued]: for our country.It's not globally applicable.But it's extremely applicable in the United States.And the fact that we can't ask and answer basic questionslike is it going up, is it going down, iswhat we're doing effective is bothersome to me,that we can have something that has

  • 03:28

    ADAM PAH [continued]: such an impact on our society and that we can understandso very little of it.So like any good research project, random search.It's a random walk.

  • 03:49

    ADAM PAH [continued]: I was fortunate enough to start poking around.I saw that Wikipedia was keeping a record of events.So that was pretty interesting.I did my literature review, went through,saw what other people had written and were working on,and quickly realized that lots of people, at some point,

  • 04:11

    ADAM PAH [continued]: had some of the same questions, and thencreated their own answer to it.There is this collective one on Wikipedia.There's essentially some that aredone by academics or one that's done just for K12as an actual National Center.And then the rest were mostly news organizations that

  • 04:34

    ADAM PAH [continued]: did it once for an article.And then others were essentially interest groupsthat we're trying to keep track of this data.But everyone started at some point.And everyone stopped at some point.And I said, OK.Well, at least six other people have tried this before me.

  • 04:56

    ADAM PAH [continued]: Those were the largest collected data sets.I was like, let's see how much everyone agrees.So I just pulled it all together and said, OK, thisshould be pretty easy now.If I stitch these together, I should have one really longcomplete series.They probably overlap a lot.And I quickly found out that it that wasn't easy.

  • 05:22

    ADAM PAH [continued]: Lots of errors, as happens in data,minor errors, some big errors.It doesn't allow you to just put it together seamlesslyand quick and easily.And then even when I did it, I found outthat the overlap really wasn't even that great.Even when they were pulling events for the same year,they didn't agree.

  • 05:43

    ADAM PAH [continued]: And then you look at how they actuallylook as the trends over time, and you see,oh, they actually even disagree on that.So some say it's going up.Some say it's going down.And suddenly, I'm hit with this realization that, well,of course we have no agreement.This is the first thing that makes perfect sense to me.Everyone's done something on their own.

  • 06:05

    ADAM PAH [continued]: It doesn't line up.And so whichever one you pick, that's the answer you get.I got to learn probably what every undergraduate sociologystudent does, which is how to curate small data sets.

  • 06:27

    ADAM PAH [continued]: So as best we could, I cleaned it up computationally.There were errors in locations.So that makes it hard to actually add things upif there's an error in school name,if there's an error in the city or the state.Sometimes there's an error in the years of the events.So a lot of it is just putting events against events,

  • 06:49

    ADAM PAH [continued]: seeing how similar they are, and then just culling it down.What's the most probable?Are these the same, or are these not the same?And then after that, I got to do manual coding.So fortunately, I had a lot of wonderful helpfrom two undergraduates here at Northwesternand another graduate student.

  • 07:09

    ADAM PAH [continued]: And we went through and did it again.But we got to start from something that was solid.So we put everything together to create one master set.And then each one of use went through and actuallyverified each event.The other thing that we really cared aboutis the definition and defining it.And so there was one thing that I really cared about,

  • 07:32

    ADAM PAH [continued]: which was that K12 and college, post-secondary should count.At this point, post-secondary is as mucha part of the educational pipeline in the United Statesas high school is.And so for an organization that'sonly concerned with high schools, they didn't do that.But that was something I felt was really important.

  • 07:54

    ADAM PAH [continued]: The other thing that I really cared aboutis that I didn't care if someone was actually good at shooting.I really only cared that the gun went off,both because it poses a danger--so coming from a public health perspective,that's really what I care about, enclosed spaces,physical danger at a place that you're just required to be.

  • 08:15

    ADAM PAH [continued]: Also because it would increase the likelihoodof being reported--so there's nothing that I can reallydo, as much as I may want to knowif someone just brings guns but doesn't display them or usethem.And then the last one was just that ithad to happen at a school.That was my reasoning.It had to happen at a school.It had to involve teachers, students,

  • 08:37

    ADAM PAH [continued]: staff in some capacity.And that's what mattered to me.The amazing thing is after all that,that's when the work actually started .Tons of effort goes into data cleaning and preparation.

  • 08:58

    ADAM PAH [continued]: I was right away interested in the most basic questions.Really, is it going up or is it going down?And so for that, I really believe in,for small data sets, using essentially Bayesian modelingand Bayesian inference, so trying to specify models

  • 09:19

    ADAM PAH [continued]: and saying, OK, fit this as best you can.And then against all of these different versionsof the possible world, which one makes the most sense?And so it's a pretty simple but consuming analytical processwhere you just try to say, OK, howmany different distinct periods are there.

  • 09:41

    ADAM PAH [continued]: So you try just one period then two periodsthen three periods and so on and so forth.And you can look at that and build up this model.And what we ended up finding was thatwhat fits best is really having four distinct periods.And what those periods are is a periodwhere there's a distinct characterized rate

  • 10:03

    ADAM PAH [continued]: of occurrence, so how many times a month a gun violence incidentis occurring.And what we found is that when we started--so our data set covered, at this point, from 1990 to 2013--that from 1992 to 1994, there was a dramatic increasefrom 1990 to '92, almost four times as frequently.

  • 10:27

    ADAM PAH [continued]: And then things went back down.And they kind of stabilized for a really long time from '94to 2007.And then they went back up.It almost doubled in frequency again from 2007 to 2013,the end of our data set.And so that was a satisfying point, extremely simple.

  • 10:50

    ADAM PAH [continued]: But out of all the different models and versionsof the world that we tested, thisis the one that made the most sense.Mathematically, it was the most parsimoniousin terms of its answer.And we were starting to get an answer to at least onequestion, that it was going up.And then we could do the exact same thingwith the number of deaths per incident.

  • 11:10

    ADAM PAH [continued]: And we find that it's actually consistent all the way through.So now we were able to put together these two facts justto start with and say, OK, so it's not getting deadlier.There's extremely deadly events.It's at the very long tail.It's not that long, but it's at the tail.

  • 11:32

    ADAM PAH [continued]: But it's not the new normal.But they are becoming more frequent, at leastduring those last six years.And that's something that we can care about.And then we can work backwards and try to understandwhat's driving this.

  • 11:55

    ADAM PAH [continued]: So I use Python.It's a wonderful language and framework for actually tacklingthese kinds of questions.I use lots of, essentially, Bayesian models,so PyMC is a wonderful package to set thatup and make that easy.Good old regressions are always a part of this.

  • 12:16

    ADAM PAH [continued]: So when we finally got to the pointwhere we were trying to figure outwhat drives it, trying to see what does or does notcorrespond--and then go it along with that, justlearning how to set up and test models of the world.So what we ended up finding that really mattered

  • 12:38

    ADAM PAH [continued]: is it's economic insecurity.So when economic insecurity goes up, so does the rate.And that's what corresponds to these periods of increasedfrequency.But the thing is, when you have something thatdoesn't happen that frequently.So as many incidents as there are, there's not that many.There's only 382 across those 23 years.

  • 13:05

    ADAM PAH [continued]: And so that's typically what would be a Poisson process.And it's extremely noisy.So even when we fit this and it's significantand it works well, the overall fitis pretty low, at least by natural science standards.It's around 0.074, 0.075, something like that.

  • 13:26

    ADAM PAH [continued]: And so the answer then is, well, is that bad or is that good.And so you can set up versions of this ruleand say, OK, if everything was perfect,if the unemployment rate every month perfectly predicted--set the mean of the distribution for the number of gun violenceincidents, how good could it be?

  • 13:46

    ADAM PAH [continued]: And then you find out that, well, it's only 0.14,even in the absolutely perfect world.So learning how to setup and codethose models, run simulations to kind of bootstrapoff the results that you already have,I think that that's really important to help actually

  • 14:07

    ADAM PAH [continued]: develop your understanding of the entire problem space,because one number on its own doesn't actuallycharacterize it for you.Saying it's an R-squared of 0.07,it's really not that meaningful, because you onlyhave one reference point, zero.You need to know what's the maximum too.So 50% of an absolutely perfect world is insane.

  • 14:32

    ADAM PAH [continued]: That's much better than 0.07.But you have to take that next step.And that's what matters, I think, a lot,is understanding that it goes beyond any one tool.And it's this much more general skillof figuring out how to setup these possible simulationsand explore the limits of whatever question or problem

  • 14:56

    ADAM PAH [continued]: you're interested in.The setup is actually trying to understand something largerwhile drawing on that generalized knowledge.When you know that something's changing,the question then is how is it changing,how quickly is it changing.

  • 15:18

    ADAM PAH [continued]: And when something is kind of changing but stable,that really suggests that there'ssome kind of exogenous driver.If it's something that's internal,one person affects another person,you don't expect it to be so consistent for so longtypically.And so you need to start looking at,well, what are the external circumstances.

  • 15:41

    ADAM PAH [continued]: What else is happening in the world that could actuallybe causing these kind of long-term forces, pushes,stresses that are moving this system.And at that point, it really helpsto actually talk to people.So fortunately, I found a wonderful collaborator,John Hagan in sociology, who for a long time

  • 16:03

    ADAM PAH [continued]: has been interested in this breakdown and, essentially,the school to work transition.And so we were able, actually, to come together.He had something that he had been workingon in theory for decades.And I had results that I was lookingto actually go one step further with and contextualize.

  • 16:26

    ADAM PAH [continued]: And so when we brought it together,we were actually able to create a lot more.And that was the exciting part.And that's the benefit.You're sitting at, essentially, the boundaries of disciplines.You're supposed to reach into those disciplinesto actually meet and talk with people, to draw

  • 16:47

    ADAM PAH [continued]: on their expertise and share what you're doingand what you know.And it adds up to quite a bit more.This data is real people's lives.It's sad.It's depressing.

  • 17:09

    ADAM PAH [continued]: So it takes a mental toll.I asked a lot of my undergraduatesto actually go and read through all of these.I can ask myself, that's easy.But asking them, it's a lot of difficult workto actually sit through and read so much about awful things.

  • 17:31

    ADAM PAH [continued]: But the reality is, whether it's gun violence,whether it's travesties and disasters in war zones,we have to look at it.Someone has to sit down and go through this.And it's hard to anticipate beforehand the kind of tollthat it takes when you just sit thereand read something that's so awful for so long.

  • 17:53

    ADAM PAH [continued]: But it needs to be done.Someone has to do it.Take breaks and remember why you started.

  • 18:14

    ADAM PAH [continued]: [MUSIC PLAYING]


Adam Pah, PhD, Clinical Assistant Professor of Management and Organizations at the Kellogg School of Business and Associate Director of the Northwestern Institute on Complex Systems discusses his research on gun violence in schools, including challenges with data collection, analysis, tools and techniques used, and research findings.

Looks like you do not have access to this content.

Adam Pah Discusses Advances in Computational Social Science

Adam Pah, PhD, Clinical Assistant Professor of Management and Organizations at the Kellogg School of Business and Associate Director of the Northwestern Institute on Complex Systems discusses his research on gun violence in schools, including challenges with data collection, analysis, tools and techniques used, and research findings.

Copy and paste the following HTML into your website