Authors:
Tristan L. Botelho – Yale School of Management
Mabel Abraham – Columbia Business School

Interviewer:
Winnie Jiang – Yale School of Management

Article link: http://journals.sagepub.com/doi/abs/10.1177/0001839217694358

---

Listen below to the December installment of the ASQ Blog Podcast Series:

 

Transcript of Podcast:

Winnie Jiang (WJ): Hello, ASQ bloggers! My name is Winnie Jiang. I’m a doctoral student at the Yale School of Management and part of the organizing committee for the ASQ blog. This is the December issue of our ASQ podcast series in 2017. Today, I’m interviewing Tristan Botelho and Mabel Abraham about their recent ASQ article, “Pursuing Quality: How Search Costs and Uncertainty Magnify Gender-based Double Standards in a Multistage Evaluation Process.” Tristan and Mabel, thank you so much for joining us today. Maybe we could get started by having you tell us a little bit about yourselves and your research focuses.

Tristan Botelho (TB): Well hi and thanks for having us. My name is Tristan Botelho and I’m a first-year faculty member here at Yale SOM. My research really centers on trying to answer interesting questions, trying to look out into the real world, see where there’s a puzzle, and trying to insert my own expertise and knowledge and trying to figure out what’s going on. Currently, I’m doing most of my work in either digital platforms in entrepreneurship, looking at questions around social evaluation—like this paper that we’re going to be talking about today—careers, strategy, and innovation, more generally.

Mabel Abraham (MA): Hi, thank you for having us. My name is Mabel Abraham, as you mentioned. I am on the faculty of the Columbia Business School. I am starting my third year this year. So, assistant professor there, after finishing my Ph.D. at MIT. In terms of my research, to summarize, most of it has really focused on gender inequality—specifically, thinking about how we can move beyond documenting the fact that inequality exists. I think if I were to tell you that there’s inequality, men and women experience different outcomes, you wouldn’t be that surprised. The thing we know less about is what the levers are that are driving inequality—what are the different things that we can see that cause there to be more versus less inequality across contexts?—with most of my current work really sitting in the space of entrepreneurship, and then thinking about social networks and how resources are exchanged and how that leads to differences for men and women.

WJ: Fascinating. Thank you very much and welcome again. The first question I have is that we are very interested in learning more about the background story of your paper, specifically what motivated you to undertake this project? Where did the idea come from, and what were some major obstacles or hurdles you had to overcome while developing this paper?

TB: I’ll kick this one off. This is a great question because it’s really rare to get an insight into how a project starts. We usually just read the finished product and think, “Oh, they must have thought about this one day and then they wrote this paper,” and that’s kind of it. Whereas for us in this project, I was working on another project related to this data. This is the data I used in my dissertation, and I was actually speaking to someone who uses the platform and trying to gain insight into why they joined the platform and just try to get a little more information about the platform in general. It was a woman, and she had asked me, “Can you tell me how my ratings are on the site?” I was interested. They’re like, “How many people are viewing my recommendations?” I thought, “Yeah, I can actually do that. Let me get back to you.” I looked it up, and she was getting below average viewership for her ideas. I thought “That’s interesting, let’s look at her performance,” because being that this is in the financial markets industry, I could actually tie back her recommendations to is she above average or below average. I found that she was actually above average. So, she’s an above average performer, lower than average viewership, but then of course I thought, “This is probably anecdotal. I’m not sure if this actually holds.”

I came over to Mabel one day, talking about this conversation, talking about that anecdotal example, and of course, Mabel was working on gender. She said, “Oh, this is really, really interesting. We should talk more about it.” We both wrote this paper while we were Ph.D. students at Sloan, and we had this working group. We presented just the initial version of this idea to people. People were really interested in thinking about the evaluation process and the gender effect given that we were able to control for so many cool variables.

In terms of obstacles and hurdles, I think this paper is special for a lot of reasons—one of which, from the obstacle perspective, it was an initial project for both of us. Both being Ph.D. students, having limited experience of writing papers and getting them published, you face all the possible hurdles and obstacles you can imagine on a project. The data was so rich, but of course that comes with a lot of cleaning and matching, where we’re talking about getting all of this financial information about the firms they’re recommending, about the stock market. It was a lot of long nights, just getting the data set together. I think those are two of the bigger obstacles.

MA: Yeah, the one thing I would add that I think is important—and we lose sight of sometimes—is it was opportunistic in some ways in which Tristan was hearing this story and definitely knew that there was probably something interesting there. Here’s a setting where we have performance information, and we’re still seeing this gender difference. When he told me about it, I had an immediate reaction where this is the kind of question that is really difficult to answer in gender research.

One of the most common critiques I would say of gender research is that we compare men and women, but there’s often unobservables that we can’t get at. We say that women are doing worse than men, and we think we’re controlling for a lot of quality information, but still missing some other things. This provided a really clean setting to get at this. We were able to say, here’s a case where all you have is the person’s name. You have their performance information. There’s literally no reason you should rely on the person’s name more than this performance information, yet we still saw this difference. I think identifying those opportunities where you can make a contribution, given a setting that really fits that question, was really unique in this case here.

WJ: That’s very interesting. It seems like it comes from a specific case you ran into and also the theoretical expertise you already have that makes this context become very suitable for the questions you both are interested in answering.

MA: Exactly.

WJ: That’s great.

TB: The other thing I would add is that, coming at it from . . . Mabel had a lot of great knowledge about gender more generally, but thinking about it also as an evaluation process. One thing that really got us excited was the fact that often, when you read research that looks at either a rating system or evaluations more generally, it’s this one cross-section where one candidate’s being evaluated. Whereas here, as I’m sure we’ll talk about later about the paper, and I’m sure everyone listening has read the paper . . .

MA: Obviously.

TB: . . . is that the same individual is going through a multi-stage evaluation, which we rarely get insight into. We thought that was really exciting, to be able to, like Mabel said, control for this objective performance where you don’t expect something like gender or someone’s name to elicit such a strong feeling that they’re going to evaluate one person differently from someone else when you have objective information. But then also trying to see, does this differ by the stages of the evaluation. More generally, no one ever evaluates in this singular stage. We think about evaluations across these stages, more generally.

WJ: That’s right. Thank you. My second question is that in this paper, you focus primarily on the effect of recommender’s gender, finding that the female recommenders had a disadvantage in receiving attention from the evaluators, especially when search costs and uncertainty were higher. Have you considered the effect of the evaluator’s gender? How do you think the female evaluators would react differently from male evaluators to the recommendations made by female versus male recommenders?

MA: That’s a really interesting question and I think an astute observation of the paper, because we definitely thought about that. When we think about social sciences, sociology, one of the most central tenets is that people fall into the trapping, for lack of a better word, of homophily. We’re attracted to people who are like us. I think one interpretation of what you’re describing is that we can imagine a world in which the men on the platform were biased against the women, but maybe the women were not. Or maybe vice versa: we’ve heard some arguments about queen bee type syndrome where maybe the women would actually be more biased. We definitely thought about this.

At the time we were doing this study, we didn’t have data that allowed us to look at whether the bias varied by the gender of the evaluator. If you remember in the paper, the main effect of gender penalty was happening at the attention stage: that’s whether or not someone clicked on a recommendation. At that stage, we didn’t have data on who was doing the clicking. Subsequently, Tristan ended up getting some data that let us look at a subset of that and get some clarity as to whether or not it was going on, but we weren’t able to do it in the study.

TB: The one thing I’ll add about that—because as we talked about before this started, one of the main audiences for these podcasts are students—is, when I got these data, it’s from an online platform, very entrepreneurial type firm who’s really interested in answering any question they can with their data. I thought of course they’re collecting this clickstream data, so they have to know this. At the time, they actually didn’t collect it at all. When I said it, they were almost like, “Why do you need that?” When I told them, they’re like, “Oh, wow. I can’t believe we haven’t collected this yet.”

So, I pushed them and pushed them and pushed them, but it took a long time, where this study was completed before I actually got that data. That’s why it’s only the small subset for the very end of the time frame. That’s one thing—if we’re going to be giving some armchair advice to students, especially—is that when you’re talking to these firms and you think that there’s data that’s very valuable, just be persistent. Just because they’re not collecting it doesn’t shed some kind of doubt on the need for it to be collected. I find that a lot of firms are either collecting the bare minimum because they think it’s hard to collect a lot of data, or they’re collecting almost too much and they don’t know where it is. You really have to push deeply.

But to get back on track, what we’re trying to do now in a subsequent project is actually understand how the characteristics of both sides—let’s call this a dyad—is affecting evaluative-type processes. We’re using entrepreneurship data coming from a network of entrepreneurs who exchange resources. We’re actually looking at both a more social network where people are getting to know one another, as well as a resource exchange network, and trying to understand the characteristics of both the giver and receiver are actually affecting how these networks play out, the resources that they get.

We do agree with you, it’s really, really important to try to understand and build our knowledge base from both the . . . evaluator side and the evaluatee side of these equations. So while we couldn’t do it fully in this paper, I think it’s an important question going forward.

WJ: Great. Thank you very much. You suggested in the paper that the more experienced evaluators in the study setting are likely to rely less on gender in their evaluation process. However, given that investment management is a highly male-dominated field, individuals might become socially molded and develop stronger gender bias as their experience in this industry increases. Have you considered looking at behaviors of evaluators with different levels of experience? How do you think their experience will influence this process?

TB: Again, that’s a great question and another example of your very close read of the paper. First, to be clear, when we use the term “experienced evaluator,” really we’re talking about an evaluator who has intimate knowledge of what’s being evaluated. They are best able or very able to discern quality. We use that to contrast previous research, especially the lab-based work, which uses an evaluator but not necessarily one who has any experience with the thing they’re evaluating. We think it’s really important when we move theories, especially from the lab to the field, to understand how characteristics about the field should affect our predictions.

What we’re trying to say there is that when you’re experienced in terms of, this is your day-to-day job—these investment professionals, when they’re not picking stocks, they’re evaluating other people’s stock recommendations—we think that is a really important feature of the setting that we have to theorize about. Like I said, when we’re trying to bring the theory from the lab to the field, that’s one thing that we saw as a disconnect, the advantage of using a more competitive context.

Then the other thing is that, the data is coming from a digital platform. Why that’s important is because if you think about who is going to be these individuals who are more, as you say, socially molded, I think it’s going to be those who have been in the industry for the longest period of time. Being a digital platform, I would assume that it’s either going to be very representative of the industry, or if it’s going to be skewed in any way, it’s going to be skewed towards younger investment professionals who are less likely to have this, be part of this social mold. It’s not going to be your 25-year veterans. They’re not going to be over-sampled or anything. They’ll be under-sampled. I still think we’d find a very similar effect.

At the time, the platform was not collecting experience, in terms of how many years you’d been in the industry, so we weren’t able to discern these differences. I do think that, and it’s a question I think we’ve gotten from students and from others—actually Mabel, I’ll let her tell this part. I think you were telling me the other day about a student saying about, “My generation of an evaluator, versus . . .” You tell the story.

MA: Yeah. I was talking to one of my MBA students, who’s going to be doing an independent study with me. He was really interested in this paper and the idea that we were seeing bias in evaluations among these people who we’d call, what Tristan is basically summarizing as experts. They’re expert evaluators in this way. He was like, “Well, do you think this still matters among our generation?”—the people who are in their late 20s to mid to late 30s. I do think there’s something there. I absolutely think that if we were looking at a pool of people who were mostly of an older generation—I won’t put an age number on there, but an older generation where they really were subjected to bias being the norm—we would expect it to be even stronger. I think if I had to go out on a limb a little bit, I think we’d see a stronger effect there. We don’t have that information here, as Tristan mentioned.

I just want to say one other thing about who these people are and what our predictions were going in. If you think about these individuals as being experts, where like Tristan mentioned, they’re basically doing this for a living, day in and day out, evaluating the performances of these stock recommendations, we would basically expect there to be less of a bias. The reason for that is when you’re presented with information, if you’re a novice—if you imagine a Ph.D. student, an undergraduate student, and an M Turker, where a lot of these lab-based studies were done—if they’re evaluating quality, they might not actually have the toolkit that they need to evaluate whether or not one thing is better than another. When you don’t have the right toolkit, you can’t disseminate that performance information, you would expect the bias to be stronger because you fall back on the thing you do know. You think, “Well, women are not usually in finance, so I’m going to prefer the man since I can’t really make sense of this other information.” In this way, I think the fact that we’re looking at experts made us feel like it would be less likely we’d see a bias.

The other thing is, we’re in a market setting. This isn’t in a lab. People are actually incentivized to find the best-performing recommendation. When we did some of these interviews, I remember one person specifically saying, “I just want the one that’s going to make me money. I don’t care about anything else. Why would I care about anything else?” Given these two factors, the fact that they’re experts and they’re in a market setting, we thought that this actually represented a lower bound for when we’d see bias, especially in a male-typed field. We’re in a male-typed field, but we expected it to be a lower bound.

TB: Right. Then the other thing I’ll add is that when we think about experience and expert evaluators, there’s a lot of dimensions where someone could have experience and be an expert. There are some industries where you’re tagged as being a judge or a critic, then there’s the more amateur evaluator that might have a lot of experience evaluating. We could expect that there’s an important interaction, not only about knowing the product, good, or service being evaluated, but also being told, “You’re going to judge this process” versus “You’re not going to judge.”

I think that this question is important because a lot of work we have on social evaluation, and especially as these rating systems are moving more and more online, we’re going to start wondering about these different dynamics, about the evaluator’s experience evaluating, versus are they a professional evaluator versus not, and how do they interact.

WJ: Great. Thank you very much for the detailed explanations. The question four I have is that in this paper you suggested that search costs and uncertainty are two conditions that would magnify gender-based double standards in the evaluation process. How did you come to focus on these two conditions as you developed this study? Did you have them in mind before analyzing the empirical setting, or were they identified as the analysis evolved?

MA: I love this question, I have to say. I love this question because I think it’s something we’ve talked so much about, because as grad students, we have very specific perceptions of what we think it means to make a theoretical contribution. The first thing I would comment on is, even making it open and a possibility that it could emerge from the analysis is something that should be said out loud. Very often, we’re doing studies and we have a main theoretical contribution in mind. We run some analyses and we’re like, “Oh, the effects are actually heterogenous in this way and I wouldn’t have expected that.” That’s okay. I think just making that okay is important and an important thing to say.

In this case, the idea to look at these two conditions actually came from the theory. It’s not the case that we just found it in the results. It came from the theory in a very specific way. We started with this notion of double standards. We were thinking about “double standards” is a term that’s thrown around in popular press, and it’s also a term in the theoretical literature that tells us that men and women might be held to different standards. Specifically, women being held to a higher standard where they have to outperform men to get the same assessment. That’s the common idea.

What double standards theory would predict is, as performance information is introduced, we no longer need to rely on gender. Once we see that Tristan and I perform similarly on a task, why would you favor Tristan over me? You would basically think we’re equal performers so we’re equally worthy of getting the same rating. But what that research was actually finding was that sometimes gender still mattered.

As we were looking at that research, we started thinking, well what exactly is double standards theory based on? It’s based on this notion that you don’t need to rely on status signals when you have performance information, and that it has to do with uncertainty. Once you have this performance information, you have less uncertainty, therefore you don’t need to fall back on signals such as gender to make sense of whether or not a person will be a strong or a better performer than someone else.

TB: Yes. What I would add to that is, when we think about all the theories we have out there about these processes, that’s very similar to what status theory more generally would tell us. What this performance information is doing is reducing uncertainty. Where we really want status to matter is when you don’t understand quality, because if you had a perfect barometer of quality, then it is unclear what the status signal is doing in a market context.

We saw that as an opportunity to not only speak to double standards theory but almost start this bridging process among very similar types of theories: double standards theory, as Mabel mentioned, status theory, which has some baseline predictions about the role of uncertainty, and then also status characteristics theory, which kind of bridges the idea of women being seen as more or less competent in a market setting. We went in with all this knowledge about status theory and how it applies to evaluations. We just saw it as the perfect setting to not create our own theory but start bridging some of the existing theories to tell a more cohesive story about the role of status and evaluative outcomes.

MA: I just want to add one more thing: that’s really important. It’s important especially since this is targeted toward students, right? To recognize how that is a contribution. Going back to the first part where not all theoretical contributions are these grandiose, grand theories that we think up and come up with, but also that not all theoretical contributions really stem from developing new theory. I think I remember at least as a Ph.D. student often thinking that the only way you can make a theoretical contribution is if you propose a new theory. The reality is there are thousands of theories already, so finding opportunities to bridge, like Tristan just described, is really important and in some ways is more valuable. It’s organizing the literature, and it’s helping us think about how does what we know over here actually speak to a point on something we’re working on?

To just take it a step farther in terms of the conditions, when we recognized this link between double standards theory and this role of uncertainty, we started thinking, what other factors could drive there being more versus less uncertainty? The most related one had to do with the amount of information that was being presented. We started thinking, is having performance information a binary variable? Is it really the case that either you don’t have it or you do? Or, is it the case that you could have varying amounts of actual quality or performance information?

That led us to look at the different stages of the evaluation. In the first stage, we see a performance metric. This recommendation has performed X to date. But then once you go to the next stage, you click on the recommendation, you had a lot of information. You had the full model that was supporting that recommendation, all the context and detail that was written up. We thought, that’s actually even more information. Unsurprisingly, we ended up finding that there is no gender difference at that stage, where there’s a lot of information. That would basically be the uncertainty piece. The other factor that’s related to uncertainty is search costs. There’s more uncertainty, it’s more challenging in some ways to make sense of information when you have 100 candidates to look through. It’s not so difficult if you have two; you can actually look at them really closely. Thinking about that tenet in the literature about uncertainty led us to think of the two conditions we thought were most prominent and make predictions based on that.

TB: Then what I’d add to that is, while as Mabel said, these predictions can come from the data, what they did come from though is our own experience before entering academia. I had worked in finance. There was one position where we hired, I think we got something like 700 applications for one slot. We started thinking about what do you do in cases like that? Did I sit there and spend 20 minutes per application and basically my whole six months? No. You start using certainly signals to start winnowing down. Maybe it’s where someone went to school. Maybe it’s where they had their internship. These are as related to status as they are related to quality. But then you start wondering, are we even subconsciously using other things, such as gender? Because we do have, unfortunately, these societal norms of thinking that women don’t work in finance. In my office that I was talking about, we had three women, two of which were administrative assistants. Everyone else was a man, besides this one associate. I think we start using these things, if we know it or not. Getting a chance to actually take some of the things we saw in the hiring process—even these multistage evaluation processes—and bringing it to where we had really good data and being able to control for performance, and bridges there, we thought was going to be a really nice contribution.

WJ: Great. That’s very helpful. Thank you very much. The next question is that, we really like the various strategies you took to conduct robustness checks and rule out alternative explanations. What advice would you give to students who are struggling to come up with techniques to address alternative explanations?

TB: I think we all fall into what I would call almost a trap, where robustness checks are simultaneously extremely important and way too overemphasized. I almost chuckle to myself sometimes when you’re at a talk or read a paper and it’s robustness check 17. You almost don’t even see the connection between what they’re ruling out and what they’re actually trying to propose in their main theory.

The one piece of advice that I got the whole time I was doing my Ph.D. was, papers are remembered for one thing. When you start ruling out an alternative explanation to hypothesis 7D, I think you’re losing track of what these are all about. But like I said, they’re really, really important because the data generated by individuals and firms are messy. That’s what makes them so intriguing is that we’re trying to answer important questions for, it could be an individual, it could be for an organization, it could be for society. If you’re getting data from these sources, they’re not going to come in a very clean CSV file that you can just pop into R, Stata and run some regressions.

MA: Oh, mine usually do.

TB: Oh, see? That’s why I picked Mabel as a coauthor. I think what actually happens is that the robustness check process, like the life of the paper, goes through various ups and downs at various stages. I think that when you think of a question—how does X affect Y or something of that nature—you automatically start thinking of simple alternatives. I think those are really important to rule out.

When I’m working on a paper, I stop there and then I start talking to people. I think especially as students, we have this . . . I’ve heard dozens of excuses, from very relevant to very crazy, about why people don’t talk about their research. For example, people on the more—what I agree with, especially students, they don’t think that the research seems ready, and they want to make a good impression. I completely understand that. On the other side, a little crazier is that they don’t want to get scooped, which means they don’t want someone else to steal their idea. I’ve never had that happen to me, and I talk pretty freely about my research. But I think that as students especially, we need to think about how can we get a project moving forward, and that comes from talking to people. Start with your friends, then move to your colleagues in your department, and then move maybe to seminars or to conferences more generally. What happens with these papers is that people start asking you questions. Then, you can almost start triangulating on what are the more popular alternative explanations. They could also come from reviewers, too. There’s a lot of ways where robustness checks come.

I remember for this paper, one of the first things that I was thinking about when thinking through the analysis was men have more female names. That was a very early on robustness check. You can’t randomly assign gender, but this was a really cool robustness check. But then, we did another robustness check about the census because of the algorithm we were using for the naming. That came much later. A reviewer suggested the robustness check about looking at industries with fewer ideas to see if it’s selection bias. Then obviously, Mabel brought some ideas, and we talked about those, and we talked about some of my ideas. Some of them made the paper, some of them are sitting in a “do” file somewhere.

I think they come from a lot of places, and I see research—or what I hope about this career doing research is that it’s a community. Science is a community. The once piece of advice I would give on this front is talk to other people. You’re never going to come up with all the robustness checks just sitting in your office, and half of the ones you come up with, you’ll never see again.

MA: I want to say one thing, sort of backtracking a little bit to Tristan’s first point about how the goal is not to have a laundry list of robustness checks. It is not the case that your paper is better because you run 25 robustness checks. I think one way I often think about this is really being careful about finding research settings that answer the question well. You alluded to this in our first question, we were talking about how it was well suited for this . . . this data set was well suited for the specific question we had in mind. I think that’s so important and really underemphasized.

At the end of the day, if you pick a data setting, a research setting or data that is well matched to the question you’re trying to answer, the likelihood that you have to do fewer of these statistical Olympics, try to rule out all these different things, is much higher. You’re much more likely to be able to make the argument you’re trying to make with some well-designed robustness checks: you always need them, but not having to do these far-fetched crazy things.

The one added thing, to take it a step farther, is even when you do the robustness checks and you still have some alternative that you can’t rule out, that’s not the death wish for your paper. It doesn’t mean that your paper is not going to be successful or that you can’t make a claim. I think what it really means is that you have to be careful about what the claim is that you’re making. You want to be able to take that alternative that you just can’t rule out and include it in your explanation. It doesn’t become a detriment; it becomes that you can’t maybe claim the very specific thing you wanted to claim, but you can claim something a bit broader that allows for your explanation that maybe you set out to find or you thought was going to be there, and this alternative that could very well be going on but you can’t rule out. So I think it becomes really important to not over claim. If you’re careful about the claims that you’re making, it becomes less of a detriment when you can’t perfectly rule out every single thing that you’re trying to rule out.

TB: All I would add to that is, I’m often wary of papers that—at least in their writing or in their presentation—it’s made to seem that this is the end-all be-all, that we now understand this question and I’ve thought of every alternative, my data are perfect, so Q.E.D., take it as is.

Again, this is how I approach research, and I know Mabel does as well: the goal is really to push knowledge forward. This is not going to be the last paper on the effect of gender in evaluation processes. This is one setting where we think we have really good data on objective quality and trying to understand what can bias the evaluation process above and beyond that. But at the end of the day, we’re really trying to triangulate. We hope that this is one paper in what is going to be several or a dozen subsequent papers that are really just trying to isolate when there’s an effect, how there’s an effect, where does it come out in the data, in organizations, in markets more generally.

While we’re very proud of the paper, we know that there are limitations, but we’re proud of those limitations too because that means we’re not done yet. We’re still trying to figure out when we can—we address gender and equality in this paper, but it could be racial inequality in another paper or it could be something completely different, depending on what the research question is. I think it’s very important not to think, “Oh, I’ve written the paper. I have all these robustness checks. There’s nothing else to be done here.” I think that’s the wrong way to approach a question.

MA: It’s just being honest, right? It’s sort of moving past the mindset of perfection and just being honest about what you can do, being transparent about what you can do, and allowing yourself room for there being space for other work to contribute as well. I think that’s just really what it comes down to.

WJ: That’s great. Thank you very much. The last question I have is about your teamwork and collaboration on the paper. How did your partnership for this paper develop, and how did you work as a team throughout the process of developing it? As doctoral students, much of our research is either independent or conducted with our advisors. Is there any particular advice you would give to students partnering with colleagues who are at similar points in their careers?

TB: Yeah, of course. Collaboration in this field is very, very important. I think the key reason is that it’s a roller coaster process where you have an idea on a random Monday night but then you don’t see the paper in print for two or three years. There’s a lot of things that happen in between those moments, so having someone or more than one person there with you just makes the process better.

MA: More fun, if nothing else.

TB: More fun, yeah. I think if you look at people in the field and you look at their CVs, very, very few write more than a handful of solo author work in their long career. I think it’s because again, this view of science as being more of a community, it’s very hard to write a very well-rounded paper without getting the opinions of others, and having the opinion of others being part of the project helps that along.

As far as this partnership, I think one thing that made it work well is that we have very similar work styles. We like being part of every stage of the process. I know some people like to divide and conquer, where I do this and you do that and we put it together. Whereas we really—in all of our projects with each other and with others—we like thinking about the entire process together as one whole unit and contributing where we can to all the different facets of the paper. I think having a similar work style is really important in fostering a productive collaboration. This was a lot of fun. Like we alluded to earlier, we’re working on another project, so I guess we didn’t hate it that much, working together.

MA: You might be more or less surprised knowing that we’re siblings. Tristan is my brother, and the fact that we have this much fun, for me, isn’t surprising because we’ve always gotten along really well and had fun doing a lot of different things together. Depending on your own experience with siblings, you might have a very different reaction like, “How can they work together? How can siblings do this?”

TB: Especially, as for the record, being the younger brother. We know those stereotypes about older sisters.

MA: He just wants to make it known that he’s younger. That’s what that was. But I do think work styles are really important. I don’t know that I could actually work successfully in a model where somebody does the front end, someone does the back end, for example. I just don’t see how that leads to a synergistic product for me. I know people who have done it extremely well and have great papers that I admire and respect and who have taken that approach, but I think that finding someone who has a similar work style as you is probably one of the most important elements.

The other two things I would add have to do with respect and liking. I don’t think I could work with anyone who I don’t respect as a person, but even beyond that, just respect their work style, their work ethic, their approach to science, what their core ideas and thoughts are on doing scientific research, their philosophies. You just have to know that you trust that person, so the pieces that they are working on, the pieces that they might be driving at any given moment in time, you’re going to support, buy into, not second guess, and feel comfortable with.

I would say for me, maybe even more important than that is liking the person. Tristan alluded to the fact that it’s a roller coaster. I think he gave the stat that two to three years later—we all know that sometimes it’s even longer than that. What that involves is everything from working with the data and doing the analysis, reading the literature, dealing with God only knows how many rounds of revisions you might have to face. With all of those different elements in place, if you’re dealing with a person that you’re going to butt heads with in an unproductive way, where you’re just contentious, that could be really miserable. You’re better off working alone at that point. But if instead, you’re working with a person that you really like and enjoy spending time with and look forward to having those conversations with, it’s much more enjoyable. You actually feel like that long process is made less painful for having that person there.

I would encourage students to not be afraid of working with peers, either same-level peers or slightly more senior peers, but to be careful in choosing those partnerships. I think that partnerships can go really badly, and once you’re scarred from having a bad relationship, I have heard from other people that you are very resistant to then engaging in a new relationship. Since this work is so collaborative, since it is so much about inputs from multiple audience members—both collaborators on the project and a broader set of people you present to—you don’t want to get tarnished in that way. You don’t want to have this perception that you shouldn’t work with other people. Just being careful early on in who you select I think can lead to future success.

TB: And then some other things I’d add to that, I think the relationship is important. I know on this project, and I carried it on to other projects as well, is that you just have to have this open line of communication. You often hear the stories about “Me and my coauthor meet at very set times, on very set days,” and it’s almost too business-like. Whereas I will just text a coauthor a question or just call them on a Saturday or on a Monday night—it doesn’t really matter—and they’ll do the same. I think it’s really important to have a fluid relationship. Of course, you need boundaries, you need obviously to set meetings sometimes, but not treating it so much as a fixed schedule and just going with the flow a little bit allows I think for a better collaboration.

The other piece I would add to partnering with colleagues at similar points in your career. I’m only one data point so it’s tough. I’ve gone the more independent route. I’ve never written a paper with an advisor, so I can’t tell you what the differences are. One of the reasons why I think I’ve gone this route is if you have a question that you find very interesting, and you have a rich set of data, why not go at it with someone at your stage? I think that the end product will feel just immensely rewarding. Not that working with your advisor isn’t, but I think there’s something special about doing every part of the project by yourself or with someone at your stage where you’re learning together. Whereas we all know, or I can imagine, how the advisor relationship goes: when you hit a road block, they’re going to help you through that road block. It’s needed, and you probably should be able to go to an advisor or mentor, but going through the research project with the only other person having a very vested interest is someone at your same stage, I think it’s just a rewarding yet tumultuous process, obviously. There’s a lot of variance and emotions.

MA: Can I add one thing to that? I’m thinking about what you just said.

TB: Sure.

MA: I think it’s also important because there’s some sort of impression management that happens when you’re working with an advisor. I think to the point Tristan was making earlier, we’re sometimes close to the vest about certain things. We’re certainly close to the vest about our weaknesses, our challenges. We don’t want to always share that right away. I think you’re much quicker to share with a peer, especially maybe your brother. If I get stuck somewhere, I have no issue just saying, “I can’t figure out why this model isn’t converging. Take a look at my code.” I’m not sure that you’re as quick to do that if you’re working with a really senior mentor. It forces you to face the challenge, and it also gets you to a resolution sometimes in a more effective way because you’re being transparent right from the beginning and not trying to show that you can figure it out on your own. There’s very little of that that happens with peer collaborations in my experience.

TB: Yeah . . . I think that’s exactly right. One of the last things I’ll add about this is that when you think about peer collaborations, you have—and this probably goes back to what Mabel just said about impression management—you have a more open dialogue about the project itself. Whereas I know I felt pressured when an advisor or a senior faculty member would say, “I’m working on this project. Do you want to work on it, too?” You almost think it has to be a good project because they’re obviously senior and they know what they’re talking about. Whereas when you talk to a peer, you’re more likely to push back and really try to focus on what the question would be and what the research is going to be.

That goes to, I guess my last piece of advice is, feel free to say no to things. I think as students, we feel this need to say yes to everything. What ends up happening is now you have an impressive pipeline. We all see CVs with nine working papers at various stages. I think it’s much more important to have a core set of projects that you’re working on that you know everything about, that you know are going to reach the finish line, and not a lot of projects that you’re only one-fifth involved in and not really sure where it’s going to go. I think focusing on a core set of projects and working with peers—to Mabel’s point, again—I think that it is really helpful in just understanding what we’re doing for research. This is the career we’ve all chosen, so I think the best way to learn about all of it is to face every challenge head on. I think that was easier, at least for me, doing it with someone at my same career stage.

WJ: This was wonderful! Thank you so much Tristan and Mabel. Thank you again for joining us today and sharing your thoughts and experiences about your paper. I found it truly fascinating, and I expect that many of our readers and listeners will find it similarly intriguing and helpful.

[Thank you to the ASQ blog community for listening. As always, if you’re interested in getting involved in the ASQ Blog, email asqblog@gmail.com for information about joining our community of scholars and bloggers.]