What do industry researchers do, anyway? Part 1 -- How to choose a team

My momma always said, "Life was like a box of chocolates. You never know what you're gonna get."

The 1994 movie Forrest Gump is timeless, not only because it is full of comedic gems that reflect on the real world that we live in, but also because it is littered with soundbites of wisdom that continues to mesmerize the audience even after the movie ends – like the one quoted above.

As one of the two food-related quotes I distinctly remember from the movie (the other being about a fruit company), Mrs. Gump’s wisdom has helped me put in perspective how one makes work and life choices, and what situations they end up being in down the road. My current take of this quote is this: life is like a box of chocolates. You can pick and choose what you want, but you are only allowed to see the shape and color of the chocolate at the time of choosing. It is only when you actually put that piece of chocolate in your mouth and taste it, that you understand the full layers of flavor of the chocolate you chose, which you cannot fully predict ahead of time.

While I appreciate the wisdom behind this analogy, I do have a small problem with this view of the world, if I am allowed to geek out for a second. This “box of chocolates” can lead to two plausible but overly simplistic interpretations about our lives. In one interpretation, you are given a once-in-a-lifetime chance to choose a chocolate from a box, and you live with the consequences thereafter without being able to do much about it. In another, you are handed a (finite?) set of chocolates at birth, and the only agency you have in the process is to choose which order you’d like to pick and enjoy them. Your choices of chocolates are largely independent from each other except for the fact that you can’t get again the chocolates you have already devoured. Both of these seems to leave something to be desired.

I think a more accurate description is something akin to the following, which I dub my life is a box of (boxes of) chocolates analogy.

Life is like a box of chocolates, where each chocolate comes with a silver ticket.¹ Once you choose, you leave your current box of chocolates behind, and enjoy the chocolate you just chose. You then redeem that silver ticket for another box of chocolates, which is somewhat related in flavor to the chocolate you just had. Instead of choosing just once or just getting a single box for your lifetime, this process repeats itself and you are always given brand new boxes of chocolates to choose from.

But you still never really know what you’re gonna get.

Side note: If you are one of those (un)fortunate enough to have been inducted to the world of stochastic processes, you might have realized that this is exactly what I have described.

This analogy has a few corollaries:

1. There is no perfect information. This part is consistent with the original box of chocolates wisdom. You can try your best to probe your chocolate candidates, learn from your past experience or other people’s chocolate-choosing experiences, or even bite off a small piece of your chocolates – none of these will give you the full picture until you actually eat it. You only determine part of your journey by making choices, the rest is sheer luck.
2. The world is not constant and your choices have effects on your future pool of choices. It is easy to lose sight of all those chocolate boxes in the future, and be tempted to leverage all the information you have at hand to find and devour the kinds of chocolate you enjoy the most today. Either positively or negatively, significantly or slightly, this is shaping your future chocolate boxes and what flavors will be available in those when they eventually get to your hands.
3. But this doesn’t mean you have no control, or can’t make predictions. The Mega-Chocolate Universe (not to be confused with other MCU) is fortunately somewhat predictable, at least in the sense of likelihoods. So even if you still don’t fully control your chocolates and which boxes come next, you can nevertheless try to improve the probability of getting the chocolates you truly enjoy in the future.

If you are unsure about what this box of (boxes of) chocolates, or stochastic process, looks like, the tweet quoted below has one of my favorite visualizations.

If you have read this far, and are genuinely interested in what I have to talk about, this is a good time for an obligatory reminder for why you should or should not read on, written in the “0th” part of this series. My experiences might not be useful or applicable to you, so take it with a pinch of salt.

So, what should you think about when choosing a team in the industry to work on research-adjacent projects? Below are a few factors I wish someone had laid out clearly and explained the pros and cons to help me make decisions.

Product-driven or curiosity-driven?

One of the first things that comes to mind for many people is this – does the team you are joining do more research driven by actual product needs, or by curiosity of the team members as long as they are vaguely related to the company’s goals? As mentioned in the previous post, industry research groups and positions can be as heterogenous as they come. That said, teams and projects can often be projected along the axis of product-driven vs curiosity-driven, and each end of the spectrum has its own pros and cons.

These pros and cons follow some simple first principles:

1. Products generally yield financial returns, which is generally welcome by companies and necessary to in turn fund R&D efforts.
2. Product building often come with timeline commitments (either from external customers or for the sake of coordinating a large team), and meeting these usually come with prioritization and tradeoffs for everyone involved, including researchers.
3. Your career progress (if that’s what you are after) and job security (especially in poor economic conditions²) as an industry researcher are likely correlated with your financial contributions (direct or indirect). This might sometimes also affect the kinds of problems you get to work on.

On the purely curiosity-driven end of the spectrum, you have a lot of research freedom, sometimes even better than those in academic institutions because you’d typically have more resources readily available to support you. Your research will more likely be driven by publication timelines (paper/journal submission deadlines) rather than anything else. But when and if you need to earn your keep with the company, or seek your promotion up the ladder, it will typically be more of an uphill battle. You would need to “sell” your idea and results and try to convince someone that works on a product to adopt your idea, and work with them to find the right resources to implement it in production code – which is usually no small feat.

At the other, the purely product-driven, end of the spectrum, you are often more bound by timelines that go hand in hand with product delivery timelines, which are usually more aggressive by curiosity-driven research standards. As a result, you cannot always wait for the “Aha!” moment for the most original ideas, but instead bias towards adopting and improving upon existing approaches to meet product needs most of the time. In the meantime, it is usually a lot easier to demonstrate that you have done something significant in the critical direction the company or larger team is heading in. You don’t need to convince your partner teams your work is valuable because they constantly rely on you and your expertise, and leveling up in the hierarchy is comparatively more straightforward.³

In reality, industry research teams are usually organizationally aligned with one of these modes of operation, in that they either have strong affinity to product teams, or they don’t. I think the underlying logic is simple – as a product team, I would pay for an in-house research team only when I know I can count on them to prioritize product support needs in the form of time commitments and/or constant brainpower support. But this organizational affinity doesn’t mean industry research teams are at the extremes of the spectrum all the time. The situation actually varies a lot from team to team, and even for the team, it can vary over time depending on where the product is, what the company’s financial situations are, how challenging the research project is, etc.

For instance, when a curiosity-driven research team needs to take on a large project that requires a lot of researcher’s efforts, it’s unlikely that everyone’s research interests will automatically fit tightly like puzzle pieces, or that a group of people will automatically function as a well-oiled machine. Furthermore, with more people working on the same project, there can be more structure, more timeline commitments that might be deemed aggressive to keep the larger group in sync, and less research freedom for the individuals involved on average. On the flip side, when a product-driven research team is not on hyperdrive towards product deadlines (and especially when research interns are around), there can be great opportunities to think about big research ideas and take actions on them. One nice thing is that if these research ideas originate from the product (which they quite often are), these teams are in the best position to test them out and helping deploy these research solutions to help benefit real people.

It is usually useful to learn about how your prospective team’s time spreads on the spectrum, and how that matches your own expectations.

Topics and fields

Should you look for a job opportunity that exactly matches your current research interests, or one that expands it a little in a way you did not plan for? Should you take the opportunity of your full-time job to re-invent your research direction and learn everything from scratch?

While I do not have the perfect answer since everyone’s situation is different at any given time, there are a few factors I would consider when it comes to research topics and fields.

1. Are these topics/fields entirely new to you, or way too familiar? If you need to work on something brand new, you will need more time to learn about the background knowledge, build intuition, solve relatively straightforward problems, before you can unleash your full research power as you would if you were working in your very own field of expertise. In the other extreme, when a topic is too familiar and/or you have been working on it for too long, you might encounter researcher fatigue and find less interest in the research work your team does.
2. What’s your runway to ramp up? If you are entering into something that isn’t immediately aligned with your expertise, what’s the gap in your knowledge/expertise, and what buffer is available to you to make up for this gap? I would like to think of full-time jobs as slightly less forgiving compared to the safety of being a PhD student, so you’d want to be able to produce something to contribute to your team’s goals within a reasonable amount of time (which varies from team to team, but typically would not exceed six months).
3. Could the team’s future direction lead to what you care about? This is difficult to predict, but not entirely unpredictable. Given the team’s size, current makeup, curiosity-driven or product-driven nature, the product(s) it currently supports, if any, and the technical assets it has accumulated over the years, you can make an educated guess about what the future might and might not hold for the team. It’s unlikely that a product-driven large language model (LLM) team suddenly reinvents itself to do medical imaging, but more likely that they will build more interesting things on top of LLMs. When joining Amazon, I knew I would be interested in knowledge-intensive conversational agents, which is where my research interests were leading me to at the time. I joined a team that worked on product-driven research for an enterprise search engine, which I reckoned will be a critical part of knowledge-intensive chatbots down the line. Fortunate of me, this turned out to be true, in fact much earlier than I had anticipated, though it was never guaranteed to happen.

Company size

How does company size affect your experience as an industry researcher? The best analogy I can think of is ships at sea.

Ships (or even fleets) with a larger crew are often larger in size and heavier, which makes them less vulnerable to rocks and waves. This crew probably didn’t all materialize onboard at the same time, so there’s a lot of folks with a lot more experience with the ship, and everything you do likely has precedents and standard operating procedures to follow, for the safety of the ship and your fellow crew members. To effectively organize such a large crew, there’s typically layers of commanding hierarchies. You typically have clear guidance on how to rise through the ranks, and everyone’s responsibilities are reasonably separated and confined. Unless you are already near the top of the commanding chain, any individual’s effort (or lack thereof) has a very limited effect on the ship’s heading, course, or speed. But when you are bored with one post, there are usually plenty other posts on the same ship you might be interested in.

On a smaller ship, things are quite different. There is a lot less organization and hierarchy to speak of, everything is very new and you are usually welcome to solve “other people’s problems” and be recognized for doing so. There’s a hole in the hull and nobody’s around to fix it? You can learn to fix it and help keep the ship afloat for longer. There’s no mast? You can build the mast and be the go-to mast person from then on. Rank progression usually comes with increased tenure and a growing crew – but even if you make Captain or First Officer, to put in perspective, it’s still a small ship, and a tropical storm can still tear it into oblivion with a few wrong moves.

Team/Project/Product maturity

The maturity of your team, project, and product functions very similarly to company size, where larger teams, longer-running projects, and more mature products can provide more stability and sometimes more research freedom in exchange for opportunities to potentially make bigger differences.

Within the same company, the differences between mature and new teams can be, say, 2-3x difference on risk/reward profile, where starting new things don’t usually work out, but when they do, they could come with raises and/or promotions. To put this in perspective, though, company size is a much larger multiplier on this profile, which can be as much as 10-100x.

One note about mature teams and businesses is that they are also at higher risk of being leapfrogged by emerging technology, business models, alternatives, etc., because their vision can be fogged by their own early success and they end up spending every possible resource on doing the same thing 1% better. It doesn’t happen that often, but when it does, mature teams and businesses are usually too slow to turn and sometimes face catastrophic failures (see the story of American retailer Blockbuster in the face of Netflix’s rise).

Business direction

I’m probably not the first or the last person to use the phrase “think like an investor” when it comes to choosing (private sector) jobs. Even if you can’t think like a seasoned investor, this prompt can help you think outside of your researcher box and view your job like, well, (part of) a business. Just like your job can’t exist outside of the context of your company’s products and business, your team’s product and your company can’t exist without the business and economic context it is in. Does your company’s business make sense? Is your team building a product that people understand and will want to buy? Is your team supporting a product that has sustained demand, or is it on its way out to be replaced in the market? Is your team supporting a research project that has significant financial value in its future? What’s the best or the worst that could happen to your company and your team’s work, how would that impact the company financially? A lot of this ties directly back to the stability of your research environment and the security of your job, so it doesn’t hurt to think for a moment, especially if your professional insight as a researcher can make any difference.

People!

By joining an industry research team, you are typically joining a (small or large) group of other researchers that are working towards a shared goal. You will see these people all the time in your day-to-day, collaborate with them, and if you are lucky, even make friends with them. The sole researcher mode typically does not play out well in the industry, so you almost always have co-workers that can significantly influence your day-to-day experience (positively or negatively).

Do you find them inspiring? Are they the people you’d like to spend time with on things you care about, be it deeply philosophical discussions, solving challenging real-world problems, overcoming difficulties in research, or just chat about life in general?

Would you find enough support where you need it? Do you have people to guide you through challenges when needed, to put up thought-provoking challenges to your ideas, to help you navigate the corporate environment, or to assist you in finding good projects to work on?

Do your prospective team members have the relevant expertise, experience, methodologies, and/or knowledge it takes to achieve the team’s goals? Are they struggling on key problems they need to solve without new additions like you, or are the basics covered reasonably well, and they are looking to expand research horizons?

Would you have enough room to grow? Are people helpful enough but not overwhelming, willing to give and share credit, easy to communicate with and low on ego? Is the team growing in size or hiring interns consistently, so that you can get the chance to potentially mentor others at your work?

These are just some sample questions you can ask about the people on your team, and the list can go on. Not everyone will write down the same list, but you likely have a list of characteristics you find important in research teammates. If you stumble upon a team where a lot of people matches your ideal descriptions, definitely consider it very seriously – the team is likely doing something right to keep all of these amazing (at least to you) people around, and that is not very easy to achieve in a sustainable manner.

Culture

One of the intangible assets that plays a big role in attracting and retaining people is culture. Put in my own words, culture is a community’s agreement on how we should do/prioritize things, either in written rules or manifested by members of the community and accepted as a norm. Culture has huge influences on how we make decisions, especially during hard times, when they are more consequential.

You might have heard conflicting information about the culture of companies, organizations, and the specific teams within them. These can all be true at the same time, because after all a community’s culture is determined by how an average person operates within that community – when you take averages over different populations, the result can be quite different. In this view, it is also unsurprising that culture will gravitate towards the underlying larger group, or regression toward the mean. Small pockets of distinct and very different cultures can exist for a period of time, but they typically take someone (which could be all members of that small community) significant effort to upkeep. When they come into contact with others in the larger community, it is usually easier for the larger population to influence the smaller one, not the other way around.

Talking to prospective team members and learning what their experiences are like on the team and how they approach various situations can be a good way to help you gauge team/company culture to some extent.

Closing thought: keep an open mind and embrace change

With everything mentioned in this post, it is important to keep a growth mindset. As Heraclitus famously put it, “There is nothing permanent except change”. The environment around you can change and you cannot perfectly predict it. What’s perhaps surprising to many is that your own interests (both research-wise and career-wise) can change faster than you might anticipate. Always aim to gather enough information as you need, since as companies are screening you, you are also choosing among your options. With this information, make your best guess of which chocolates you might like in the future, and which choices today would most likely lead to boxes with those chocolates. It’s helpful to remember jobs don’t have to be permanent – so these decisions might not be as consequential as they seem, don’t overthink it!

Footnotes