The Intricate Puzzle of Balancing Talent
In which we discuss various organizational styles and how they apply to products that require the presence of very different kind of talent.
One of the reasons I left $previous_employer is that I perceived that my learning S-curve was saturating. There was a whole lot I could have learned about managing larger systems or larger software engineering projects but those things didn’t feel very appealing to me.
You know the adage “the meeting doesn’t start until $role is in the room”? At $previous_employer, $role is, effectively, “engineer”. It’s clearly flattening a lot (so hold it lightly) and it’s rarely the case at the every top (the current CEO, for example, is not an engineer although the previous two both were), but it feels significant to me.
What was appealing to me about $current_employer is the obvious need to balance very different kinds of talent in order to succeed. Here’s a thought provoking quote from a document I recently read: MMO video games have “all the problems of a movie studios combined with all the problems of an online service”.
I can see a whole fractal world of tension and complexity in there.
One realization was obvious to me only in retrospect: movies don’t need patching. Movies need to bring all sorts of people and talents and logistics and scheduling together, but once the thing is made, it’s done. People move on. The movie is effectively an immovable artifact that doesn’t need tweaks, adaptation, fixes, “keeping the lights on”, etc.
Because of this, “scalability” is not a problem of those making movies, it’s the problem of those distributing them. There are (at least on the surface) no “growing pains” in movie production. The production effort is structurally decoupled from the distribution effort. The number of people working on a movie is a function of the movie itself (and its funding), not a function of the success of the distribution. I mean, sure, good things happen when the two are correlated, but it is absolutely possible for a small-budget movie to have large success thus distribution needs. This doesn’t change a thing about the organizational requirements of making the movie (because it happened already).
On the other hand, “online services” have very different needs. Something like an online payment API needs people to care about availability and cost of running the service, the pain of growing the organization, the concern of business moats, etc. It still needs to acquire and retain creative talent in those areas (I believe there is no less creativity need in engineering than in other more immediately perceivable activities such as visual entertainment) but the load-bearing part of the business is effectively about growth, sustainability and adaptability. In SaaS, there aren’t even artifacts to distribute!
A fun, meaningful, appealing online community simulation (what my $current_employer is working on) touches both aspects simultaneously: all the creative and artistic needs of an interactive movie and all the development, growth and sustainability needs of an online service.
There is a fun puzzle in all this: what is the best way to design an organization in which decision making needs to be balanced between various needs for which locally optimal solutions are pretty much guaranteed not to be optimal for the entire effort?
The Matrix Organization
$current_employer is organized as a matrix organization: a cross between the “functional organization” (hierarchies organized around skills) and the “divisional organization” (hierarchies organized around output).
Note that “pure divisional organizations” are rare. What tends to happen in practice is that some functions and up being run like product divisions. For example, $previous_employer is ran as a “divisional organization” in which some divisions are functional. Things like Finance, HR, Legal, Sales, Research are effectively “functions” but operate as “divisions” that collaborate across other divisions organized around key products areas (which, in fact, were called PAs).
The most notable (and jarring) difference of a matrix organization from these other two models is that most people effectively end up having two bosses: one that is responsible for the row of the matrix (the output) one that is responsible for the column of the matrix (the skills). Having multiple bosses does indeed happen in a divisional organization (so called “dotted line” reporting), but it is very much the exception and I have never seen it below middle management.
At $current_employer, the figure without which meetings don’t start is the owner of the “row” of the matrix which is called “team captain” and often is a “producer”. A producer is a title that effectively doesn’t exist at $previous_employer: to me, it feels like a hybrid between a product manager and a project manager but operating with executive powers above everyone else’s at their level.
I will admit that it feels somewhat disorienting to experience the social dynamics of decision making in which engineering leadership doesn’t dictate engineering priorities or allocate engineering resources.
Here’s an example of a practical translation of the above: as a software engineer at $current_employer, if the technical lead of my team tells me to do A and my team captain producer tells me to do B, if I do A before doing B, I will get in trouble (and my TL as well). For engineers coming from engineering-centric decision making, it may feels bizarre to build technology this way.
But there is one intriguing potential benefit here: since producers effectively call the shots and are ultimately responsible for outcomes but they are not experts in any of the various skillsets required to “build the thing”, the various functions need to describe their work, their needs and their concerns in ways that can be evaluated and ranked across all the different functions. The producers’ daily job looks like project management on the surface (“running the scrum”, “making sure everybody knows what to do”, “keep everybody on the same page”, etc.) but they ultimately hold the power of “no”. They decide how to allocate resources. This require them to build and finetune a ranking function of priorities. This, in turns, decides what gets done and what doesn’t.
At $previous_employer, most of the time are engineers the ones that have this “no” power. If engineers calls the resource allocation shots, every other function needs to adapt to their view of the world… or fail to get what they want/need. That perpetual joke about the never-ending $previous_employer graveyard of discontinued products? It is important to recognize how it is locally optimal to avoid wasting engineering resources on things that look like aren’t growing while there are others that are. Brand damage externality, you say? I think it’s likely that those concerned about that were not able to make a strong enough case to sway engineering-driven resource allocation.
At $current_employer, it doesn’t feel it would ever be a problem: keeping engineers happy and their promotion/career primed is structurally secondary to the balanced operation of the team as a whole. And this is the same for designers, artists, quality assurers, data analysts or whatever other set of skills are deemed useful by the producer to maximize the likelihood of successful outcomes.
Producer
I find myself fascinated by this “producer” figure because it’s completely unexplored territory for me. I still don’t know enough, for example, to distinguish a good producer from a great one.
I’ve been told (by a producer) that this figure of the producer at $current_employer is inspired more by music production than by visual entertainment production. Music producers help the individual musicians execute their talent to generate an optimal ensemble of their individual outputs. Sometimes it’s direction, sometimes it’s balance, sometimes it’s bringing in additional talent or instruments or ideas or changes. Sometimes it’s getting out of the way and let them find their groove.
What feels intriguing to me here is the idea of a neutral party that can effectively balance all the talents to optimize their ensemble. $previous_employer, for all its technological might, feels organizationally imbalanced to me. The same way professional tennis players end up with one arm bigger than the other, no matter how much they try to compensate with training or exercise. Can a company function more like a professional swimmer, forced by the nature of the activity to exercise its muscles more harmoniously?
The music production parallel is also generative. It is one thing to produce the music of a rock band of 4 people, it is likely very different to produce the music of a symphonic orchestra composed of 100 people. For orchestras, the figure of a “conductor” is crucial to the value of the output: the very same composition played by the very same musicians can result in very different artistic value when conducted by different people.
Listen to “Dies Irae” from Mozart’s Requiem conducted by Claudio Abbado, conducted by Herbert von Karajan, conducted by Colin Davis, conducted by John Eliot Gardiner and conducted by Franz Welser-Möst. It’s obviously the same music, but the feelings it evokes are very different (at least for me).
I can’t help feeling that there is interesting ground to investigate around the interplay between matrix-org production and orchestra conduction. How this interacts with game or tech design feels also very intriguing to me.
Size
Small teams work actually better with minimal organizational overhead. For example, a conductor would probably make things worse for a quartet.
But as teams grow, they need organizational support structures because coordination costs scale with the number of human pairs, which grows with the square of the team size.
The first obvious step is the partitioning of the org in smaller teams independently ran. In hierarchical orgs, each team gets a single person responsible for the outcome of their teams. In matrix orgs, each team gets a captain and a lead for each discipline.
This is where things start to get tricky: not every team will have the same representation from various disciplines. Some teams might not need artists, while others might not need analysts. This means while each captain will have a team, discipline leads might supervise people on that team but not be part of it. 5 weeks into the job, it is likely my mental model is not yet properly generalizing, but it feels to me that outcomes hinge on the the health of the social relationship between team captains and discipline leads. A poor relationship between captains and discipline leads could have dramatically bad impact on things like morale, retention, career growth, coaching. On the other hand, a great relationship between captains and discipline leads could have strongly beneficial effects for inclusion and performance calibration.
Teams of Teams
As an organization grows in size and partitions, it starts to experience the negative effects of such partitioning: misalignment, “hot potatoes” and erosion of agency for change that impacts multiple teams.
This calls for the creation of teams of teams, which at $current_employer are called “initiatives”. Initiatives are assemblies of teams that share some common “output-centered” focus.
The transition from “flat list of teams” to “flat list of initiatives” is underway at $current_employer so it’s even harder for me to distinguish temporary growing pains from structural concerns.
What I can say is that it feels like a “phase transition” where all the things that were successful up to this point start to break down in unexpected ways or get in the way of progress. The operation of initiative-level leadership feels to me to be more difficult because of an extra level of agency indirection, but also because it requires different skills and flex different muscles.
One concern I have is that initiatives risk creating more barriers for changes that need to cross them to provide global balance. For example, an initiative that is focused on user engagement as a metric for success might end up optimizing for things extra rapid iteration cycles to understand what players perceive as fun (which might have negative impact on code maintenance), while an initiative that is focused on maximizing scale might focus on minimizing cost of ownership and maintenance (which might have a negative impact on the ability to perform rapid iteration). It is only by evaluating the two aspects together that the tension appears and decisions can be made that are globally optimal (even if they appear like a compromise locally).
It’s early days but I am starting to feel it’s this interplay between organizational partitioning and cross-cutting concerns that would benefit the most from my attention.