Back to table of contents

Software organizations

The photo above is candid shot of some of the software engineers of AnswerDash, a company I co-founded in 2012. There are a few things to notice. First, you see one of the employees explaining something, while others are diligently working off to the side. It's not a huge team; just a few engineers, plus several employees in other parts of the organization in another room. This, as simple as it looks, is pretty much what all software engineering work looks like. Some organizations have one of these teams; others have thousands.

What you can't see is just how much complexity underlies this work. You can't see the organizational structures that exist to manage this complexity. Inside this room and the rooms around it were processes, standards, reviews, workflows, managers, values, culture, decision making, analytics, marketing, sales. And at the center of it were people executing all of these things as well as they could to achieve the organization's goal.

Organizations are a much bigger topic than I could possibly address here. To deeply understand them, you'd need to learn about organizational studies, organizational behavior, information systems, and business in general.

The subset of this knowledge that's critical to understand about software engineering is limited to a few important concepts. The first and most important concept is that even in software organizations, the point of the company is rarely to make software; it's to provide value (Osterwalder et al. 2015). Software is sometimes the central means to providing that value, but more often than not, it's the information flowing through that software that's the truly valuable piece. Requirements, which will discuss in more detail soon, help engineers organize how software will provide value.

The individuals in a software organization take on different roles to achieve that value. These roles are sometimes spread across different people and sometimes bundled up into one person, depending on how the organization is structured, but the roles are always there. Let's go through each one in detail so you understand how software engineers relate to each role.

• Marketers look for opportunities to provide value. In for-profit businesses, this might mean conducting market research, estimating the size of opportunities, identifying audiences, and getting those audiences attention. Non-profits need to do this work as well in order to get their solutions to people, but may be driven more by solving problems than making money.
• Product managers decide what value the product will provide, monitoring the marketplace and prioritizing work.
• Designers decide how software will provide value. This isn't about code or really even about software; it's about envisioning solutions to problems that people have.
• Software engineers write code with other engineers to implement requirements envisioned by designers. If they fail to meet requirements, the design won't be implemented correctly, which will prevent the software from providing value.
• Sales takes the product that's been built and try to sell it to the audiences that marketers have identified. They also try to refine an organization's understanding of what the customer wants and needs, providing feedback to marketing, product, and design, which engineers then address.
• Support helps the people using the product to use it successfully and, like sales, provides feedback to product, design, and engineering about the product's value (or lack thereof) and it's defects.

As I noted above, sometimes the roles above get merged into individuals. When I was CTO at AnswerDash, I had software engineering roles, design roles, product roles, sales roles, and support roles. This was partly because it was a small company when I was there. As organizations grow, these roles tend to be divided into smaller pieces. This division often means that different parts of the organization don't share knowledge, even when it would be advantageous (Chilana 2011).

Note that in the division of responsibilities above, software engineers really aren't the designers by default. They don't decide what product is made or what problems that product solves. They may have opinions—and a great deal of power to enforce their opinions, as the people building the product—but it's not ultimately their decision.

There are other roles you might be thinking of that I haven't mentioned:

• Managers exist in all roles when teams get to a certain size, helping to move information from between higher and lower parts of an organization. Even engineering managers are primarily focused on organizing and prioritizing work, and not doing engineering (Kalliamvakou et al. 2018).
• Data scientists, although a new role, typically facilitate decision making on the part of any of the roles above (Begel & Zimmermann 2014). They might help engineers find bugs, marketers analyze data, track sales targets, mine support data, or inform design decisions. They're experts at using data to accelerate and improve the decisions made by the roles above.
• Researchers, also called user researchers, also help people in a software organization make decisions, but usually product decisions, helping marketers, sales, and product managers decide what products to make and who would want them. In many cases, they can complement the work of data scientists, providing qualitative work to triangulate quantitative data.

Every decision made in a software team is under uncertainty, and so another important concept in organizations is risk (Boehm 1991). It's rarely possible to predict the future, and so organizations must take risks. Much of an organization's function is to mitigate the consequences of risks. Data scientists and researchers mitigate risk by increasing confidence in an organization's understanding of the market and it's consumers. Engineers manage risk by trying to avoid defects and moving fast.

Open source communities are organizations too. The core activities of design, engineering, and support still exist in these, but how much a community is engaged in marketing and sales depends entirely on the purpose of the community. Big, established open source projects like Mozilla have revenue, buildings, and a CEO, and while they don't sell anything, they do market. Others like Linux (Lee & Cole 2013) rely heavily on contributions both from volunteers (Ye & Kishida 2003), but also paid employees from companies that depend on Linux, like IBM, Google, and others. In these settings, there are still all of the challenges that come with software engineering, but fewer of the constraints that come from a for-profit or non-profit motive.

All of the above has some important implications for what it means to be a software engineer:

• Engineers are not the only important role in a software organization. In fact, they may be less important to an organization's success than other roles because the decisions they make (how to implement requirements) have smaller impact on the organization's goals than other decisions (what to make, who to sell it to, etc.).
• Engineers have to work with a lot of people working with different roles. Learning what those roles are and what shapes their success is important to being a good collaborator (Li et al. 2017).
• While engineers might have many great ideas for product, if they really want to shape what they're building, they should be in a product role, not an engineering role.

All that said, without engineers, products wouldn't exist. They ensure that every detail about a product reflects the best knowledge of the people in their organization, and so attention to detail is paramount. In future chapters, we'll discuss all of the ways that software engineers manage this detail, mitigating the burden on their memories with tools and processes.

Further reading

Begel, A., & Zimmermann, T. (2014, May). Analyze this! 145 questions for data scientists in software engineering. In Proceedings of the 36th International Conference on Software Engineering (pp. 12-23).

Boehm, B. W. (1991). Software risk management: principles and practices. IEEE software, 8(1), 32-41.

Chilana, P. K., Ko, A. J., Wobbrock, J. O., Grossman, T., & Fitzmaurice, G. (2011, May). Post-deployment usability: a survey of current practices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2243-2246). ACM.

Clegg, S. and Bailey, J.R. (2008). International Encyclopedia of Organization Studies. Sage Publications.

Kalliamvakou, E., Bird, C., Zimmermann, T., Begel, A., DeLine, R., German, D. M. What Makes a Great Manager of Software Engineers? To appear in IEEE Transactions on Software Engineering. IEEE.

Ko, Andrew J. (2017). A Three-Year Participant Observation of Software Startup Software Evolution. International Conference on Software Engineering, Software Engineering in Practice, to appear.

Lee, G. K., & Cole, R. E. (2003). From a firm-based to a community-based model of knowledge creation: The case of the Linux kernel development. Organization science, 14(6), 633-649.

Li, Paul, Ko, Andrew J., and Begel, Andrew (2017). Collaborating with Software Engineers: Perspectives from Non-Software Experts. In the Proceedings of the 10th International Workshop on Cooperative and Human Aspects of Software Engineering.

A. Osterwalder, Y. Pigneur, G. Bernarda, & A. Smith (2015). Value proposition design: how to create products and services customers want. John Wiley & Sons.

Yunwen Ye and Kouichi Kishida. 2003. Toward an understanding of the motivation Open Source Software developers. In Proceedings of the 25th International Conference on Software Engineering (ICSE '03). IEEE Computer Society, Washington, DC, USA, 419-429.