Sunday, November 1, 2009

Software Architecture, the US, Outsourcing, and R&D

The U.S. Is Outsourcing Away Its Competitive Edge is an interesting article. I'm not a business person (yet!) so I won't comment on the core message of the article. But this argument resonated on me:

“In many cases, R&D and manufacturing are tightly intertwined. Unless you know how to manufacture a product, you often cannot design it. And, to understand how to manufacture it, you have to have manufacturing competencies and experience. The notion that you can design a product in the serene world of the R&D laboratory without any knowledge of the rough and tumble world of production is ridiculous”

— The U.S. Is Outsourcing Away Its Competitive Edge, Harvard Business Review


I drew the obvious connection with that in our field is called as ”Software Architecture“ which, while reasonable in theory, ends up in practice with a bunch of Architecture astronauts: People who stop thinking on the real problems and solve abstract issues using whatever high-level “design language” they think is the equivalent to what the real architects draw.

For these folks, coding is too boring. And you know, they already did their share of coding and finally got away from that. Which tells me that they never liked coding too much, by the way.

So the solution is to "outsource" coding to the more junior coworkers. Which is were I draw the parallel with the article which talks about the US and their massive outsourcing of technology production while still wanting to keep the edge on R&D. Which sounds like a problem.

If you think that coding is sometimes boring (and I can agree about that!) and you feel tempted to take an software architect role which is being sell as the position in which you will still do the entertaining part of the work (the one in which your brain gets some exercise) while leaving the rest to something else, think about that again. It will probably work at first (since your experience with real coding is still fresh) but it won't work at long term. You can't tell people how to code and what to code if you stopped coding and solving real problems a while ago.

Not to mention than “coding” with diagrams is boooring, improductive and and almost useless anyway.

So I'm on the camp which thinks that the right answer is the technical leader, not the architect. One who will need experience, for sure. But who won't throw away that experience by completely change his role! He will still code and test and make releases and make mistakes (which is when he will continue building experience!)

“Experience is what you get when you don't get what you want”

— Dan Stanford


And this in turn takes me to the agile camp. Not only because I agree with the core principles like people-over-processes. But because I think that the whole parallel with civil engineering made by the traditionalist and waterfall-ish model is dead wrong. Civil engineering need to plan a lot because changing a building on the fly isn't exactly easy. But we not only can change our systems on the fly, but people expect to be able to change them.

We don't need architects who aren't up to get their hands dirty and participate in the coding process. Because, in the software-engineering-as-civil-engineering parallel, the real equivalent of what the civil architects do is the code, not the diagrams.

Which brings us back to something I mentioned before in passing: Coding can be boring, tedious sometimes. In my view, the appropriate solution is to change to a better language that let you write code which is closer to a description of the underlying design. DSLs and that sort of stuff.

And yeah, diagrams can be a useful tool in the thinking process (and even better tools as high-level, introductory views of the systems), but stating that they are the design/specification is admitting defeat.

This is also why I think that agile practices are somewhat dependent on programming languages. They can't be completely separated like a set of practices that you can use regardless of what you are using to build whatever you are building. But I'm digressing to a different topic, and this is enough for today.

My final advice, however, is that if you like to program but find it increasingly boring and the offered “solution” by your current company is to become an architect, counter by offering the real solutions. Or move to a better company, if that doesn't work.

9 comments:

Abraham said...

completely agree with you that the architect must not ignore the code. anyway, not so easy to change into a better language due to customer requirements, but the solution to switch to a better company is relatively good, as long as possible, the point is not for ever, I think

Leo Soto M. said...

Yeah, I know that switching the customer to a different platform is a problem.

Multi-languages platforms are the right solution IMHO (See how both .NET and the Java Platform are moving into that direction): It allows the big corps to stay on one coherent platform which they know how to manage while still giving their vendors the chance to move with agility, which is on their own best interest.

Of course lot of big corps don't see this point and it is our job to make them see it. We are the experts. They are paying us not just to always agree with them but to also guide them on how to run their TI in the best possible way.

Javier Rojas Goñi said...

Hi Leo,

"Civil engineering need to plan a lot because changing a building on the fly isn't exactly easy"

I disagree... in construction you plan a lot JUST BECAUSE YOU CAN. An architect envisions a building with enough precision to make detailed plant that workers can build with only a basic knowledge of the general plan. In software happens one of two things (I don't know which of them rules): we aren't able to envision properly the complete system or lack the proper representations to make a detailed and complete specification OR in the case of software the devil is in the details, so finally the constructor of the software, i.e. the programmer, ends taking decisions as transcendent and important as the designer's.

Jorge.Rodriguez.Suarez said...

Wow !, this is a great article about the "software architect" inflated term !

"Software architect" is evil !...

and Yeah, diagrams are useful too for communications and 10.000 feets view, and can change all the time in the development cycle.

I think that agile languages can be used to design and prove that you are in the right o wrong direction in the solution !

Leo Soto M. said...

Javier:

Interesting point. If I'm understanding it correctly you mean that software is way more complex than civil engineering, making software almost impossible to fully plan in advance.

I kind of agree. Of course there are counterexamples (both on the civil engineering field and on software engineering on projects from the defense sector, NASA and the like). But I guess the crucial point is that in the BDUF approach only does economical sense on a very small portion of the software being constructed.

That's a shame: one part of me would love to use mathematical models to prove programs correct (and that sort of fun stuff with static analysis), take the pride of building zero-bug systems.

But it's reality, and we need to build solutions for the real world.

Leo Soto M. said...

Jorge:

I agree with the fact that one way to use modern, more flexible languages is to design by prototyping a solution and prove if it shows promise or not.

On the other hand you will probably need to throw away the prototype if you want to create a really good quality system.

Javier Rojas Goñi said...

"If I'm understanding it correctly you mean that software is way more complex than civil engineering, making software almost impossible to fully plan in advance"

Non exactly Leo. What I mean is that we in software lack the proper tools (and established, standard methodologies) for proper planning and detailed specification that are present since hundreds of years in the construction industry. I'm afraid the continuous changes of paradigms and technologies in the software industry had made it difficult to evolve.

Leo Soto M. said...

Jorge,

OK. So you think that we lack tools/processes to make our field more like civil engineering. But how those tools/processes would look like? I fail to envision them

[And again, remember that the airspace and defense industry do have a civil-engineering-like process to build software. But it's really painful and hardly applicable to the rest of the word, AFAICS]

Javier Rojas Goñi said...

Leo,

It's difficult to envision the tools and processes we need. I agree with you that NASA does it, but that it's obviously not practical for our everyday business system.

What I feel is that software engineering is a goal we must keep pursuing, we need to be able to offer customer a clear understanding of what they will get and reliable schedules. We need to be able to completely and properly define a system to be built in order to outsource its building and get it done in fractions of the price.