Todd Biske: Outside the Box

Archive for May, 2007

I’ve been meaning to post on this subject for some time, so it’s good that I got to the airport a little earlier than normal today. There’s nothing like blogging at 5:30 in the morning.

As I mentioned in my last entry, I just finished listening to a podcast from IT Conversations on the drive to the airport which was a discussion on user experience with Irene Au, Director of User Experience for Google. One of the questions she took from the audience dealt with the notion of having a centralized group for User Experience, or whether it should be a decentralized activity. This question is one that frequently comes up in SOA discussions, as well. Should you have a centralized service development, or should your efforts be decentralized? There’s no right or wrong answer to this question, however, it’s certainly true that your choices can impact your success. In the podcast, Irene discussed the matrixed approach at Yahoo, and how everything would up being funded by business units. This made it difficult to do activities for the greater good, such as style guides, etc. The business unit wanted to maximize their investment and have those resources focused on their activities, not someone else’s. Putting this same topic in the context of SOA, this would be the same as having user-facing application teams developing services. The challenge is that the business unit wants that user-facing application, and they want it yesterday. How do we create services that aren’t solely of value to just that application. At the opposite extreme, things can be centralized. Irene discussed the culture of open office hours at Google, and how she’ll have a line of people outside her office with their user experience questions in hand. While this may allow her to maintain a greater level of consistency, resource management can be a big challenge, as you are being pulled in multiple directions. Again, putting this in the SOA context, the risk is that in the quest for the perfect enterprise service, you can put individual project schedules at risk as they wait for the service they are dependent on. These are both extreme positions, and seldom is an organization at one extreme or the other, but usually somewhere in the middle.

In trying to tackle this problem, it’s useful to think of things as either horizontal or vertical. Horizontal concepts are ones where breadth is the more important concern. For example, infrastructure is most frequently presented as a horizontal concern. I can take a four CPU server and run just about anything I’d like on it these days, meaning it provides broad coverage across a variety of functional domains. A term frequently used these days is commodity hardware, and the notion of commoditization is a characteristic of horizontal domains. When breadth becomes more important that depth, there’s usually not many opportunities for innovation. Largely, activities become more focused on reducing the cost by leveraging economies of scale. Commoditization and standardization go hand in hand, as it’s difficult to classify something as a commodity that doesn’t meet some standard criteria. In the business world, these horizontal areas are also ones that are frequently outsourced, as all companies typically do it the same way meaning there is little room for competitive differentiation.

Vertical concepts are ones where depth is the more important concern. In contrast to the commoditization associated with horizontal concerns, vertical items are ones where innovation can occur and where companies can set themselves apart from their competitors. User experience is still an area where significant differentiation can occur, so most user-facing applications fall into this category. Business knowledge, customer experience (preferably at a partnership level to have them involved in the process), are keys at this level.

By nature, vertical and horizontal concerns are orthogonal and can create tension. I have a friend who works as a user experience consultant and he once asked me about how to balance the concerns that may come from a user experience focus, clearly rooted in the vertical domains with the concerns of SOA, which are arguably focused on more horizontal concerns. There’s no easy answer to this. Just as the business must make decisions over time on where to commoditize and where to specialize, the same holds true for IT. Apple is a great example to look at, as their decision to not commoditize in their early days clearly resulted in them being relegated to niche (but still relevant) status in computer sales. Those same principles, however, to remain more vertically-focused with tight top-to-bottom controls have now resulted in their successes with their computers, iTunes, Apple TV, the iPod, and the forthcoming iPhone. There are a number of ways to be successful, although far fewer ways than there are to be unsuccessful.

When trying to slice up your functional domains into domains of services, you must certainly align it with the business goals. If there is an area of the business where they are trying to create competitive differentiation, this is probably not the best area to look for services that will have broad enterprise reuse, although it is very dependent on whether technology plays a direct role in that differentiation or an indirect role, such as whether the business to consumer interaction is solely through a website, or if it is through a company representative (e.g. a financial advisor). These areas that are closest to the end user are likely to require some degree of verticality to allow for tighter controls and differentiation. That’s not to say they own the entire solution, top to bottom, however, which would be a monolith.

As we go deeper into the stack, you should look for areas where commoditization and standardization outweighs the benefits of customization. It may begin at a domain level, such as integration across a suite of applications for a single business unit, with each successive level increasing the breadth of coverage. There is no point where the vertical solutions stop, and everything beneath it has enterprise breadth. Rather, it is a continuum of decreasing emphasis on depth and increasing emphasis on breadth. A Internet company may try to differentiate themselves in their end-user facing systems that the users interact with, allowing a large degree of autonomy for each product line. The supporting services behind those user interfaces will increase in the breadth of their scope, but still may require some degree of specialization, such as having to deal with a particular region of a country or even the world for global organizations. We then bleed into the area of “applistructure” and solutions that fall more into the support arena. A CRM system will have close ties to the end-user facing sales portal. The breadth of CRM coverage may need to span multiple lines of business, unlike the sales portal, where specialization could occur. Going deeper, we have applications that may have no ties to the end-user facing systems, but are still necessary to run a business, such as Human Resources. Interestingly, if you’re following my logic you may be thinking that these things should all be outsourced, but the truth is that many of these areas are still far from being commoditized. That’s because they involve user facing components, which brings us back to those vertical domains where customization can add value. An organization that outsources the technology side of HR, but doesn’t have an associated reduction in HR staff may have a potential conflict when they want to have specialized HR processes, but are dealing with commodity interfaces and systems. Put simply, you can’t have it both ways.

The trend continues on down the stack to the infrastructure and the world of the individual developer. If you’re truly wanting to adopt SOA from top to bottom, there should be a high degree of commoditization and standardization at this level. Organizations where solutions are still built top-to-bottom, with customized hardware platforms, source code management, programming languages, etc. are going to struggle with SOA, because their culture is vertically-oriented to an extreme.

While the speed of change, business decisions on what things are core competencies and what things are not do not change overnight. Taking an organization where each product group had its own staff (vertically-oriented) and switching it to a centralized sales organization (horizontally-oriented) is a significant cultural change, and often doesn’t go smoothly. You only need to look at the number of mergers and acquisitions that have been deemed successful (less than 50%) to understand the difficulty. Switching from vertically-focused IT solutions to horizontally-focused IT solutions is just as difficult, if not more difficult. Humans are significantly more adaptable than technology solutions, which at the core, are binary, yes/no environments. The important thing is to recognize when misalignment is occurring and take action to fix it. That’s all about governance. If users are trying to apply significant customization to a technology area that has been deemed as a commodity by the business, push back needs to occur to emphasis that the greater good takes precedence over individual needs. If IT is taking far too long to deliver a solution in an area where time to market and competitive differentiation is critical, remove the barriers and give that group more control over the solution, at the expense of broader applicability. If you don’t know what your priorities and principles are, however, for each domain, you’ll end up in and endless sequence of meetings that are rooted in opinions, rather than documented principles and behaviors desired by the organization.

I just listened to another great podcast from IT Conversations. This one is from the Adaptive Path series, and is a chat with Irene Au, the Director of User Experience at Google. Irene’s also a fellow alumnus of the University of Illinois (go Illini). User experience has always been a passion of mine, and it was interesting to hear what Irene’s done in her career at both Yahoo and now Google. The discussion was not so much about User Experience per se, but the way her and her teams have been utilized. If you’re interested in a little bit of insight into Google (the “testing on the toilet” part was great), I encourage you to give it a listen.

The latest ZapFlash from ZapThink discussed what the real barrier to SOA Adoption is, and Ron points directly at IT as the source. He states:

What ZapThink is finding is that the primary barriers to SOA adoption do not come from business management … but rather from within the IT organization … too many people in the IT organization conceive SOA as a technology concept only, and as such think of SOA as just a set of technologies and infrastructure for exposing, securing, running, and managing Services.

As always, Ron walks the line of controversy with this entry. In reading the article, there’s certainly not much you can argue about as far as some generalizations of IT go. I have seen organizations that are clearly burying SOA within IT, thinking of it as just another application development technology, just as Ron describes it. On some recent work, I recently had created a slide that showed a picture of a bunch of monolithic applications and then breaking apart those monolithic into groups of services. Some may think that this is doing SOA. I’d argue that if that’s the only thing that changes, you’re probably not. In most organizations, there does need to be some fundamental changes in the way IT operates. That being said, what causes that behavior in the first place? Ron made one statement in the above quote, which I intentionally left out at first. Here’s the full text:

What ZapThink is finding is that the primary barriers to SOA adoption do not come from business management, which by and large realize the benefits of an agile, reusable, and loosely coupled architecture (even if they don’t call it that), but rather from within the IT organization …

The italicized text is what really surprised me. Previously, I had a post that said that the organizations that are claiming success with SOA probably had a culture that already had IT and business working together at a higher level. Therefore, I would expect that a business that realizes “the benefits of an agile, reusable, and loosely coupled architecture (even if they don’t call it that)” has a mature Enterprise Architecture practice, with IT having a seat at the strategic planning table. If that’s the case, it would be even more surprising to find a huge gap between IT management and architecture and the actual IT execution. That tells me that some CIO, CTO, or Chief Architect is not doing their job very well.

In organizations where SOA is buried within IT, I would have expected that IT is buried within the organization. So, while all the problems that Ron describes would be accurate, the cultural issues actually ripple beyond IT. Those cultural barriers are a huge barrier, and it’s not just an IT problem, it’s a problem for IT and the business. If the business doesn’t think strategically, which you’d think would be required in order to understand the benefits of an agile, reusable, and loosely coupled architecture, how can can IT be expected to think strategically?

My pragmatic nature tells me that there are surely cases where IT is the problem. There are also cases where the business is the problem. There are cases where both groups are the problem, and there are cases where everything is well. The only real problem is when no one recognizes the problems that exist. It’s always better to deal with your own problems before you begin finger pointing at another organization. IT can be a barrier, and Ron’s article does call out some common problems so it’s definitely worth the read. Truth be told, however, if you’re struggling with your SOA efforts, odds are that there are things that need to be fixed both within the business and IT. Don’t try to boil the ocean, but rather, understand where you’re at, the things you need to work on, and set a realistic approach in motion for doing so.

Just received an email that David Chappell, author of the ESB book and formerly with Progress/Sonic, has been named Vice President and Chief Technologist for SOA for Oracle. Dave had been pretty quiet over the past 12-18 months, pretty much from the time when Progress acquired Actional. I was curious how things would pan out having both Dan Foody of Actional and David setting strategy in a space that had some overlap, as existed between some of the features of Actional SOAPStation and the Sonic ESB. All of that may have had absolutely nothing to do with David’s decision to go to Oracle and we’ll probably never know. In any case, I met David at the InfoWorld SOA Executive Forum in March of 2006, and have enjoyed what’s he’s had to say. I’m sure he’ll be successful in his efforts at Oracle.

It’s always refreshing to see someone else thinking along the same lines as yourself. The latest case for me was Om Malik in the April 2007 issue of Business 2.0. His column for that issue is titled, “Putting Widgets to Work.” He feels that there is significant promise for corporate widgets. This sounds very familiar to my post last week on Widgets, Gadgets, etc. You can read the teaser on his blog here, or read the full article over at CNNMoney.com.

Strategy is an interesting beast. It typically begins at such a high level that it’s possible to make connections between virtually any goal and any effort of a strategic nature, like SOA. At the same time, simple connections between those goals and efforts can be of, well, strategic importance. For this entry, I wanted to take one of those typical goals and try to bring it into the world of SOA and make some of those connections. Some of you may read this and just go, “Well duh, that’s common sense” while others may find something of significant value. I’d really like to hear your feedback on this entry. Strategy is one of those areas where people would many people in the trenches want to contribute, yet ultimately feel disconnected from it or that it doesn’t impact them. At the other end of things, the strategy makers want things to ripple down the trenches and for individuals to know that they are contributing to strategic success, yet their work is easily dismissed as being “fluff” or “ivory tower.”

The topic for today is knowing your customer. There’s no doubt that many, many companies likely have some strategic goal that can be tied back to customer support. The March 2007 issue of Business 2.0 had an article on “The Quest for the Perfect Online Ad” and it all comes down to better targeted advertising, which in turn, means knowing your customers. I would venture to guess that most companies try to do this through some form of data warehouse and business intelligence system. So how does SOA come into the mix? There’s actually two ways that I envision it having a role. First, it can enable a greater level of visibility into customer actions, presuming there’s some system interaction associated with the customer’s actions (sorry brick and mortar guys, not much SOA can do). Let’s presume that the system they use is a monolithic application. Your only source of input are the boundaries of the solution. If it’s a web application, you’ll have a clickstream, which can be pretty good, as evidenced by the levels of personalization from Amazon.com. If it’s a thick app, you may be limited to just extracting information from your transactional store, representing only the end outcome of the user’s efforts.

SOA, if done properly, should make it much easier to capture the user’s actions for later analysis. You’ll break down that monolithic application into a collection of services. These services represent boundaries within the solution, and at those boundary points you can leverage standardized infrastructure to log incoming requests and outgoing responses. This is even better than a web clickstream, because there you have to recreate the web page and understand what data elements were shown to the user on the page. If you’ve got the service request and response that returned the raw data tagged with the customer identifier, you’ve saved yourself a bunch of work associated with removing the presentation components. By pulling those messages into your BI system and analyzing the content of the message, you should begin to gain more knowledge about your customers.

The second way that SOA should come into play is in providing access to the data warehouse and business intelligence system to personalize the user experience. Rather than strictly being a source of reporting information for some person to look at, the information gleaned from those systems can be made available as services and factored back into the interface presented to the customer.

You may read this and think this is all common sense, and I actually hope that’s the case. Sometimes, however, we’re so buried in the technology or tactical issues that we fail to take the time to think strategically. It would be very easy to build the application in a service oriented manner and never bother to make use of the fact that this content is available and could provide valuable information for improving the customer experience. This is part of thinking outside of the box and beyond the functionality at hand. Hopefully reading this entry, whether you think it is common sense or not, will help you do that!