Todd Biske: Outside the Box

Archive for February, 2006

On an SOA mailing list I follow, someone recently posted a very simple question. What is a well-defined interface? The notion of a well-defined interface is frequently cited as a requirement for a good service, but the discussion usually heads in the technology direction of WSDL, IDL, WS-Policy, etc. In terms of importance to an enterprise SOA, I think that the use of standard description techniques is far less important than the qualitative aspects of being well-defined. As I see it, a well-defined interface is one where the actions that will be performed by the service provider are clear and unambiguous. Does using WSDL and WS-Policy ensure this? Absolutely not. Just as the use of standard look and feel guidelines and common user interface toolkits do not guarantee a highly-usable user interface, the use of WSDL and WS-Policy do not guarantee a highly-usable service interface. After all, the reason for defining something well is so that others will use it.

Defining an interface so that the behavior is clear and unambiguous is a very difficult thing to do. Rooted in binary technology, typical IT systems are not designed to deal with ambiguity. Where ambiguity exists, we design systems take a decision tree based approach, always dealing with yes/no decisions. Systems break down when they enter an unexpected state, a situation not modeled in the decision tree. When this decision logic is represented in source code, the cost to go in and add new logic to handle the unexpected state is high.

A very simple example that is timely for those of us in the USA is tax filing. Those of us who do our own taxes often use a system like TurboTax or TaxCut. The final step in the process is the electronic filing of a return. Let’s suppose I am providing an tax return filing service. There are state and federal fees to be collected. How should these fees be represented in the service interface? Hypothetically, the fee could be paid by extracting the fee from any refund the filer is due, they could supply a credit card, it could be electronically deducted from a bank account, a payment service like PayPal could be utilized, or a filer from previous years may have a payment preference on file. What happens if the credit card is rejected? What happens if the tax return is rejected?

When working with human-to-system interactions, systems can deal with this ambiguity by having a very generic return type, such as an HTML page. The interpretation of the response is left up to the human, and we’re good at processing ambiguities. This doesn’t work for system-to-system interaction. The analogous approach would be a web service that returns a message that can be any well-formed XML document.

Choreography is a big part of this. While we may try to decompose things to simple request-response patterns, the real world doesn’t work that way. What may have looked like a simple placeOrder operation may need to be modeled as a conversation, with only one path being a simple request-response. When conversations are involved, we now have to deal with system behavior when one party stops communicating. If I’m a cashier, and I tell a customer that they need to give me more money, I can infer that when they walk away from the counter without giving me the additional money, they’re not going to buy the product any more. I don’t sit there waiting for a response from them, and require the store manager to find another cashier to wait on customers. So what’s my advice? There are two things.

First, we still need to strive to take as much ambiguity out of the system-to-system interaction as possible. Systems must have some level of agreement on the semantics of the interchange and the policies associated with the interchange, including the semantics of the policies (we need domain specific policy languages). That being said, the ambiguity will never go away. If we only focus on removing ambiguity, we’re doing ourselves an injustice. I would argue that removing ambiguity results in more rigid systems, which is exactly the opposite of where we want to be. We’re adopting SOA to allow for increased flexibility and agility, not increased rigidity.

While embracing standards, we must also know that change will continue to occur and ambiguous situations will continue to arise. As a result, the second thing must be a focus on efficient management and incorporation of that change. While SOA and Web Services allow us to remove ambiguity, the worlds of BPM and EDA allow us to efficiently address ambiguous situations and changes to the interaction model. BPM and BAM technologies are allowing the interaction to be modeled and executed by run-time engines, including the ability to run simulations based upon new process variations and new interactions. Event processing allows analysis of a flowing stream of events to infer state (the key to recognizing an ambiguous situation), that can now drive new decisions in the process or new service invocations. Brenda Michelson has a great blog entry that goes into this in more detail, with some feedback from Joe McKendrick at ZDNet. For more on the well defined interface, check out Miko Matsumura’s comments on the same thread and pizza shop blueprint as well.

James McGovern has a few

outstanding questions on becoming an industry analyst. One of his five questions wasn’t directed at becoming an analyst, but interacting with analysts:

I have seen many bloggers comment on how software vendors should interact with industry analysts but have never seen anyone really explain how us folks in corporate America should? Come someone please blog about best practices in this space?

As a corporate America user of research from analysts, here’s my take on this. As with most things in this world, there are companies who are leaders, followers, and laggards. The 80/20 rule applies, with the “follower” category taking up the 80%, and the remaining 20% equally split between the leaders and the laggards. The leaders are taking risks, trying to innovate, and, when they choose wisely, reaping the rewards. The followers sit back and watch, and then try to follow a similar path of the leaders, albeit at less risk. The laggards, well, they’re distracted doing something else and eventually get left behind to wither and die.

The challenge for companies in this follower region is to identify the trends that lead to success, versus the trends that were not successful. SOA is a great example to use here. Virtually every single presentation I’ve seen on SOA begins with a definition. This is because there are so many different ways of applying service-oriented principles, and the right approach is dependent on the company’s strategy. When the CIO simply makes a statement that the company is going to embrace SOA, someone has to go out and figure out what that means. There are two common ways of doing this:

1. Hire someone who has helped other companies embrace SOA, effectively outsourcing the strategy and execution. When the consultants leave, so does the thought leadership. While some of that knowledge will carry over, the ability to analyze and implement primarily stays with the consultants. They go on to the next big thing, while the corporate workers continue to maintain the status quo.
2. Seek out information to build the internal thought leadership. This is done through books, magazines, whitepapers, webinars, and conferences. Of these, the best sources are going to be the ones that involve the widest coverage of case studies. Who has this? Analysts.

If we didn’t have analysts, the source of these items would lack breadth. The only entity with significant coverage would be vendors. Vendors’ bottom line is selling product, period. While they publish whitepapers, host conferences, etc., and these can be at a very high quality, it is all comes down to selling product. Outside of vendors, consulting firms are next in line. They, however, are more interested in sharing resources than they are in sharing information. If they share information, their resources may not be necessary. User groups and conferences that are not vendor sponsored tend to be local, which may constrain the amount of information that can be shared due to local competition for resources. So, there’s a big gap that needs to be filled, and that’s where the analysts can help.

Back to the original question: how should corporate America work with them? Clearly, the communication should be a two-way street. Just as the best service interfaces will probably come about through close communication between consumers and providers, the best analysis will come out through close communication between analysts and customers. The question, then, is how does the funding model reflect this? Do companies that share their information with analysts get discounts? Using the 80/20 rule again, 80% of the companies will likely be consumers of information only. That’s probably enough to sustain a healthy revenue stream. For the 20% that do share information, I would expect that discounting would be involved. If those companies fall into the leadership category, analysts may even need to pay for access to those companies. After all, these are the companies paving the way without support of external analysis, and they are the ones that the followers want to emulate. I’ve never had a research firm offer to pay me for information, so I don’t even know if this occurs.

Fortunately, analysts don’t publish evaluations of their corporate customers, so there isn’t the same conflict of interest that exists in the vendor space. There is potential for conflict of interest when two competing companies work with the same analyst firm, but again, the analyst research typically speaks in generalities, so this risk is mitigated. Additionally, many of the leaders are usually willing to talk about their experience, and are encouraged to do so. James’ should be familiar with this practice at The Hartford. This is further supported by the existence of research companies like the Corporate Executive Board who work to establish networks of companies for direct information sharing.

Ultimately, analyst firms exist to facilitate information sharing. Reading publications, case studies, vendor strategies, etc. is a full time job, and most companies are focused on providing solutions for their customers. Every company will have a different stance on how much “overhead” they want to fund. If the preference is to focus resources on execution rather than analysis, external research analysts will fill the gap. If a company is willing to fund analysis activities, there is a ton of free and low-cost information available. You may have every vendor knocking down your door, but it can be done.

Two days ago, I posted about my own aha moment with regards to SOA. This was in response to the latest ZapFlash from Ron. Today, they released their latest podcast
, which was further discussion about policies, contracts, SOA, and Web Services with Eyal Ma’or, CEO and co-founder of Silver-kite.

My own experience had to do with a realization on how aspects of things that we used to put into source code can be extracted and enforced through policy-based infrastructure. This is exactly the scenario that Eyal discussed with Ron and Jason. They discussed the lack of domain specific policy languages as a current barrier. To me, that’s one half of the problem, the other half being corporate culture. How many organizations have the maturity to properly express technical policies, let alone business policies? I fully believe that this is the way of the future, however, I don’t think I’d put this on my list of predictions for 2006 or 2007. My questions to my readers out there (and I’ve emailed it to Eyal, Ron, and Jason as well) is what policy domains can we apply this in today? Clearly, the most advanced domain is security. Whether through proprietary languages as part of an identity and access management infrastructure or through WS-SecurityPolicy, it can be done today. Beyond security, how would you prioritize the policy domains (and what are the policy domains)? Mine are:

1. Security
2. Routing/Accessibility (I’m not sure if you routing is the proper term, but this gets into the domain of load balancing algorithms and the drivers beyond them.)
3. Auditing/Compliance
4. Integration (i.e. transformations)

None of these begin to get into the “business” space, and I’m sure this is far from a complete list. What would be on your list? Leave comments or a trackback!

In the latest ZapThink ZapFlash, Ron Schmelzer provides some guidance on services, contracts, and policies. He speaks of the “aha” moment that many of us experienced when learning object-oriented programming languages, and compares this to a migration to service-oriented development.

Ron goes on to discuss the differences between a policy and contract based approach versus an object API-based approach. He discusses the need for a many-to-many relationship between policies and contracts, as well as between contracts and implementations. His analysis is correct, but what he didn’t go into is how the “aha” moment can occur.

In my own case, the key to this was when container-managed code was introduced with the J2EE platform. One of the goals of J2EE was to remove the responsibility for security, transaction management, and more from the developer by declaring the policies surrounding this in a separate configuration file. When J2EE was introduced, Sun also outlined the roles and responsibilities associated with a development project. One of these, the Application Assembler, was intended to create this descriptor file, while the Application Component Provider was responsible for the implementation. In my experience, these two roles were typically played by the same person. There’s even recognition of this in the C# and .NET framework where these policies are implemented by annotating source code. This is also now common practice in the Java camp as well, first through JavaDoc tags, and now through J2SE 5.0 Annotations.

When I began investigating Web Services, the “aha” moment occurred when I realized that the declarative policies that were established in a platform-specific manner can now be utilized independent of the platform. In fact, the entire “container” concept can be externalized from the actual implementation container. Since Web Services are sent as XML messages on the wire, policies can be enforced anywhere between the consumer and provider. Working on the infrastructure side of things helps this understanding, because some policies, typically routing, are already configured on a separate device from the application container.

This truly is a light bulb moment, because of the implications. As soon as policies can be externalized from the source code and execution container, we begin down a path where policy definition can be separated from policy enforcement points. Once policies are separated from the enforcement points, a new breed of tooling can be created for policy definition appropriate to the policy setters, rather than appropriate to the enforcement point.

For example, an organization may have an Information Security department that sets policy, but it may require an IT operations staff member to express that policy in a way that can be enforced by the infrastructure. If that same infrastructure also enforces routing policy, there is a dilemma. The person configuring routing policies should not be able to configure security policies. The key to this, as Ron suggests, lies with standards and tool support. By creating standard policy languages, an interoperability framework for sharing policies between editing, management, and enforcement points can now be created.

The more I think about this, the more there is to say. Some more thoughts:

• To have success as a good service provider, you need to provide a good service.
• Services developed in Corporate IT are inherently more difficult than many Web 2.0 services because of the user-developer disconnect. In the case of Web 2.0, many services are consumer oriented. Guess what? Most developers are also consumers. In the corporate world, services are business oriented. Chances are, you don’t also play the role of the business user. Interestingly, however, we still are always moving in a direction of allowing the actual end user to build their own systems. Arguably, many tech-savvy users come up with new ways to utilize technology beyond what the IT department provides, whether it’s as simple as a good set of bookmarks in their browser, or some sophisticated use of scripting.

One last thought- I recently read An Introduction to Enterprise Architecture by Scott Bernard. One of the points he makes early in the book is that one element of enterprise architecture is social theory. Given this debate and my own experiences, I couldn’t agree more.

There’s some interesting banter going back and forth between Dion Hinchcliffe and Steve Vinoski, stemming from Steve’s IEEE Internet Computing column, entited The Social Side of Services.

In this blog entry, Dion states that socializing SOA is “not the first order of business or the most important aspect of adoption.” Steve counters in his own blog.

Whether you agree with Dion or Steve, this is a very interesting conversation. I, for one, fall more on Steve’s side. In 1998, a presenter at SD East spoke on implementing reuse in a corporate IT shop, and his experience at several large firms was that marketing and communications was the number one factor in achieving success. I tend to agree. The degree of communication across IT groups is simply not what it needs to be to foster reuse. Dion dismisses this, and more blames it on “software that is actually extremely hard to mesh together.” Here, I disagree. I don’t think that integrating a typical corporate Web Service using SOAP/WSDL is all that more difficult that integrating a REST-based service, especially for system-to-system interaction. I’ll give REST or simple XML over HTTP the nod for AJAX-to-service interaction.

Dion does make some good points that corporate developers do need to embrace, however. He states that “Web 2.0 approaches to service design dictate an almost extreme preference for encouraging unintended uses.” Here, he hits the nail right on the head. Roy Schulte commented in a webinar that we need to switch from “build to last” to “build to change.” If an internal IT team builds a great service, they need to understand what it means to be a service provider, and encourage any usage they can get. If they take a rigid stance with regards to their customers, they won’t be successful.

Where I see the gap in this debate is really the cultural differences between Web 2.0 development and corporate development. Corporate developers are working to the demands of the business according to a project schedule. I’m willing to bet that the typical Web 2.0 Google Maps mashup is not, or at least not in the early days. On top of that, the mashups are creating new applications out of existing parts that would be too expensive to go and build from scratch. I don’t think any Web 2.0 programmer (unless they work for MapQuest, Yahoo, or the mapping provider…) could go out and write Google Maps on their own. For the services being reused in corporations, this isn’t the case. Often times, the developer has everything they need to write it themselves. In this situation, it absolutely is about marketing and communication. If the developer doesn’t know someone has already done it, they’ll go off and do it themselves. If they do know it exists, now it’s more of a trust and control issue.

In the Web 2.0 world, we’re encouraged to communicate, socialize, and share. (hmmm, isn’t this what Steve was saying is a critical factor?) If the service needs to be tweaked, we’d contact the service provider and get it tweaked. Even in the Web 2.0 world, if the group was unresponsive, that service will quickly fade away.

In the corporate world, the team providing the service does not have resources assigned to that project, and this makes project managers very, very nervous. They’re now dependent on something outside of their control, and unless the corporate culture changes to allow that, they may be apt to do it themselves, strictly to control their own destiny. If your organization can’t built the internal trust to get over this, they won’t be successful with SOA.

Just a quick note to thank Brenda Michelson for coming to St. Louis last week, and for the good people at CAIT (Center for the Application of Information Technology from Washington University) for setting it up, including my good friend Fred Domke. It’s a shame that her luggage couldn’t make it, but she still made the best of it. As Brenda reflected in her blog, events like these (the CAIT event, not loosing your luggage) can only help all of our work get better. I look forward to CAIT’s continuing workshop on SOA and being a part of it. As if that wasn’t enough, Thomas Erl also paid us a visit courtesy of Microsoft, including copies of his latest book.

Both presentations were very good. I always enjoy these exchanges on best practices and approaches to SOA. Everyone comes at SOA in a slightly different manner, and it’s interesting to see what points others emphasize or include. Thomas clearly came from the technical side of things, which was appropriate for the audience. Brenda, as I had hoped, brought the user element back into the mix. I was glad to see this as well as recent articles that also touched on this from Ron Schmelzer of ZapThink and JP Morgenthal (blog) of Avorcor.

Anyway, it was a good week for SOA in St. Louis. Hopefully, it’s the beginning of a good trend, although I certainly don’t mind having to travel to Orlando, San Francisco, etc. to discuss SOA.

I’ll be part of a customer panel in Phil Windley’s SOA Governance session at the upcoming InfoWorld SOA Executive Forum. This event, as well as recent publications on SOA Governance (Phil’s article in the Jan. 23rd print edition of InfoWorld, Jason Bloomberg’s Butterfly Effect ZapFlash), have me thinking more and more about this topic.

One very simple analogy to make regarding SOA Governance is to compare it to traditional government. What does Congress do? What do your local city leaders do? This is actually a very good analogy, because it works for all definitions of SOA Governance, whether applied to strategic planning, design-time governance, or run-time governance. Recently, I’ve started thinking about more intricate details of this analogy. It is certainly true that SOA Governance has to do with setting and enforcing policy, just as does traditional governance. What’s interesting, however, are the types of policies being set and the communication involved in those processes.

A great illustration is my own experience. From mid-1994 to late 1997, I lived in California. During that timeframe, California had ballot initiatives on discrimination in state university admission processes (Proposition 209) and on basic services (i.e. health care) for illegal immigrants (Proposition 187). As part of the voting process, California would provide a sample ballot, as well as statements on the proposed initiatives and candidates. There would typically be a statement for the initiative/canditate along with a rebuttal, as well as a statement against the initiative/candidate along with a rebuttal. I found this to be extremely valuable in my own processes to be an educated voter.

In late 1997, I moved to Missouri. The most highly contested initiative for my first ballot as a Missouri resident was cock fighting restrictions. Missouri politicians are only now beginning to debate things like smoking bans in restaurants. Thankfully, I’ve moved to southern Illinois where we at least have the influence of Chicago on statewide issues to keep things a bit more progressive. Neither state, however, provides anything to help me be an educated voter. I have to rely on my own research, in addition to the normal media.

What this demonstrates is that in addition to figuring out what SOA means to your company, you also need to figure out what SOA Governance means to your company. Should you be progressive about governance and focus on cutting edge concerns, or do you still need to cover the basics? What information do you need to provide as part of the governance process, and how much do you involve the people who will ultimately be governed by the policies being set? Ultimately, success will largely depend on how well your approach to these topics match your corporate culture. If your company is conservative, adopting a progressive stance on governance may be too much shock for the company to take right off the bat. Likewise, A conservative approach on governance may put a halt to the rapid innovation occurring at a progressive company. The same holds true in traditional government. The focus of media attention in the middle east has been on the fact that voting is occurring, rather than on what topics are on the ballot. Simply having a vote for the highest levels of government is enough of a culture change. Anything more would have been a failure.

As with any change, and SOA is most definitely a change, the key is understanding where you want to go, creating a plan to get you there, and then executing that plan. SOA Governance is simply another element to be addressed. Take the time to understand what your needs are and the pace that they can be addressed. If you’re currently using punch cards, hanging chad and all, but voting on things that have minimal impact on society, switching to an electronic voting machine isn’t going to help.

Tags: SOA, Governance