Todd Biske: Outside the Box

Archive for September, 2008

Just a note of thanks for Brenda Michelson and the SOA Consortium for a great meeting and the opportunity to participate in the Soapbox Derby. It was a bit like an un-conference in that the participants were free to talk about anything SOA-related for 15 minutes, followed up by 15 minutes of group conversation. I’m pretty sure that these will likely be posted as a podcast, but I hope they decide to repeat this effort at another meeting. We had some great conversations around governance, actual project experiences, organizational change management, SOA and model-driven architecture, EDA, and more. I think it could easily be a great draw again at a future meeting. A great part of organizations like the SOA Consortium is the opportunity to share experiences with fellow practitioners, and I thought the Soapbox Derby was a great way of doing it. I encourage my fellow practitioners to keep an eye out for the next time they do this and take the time to share for a few minutes.

During my soapbox derby discussion at the SOA Consortium meeting, I chose to discuss SOA Governance, and I thought that one of the messages I delivered would be another appropriate post to highlight some of the content in my upcoming book.

As I’ve said in this blog, SOA governance is the combination of people, policies, and processes that an organization uses to achieve the desired behavior associated with SOA adoption. This post, not surprisingly , will focus on the process component. Previously, I had a post explaining that governance does not imply command and control. Those two words only bring to mind one of the four processes: enforcement. There are three additional processes that your governance effort must also implement. The four processes are:

• Policy definition
• Education and communication
• Enforcement
• Measurement and feedback

Policy definition is concerned with establishing the policies that the governance team feels will result in the desired behavior if they are followed. Without policy, the rest of the organization must either guess what the correct decisions are to get to the desired outcome, or involve someone from the governance team on every single project. The first option is unlikely to lead to success, and the second option has both scalability issues as well as being prone to variation based upon the “tribal knowledge” of the particular person from the governance team involved. Defining and documenting the policies is step one toward gaining consistency in the outcome.

Education and communication is the next step, not enforcement. Just because the governance team has reached agreement and documented the policies doesn’t mean they’re going to be followed, or even known for that matter. A formal, planned communication effort to educate the organization on why you’re adopting SOA, the desired behavior you hope to achieve, and the policies that are being put in place to achieve them is required. It’s not a one time presentation to all of IT, but rather a series of targeted communications for the various roles in the organization, large group presentations, small team presentations, blogs, wikis, and appropriate surveys and followups to ensure that the communication is effective.

Enforcement is the third step. Even if your communication efforts are incredibly successful, you still need to put processes in place to ensure the policies are being followed. What you will find, however, is the better job you can do on communication and education, the easier your enforcement processes can be. If education is poor, enforcement will likely need to be more heavy-handed. Where possible, automated testing and reporting can certainly make the processes more efficient and cost-effective.

Finally, the governance group must have measurement and feedback processes to ensure that progress is being made toward the desired behavior. If the desired behavior is not reached, something needs to be changed, and it could easily be the policies, the processes, or the people involved with governance. Accountability is lost if the team puts policies and processes in place, but then does nothing to verify that all that effort actually paid off.

I’m here at the SOA Consortium meeting in Orlando. The first speaker at the event was Jeanne Ross from the Center for Information Systems Research at the MIT Sloan School of Management. They had recently completed a study on SOA adoption, and she presented some of the findings to the consortium. If you’re not familiar with Jeanne Ross, I thoroughly recommend two of her books, IT Governance and Enterprise Architecture as Strategy. I continue to refer to some of the concepts I learned from these books as part of my day-to-day job.

One of the interesting takeaways I had from Jeanne’s presentation was the information on the value companies are getting out of reusing services. What the study has found is that while there is definite measurable value that comes out of reuse, in many cases that effort comes at a very significant cost, a cost that often usurps the value generated. This made me think about a discussion that I frequently have with people concerning “enterprise” services versus “non-enterprise” services. In these discussions, it’s always been a black-or-white type of discussion where my stance is that we should assume a service has the potential to be an enterprise service unless told otherwise while the stance of the person I’m speaking to is frequently the exact opposite. It turns out that neither of these positions is the right way.

If you took my old stance, the problem is that you incur costs associated with building and operating a service. If that service doesn’t get reused, you won’t recoup that cost. If you took the opposing stance, you avoid the initial cost, but then you’re at risk of incurring an even higher cost when someone wants to reuse it. To maximize the value that you get out of your services, we need to find something in the middle. There does need to be a fixed amount of cost that will get sunk into any service development effort, but it needs to be focused on answering the question of whether or not we should incur the full cost of making it an enterprise service with full service lifecycle management, or if we should simply focus on the project at hand and leave it at that. I’m going to give this topic more thought and try to determine what that fixed cost should be and hopefully have some future blog entries on the subject. In the meantime, if you have ideas on what you think it should be, please comment or trackback. I’d love to hear more on how others are approaching this.

I will be attending the SOA Consortium meeting next week in Orlando. As Brenda announced on the SOA-C blog, I’ve agreed to be a soapbox derby participant.

I haven’t yet decided which soap box I want to climb on. There’s the obvious and likely SOA Governance topic, given my new book, but I’ll have to find the angle that can stimulate discussion. I could talk about how just because the media has shifted discussions toward WOA and other shiny new things, doesn’t mean that SOA should be put out to pasture. It’s safe to say that I will be talking. In my role as an enterprise architect, I understand that it is my job to have an opinion and provide direction. There’s nothing that drives me nuts more than a meeting where everyone knows something needs to be done, they may even agree on what needs to be done, but no one wants to step up and do it. There’s a soap box for you right there!

Anyway, I’m really looking forward to attending one of the face to face meetings for the first time. I’m especially looking forward to the keynote from Jeanne Ross on SOA adoption and value. If you’re going to be at the meeting, please at least stop by and introduce yourself.

With the upcoming publication of my SOA Governance book, which will be shamelessly plugged on this blog, you’ll be seeing more posts on SOA Governance, whether they are teasers to the book’s content or complementary material.

Many of the discussions I’ve had with colleagues on the topic of SOA governance is how to get started. Everybody’s heard from all of the analysts, bloggers, and other pundits on the importance of governance, but they don’t have a clear plan on how to put it in place at their organization. This isn’t a surprise, because organizations at this points are now facing the need to change head-on.

My definition of governance is the people, policies, and processes an organization uses to achieve a desired behavior. If you’re not achieving the desired behavior, than change is needed.

It is at this early stage where the first breakdown can occur. All too often, the steps of articulating the desired behavior and the policies that will lead to that behavior are not done, or done insufficiently. Rather, the focus immediately jumps to enforcement. Not surprisingly, the people involved with governance at organizations in this situation make statements like, “The developers are going to do whatever they want, and we can’t stop them.” Strong enforcement may catch things before they go live, but it doesn’t address the behavior that did things the wrong way to begin with. While it may result in some behavior change the next time around, it is unlikely, because it did nothing to change the understanding of the project managers, architects, and developers on what they should be evaluating themselves against as the right thing to do. If change is needed, but you’re not stating what you want to change into, the behaviors are unlikely to change. This truth applies whether we’re talking about pre-project governance, project (design-time) governance, or run-time governance, although it’s most easily understood in the world of project governance.

All too often, architects and developers lament the fact that the only concern of the stakeholders is that the project is delivered on time and on budget. If this is an impediment to success with SOA, then that mentality needs to change. If it does not, you’re always going to have this tension between the project manager and the technical leadership on the project. There’s a ripple effect to this, however. The resistance to such a change is that, in general, IT has not been able to demonstrate how adding technical concerns into the success criteria has an impact in IT’s ability to deliver solutions in a timely manner. If you are able to change the success criteria to where projects can be delayed in order to address architectural concerns, you’d better be collecting metrics to show that things are improving over time. This again comes back to establishing your desired behavior and policies first. If you have these in place, now you have something to measure against. If you’re not achieving the desired behavior, it doesn’t mean that you need to scream louder. A change in policy, people, or process may be what is needed.

So, if you’re looking for a place to start, my recommendation is not to focus on enforcement. My recommendation is to define the behavior you’d like to see out of your organization, the policies that will help guide that behavior, and then focus first on education of the organization on those items. If your staff is better educated on the outcomes the organization wants to achieve, they’re more likely to comply with the policies that will lead to that behavior, lessening the need for strong enforcement.

I can finally let my secret out. For the past few months, I’ve been working on my first book, SOA Governance, and I’m now happy to say that it is available from my publisher, Packt Publishing. It can also be purchased from Amazon.

If you’ve followed this blog, you’ll know that my take on SOA governance is that it’s all about using people, policies, and process to achieve a desired behavior, and that same theme carries through the book, showing how it applies to all aspects of your SOA efforts, ranging from project governance, to run-time governance, and finally to what I call pre-project governance.

The style of the book is inspired by the great books of Patrick Lencioni, including Five Dysfunctions of a Team, Death by Meeting, and Silos, Politics and Turf Wars. In it, he presents a fable that illustrates the concepts and points. In my book, I present the story of a fictitious company, Advasco, and their journey in adopting SOA. Along the way, the book analyzes the actions of Advasco and the role of SOA governance in the journey.

Please check it out, and feel free to send me your feedback or questions. I hope it helps you in your SOA efforts.

I’m sure David Linthicum is growing to expect a follow up blog from me after his “SOA Governance Monday” posts. True to form, I couldn’t let this week’s entry go by without commenting. There’s a good reason for that, however. Thanks to the good people over at Packt Publishing, my first book, SOA Governance, is now available for pre-order from Packt, with expected availability in October. It will also be available from your favorite online bookseller. Anyway, back to Dave’s blog entry.

The theme of this entry is SOA Governance as a service, which Dave states could be “a complete design time and runtime SOA governance system delivered out of the cloud.” I couldn’t help but think of the animation from Monty Python’s Flying Circus with the hand (or more frequently, foot) of God emerging from the clouds and squishing something beneath it. All humor aside, my first thoughts immediately came back to my usual comments on Dave’s past governance posts, which is that he’s focusing too much on the tools, and not on governance. His post probably should have been titled “Registry/Repository as a Service.” Of course, that’s what the original authors of UDDI tried to push, and that never came close to realizing that vision, but admittedly, the landscape has changed.

I have no issues with providing a registry/repository as a service. Certainly, the querying interface must be available as a service for any company exposing service outside their firewall. Likewise, if I’m consuming many services from the cloud, it would be great to let someone else handle putting all of those services into a common, queryable location, rather than me having to establish some form of federation or synchronization between my internal registry/repository and the registries/repositories of the service providers in the cloud. This is no different than the integration problem faced by a company that builds some services from scratch, but gets others from third party products like SAP that may have their own registry/repository, like SAP ESR.

Exposing the publishing interface is another story. If we get into the public service marketplace, we’d need this, but I think that’s a niche market today. Even if we consider a scenario where I deploy services into the cloud, I would argue that the publishing service is internal to the cloud provider. In short, the registry/repository capabilities would be part of the service hosting service (is that confusing or what?) made available by the cloud provider.

Back to governance. My constant theme on SOA governance is it is about people, policies, and process. The only role of tools is to make the processes more efficient. The cloud can only provide tooling. The degree to which you will need a registry/repository in the cloud will be completely dependent on the degree to which the rest of your tooling is in the cloud. If your services are deployed in the cloud, then it follows that the cloud must provide policy-driven run-time capabilities, such as request throttling, security, metric collection, etc. If your services are developed in the cloud, then the cloud must also provide metrics, code inspection, automated testing capabilities, and the same things that the design time tools used inside the enterprise provide. The cloud can not provide people, although I guess if a company wanted to outsource all of IT, and take a crowd-based governance model where everything they did would be open to the scrutiny of the developer community at large, then putting it all in the cloud would be a necessity.

The one upside to this notion of a registry/repository and all of the associated metadata associated with consumer/provider relationship out there in the cloud is that some large company with massive data centers (i.e. Google) could run their magic over all of that data and begin to extract out best practices for governing consumer/provider relationship, codifying policies, etc. Dave did call this out in his post, stating:

As the services are revised so are the design artifacts and the policies, both shared and private. In short, you’re taking advantage of the community aspect of SOA governance delivered as a service to do most of the work for youÖ100,000 heads are better than one.

While I don’t think the majority of large enterprises would be willing to allow their data to be analyzed in that manner today, it won’t surprise me at all if it happens in the future. If we think about it, this type of analysis on vendor contracts with enterprises is already done by companies like Gartner, so why shouldn’t a company like Google do the same for consumer/provider interactions with SOA? Even if that happens, it’s still not governance, it’s only analysis, which at best, provides better tools information for having good governance, but it won’t govern for you unless you choose to let it, at which point, you’re stating that the utilization of technology is a complete commodity for you, and you’ll just take whatever the big brother, be it Gartner, Google, or anyone else, tells you is the norm.

I saw the press release from SoftwareAG that announced their “strategic OEM partnership” with Progress Software for their Actional products.  While I’m not going to comment on that particular arrangement, I did want to comment on the challenge that we industry practitioners face when trying to leverage vendor technologies these days.

There has been a tremendous amount of consolidation in the SOA space.  There’s also been a lot of consolidation in the Systems Management space, another area where I pay a lot of attention. Unfortunately, the challenge still comes down to an integration problem. The smaller companies may be able to be more nimble and add desired capabilities.  This approach is commonly referred to as a “best of breed” approach, where you pick the product that is the best for the immediate needs in a somewhat narrow area.  Eventually, you will need to integrate those systems into something larger.  This is where a “best fit” approach sometimes comes into play.  Here, the desire is to focus more on breadth of capability than on depth of capability.

The definition of what is appropriate breadth is always changing, which is why many of the “best fit” vendors have grown by acquisition rather than continued enhancements and additions to their own solutions.  Unfortunately, this approach doesn’t necessarily make the integration challenges go away.  Sometimes it only means that a vendor is well positioned to offer consulting services as part of their offering, rather than having to go through a third party systems integrator.  It does mean that the customer has a “single throat to choke,” but I don’t know about you, I’d much rather have it all work and not have to choke anyone.

This recent announcement is yet another example of the relationships between vendors that can occur.  OEM relationships, rebranding, partnerships, etc.  Does it mean that we as end users get a more integrated product?  I think the answer is a firm maybe.

The only way that makes sense to me is to always retain control of your architecture.  It doesn’t do any good to ask the questions, “Does your product integrate with foobar?” or “How easy is it to integrate with such-and-such?”  You need to know the specifics of where and how you want these systems to integrate, and then compare that to what the vendors have to say, whether it’s all within their own suite of branded products or involves partners and OEM agreements.  The more specifics you have the better.  You may find that highly integrated suites, perhaps are integrated in name only, or maybe you’ll find that the suite really does operate as a well-oiled machine.  Perhaps you’ll see a small vendor that has worked their tail off to integrate seamlessly into a larger ecosystem, and perhaps you’ll find a small vendor that is best left as an island in the environment.

Then, after getting answers, go through a POC effort to actually prove it out and get your hands dirty (you execute the POC, not the vendor).  There are many choices involved in integrating these systems, such as what the message schemas will be, and the mechanisms of the integration itself- are you integrating “at the glass” via cut and paste between applications?  Are you integrating in the middle via service interactions in the business tier?  Or are you integrating at the data layer, either through direction database access or through some data integration/MDM-like layer?  Just those questions alone can cause significant differences in your architecture.  The only way you’ll see what’s really involved with the integration effort is to sit down and try it out, just do so after first defining how you’d like it to work through a reference architecture, then questioning the vendors on how well they map to your reference architecture, and finally by getting your hands dirty in a POC and actually trying to make it work as advertised in those discussions.