Todd Biske: Outside the Box

Archive for November, 2007

Robert Annett had a good post asking the question, “Do you need to build it yourself?” He states:

I’ve worked in many organisations and one thing that never ceases to amaze me is how often teams build unnecessary tools and frameworks … As a software/systems architect it’s your job to deliver value to the business you work for. Whenever someone suggests creating a bespoke tool ask yourself the following:

• Can I buy something to do this?
• Can I use or tailor an open source product?
• Can I use or create a template for a standard tool like Eclipse, Word or Excel?

I think that this is a challenge for corporate IT departments. I managed a frameworks team at a previous employer, and it’s extremely difficult to figure out where to invest your time. It was a continual game of guessing as to what things the vendors would add in the next release, what things would take off in open source community, and what things would be woefully underserved by third party products, whether open source/closed source or freeware/commercial products. It’s also not limited to low level frameworks, either, although the higher you go up toward business applications, the fewer choices there are and the more they cost.

Every company is going to be different with how they approach this problem, and a lot of it is dependent on company size. If there are IT resources to burn, you may be able to afford having a frameworks team, potentially even allowing developers to contribute to open source frameworks on company time. If IT resources are scarce, however, you have to balance the cost of a third party product and the cost of not having resources working on business solutions where custom coding is required.

I expect that to many of my readers, this is nothing new, but I do think that it a reminder now and then is good. Our responsibility is to deliver the best business solutions for appropriate costs, whether it is something off the shelf or something custom built. It is our job to be good corporate stewards and do our jobs wisely.

At the upcoming Gartner Application Architecture, Development and Integration Summit, I’ll be part of a panel discussion on funding SOA. I’ve previously posted on this in this entry, but I thought I’d bring it up again with the upcoming conference.

I’m very interested in hearing the experience of others on this topic. While there’s a lot of discussion about funding models, I still have yet to run into an organization that has had to actually implement one of these models. More often than not, I’ve seen one of two things:

1. A program of large enough scale where a number of services will be created with some in use by more than one consumer
2. Project-level SOA where a single project develops both the consumer and the service

There’s nothing wrong with either of these, but the thing to note is that these efforts did not require any change to the way funding of IT efforts occurs.

In discussing this with some colleagues, it seems that changes in how projects are funded really only comes about when reuse gets involved. In many ways, I feel that it is no different than dealing with shared (reused) infrastructure except that it is a bit more difficult to figure out how to partition the responsibilities.

A key question in all of this is how many services will be reused? If only 5% or 10% of services are reused, it is hard to justify changes to the funding model. But is this a chicken versus the egg scenario? Perhaps it is the way IT projects are defined to begin with which hampers the organization’s ability to identify services with reuse potential.

The point of this is that we’re still in the very early stages of SOA adoption. My sample base is very small, so I’m very interested in whether what I’m seeing is just an artifact of my small sample base, or if SOA funding is still a topic ahead of its time. This topic came up in a recent SOA Consortium call, and in order to reach out to a broader sampling base, I assisted in the development of a survey on the topic. It’s relatively high level and shouldn’t take very long to complete. I, and the other members of the SOA Consortium would certainly appreciate the input. It is intended for corporate SOA practitioners. You can access it here. Thanks!

It’s amazing how long it can take for some things to become a reality. Back in the corn fields of central Illinois in the early 90’s, I was in graduate school working for a professor that was researching visual programming languages. While the project was focused on building tools for visual programming languages, and not as much on visual programming itself, it certainly was interested. Here we are, almost 15 years later, and what’s the latest news from Microsoft? Visual programming languages. Okay, they’re calling it model driven development (which isn’t new either).

It will be very interesting to see if Microsoft can succeed in this endeavor. I suspect that they will, simply because Microsoft has a unique relationship with their development community, something that none of the Java players do. While there were competing tools for quite a few years, you’d be hard pressed to find an organization doing Microsoft development that isn’t using Visual Studio. You’d also be hard pressed to find an organization that isn’t leveraging the .NET framework. While the Java community has Eclipse, there’s still enough variation of the environment through the extensive plugin network that it’s a different beast. So, if Microsoft emphasizes model driven development in Visual Studio, it’s safe to say that a good number of developers will follow suit. As a point of comparison, BEA did the same thing over 5 years ago when they introduced WebLogic Workshop in February of 2002. This article stated:

BEA is calling WebLogic Workshop the first integrated application development environment with visual interfaces to Java and J2EE… “We’re radically changing the way people development applications,” said Alfred Chuang, president and CEO of BEA… Chuang said WebLogic Workshop could improve the application development and deployment cycle by as much as 100 times.

Hmmm… I’m getting a sense of deja vu. I’m hopeful that Microsoft can achieve better successes than BEA did. Point of fact, I don’t think it had anything to do with the quality of BEA’s offering. At the time, I had someone working for me look into Workshop. I was very surprised when the answer was not, “This is just a bunch of fluff, we need to keep writing the code” and instead was, “Wow, this really did improve my productivity.” Unfortunately, many developers will take their text-based editor to the grave with them so they can continue to write code.

In a similar vein, Phil Windley had a post on domain specific languages or DSLs. He points out that he is “a big believer in notation. Using the right notation to describe and think about a problem is a powerful tool.” Further on, he states, “GPLs (General Purpose Languages) are, by definition, general. They are designed to solve a wide host of problems outside the domain I care about. As a result, expressing something I can put in a line of code in a DSL takes a dozen in most GPLs.” Now Phil focuses on textual notations in his post, but I’d argue that there’s no reason that the notation can’t be symbolic. It may make the exercise of writing a parser more difficult, but then again, I’m pretty sure that the work I did back in grad school wound up having some in-memory representation of what was graphically shown in the editor that would have been pretty easy to parse. Of course, the example I used in my thesis wound up being music notation, which didn’t have such an internal representation, but I’m getting off the topic. If there are any grad students reading this blog, I think a parser for music notation is an interesting problem because it doesn’t follow the more procedural, flow chart like approach that we all too often see.

Anyway, back to the point. I 100% agree with Phil that a custom notation can make a smaller subset of programming tasks far more efficient. So much of what we do in corporate IT is just data in/data out type operations, and a language tuned for this, whether graphical or textual can help. The trend right now is toward graphical tools, toward model driven development. There will always be a need for GPLs, and we’ll continue to have them. But, given the smaller subset of typical corporate IT problems, it’s about time we start making things more efficient through DSLs, and model driven development. Let’s just hope it doesn’t take another 15 years.

InfoWorld published a collection of end-user stories on SOA this week, and the discussion on Leapfrog Enterprises caught my attention. In the article, Galen Gruman states that:

Leapfrog had many of the same goals that typify a typical SOA initiative: greater reuse of code, faster development time, and easier integration. But the company did not want to approach SOA as simply a changing of the guard for development tools and integration platforms. Instead it wanted to free its developers from conforming to a platform’s idea of best practices so they could focus on the applications’ functionality and use a wide range of development technologies as best for each job.

[Eugene] Ciurana [director of systems infrastructure for Leapfrog] did not want to force developers to all use the same transport. “The transport doesn’t matter,” he says. He chose to use the open source Mule ESB as a messaging backbone, relying on it to deal with transport interfaces. That way, “developers could focus as little as possible on the implementation of services,” he explains. Instead, their focus is on the functionality they are trying to achieve. The result is that developers tend to use HTTP as their transport mechanism, but some use REST (Representational State Transfer) and SOAP — “whatever works best or they’re most comfortable in.”

This caught my attention because it appears to be contrary to what I’ve seen at other organizations. Typically, the organization wants to constrain the platform to ease the integration problem and get away from the notion of “any-to-any” integration hubs. This may just be my misinterpretation, but it does raise an interesting question. How many constraints should be put in place? Interestingly, I’ve yet to run into an organization that has had to drive adoption of XML-based integration via enterprise architecture. The developers have slowly migrated away from whatever distributed object technology they were using and toward XML. The bigger challenge has been whether or not the XML contained the right information for broad consumption, not on whether or not XML was used. That being said, many EA teams are focused on the latter constraint (when to use XML or not). Knowing that there’s only so much that can be governed, what are the critical factors that EA teams should be making sure we get right as compared to the broad spectrum of things that we could be governing? Is it worth the pain to create policies regarding SOAP/HTTP versus XML/HTTP? The approach draws parallels to the big government/small government discussions that go on between the Democrats and the Republicans. The right answer is very dependent on the culture within IT, as I’ve stated often in this blog. Personally, I’m not a big fan of the integrate any-to-any approach. That being said, I also recognize that not everything is going to adhere to the constraints. I think it’s important to differentiate between your connectivity (mediation) infrastructure and your service enablement activities. Connectivity is about connecting two parties that adhere to the constraints for communication that have been adopted by the organization. Unless there’s only one way to communicate, there will be a need for mediation, for example, HTTP to JMS bridging. It’s important that the set of technologies be small, however. Service enablement is about taking something that doesn’t adhere to the standards and exposing it in a way that it does. We should strive to reduce the amount of service enablement over time, but the need for connectivity and mediation will always be there.

I was recently sent a copy of “SOA and WS-BPEL” by Yuri Vasiliev and Packt Publishing for review. The book is subtitled, “Composing Service-Oriented Solutions with PHP and ActiveBPEL.” After reading the book, the subtitle is much more accurate than the primary title. The book is first and foremost a guide for constructing Web Services using the SOAP extension for PHP and building BPEL processes using ActiveBPEL. Secondarily, there is a discussion behind the principles of SOA and WS-BPEL. Clearly, the right audience for this book is the developer community. If you’re removed from day to day coding, the book may not be as valuable.

If you’re looking for hands-on examples, this book has plenty of them. It includes all of the building blocks necessary from building your first service in PHP to creating an orchestrated process using BPEL. I felt that there was more emphasis on the coding efforts than necessary, however, and not enough on some of the theory behind it. This was evident in some of the early examples. In the sections on PHP, the examples result in a service that stores an XML representation of a purchase order in a database. The examples in the book take the incoming XML, create a PHP array representation of the XML, then convert it back to a DOM representation for storage in the database. While I do not know whether this approach was due to a limitation of the SOAP extensions, as an architect, it left me shaking my head. If the service is simply a pass-through to a database, there’s no reason to take all the time to parse the XML, bind it to some internal data structure, and then turn that internal data structure back into XML. Continuing on, the example then added XML schema validation to the mix, but it was performed by a stored procedure in the database. If you understand the role XML Schema validation has in security, odds are that XML schema validation will have occurred long before we even reach the back end database.

The section on WS-BPEL followed a similar vein. The bulk of the book was simply focused on walking you through the steps necessary to perform the actions in ActiveBPEL, rather than on building a sound understanding of WS-BPEL. There were pages upon pages of instructions on what menus to select, items to click, etc. I was very surprised at the lack of screenshots or graphical representations of the processes in the book. More often than not, they recommended using the Source tab in ActiveBPEL to compare the BPEL document in the book to what should have appeared after going through all the actions. My personal view on BPEL is that I don’t ever want to see it. I want to leverage the modeling capabilities of any BPM tool, and let the tool worry about BPEL behind the scenes.

Overall, for a book heavily focused on the developer community using PHP and ActiveBPEL (somewhat of a narrow audience, in my opinion), it’s certainly a good walkthrough to get your feet wet. It’s not going to give you the architectural skills you need, but it will move you through a lot of material very quickly. For people who are quick studies and just want some solid examples, you may find this a decent investment. For someone looking for more theory and architectural principles behind SOA and WS-BPEL, I’d probably look elsewhere.

Disclosure: This book was provided to me at no cost for the purposes of reviewing it. If you’re interested in having me review a book, please contact me at todd at biske dot com.

Okay, no bears, sorry. I read post from my good friend Jeff Schneider regarding SAP’s Enterprise Service Repository (ESR). He states:

At the core of the SAP SOA story is the Enterprise Service Repository (ESR). It is actually a combination of both registry and repository. The registry is a UDDI 3.0 implementation and has been tested to integrate with other registries such as Systinet. But the bulk of the work is in their repository. Unlike other commercial repositories, the first thing to notice is that SAP’s is pre-populated (full, not empty). It contains gobs of information on global data types, schemas, wsdl’s and similar artifacts relating to the SAP modules.

This now brings registry/repository into the mix of infrastructure products that SAP customers must make decisions regarding adoption and placement. Do they leverage what SAP provides, or do they go with more neutral products from a pure infrastructure provider such as BEA, HP, SOA Software, or SoftwareAG/WebMethods? The interesting thing with this particular space is that it’s not as simple as picking one. Jeff points out that the SAP ESR comes pre-populated with “gobs of information” on assets from the SAP modules. Choose something else, and this metadata goes away.

I hope that this may bring some much needed attention to the metadata integration/federation space. It’s not just a need to integrate across these competing products, but also a need to integrate with other metadata systems such as configuration management databases and development lifecycle solutions (Maven, Rational, Subversion, etc.). I called this Master Metadata Management in a previous post.

Back when Gartner was pushing the concept of the ESB heavily, I remember an opening keynote from Roy Schulte (I think) at a Web Services Summit in late 2005. He was emphasizing that an organization would have many ESBs that would need to interoperate. At this point, I don’t think that need is as critical as the need for our metadata systems to interoperate. You have to expect that as vendors of more vertical/business solutions start to expose their capabilities as services, they are likely to come with their own registry/repository containing their metadata, especially since there’s no standard way to just include this with a distribution and easily import it into a standalone RR. It would be great to see some pressure from the end-user community to start making some of this happen.