Archive for the ‘Repository’ Category
All content written by and copyrighted by Todd Biske. If you are reading this on a site other than my “Outside the Box” blog, it’s probably being republished without my permission. Please consider reading it at the source.
David Linthicum continued the conversation around design-time governance in cloud computing over at his InfoWorld blog. In it, he quoted my previous post, even though he chose to continue to use the design-time moniker. At least he quoted the paragraph where I state that I don’t like that term. He went on to state that I was “arguing for the notion of policy design,” which was certainly part of what I had to say, but definitely not the whole message. Finally, Dave made this statement:
The core issue that I have is with the real value of the technology, which just does not seem to be there. The fact is, you don’t need design-time service governance technology to define and define service policies.
Let’s first discuss the policy design comment. Dave is correct that I’m an advocate for policy-based service interactions. A service contract should be a collection of policies, most if not all of which will be focused on run-time interactions and can be enforced by run-time infrastructure. Taking a step backward, though, policy design is really a misnomer. I don’t think anyone really “designs” policies, they define them. Furthermore, the bulk of the definition that is required is probably just tweaking of the parameters in a template.
Now, moving to Dave’s second comment, he made it very clear that he was talking about governance technology, not the actual governance processes. Speaking from a technology perspective, I’ll agree that for policy management, which includes policy definition, all of the work is done through the management console of the run-time enforcement infrastructure. There are challenges with separation of concerns, since many tools are designed with a single administration team in mind (e.g. can your security people adjust security policies across services while your operations staff adjust resources consumption while your development team handles versioning, all without having the ability to step on each other’s toes or do things they’re not allowed to do?). Despite this, however, the tooling is very adequate for the vast majority (certainly better than 80-90% in my opinion) of enterprise use cases.
The final comment from me on this subject, however, gets back to my original post. Your SOA governance effort involves more than policy management and run-time interactions. Outside of run-time, the governance efforts has the closest ties to portfolio management efforts. How are you making your decisions on what to build and what to buy, whether provided as SaaS or in house? Certainly there is still a play for technology that support these efforts. The challenge, however, is that processes that support portfolio management activities vary widely from organization, so beyond a repository with a 80% complete schema for the service domain, there’s a lot of risk in trying to create tools to support it and be successful. How many companies actually practice systemic portfolio management versus “fire-drill” portfolio management, where a “portfolio” is produced on a once-a-year (or some other interval) basis in response to some event, and then ignored for the rest of the time, only to be rebuilt when the next drill occurs. Until these processes are more systemic, governance tools are going to continue to be add-ons to other more mature suites. SOA technologies tried to tie things to the run-time world. EA tools, on the other hand, are certainly moving beyond EA, and into the world of “ERP for IT” for lack of a better term. These tools won’t take over all corporate IT departments in the next 5 years, but I do think we’ll see increased utilization as IT continues its trend toward being a strategic advisor and manager of IT assets, and away from being the “sole provider.”
All content written by and copyrighted by Todd Biske. If you are reading this on a site other than my “Outside the Box” blog, it’s probably being republished without my permission. Please consider reading it at the source.
David Linthicum started a debate when he posted a blog with the attention grabbing headline of “Cloud computing will kill these 3 technologies.” One of the technologies listed was “design-time service governance.” This led to a response from K. Scott Morrison, CTO and Chief Architect at Layer 7, as well as a forum debate over at eBizQ. I added my own comments both to Scott’s post, as well the eBizQ forum, and thought I’d post my thoughts here.
First, there’s no doubt that the run-time governance space is important to cloud computing. Clearly, a service provider needs to have some form of gateway (logical or physical) that requests are channeled through to provide centralized capabilities like security, billing, metering, traffic shaping, etc. I’d also advocate that it makes sense for a service consumer to have an outgoing gateway, as well. If you are leveraging multiple external service providers, centralizing functions such as digital signatures, identity management, transformations, etc. makes a lot of sense. On top of that, there is no standard way of metering and billing usage yet, so having your own gateway where you can record your own view of service utilization and make sure that it’s line with the what the provider is seeing is a good thing.
The real problem with Dave’s statement is the notion that design-time governance is only concerned with service design and development. That’s simply not true. In my book, I deliberately avoided this term, and instead opted for three timeframes of governance: pre-project, project, and run-time. There’s a lot more that goes on before run-time than design, and these activities still need to be governed. It is true that if you’re leveraging an external provider, you don’t have any need to govern the development practices. You do, however, still need to govern:
- The processes that led to the decision of what provider to use.
- The processes that define the service contract between you and the provider, both the functional interface and the non-functional aspects.
- The processes executed when you add additional consumers at your organization of externally provided services.
For example, how is the company deciding what service provider to use? How is the company making sure decisions by multiple groups for similar capabilities are in line with company principles? How is the company making sure that interoperability and security needs are properly addressed, rather than being left at the whim of what the provider dictates? What happens when a second consumer starts using the service, yet the bills were being sent to the first consumer? Does the providers service model align with the company’s desired service model? Does the provider’s functional interface create undue transformation and integration work for the company? These are all governance issues that do not go away when you switch to IaaS, SaaS, or PaaS. You will need to ensure that your teams are aware of the contracts in place, and don’t start sending service requests without being properly onboarded into the contractual relationship. Your internal allocation of charges takes multiple consumers into account, if necessary. All of these must happen before the first requests are sent in production, so the notion that run-time governance is the only governance concern in a cloud computing scenario is simply not true.
A final point I’m adding on after some conversation with Lori MacVittie of F5 on Twitter. Let’s not forget that someone still needs to build and provide these services. If you’re a service provider, clearly, you still have technical, design-time governance needs in addition to everything else discussed earlier.
I’ve been invited to participate in SearchSOA.com’s “Ask the Expert” series and will be fielding questions primarily on BPM technologies in the context of SOA, but I hope to see some EA related questions as well. My first response was posted on November 3rd, answering the question, “What is a key characteristic I should look for in BPM modeling tools, especially when looking to pair them with SOA?” You can read my response at SearchSOA.com.
In a two-part podcast series, Dave Berry from Oracle’s Fusion Middleware team and Mike van Alst, a consultant with IT-eye, discussed some remarks I made in an earlier OTN Arch2Arch podcast regarding SOA and reuse. Specifically, I tried to de-emphasize the reuse aspect of SOA. Many reuse programs that I’ve seen or read about have two key elements:
- Building things in a reusable manner
- Making those things visible
While noble goals, these approaches are at significant risk of producing the intended results. The first item has a fundamental problem in that it is all but impossible to define exact what “building in a reusable manner” is. We can use open, interoperable standards rather than closed, proprietary ones, but is this the key barrier to reuse? There’s probably some low hanging fruit that this will capture, but there’s so much more to reuse than this. From a technical standpoint, one must also consider the structures of the information being exchanged and the varying granularity of the information being exchanged, among other things.
On the second item, visibility is important, there’s no doubt about it. But visibility without context will not be successful. It’s a matter of providing the right information at the right time. Too many initiatives that are associated with the collection of IT artifacts, be it reuse, SOA, portfolio management, ITSM, or any of the like, fail because the information is never put into the context of the processes that need that information. How many times have you seen the information collected as part of a fire drill for an immediate need, only to grow stale once that fire drill is completed.
The two things I recommend are service ownership and linkage to key IT processes. If you’ve heard me talk on panel discussions at conferences, you’ll know that my answer to the question, “What’s the one piece of advice you have for companies adopting SOA?” has always been, “Define your service owners.” Someone is given the responsibility for a functional area, providing capabilities to the rest of the organization and accountable for driving out the redundancies that may exist. This is a tricky exercise, because service ownership has a cost associated with it. Expending that cost for a service that is only used by one consumer can lead to waste, so it’s not a silver bullet. It does, however, being the cultural change from a project-driven organization to more of a product-driven/service-driven organization. Without having someone accountable for the elimination of redundancy in a domain and serving the needs of consumers, it won’t happen.
The second piece of advice is the process integration. To avoid creating repositories that see infrequent use after initial population, you have to define the role of that information in the IT processes. If you have a service repository, when do you expect project architects and designers to look into that repository for services that may be appropriate. How about it the strategic planning process? The scoping effort for a project likely begins long before a project architect is assigned? How is the service repository used in those activities? By defining the links with key IT processes and ensuring that those processes are changed to use the repositories involved, with appropriate governance to make sure those changes are occurring, you will make sure that your services are visible, and more importantly, that the right people are looking for them at the right time.
Here’s a topic for which I’d really like some community input, and I think it’s something that many of my readers have probably had to do, are doing, or would be interested in the result. If you’re adopting SOA, you’re likely using a Service Registry/Repository of one form or another. It can range from a set of scribbled notes on a whiteboard or post-its in some architect’s office/cube, to Excel, to one of the many vendor products available for this purpose. So, assuming you are actually using one of these mechanisms, what are you recording about your services, the consumers of those services, and how/where are you capturing the relationship between the two? In this post, I’m going to start with the first question, the answer to which constitutes what I call the technical service record. Please note that the focus of this is on services that have a programmatic interface, and not the broader business service or ITIL service space, although I am very interested in the overlap between this record and the service record that would existing in an ITIL v3 Service Portfolio.
Here’s a list of items that could be recorded about a service to get the discussion started. For each item, I’ve provided a description of what that item is, whether it is optional or not, the visibility of that item (public, consumers only, service manager only, etc…). Please contribute your thoughts on other attributes that could/should be captured along with its optionality (is that a word?) and visibility.
|Name||Human-readable name of the service||Yes||Public|
|Description||Human-readable description of what the service does||Yes||Public|
|Owner/Manager||The person accountable (in the RACI sense) for the service. At a minimum, this is the person to contact in order to begin using the service.||Yes||Public|
|Question: Should the owner be public, or only visible to registered consumers? A registry/repository could facilitate interaction with a potential consumer without publicly revealing the owner’s name.|
|Interface Type (or should it be types?)||The technical interface type, such as SOAP, REST, POX/HTTP, etc.||Yes||Public|
|Internal/External||Is the service exposed internally, externally, or both?||Yes||Public|
|Note: External users can only see services exposed externally.|
|Service Type||Taxonomy classification for purposes of mapping to technology platform||Yes||Internal Only|
|Production WSDL URL||URL for the production WSDL (Required for Web Services)||No *||Consumers|
|Deployment Platform||On which logical platform is the service hosted?||No *||Internal Only|
|Deployment Location||What is the physical location(s) of the service? Preferably, this should be a link into the CMDB.||No *||Internal Only|
|Test Plan/Scripts||A link to a test plan or specific test scripts for the service as provided by the provider.||No *||Internal Only|
|Performance Profile||The expected resource utilization of the service.||No *||Internal Only|
|Development Cost||The cost incurred in creating the service.||No *||Internal Only|
|Estimated Integration Cost||Expected cost for consumers to integrate service usage.||No *||Internal Only|
|Current ROI||Current development ROI generated based upon development cost, cost to integrate, and current number of consumers||No *||Internal Only||Status||Status of the service: Planned, in development, in production, decommissioned)||Yes||See below|
|The visibility of this is directly tied to the state. For internal services, status is open to the public. For external services, a service should only be visible if it is in production.|
|Version||The version of the service associated with this record.||Yes||Public|
|Created Date||The date this record was created.||Yes||Internal Only|
|Modified Date||The date this record was last modified.||Yes||Internal Only|
Of course, now that I attempted to put this list down with some simple attributes, I’ve realized that whether or not things are required or visible to particular parties are dependent on the status of the service, whether it is exposed externally or not, the interface type, etc. It’s just hard to make that fit into an HTML table and still have this entry be readable. Anyway, if there isn’t anything proprietary or confidential about the structure of your service records, consider sharing it here. I promise to publish the end result of this effort here for all to share for free. This isn’t limited to Web Services, either. If you’re using REST, what information would you provide about the collection of resources that comprise the service to potential users of those services? I would guess that many of the above attributes would still apply, and could certainly be accessed themselves through a REST interface, since a serivce record is a resource in and of itself.
Thanks for your participation! If you’d prefer to send me your information directly without publicly posting it here, send me an email at todd at biske dot com or you can send me a direct message on twitter at toddbiske.
The press releases came out today that SOA Software has bought LogicLibrary, with blogosphere comments from Miko Matsumura, Dana Gardner, and Jeff Schneider. I see this as a step toward the bigger SOA platform players by SOA Software. At this point, most of the players in SOA platforms all now have a registry/repository offering. IBM has WebSphere Registry Repository, Oracle/BEA has AquaLogic Registry Repository (consisting of the OEM’d Systinet and purchased Flashline products), Tibco resells Systinet, SoftwareAG has the former WebMethods/Infravio, Iona has Artix Registry/Repository, SAP has their Enterprise Service Repository, and Microsoft has their Oslo efforts. I think it’s safe to say that the vendors that are trying to be the acquirer rather than the acquired have all realized that a registry/repository is the center of the SOA technology universe. Now if only they could talk to each other easily along with the CMDBs of the ITIL technology world.
In my “Future of ESBs” post, I talked about how selling an ESB on its own is a difficult proposition because of the relative value that a developer will place on it. The same thing certainly holds true for a registry/repository, and I think the market has shown that to be the case by now having all of the registry/repository providers get swallowed up by larger fish. It would be interesting to know how many times these products are sold on their own versus being bundled in as a value-add with a larger purchase.
The fascinating thing about service contract standardization, a point that you hit on at the end of your post, is that it is not substantially different from the standardization of terms and conditions that occurs for legal agreements or sales agreements in an organization.
I am a SOA architect and a member of my Enterprise Architecture team, as you are, but Iâ€™m also intimately familiar with solutions that perform Contract Generation from Templates in the Legal and Sales agreements for a company. My employer sells over 80% of their products through the use of signed agreements. When you run $3B of revenue, per month, through agreements, standardization is not just useful. It is essential.
When you sign an agreement, you may sign more than one. They are called â€œmulti-tierâ€? agreements, in that an agreement requires that a prior one is signed, in a chain. There are also â€œassociated agreementsâ€? that are brought together to form an â€œagreement packageâ€?. When you last bought a car, and you walked out with 10 different signed documents, you experienced the agreement package firsthand.
These two concepts can be leveraged for SOA governance in terms of agreements existing in a multi-tier environment, as well as services existing in an ecosystem of agreements that are part of an associated package.
For example, you could have one of four different supporting agreements that the deployment team must agree to as part of the package. All four could rely on the same â€œcommon terms and taxonomyâ€? agreement that every development and deployment team signs (authored by Enterprise Architecture, of course). And you could have a pair of agreements that influence the service itself: one agreement that all consumers must sign that governs the behavioural aspects of the service for all consumers, and another agreement that can be customized that governs the information, load, and SLA issues for each provider-consumer pair.
If this kind of work is built using an automated agreement management system, then the metadata for an agreement package can easily be extracted and consumed by automated governance monitoring systems. We certainly feed our internal ERP system with metadata from our sales agreements.
Something to think aboutâ€¦
In an email exchange with David Linthicum and Jason Bloomberg of ZapThink in response to Dave’s last podcast (big thanks to Dave for the shout-out and the nice comments about me in the episode), I made some references to the role of the service contract and decided that it was a great topic for a blog entry.
In the context of SOA Governance, my opinion is that the service contract is the “container” of policy that governs behavior at both design-time and run-time. According to Merriam-Webster, a contract is “a binding agreement between two or more persons or parties; especially : one legally enforceable.” Another definition from Merriam-Webster is “an order or arrangement for a hired assassin to kill someone” which could certainly have implications on SOA efforts, but I’m going to use the first definition. The key part of the definition is “two or more persons or parties.” In the SOA world, this means that in order to have a service contract, I need both a service consumer and a service provider. Unfortunately, the conversations around “contract-first development” that were dominant in the early days caused people to focus on one party, the service provider, when discussing contracts. If we get back to the notion of a contract as a binding agreement between two parties, and going a step further by saying that the agreement is specified through policies, the relationship between the service contract and design and run time governance should become much clearer.
First, while I picked on “contract-first development” earlier, the functional interface is absolutely part of the contract. Rather than be an agreement between designers and developers, however, it’s an agreement on between a consumer and a provider on the structure of the messages. If I am a service provider and I have two consumers of the service, it’s entirely possible that I expose slightly different functional interfaces to those consumers. I may choose to hide certain operations or pieces of information from one consumer (which may certainly be the case where one consumer is internal and another consumer is external). These may have an impact at design-time, because there is a handoff from the functional interface policies in the service contract to the specifications given to a development team or an integration team. Beyond this, however, there are non-functional policies that must be in the contract. How will the service be secured? What’s the load that the consumer will place on the service? What’s the expected response time from the provider? What are the notification policies in the event of a service failure? What are the implications when a consumer exceeds its expected load? Clearly, many of these policies will be enforced through run-time infrastructure. Some policies aren’t enforced on each request, but have implications on what goes on in a request, such as usage reporting policies. My service contract should state what reports will be provided to a particular consumer. This now implies that the run-time infrastructure must be able to collect metrics on service usage, by consumer. Those policies may ripple into a business process that orchestrates the automated construction and distribution of those usage reports. Hopefully, it’s also clear that a service contract exists between a single consumer and a single provider. While each party may bring a template to the table, much as a lawyer may have a template for a legal document like a will, the specific policies will vary by consumer. One consumer may only send 10,000 requests a day, another consumer may send 10,000 requests an hour. Policies around expected load may then be enforced by your routing infrastructure for traffic prioritization, so that any significant deviation from these expected load don’t starve out the other consumers.
The last comment I’d like to make is that there are definitely policies that exist outside of the service contract that influence design-time and run-time, so don’t think that the service contract is the container of all policies. I ran into this while I was consulting when I was thinking that the service contract could be used as a handoff document between the development team and the deployment team in Operations. What became evident was that policies that govern service deployment in the enterprise were independent of any particular consumer. So, while an ESB or XML appliance may enforce the service contract policies around security, they also take care of load balancing requests across the multiple service endpoints that may exist. Since those endpoints process requests for any consumer, the policies that tell a deployment team how to configure the load balancing infrastructure aren’t tied to any particular service contract. This had now become a situation where the service contract was trying to do too much. In addition to being the policies that govern the consumer-provider relationship, it was also trying to be the container for turnover instructions between development and deployment, and a single document couldn’t do both well.
Where I think we need to get to is where we’ve got some abstractions between these things. We need to separate policy management (the definition and storage of policies) from policy enforcement/utilization. Policy enforcement requires that I group policies for a specific purpose, and some of those policies may be applicable in multiple domains. Getting to this separation of management from enforcement, however, will likely require standardization in how we define policies, and they simply don’t exist. Policies wind up being tightly coupled to the enforcement points, making it difficult to consume them for other purposes. Of course, the organizational culture needed to support this mentality is far behind the technology capabilities, so these efforts will be slow in coming, but as the dependencies increase in our solutions over time, we’ll see more and more progress in this space. To sum it up, my short term guidance is to always think of the service contract in terms of a single consumer and a single provider, and as a collection of policies that govern the interaction. If you start with that approach, you’ll be well positioned as we move forward.
Just when you thought it was safe to build your SOA, the ESB has returned. A whitepaper from Paul Fremantle of WSO2 seems to have stirred the ESB pot once again, with a number of pundits chiming in on the future of the ESB. I was surprised at how little conversation there was about at the Gartner AADI Summit in comparison to the previous Gartner event I had attended two years prior, but that’s quickly changed.
I read Paul’s paper, and from my perspective, not much has changed from the views that I expressed two years ago back in a Burton Group Catalyst presentation and then in this followup blog post. The bulleted list of capabilities provided in that post aren’t ones that are going to go away, and it surprises me that there continues to be so much debate in this space.
Joe McKendrick recently summarized a number of recent comments in a blog entry, including ones from Roy Schulte of Gartner, Lief Davidson of IBM, ZDNet blogger Dana Gardner, and Lorraine Lawson of IT Business Edge (who has given a couple of my recent blogs some publicity, thanks for that). There were a couple things in Joe’s entry that just rubbed me the wrong way.
First is this notion of federated ESBs. Both Roy and Dana made comments around this, and I simply don’t agree. I’ll admit, there are a certain class of organizations that will have multiple ESBs. They’re the same organizations that have one of just about every technology because they’re so huge. Take them out of the equation, because I don’t believe they represent the masses. For the rest of us, what does “federated ESBs” mean? Does it mean that I have multiple ESBs performing redundant capabilities? Or does it mean that I partition up the capabilities across multiple products/devices, but yet with no two products providing the same capability, even though they may be capable of it? The latter is tolerable and likely, the former is not. For example, Roy mentioned security policies and quality of service. I may rely on an XML appliance for service security at the perimeter, and then rely on app server capabilities or some security agent within the data center. For simplicity sake, if we define quality of service as some form of intelligent routing and traffic shaping, I may rely on an ESB, a high-end XML appliance or network device, or advanced clustering capabilities of an app server. This type of partitioning makes sense. What doesn’t make sense is having one ESB (pick your favorite) providing security, QoS, etc. for services by development group A and having another ESB providing the same for services from development group B. To put it in context, routing is a core capability according to my definitions. Would you let development group A put a Cisco router in front of their services and development group B put a Juniper router in front of their services? You certainly wouldn’t do this if all of the services are hosted in the same data center. Yes, if your company has grown through acquisition and has lines of business all over the place, you may have different routers, but now we’re getting back to those super-large conglomerates I mentioned at the beginning. For those of you now envisioning the “big honking ESB,” don’t think that way. Think of it more like the network. I may have many deployments of a single ESB product, each handling a portion of the services in the enterprise, and their consumers know to direct to the appropriate ESB from design time rather than some uber-ESB. Applications point at specific databases or hostnames, there’s no reason that services can’t work the same way. Again, I don’t view this as federation.
What I believe we need to strive for in this space is centralized policy management and distributed enforcement, with standards based communication between the policy manager and the enforcement points (if only those standards existed). For example, one could make the argument that SAP could provide a second ESB that deals with services provided by the SAP infrastructure. I don’t view this as federation, however. From my perspective, I don’t even care whether SAP has an ESB or not. What I do care about is whether the entry points exposed by SAP can enforcement my QoS policies, my security policies, my versioning policies, etc. Let me centrally manage the policies, push them out, and have it just work. The challenge with this is not the enforcement architecture and federation there, but rather the metadata repository that will hold all of this policy information. This is where federation is important, because I may be getting third party products that come with their own pre-populated registry/repository of services that I need to manage.
The second statement that I disagreed with was Paul’s comment that ESBs discourage the shared ownership of services. If it does, then I think ESB ownership is the problem. Most web applications require the configuration of a load balancing farm before they can be accessed. No one considers the keeper of the farms in network operations the “owner” of those web applications, so why would the team responsible for the configuration of the ESB be considered the owner of services? Personally, I think a lot of this stems from ESBs being targeted at developers, rather than at operations teams. As I’ve commented in the past, I think the inclusion of service development capabilities like orchestration and the leftovers from the EAI space messed up the paradigm and caused confusion. Keep mediation separate from development.
So, as I climb down from my soap box, what are my parting words? I still believe that the capabilities in my post from last year are necessary, and an ESB is one way of providing those capabilities. Intermediaries are almost always used in web architectures, so there shouldn’t be such a strong aversion to having them as a front end to a service. That being said, too many intermediaries is a bad thing, because we haven’t gotten to the single pane of glass management that I think is necessary. Rather, each intermediary has its own management console, and the chances of something getting missed or fat-fingered goes up. Focus on an approach for centrally managing the policies, minimizing the number of places where policies are enforced, and keeping operational activities separated from service development activities.
I attended a session from Michael Blechar titled “The Yin & Yang of Processes and Data: Which Will Be King of the Next Generation Applications?” A big title, and as a result, Michael covered a lot of topics. While there wasn’t any new information for me, I would say that this was one of the better big picture overview presented at the conference. Some of the topics he hit on:
- The challenge of multiple metadata systems, and how some of the vendors are approaching it, in particular IBM. He specifically touched on CMDBs (Configuration Management Database), Service Registry/Repository, Software Development Assets, Database Metadata Management, and more. IBM’s approach is to provide a collection of metadata services that reside above all of this systems, providing a federation layer. Hmmm…. this sounds vaguely familiar, I think I called it Master Metadata Management and spoke more on it here.
- The challenges in determining when to use them versus pushing the logic up into a service layer. It discussed the importance of service ownership.
- The importance of information modeling and its importance in SOA.
- The importance of service ownership/stewardship.
- The importance of enterprise architecture operating as a team, and not having silos of business architecture, technology architecture, and information architecture that don’t talk to each other.
Overall, this was probably a good session for many people and hopefully helped them see a bit more of forest for the trees.
The Power Breakfast is over and done. If you saw me there, hopefully you got something out of the conversation, although it was unfortunate that the session wound up being four independent case studies without a lot of time for audience questions, rather than a discussion among the panelists and the audience. Despite this fact, there were some good points that came out, and I wanted to highlight the ones that struck home with me here:
- First, as the preliminary results of the SOA Consortium funding survey have indicated, there’s no uniform initial approach to SOA. Some leverage an SOA program, some expect service development out of existing projects, etc. One of the panelists, Leo Schuster from National Bank, indicated that they had an SOA Program. In my experience, I’ve never encountered a funded program whose sole goal was to produce SOA. I’ve always worked with business projects/programs that wanted to leverage services as part of their efforts. Personally, I prefer the latter, because as Victor Harrison from CSC pointed out, programs end. Will your SOA efforts ever stop? They shouldn’t. So, while you may have solved the initial funding problem, you haven’t solved the ongoing funding problem.
- Second, metrics are critical. Every single one of us mentioned metrics. Without them, how can we say that we’ve accomplished anything? The oft-mentioned notion of agility is frequently associated with the time to deliver a particular solution. To show that we’re more agile, we need metrics of how things have been and how things are in the future. Beyond efficiency metrics, there’s also metrics associated with service consumption: number of consumers, usage by consumers, min/avg/max response time, etc. In my own experience, merely making these metrics available, and now at a finer level of granularity than the entry points into a web application were very beneficial. This visibility didn’t exist before, so anything was better than nothing. If you’re already collecting this, now you can start to do baseline comparisons to show improvement.
- Third, service ownership is key. It was great to hear someone else say that the project-based culture of IT can be in impediment to SOA success. While your SOA shouldn’t have a lifecycle, individual services do have a lifecycle. If you simply have someone who is responsible for service lifecycle management, you’ve got the key piece of the puzzle for many of the discussions around service granularity, funding, etc. If you don’t have a service owner independent of the consumers, then each consumer is going to try to push things in the direction that has the most benefit for their project, but yet no one will own the service. This in-fighting can be very detrimental to SOA.
- Finally, lets get Service Portfolio Management (SPM) right from the get-go. There are many sessions here at the summit concerned about application portfolio management. We’re all behind the eight-ball here as all of these applications have been built/bought over 5, 10, 15, 25, … years without anyone managing them as a whole. The last thing we want to do is repeat the process all over again with services. We’re at the early stages of SOA and can now do it right. Anyone saying that they don’t need a registry/repository yet because they haven’t reached “critical mass” is potentially making a big mistake. They only need to look at their struggles in trying to do application portfolio management as an example of the path they are heading down.
To those who got up early for breakfast and attended, I hope you found some nuggets in the presentation that were valuable from me or from the other speakers. I’d be happy to have followup conversations with anyone who felt like their questions are still unanswered, or those who want more depth on some of the things they heard.
In the latest BriefingsDirect SOA Insights Edition, Dana Gardner, Jim Kobielus, Neil Macehiter, and Joe McKendrick discussed, among other things, Microsoft’s recent announcements. The conversation started very similar to some of my own comments on the subject with this sense of deja vu. Neil Macehiter made a great point, however, that shows that this isn’t simply a rehash of model-driven architecture. He stated:
…they are actually encompassing management into this modeling framework, and they’re planning to support some standards around things like the service modeling language (SML), which will allow the transition from development through to operations. So, this is actually about the model driven life cycle.
This reminded me of my trip to Redmond in 2005 for the Microsoft Technology Summit. At the summit, we were shown an internal tool, I think from the Patterns & Practices group, that presented a deployment model of a solution. I recall a number of us going, “we want that.” If Microsoft has taken steps to integrate these models into the development and run-time management tooling, this is an excellent step, and certainly something beyond the typical model-driven development of the BPM suites. At a minimum, these capabilities should be enough for people to at least track the ongoing progress of the Oslo effort.
The second thing that came up, which again was consistent with some recent blogs of mine (see Registries, Repositories, and Bears, oh my! and Is Metadata the center of the SOA technology universe?), was the discussion around the metadata repository at the heart of Microsoft’s strategy. Dana pointed out that “there really aren’t any standards for unifying modeling or repository for various models” with some comments from Neil that this is very ambitious. First, I’d have to say that Microsoft trumped IBM on this one. Remember when IBM announced WebSphere Registry Repository and stated that they’d be coming out with their own standards for communication with it? They were slammed by many analysts. Microsoft, rather than trying to operate in the narrow space of the SOA registry/repository, are talking about the importance of metadata in general. The breadth of models and associated metadata when talking about full IT product lifecycle (development and management), is far broader than what is typically discussed in the SOA space. AS a result, there are no standards that cover this completely, so the lack of standards-based integration is a non-issue, and Neil nails it by saying Microsoft is trying to get out in front of the metadata federation problem and drive others to comply with what they do.
Okay, no bears, sorry. I read post from my good friend Jeff Schneider regarding SAP’s Enterprise Service Repository (ESR). He states:
At the core of the SAP SOA story is the Enterprise Service Repository (ESR). It is actually a combination of both registry and repository. The registry is a UDDI 3.0 implementation and has been tested to integrate with other registries such as Systinet. But the bulk of the work is in their repository. Unlike other commercial repositories, the first thing to notice is that SAP’s is pre-populated (full, not empty). It contains gobs of information on global data types, schemas, wsdl’s and similar artifacts relating to the SAP modules.
This now brings registry/repository into the mix of infrastructure products that SAP customers must make decisions regarding adoption and placement. Do they leverage what SAP provides, or do they go with more neutral products from a pure infrastructure provider such as BEA, HP, SOA Software, or SoftwareAG/WebMethods? The interesting thing with this particular space is that it’s not as simple as picking one. Jeff points out that the SAP ESR comes pre-populated with “gobs of information” on assets from the SAP modules. Choose something else, and this metadata goes away.
I hope that this may bring some much needed attention to the metadata integration/federation space. It’s not just a need to integrate across these competing products, but also a need to integrate with other metadata systems such as configuration management databases and development lifecycle solutions (Maven, Rational, Subversion, etc.). I called this Master Metadata Management in a previous post.
Back when Gartner was pushing the concept of the ESB heavily, I remember an opening keynote from Roy Schulte (I think) at a Web Services Summit in late 2005. He was emphasizing that an organization would have many ESBs that would need to interoperate. At this point, I don’t think that need is as critical as the need for our metadata systems to interoperate. You have to expect that as vendors of more vertical/business solutions start to expose their capabilities as services, they are likely to come with their own registry/repository containing their metadata, especially since there’s no standard way to just include this with a distribution and easily import it into a standalone RR. It would be great to see some pressure from the end-user community to start making some of this happen.
SOA What? With all of this Web 2.0 development, it’s clear that internet scale folksonomies work far better than taxonomies. On the other hand enterprises are, for the most part, stuck with UDDI-related SOA governance tools and their strict taxonomy and categorization mechanisms. The open question though… is this really a problem?
Aside: I love the use of SOA What? That’s exactly why I try to always say S-O-A. On the subject, however, I think Dan raises an interesting question. One of the questions I’ve asked some of the registry/repository vendors is “Can you be indexed by a Google Appliance?” Admittedly, I’m not a huge fan of taxonomy-based searching. At the same time, however, a typical enterprise asset repository may not have enough critical mass to get appropriate metadata for folksonomy based searching. The Web is filled with hyperlinks. How many links to a service detail page am I going to have inside a typical enterprise?
Personally, I’d rather try to find a way to build up the metadata than go crazy building taxonomies to support direct navigation. First off, you can quickly get into taxonomy hell where there are so many variations that you try to support that it becomes difficult to present to the user. Second, people are so used to using Google, Desktop Search, Spotlight, etc. Universal search is going to be a standard part of the office toolset, and we need to find a way to ensure relevant results get returned. This will likely require analysis of software development artifacts (including source code) and building up those relationships based upon presence within project repositories and the role of the user performing the search. A developer performing a search will want to see very different results when searching on “Customer service” than a business manager.
The challenge we face is that the documents and their metadata are scattered all over the place. I previously asked if metadata should be the center of the SOA universe. Neil Ward-Dutton replied that it the center of the universe, and is inherently federated. We need intelligent crawlers that can infer the appropriate relationships and feed this into the universal search engine. Is anyone out there leveraging a Google appliance or other universal search option to facilitate searching for services and other IT assets? If so, like Dan, I’d love to hear about your experiences.