• We are implementing the Web Service binding for CMIS. While REST would be better for what we are doing, it was felt that the Web Services binding would be easier for the development team to churn out.
• As a result of that, the participating vendors are Alfresco, EMC, IBM, and Nuxeo. Microsoft wanted to participate was not sure that their Web Services binding would be complete in time.
• Each vendor will have a two issues worth of articles from AIIM’s bi-monthly publication, Infonomics.  In addition, each vendor is welcome to add their own white papers and collateral to the system.
• Users will search on metadata and/or full text. All searches will be round-robin sorted so that each repository has multiple hits on the first page, assuming that they have any content that meets the criteria.
• The system is being developed in .NET because we were able to identify a free hosting server that could support the effort.
• We, including myself, are going to be at the Expo on April 2nd to talk about it. I’ll share the exact time when I have it.

That is about it. I’ll be working and trying to get a basic search up this week. The second step will be performing this in a federated manner against multiple repositories.

I’ll share the journey as it unfolds. Until then, here is a modified version of the metadata model:

Object: AIIMContent

* Each Vendor will have the same value for all content in their repository.  It will clearly identify the vendor and may identify any other repository product information. For example, in my development Documentum repository, I’ve set this to “EMC Documentum”. Of course, EMC may actually us a different value if they desire.

Simple, Thought Not Impressive

This session was presented by David Gwyn and Mike Mohen, of EMC’s consulting services. That fact alone basically told me that this product evolved out of a series of consulting engagements. The presentation started with the standard spiel of why SharePoint is spinning out of control and needs ECM. While always overstated by ECM vendors, it is a valid point.

They did share my favorite Andrew Chapman quote about the requirements that they receive, Just make SharePoint and Documentum work together. They didn’t describe it as as a problem, but as a challenge. If that is your goal for the integration, then you need to actually define what you are trying to accomplish. Requirements are not a bad thing.

They then went into basic features and a recorded demo. From this we saw four basic capabilities in this offering:

• Web Parts Library
• Integrated Archive Support
• Shortcut Support
• Common Services

Some nice features is that the product, DCTM 4 MOSS, is 64-bit compatible and the Web Parts are theoretically 400% faster than the current Web Parts offered by Documentum. They use a Web Services layer utilizing DFS and SFX (which I need to explore in more detail) which would lead to faster performance. DCTM 4 MOSS appear to be deployed as a set of Web Services Extensions in the SharePoint Solution Framework.

Overall, it was a nice looking product, especially for a 1.0 release. The Web Parts seem fully functional, but not very pretty. It had a definite SharePoint look-and-feel (part of the problem), but you could tell that a user interface professional hadn’t take a go at it yet. Many of the partners have prettier versions, which they should as they have probably spent more time with them.

Patience is Still a Virtue

After seeing this presentation, I saw Andrew Chapman post an entry on his blog that started with the statement, I’m working on some innovative and groundbreaking stuff at work right now which is really interesting but I’m not in a position to share it just yet. I sent him a message if he was referring to this little tool. He chuckled and said that he was talking about something else and that I would like what he was working upon.

There are two product groups that I have interacted with over the years at EMC that I like and respect very much. To be fair, there are many that I have had only limited exposure to over time, so I don’t want to knock them, but these teams have been exceptional. One team is the collaboration product team, originally the eRoom team. Last winter they told me that I would like what was coming and they took some time to talk to me about it. That effort resulted in Magellan (now CenterStage).

I must say am pretty pleased with that effort so far. Now that Andrew has told me similar things about his team’s efforts, I am excited. I’ve known Andrew for years and when you dissect his wonderfully quirky sense of humor, you get solid insight and honest facts. Now if only I knew time frames.

I guess we shall have to remain patient.

In the meantime, the Documentum Connector for MOSS 2007 appears to be an improvement on what is currently offered, not to mention a mouthful to say. It has a better architecture, extensibility, and features than the current products. It didn’t appear to have any large-scale archiving functions, but what it offers to the user appears much better than what was out there before.

Oh yeah, and you can make it pretty and customize it as needed. The jury is out until it is released, but I’ll tell you right now, I’m clearing space in my lab to play with it.

In his post on the Elephant, John talks about the load of Junk DNA in Documentum. As the product has evolved over the last 15 years, things have been left behind and other things that worked, haven’t evolved. My recent post on the Audit Trail has led to a couple of posts addressing both of these aspects.

User Names versus Object ID

I mentioned in the Audit Trail that I and an index on either user_name or user_id. Rajendra wrote a post about the merits of each field in the system and how both are used. It is a nicely detailed post and comes to the same conclusion that I always had in my mind. The name is better for performance and the ID is better from a space/data consistency perspective. Denormalization of the database can be good for end-user performance as long as you test it thoroughly, which Documentum has on this feature.

Then came a comment from Andrew Goodale that explains why both are used. The ID was not guaranteed to be immutable because of how the LDAP synchronization worked. That has been resolved in D6, so going forward it will be useful. The user name is still everywhere though and it will be for years.

Just one more piece of junk DNA in Documentum.

Is the Audit Trail Secure and Complete?

As always when I talk about the internal workings of Documentum, James McGovern chimed into the conversation. He raised a few good points that I wanted to share and comment upon here.

First is security. I don’t always address security in the same way I don’t always address breathing when I tell people about my day. It is a part that I do and don’t always elaborate. James is correct on the need for maintaining security on the logs. This isn’t just limiting access to the logs, but is also in restricting the ability for users to change the log from those that can view the logs.

Documentum does a fairly good job at this. While the Audit Logs can only be stored in the same database as the rest of Documentum, it is locked down by default and super users do not have access by default, not can they grant themselves access. You can even divide your system administration in such a way that the Documentum administrators can’t access the logs. There does reach a point though where you have to trust your admins as someone will always have access on some level.

As it stands now, changing the audit trail is only available through direct SQL access using either a database system user or the schema owner account, which does not necessarily map to an actual Documentum user. Remember that database access does not equate to access to the content as that is stored elsewhere, unlike SharePoint.

Having the Audit Log stored in a second system as a feature, and not in a custom Data Shack, would be a nice extension that should be simple to implement.

Second on his list was the concern that not everything was in the log. He raised the excellent point regarding Web Services. Now, I know that in DFS a session has to be established by the calling system. In architectures that are direct integrations to systems, users usually map one-to-one. What about other systems?

Take the scenario where objects in the repository are being updated by users of a remote system that do not correspond to users in Documentum. How do you map these? We created custom audit events within Documentum and then, in our Data Shack, we transform them into one coherent trail of actions. We track the calling system and the user from that system. We also have users defined specifically to operate our Web Services so we can correlate all of the actions.

It is seems like a lot of work, but it is necessary and saved us a lot of work after we went live.

This, of course, can lead to a big licensing issue. Many clients have licenses based upon users. In theory, these remote users count against the total, even if they don’t exist within the system. There are many different licensing models out there, so when planning, be sure to check to make sure you aren’t in violation. This applies to any system, not just Documentum.

James and everyone needs to remember, DFS is still in version 1.0. 2.0 is being released as part of D6.5. I think the version that comes out with D7 is the one to keep our eyes on as it should incorporate feedback from the adopters.

• He had two presentation slots and the crowd voted for a more detailed two-part presentation rather than one high-level and one deep-dive, neither covering everything.
• Layered Technical Architecture (bottom-up)
  • Repository Infrastructure (database, index…)
  • Repository Services
  • Security
  • Compliance
  • Content Services
  • Process Services
  • Client Infrastructure Layer
• Tools, Programming Interfaces, and Configuration Interfaces across all but the Infrastructure Layer
• D6.5 adds XML Store on the Infrastructure Layer (formerly X-Hive)
  • Allows for rapid XQuery against the XML in the system
• SOA is a focus on where they are trying to move Documentum from client-server, New Paradigm
  • Easy to do from scratch, hard to transition
  • Started with 5.x with Business Office Framework
  • DFS in D6
• DFS has always been JAX-WS based in terms of its internal runtime foundation. The previous Web Services Foundation (WSF), which just project SBO’s as web services but didn’t provide out-of-the-box (OOTB) services like DFS does, was Axis-based. (From Craig Randall per comment below).
  
• They provide client-side libraries to simplify development in Java and .NET (D6.5 sp1)
  • This is to aid development.
  • Can still use WSDL right now.
• Services (first 6 are in D6):
  • Schema: Data Dictionary
  • Object: Create, Update, Delete, Copy Move, Validate…etc.
  • Version: checkin, checkout, cancelCheckout, etc.
  • Query: DQL (supports paging)
  • Search: support mulltiple repositories, can tie into ECIS
  • Workflow: startProcess, getProcessInfo, etc.
  • Access Control
  • Lifecycle
  • Virtual Documents
  • Query Store: Saved searches
  • Task Management
  • Analytics
  • Content Delivery
  • Profile (CTS)
  • Transformation
  • Policy: apply, remove, getAppliedPolicies, etc. (Retention Policy Services)
  • Electronic Signature: add, verify
  • ERP Integration: execute Action, executeAgent, etc.
  • Formal Record
• Three types of services in the Intelligent Information Infrastructure: Content, Archive, and XML
• Post D6.5
  • More services
  • Services governing strict process (ensure services ineroperability)
  • Support WS Standards
  • REST support: Define resource model and URI (complimentary to the current SOAP-based services)
  • Next generation of clients will use SOA
• May preclude DFC from the application layer in the future.
• As last year, Aspects are presented as the way to slim the system object, and thus everything we throw into Documentum
• Operation Customization D6.5 (already in there, just not documented)
  • Able to modify the behavior of action on the server-side without modifying the application
  • Import, Export, Checkout, Check-in, Cancel Checkout, Delete, Copy, Move, Validate, Transform
  • Defines steps for each operation
  • Allows operation cutomization based on Type, Format, or Repository
  • Includes modifying existing operations and creating new operations
  • Register Operation populators
  • Can add new steps and replace existing actions
  • Using BOF 2
• BOF 3 in post D6.5
  • BOF versioning, allows different versions of BOF objects to co-exist
  • Will add notion of bundling of business objects, tracking those dependencies
  • Tighter integration with Composer (I hope so because it is miserable now)
  • Simplified customization model
• Lightweight sysobject
  • Current sysobject 3K+ per instance (I always thought it was 2K)
  • Light ones allow multiple instances to share the same set of policies and metadata
  • One instance of Parent object and one smaller child object for each individual object
  • Changing the Parent impacts all children
  • Make use of Documentum as a Archive repository much more realistic
  • Will complement existing object model
  • Object look and behave like normal sysobject
  • Objects can be change to heavyweight sysobject from lightweight sysobject if needed so shared values on Parent can be changed without impacting the other children.
  • 30x footprint reduction seen in database, depends on number of parents to children
• Batching
  • Allows the grouping of multiple operations into one set of SQL statements to the database
  • IDfBatchManager interface allows you to open and then close a batch. normal DFC in the middle
  • Helps with mass import, reduces round-trips to database
• Scoping
  • Server performs checks during every operation (does this folder exist)
  • Allows for suspension of checks when performing a batch operaion
  • Default Scopes: Operation, Transaction, Batch
  • When combined with Batching, getting 3x+ performance improvement
• Partitioning
  • Allows items to be placed in logical partition to manage items and to store.
• JBoss as application server in D6.5, option or replacement? Need to ask Product Manager
• Indexing Architecture in D7
  • Lucene (Had been D6.5, very sad)
  • support for XQuery

Off to the Standards session.

Disclaimer

All information in this post was gathered from the presenters and presentation. It does not reflect my opinion unless clearly indicated (Italics in parenthesis). Any errors are most likely from my misunderstanding a statement or imperfectly recording the information. Updates to correct information are reflected in red, but will not be otherwise indicated.

All statements about the future of EMC products and strategy are subject to change due to a large variety of factors.

• Focusing on non-programming composite applications. They are going to focus on Email and “Restful and SOAP based Web Services” (A restful web service?)
• Talk about the Analyze, Deploy, Execute, and Monitor lifecycle of BPM. (This is good as a lot of developers don’t get exposed to the BPM lifecycle very much)
• Three core products ot build these apps in the back-end:
  • Process Builder
  • Process Engine
  • Process Integrator – old Business Process Services
• Composite Applications are apps that are put together from multiple services
• Glue-Code (old fashion integration) vs. Model Driven (Services etc.)
• (Very basic right now for me. fighting hard not to step out. Hoping for something useful or insightful soon).
• Can use process variables to store data about the process. They are process centric and don’t exist outside of the process. The “theory” is that the variables are from other systems or in documents and surfaced here for easy use and processing.
• Invoking Services – drag and drop
  • Configure the activity: includes rules from process data to configure the service call
  • Data mapping and transformation – can perform functions, XML transformation, support multi-value fields, and conditional nodes.
  • Supports JMS, SMTP, HTTP, SOAP, FTP/File, JDBC, Fax
• Demonstration:
  • Demo of creating a HTTP service to a Yahoo Service. (Not too difficult looking, but it will take practice. Only way to learn it is hands-on.)
  • D6.5 adds debugging and process parameters.
  • Showed creating a conditional test to determine what values are captured
  • Drag and drop mapping of values
• Receiving messages
  • Drag and drop
  • Service specific
  • Rules for copying the data over into the system
  • Asynchronous Interaction
  • Performs Correlation
    • Transport-based uses unique values to keep requests and responses together
    • Payload-based uses process specific information (PO Number)

(This all looks great. Nothing really new for me. My advice to people is to play with it. It will take more than days to deploy a new composite application, but it will still be faster than coding a new application.)

Time for lunch. Going to hit the exhibits next. You should be able to catch me near the Developer Lounge on the floor. My next session is probably going to be the D6.5 Architecture overview.

Disclaimer

All information in this post was gathered from the presenters and presentation. It does not reflect my opinion unless clearly indicated (Italics in parenthesis). Any errors are most likely from my misunderstanding a statement or imperfectly recording the information. Updates to correct information are reflected in red, but will not be otherwise indicated.

All statements about the future of EMC products and strategy are subject to change due to a large variety of factors.

This was prompted by a reading OpenText’s Enterprise 2.0 Content Management strategy. Note the placement of the 2.0. We’ll be getting back to that.

ECM 2.0, Coming Soon to an Enterprise Near You!

Well, if you look at my definition of the next generation of ECM, ECM 2.0 for fun, you’ll quickly see that we aren’t there yet. Even Bex chimes in on the Identity Management support in Oracle ECM and says that it isn’t there, yet. It is a great post and all concerns and issues there are true for Documentum as well.

Let’s take it as fact that all ECM systems are currently lacking in open support for Identity Management systems. Most can leverage external authentication, but fall short for external authorization.

Documentum can handle external groups, but not security policies. Those have to be created inside of Documentum using the external, or internally managed, groups. Documentum, like many systems, cache the external information for various performance reasons. That is fine, except that if something changes, you are out-of-date until the next synch job runs.

As if my critical eye wasn’t enough, EMC themselves aren’t at ECM 2.0 yet by their own reckoning. Way back at EMC World 2007, they defined their vision of ECM 2.0. They stated that D6 was just laying the foundation for ECM 2.0 and that it wouldn’t be realized until at least D6.5. That is this summer, so we can’t even evaluate it until then.

D6.5 will either be the Beta 2 release of ECM 2.0 or the “gold” release. It depends on who you ask and what is delivered.

ECM 2.0 and Enterprise 2.0, Not Even Related

Aside from the flashy 2.0 attached at the end, there is little relation as far as I am concerned. ECM as it currently stands can support Web 2.0 artifacts. Some user interfaces may need some updating, but the back-ends of almost all Content Management systems, Enterprise or not, can store any type of content. Being able to store a wiki isn’t that impressive. Sounds more like a new feature for either a Collaborative CEVA or a Web Content Management CEVA.

ECM systems can support Enterprise 2.0. It is the interfaces that need to be updated. Clearspace, from Jive, supports Enterprise 2.0 activities. Infovark, once released, will support Enterprise 2.0 activities. Neither are ECM systems or based upon them. I bet that I could take Documentum 5.2.5 and use it as a back-end for content for Clearspace. It may take a little development, but it could be done and nobody will EVER confuse that old version of Documentum with ECM 2.0.

OpenText is updating their user interface. That’s it! Big deal. If they have to make a lot of changes to their back-end to make this work, then they are in worse shape than I ever imagined.

Everyone, Look Outside the Traditional 2.0 Box! (Traditional 2.0? Funny.) ECM 2.0 is the next generation of Enterprise Content Management. It is not a slave to any other technology. It is a servant for the emerging Enterprise Architectures in the SOA world. It isn’t dependent on Web Services or REST. It is dependent on an undefined SOA Standard.

Look, databases have ODBC and JDBC. ECM needs an equivalent. CEVA providers shouldn’t have to write different interfaces to interact to different systems. They need one interface and ECM system will then only need one interface. Then we can get back to features and not pick a vendor because they are at least average in everything that we need.

Jed Cawthorne gets it. Billy Cripe gets it. Lee Dallas is getting it. Do YOU get it? If not, tell me why and I’ll see what I can do to help you along. Please chime in and share your thoughts. I’d love to hear from someone at EMC like Cornelia, Craig, Mark, Chuck, or someone new.

I have come back to this topic through multiple avenues. One is the concept of Invisible ECM from Billy and crew over at Oracle. It resonated very strongly with my previous discussions on Transparent ECM. We can debate terminology later, but what is important now is the shared concept.

A second avenue comes from my need to explain where ECM is going, ECM 2.0, in a simple and concise way. I can explain it and speak passionately on the topic. The need to get the concept out there in one breath has become more important as I talk to more people.

I have developed a proposed definition for your consideration. I would love feedback. I will approve all constructive comments for sharing, though I may not respond until a subsequent post. I’ll throw it out there and then discuss it briefly. Remember, I want this definition to have a soul.

Enterprise Content Management is the empowerment of all content within an organization. This is accomplished through the centralized management of content, allowing for people and systems to access and manage content from within any business context using platform agnostic standards.

Enterprise Means Everything Managed Together

The Enterprise in ECM started as being able to support various Content Applications throughout the organization with the same tool. This led to WCM, RM, DAM, and Collaboration applications all being developed or acquired by each major vendor.

The growing problem now is using content in applications not provided by the ECM vendors. These are applications that I have previously referred to as Content Enabled Applications, but Gartner and others refer to as Content-Enabled Vertical Applications, or CEVAs. (Gartner’s term pre-dates mine so they win the terminology battle.)

For example, documents in a case management system (a solid example of a CEVA) should be retained in a central repository. From here, records management policies and archival rules can be systematically applied.

Traveling the Road to ECM 2.0

When ECM was first coined, you would have been hard pressed to find a vendor that could actually deliver on the definition. The biggest hurdle these days is finding a proper project manager and plan that leads to a successful ECM implementation. People often blame the vendor, but usually it is the integrator, or lack thereof, that dooms an ECM project.

ECM 2.0 tries to make that easier. It’s very nature is to support other applications. Using supported standards, the ECM system is plugged into the back of a CEVA. If needed, a WCM or collaboration solution (RedDot and SharePoint respectively) can be plugged in as well. Organizations shouldn’t have to use the Content Applications from their ECM platform vendor. There are advantages to doing so, but it shouldn’t be required of the organizations.

Delivering an ECM 2.0 platform is a tall order. It isn’t just about scaling for volume or handling all types of content. Those things are important, but not what separates old ECM from new ECM. What is different is the way the ECM platform acts as a platform supporting the Enterprise. To get there, at least two things need to happen.

1. A standard for the ECM world needs to be set and implemented: Like ODBC and JDBC with databases, ECM systems need a standard interface. In today’s world, a SOA standard makes the most sense as it would loosely couple the systems together. If I had to pick a technology, I would go with something built on Web Services, but mostly because so many ECM vendors allow communications in that manner.
2. ECM Platforms need to leverage external user stores for authentication and security: A classic example of this is as follows…I have a Word document in SharePoint. It has a set of access rights assigned to it which is mirrored in the ECM system. When the security is changed in one place, it needs to automatically change in the other. All the users and groups that exist in one system should exist in all systems. Ideally, there is only one instance of the access control lists and users that both systems leverage.

This isn’t going to happen over night. I know several organizations, in different verticals, that are facing the problems that ECM 2.0 addresses now. Their solutions are functional, but far from ideal. They are locking themselves into a set of technology and putting themselves into a position where they cannot upgrade one piece without upgrading the whole.

The requirements are forming in the heads of the users. The first vendors that get there with a robust solution will flourish. I suspect that at least one of IBM, Oracle, or EMC will fail to deliver in time and will drop from the top three. The question is, who will rise up to take the vacated spot?

Nice Filling

What we have is a 10 page document with three pages spent on the cover, contents, and contact information. Two pages cover SOA and two more cover how Web Services fit as part of the SOA world. Since most of the actual content talks about SOA independent of ECM, I thought I would talk about it.

SOA represents the broad set of best practices that enable companies to hide the complexity of the underlying technology while at the same time providing an agile set of resources to the business.

That is the initial definition in the Introduction. Luckily for the reader, they go into much more detail, and offer different viewpoints on the definition of SOA. When taken as a whole, I think they get it correct. They set out four basic characteristics:

1. SOA is a component-based architecture
2. Components are loosely coupled through the leveraging of standards
3. Components are used to build composite applications
4. SOA is a joint initiative between IT and business

Not bad. An executive reading this may actually learn the important aspects of SOA.

They then go into the relationship between SOA and Web Services. They actually get it right. They go out of the way to say that they are not one and the same. They use this section they lay the groundwork to say that by using Web Services, the Documentum suite is standards-based.

A Hidden Definition of ECM

In the middle of a little sales talk of why SOA is good, EMC slips in a discussion as to what ECM is and what ECM suites deliver. This was a little surprising and is something that I am going to have to look a little deeper into later this month. As to what it is:

ECM refers to the category of software that helps you manage all of the unstructured information – or content – in your enterprise. ECM brings many different capabilities to the organization across the content lifecycle, including content creation, management, archiving, collaboration, publishing, and more.

Nothing new or exciting so far. Then the following shows up:

Traditional ECM suites tightly integrate these capabilities, providing value to the organization at the expense of agility. ECM Documentum, however, exposes enterprise content services to simplify development of new composite Documentum applications and content-enabled existing applications.

Good stuff. Now, I’m not going to try and match this vision to their execution today. What is important is that this vision for ECM is pretty darn good. Execution, and license cost efficiency, is what is going to matter in making this a reality.

The Actual Topic

So now we get to the three pages that I actually cared about when I saw the document. What a bunch of marketing fluff! They basically say how much work they put into the product to make it the ideal solution for the SOA world. Forget about right and wrong for a minute. They don’t really dive into the how. They let the SOA discussion carry forward into the ECM section and basically says, Apply all of that to ECM!

They do have a pretty diagram that shows how their services, as defined, fit into the Enterprise world. However, it just isn’t enough. I guess after the “detail” in the SOA section, there would be more meat talking how ECM fits into SOA and not an appetizer of EMC and SOA. I guess they used up their 15 minutes talking about SOA and just didn’t have enough time to properly prepare the main course.

Lots of Work to Do

After being initially pleased with the SOA discussion, and pleasantly surprised, my mood hit rock bottom. Let’s face it, when you go to a good restaurant and have great atmosphere, a great beverage, and a mouth watering appetizer, it is a little disappointing when your steak comes out like a piece of coal.

It also shows that I have a lot of work to do. I have a lot of feedback that I need to start providing to EMC. I need to encourage others to do the same. I can’t expect EMC to change for the better on their own, and I can’t depend on other’s to do it for me. I’m not saying that I can actually get them to do anything or that the change won’t occur without me, but if I want to see changes, I need to get on the ball.

Before I dive into the nitty-gritty details, I wanted to share an interesting observation. I started this blog to talk about things that were of interest to the EMC/Documentum crowd. That has expanded to encompass broader ECM issues such as standards. I think these topics are of interest to Documentum Architects, so it isn’t a reach. However, I’ve noticed a trend. Whenever I post on SharePoint, my hits jump way up. If I was just after hits, I’d just switch to SharePoint all the time. However, I expect this to be one of my last posts on the topic for a while as I have bigger fish to fry and I think I’ll have covered most of what I feel I need to cover for the short term.

It does make one think. I wonder how far my hit rate would jump if I included the name of a celebrity who is named after a French city? I’m not shameless enough to find out. Now on to the meat of the post…

Q: Install SharePoint or ECM? A: Yes

That sums up the White Paper. Doculabs spends quite a bit of time saying how Enterprises will get the most value if they implement both MOSS 2007 and ECM systems such as those from EMC and IBM (Note: They only refer to these two vendors during the paper). We’ll get to why over the course of this post, but I wanted you to be ready for the overall theme.

The paper starts by talking about the ubiquity of SharePoint and how organizations need to address that situation now. This is similar to observations by myself and others. In fact, they even make the Lotus Notes comparison.

Left on their own, existing departmental users will continue to implement radically different index hierarchies, design templates, and portal interfaces. Organizations should consider the potential ubiquity of MOSS 2007 implementations and put the appropriate policies and procedures in place to prevent the situation that many firms experienced with Lotus Notes.

The paper is broken into answer 10 questions. These 10 topics address both the how and why of SharePoint in your overall IT infrastructure. Let’s focus on some that why.

SharePoint and ECM Working Together

This isn’t a new thing. Both Documentum and FileNet have integrations into SharePoint. What Doculabs focuses on that a majority of users and solutions don’t require full ECM to provide Document Management functionality. They have several charts describing the functionality of SharePoint and when the level of functionality rises to the need of a full-fledge ECM system.

MOSS 2007 offers a core content management design for a broad user-population, but the system is not a substitute for functions such as support of an externally-facing transaction web portal; the authoring and publishing of complex documents; high-volume, task-intensive workflows; records management and e-mail archival.

SharePoint provides Collaborative Content Management, not Archival, Rich Media, or Transactional. The functionality that it does provide is less robust and rich in features as well. A perfect example is the Document Management functionality. While noting that Microsoft refers to this as ECM, they state that most authoring and reviewing tools that users need are supported in SharePoint. However, when that is expanded to internal and external parties, rights management, and advanced versioning, it is necessary to add a full fledged ECM system behind the scenes.

The Total Value Proposition

One of the points that they make is that SharePoint may only cost X, but the functionality delivered is less. ECM systems while costing more and delivering more, have greater complexity and lower adoption rates. They speak to the fact that by implementing both, Enterprises can take advantage of the viral nature of SharePoint to deliver ECM functionality behind the scenes. That is one thing that most ECM professionals can understand, the greater the adoption, the greater the value. The more content in the system, the deeper the impact in the organization.

However, the key is best put by them:

The critical success factor for the successful deployment of MOSS 2007 and any advanced ECM solution is this concept of proper controls, together with an enterprise strategy for how content services will be provided to the various user constituencies throughout the organization…It also requires that you develop (and implement) policies and procedures that will not impede your users.

This is a standard approach for implementing ECM systems. It isn’t just technology, though that is important. What is key is that you define policies that clearly define how to make the most out of the features of both SharePoint and ECM systems. They suggest putting all draft and collaborative content in SharePoint, but once it is final form, it should be moved to the underlying ECM system. This doesn’t mean that the document can’t be accessed from SharePoint. It only means that the storage, and the rules around managing that content, should be handled by an ECM system.

So how would I do this if I had to work with SharePoint? Simple. Plug an ECM system into the back-end of SharePoint. Let the users use SharePoint without knowing the power behind it. Hopefully this can be done using some sort of ECM SOA standard. If not, at least Web Services which easily tie into SharePoint. Then, by extending the already defined definitions, the richer ECM functionality can be exposed into SharePoint.

When will we be there? Not as soon as I would like. A better answer sooner rather than later.

And now, folks, it’s time for “Who do you trust!” Hubba, hubba, hubba! Money, money, money! Who do you trust? Me? I’m giving away free money.

- The Joker, Batman

Two Examples

I take public transportation to work. Not out of any sense of environmental friendliness, though that is a bonus, but because it is faster and more economical. The price of both the Metro and bus, without transfers, is less than the cost of parking alone. My travel time is also consistent, regardless of weather. I can even write the occasional post as I travel.

The other day, I was sitting on the bus, waiting for the driver to return from what I can only assume was a bathroom break. We weren’t running late, so I had no worries. Onto the bus came a passenger who, upon looking around, decided not to pay the $.35 fare and just sat down. After that, I watched and everyone else appeared to pay. If I had to pick five people from the riders that evening to be candidates for cheating the system, that passenger would not have been one of them. You can’t judge a book by its cover.

The other example was a joke, but it highlighted a more pressing issue of trust. I was listening to a project team discuss the upcoming expiration of a trial version of a software tool. The process to purchase the licenses was already initiated, but the team didn’t want to loose development time waiting for the process to complete. It was mentioned that the trial version wouldn’t stop working after the expiration date had passed. It was then jokingly asked, Why pay at all? After a few chuckles, the ethics of continuing to use the product after the trial expired was discussed. It was decided that since the only delay in purchasing the product was going to be from bureaucracy and red tape, the product could continue to be used in Development.

Trust at Work

The vendor above trusts their clients, and potential clients, to pay for their software and to only use it as licensed. If you buy two user licenses, they trust you not to install on four desktops. The vendor can’t track it the usage without the addition of support overhead to handle the license keys. Having worked with products that enforce licenses, as both a vendor and a user, I can speak to the productivity lost on both sides. However, the trust involved could easily be taken advantage of by users. In the above situation, a smaller, more cash-strapped, organization may have very well leaned the other way.

How do you determine who is trustworthy? Who embodies truthiness? When you join a company, you trust those you know and work with every day. They may introduce you to people and vouch for them. You can repeat this for several iterations. After a while, you are basing trust on the word of a friend’s friend’s friend’s. How strong is that trust? You can avoid this situation in your personal life, but not in large companies or projects.

Trust in the Enterprise

Now you are placing trust in this Web Services call that just came to your application. You check the digital certificate, see that it came from a valid source, hasn’t been modified from its original form, and the contents match the contract that you established. Everything has been done to enable you to validate and confirm the message. However, that can be compromised. It may take advanced technology or trickery, but there is always the risk that the call that your application just received is phony.

Even worse, what if the call is valid, but made by a rogue person on the other end? If it is from another organization, you have put your trust in that organization to only allow trustworthy, responsible people to work for them. All organizations try and hire trustworthy people. Some organizations go beyond that by locking systems down, but someone has to have a key. There is always a key.

In the meantime, we protect ourselves with better contracts, authentication, and encryption. Oh, and don’t forget trust. Without trust, we would be forced to hide in our homes. Not a pleasant thought, even if you have a Wii.

• Our old buddy James McGovern started the whole thing off. He has apparently been sharing is frustration with his significant other and he wrote a post on the sad state of WSDLs in the ECM space. They are ugly and poorly written in his experience. Not having delved into any out of the box WSDLs in ECM, I can hardly argue. It wouldn’t shock me though. Hopefully the DFS ones will measure up better. James then starts to talk about the ECM systems having a standard Document Query Language and a common WSDL built upon that structure. Sounds good to me. In fact, it is a nice, positive contribution to the whole ECM standards issue.

• Bex Huff then gets wound up. Part of the reason is that James referred to him by name when listing known ECM architects that may have contributed to current WSDLs out there. Bex reiterates his positions (No new standards, REST is bad, Microsoft killed WS-*) from previous posts with fresh examples. I don’t agree with Bex on some of it, but I will respond to that in more detail later.
• James then linked to Bex’s post, and added a nice clarification. I’m posting the entire comment as it sums-up James’ thought process to date:

Brian Huff got it somewhat twisted due to my calling out patterns across vendors and not enumerating them on a per-vendor basis. Brian, don’t worry as we are on the same page and agree on more things that appears on the surface. I was simply commenting on the horrific WSDL at large that exists. Some of the WSDL I have seen in the world of ECM feels more like a weakly defined upgrade path where they took their suboptimal query languages and proprietary APIs and simply wrapped them without any thought about doing it the right way of WSDL first. We both agree that arg0 is fugly but it is even worse as I held back and didn’t comment on the simple opportunity to enumerate all possible values that could be passed as schema does support the notion of xsd:choice. Even worse is when you run across namespaces that read exactly like the underlying java classes. Let me say that standard WSDL in the ECM domain isn’t possible until vendors at least figure out a standard query language to handle all the semantics you mentioned.

• Bex then reacts by saying that he may have read too much into James’ post. Then something amazing happened. Bex agreed with James. Bex still loves his SOAP messages and I am not in a position to dissuade him. He does provide a link to a Stellent SOAP message. I’ll make you go to his post to get a look at it.

ECM Platforms and Content Rich Applications

James then writes a great post with the input of his significant other. Go read it and come back….

My first impression is that I like his significant. She seems very rational and not bitter from repeated issues that this environment can throw at you, and has thrown at James. They, giving both credit, then talk about how ECM vendors view ECM as the center-of-the-world. They are correct, ECM is not the center, as such. ECM is a platform, not the end all, be all of life. As I stated recently, the Enterprise in Enterprise Content Management needs to refer to supporting the Enterprise, not doing everything for the Enterprise. To quote them, ECM is a participant … and should defer/support transactions in a larger context.

ECM vendors need to provide a strong ECM platform. From this, there should be the ability to plug different Content Rich Applications (CRA) into the platform, like SharePoint and WCM applications. The vendors can provide their own offerings, and even provide a tight integration as a competive edge. However, in the grand scheme of things, CRAs need to become plugable into multiple ECM platforms. This also means having the ECM vendors making their own CRAs, like eRoom, plugable as well. Imagine if I had a FileNet ECM platform and was able to plug Interwoven’s WCM CRA into it. Ok, maybe a bad example, but you get my point.

In this same way, ECM needs to be the support for serving content to other applications such as BPM engines. This is the example that they give. The process should be just like plugging in a lamp. This is the lower hanging fruit. Security, Metadata, and Content. There are details buried there, but from a high level, that is it.

I am planning another post on ECM and SOA and why Standards are needed. I have decided to convince Bex of the need for Standards. I figure if I can convince him, I can convince anyone.

[Edit 8-23: James wrote another post on this topic directly responding to Bex that already slipped off his main page.]

Here is my most recent problem. A client has a legacy Records Management solution. It works well, though is a little dated. The client is now going to implement a Web Content Management solution. The approved web pages need to be automatically declared as a record. If these were the same ECM platform, there wouldn’t be any problem. However, they aren’t. In fact, one is Microsoft based and the other is Java based. Neither of them is Documentum, though that wouldn’t change the problem significantly.

So how do I get the content and required metadata from the publishing system to the RM system? Well, it isn’t going to be straightforward. For the RM system, we are going to have to write a Web Services interface using .NET to allow us to receive the content and declare it as a record through the products API. Once done, we just send a message back to the publishing system giving either a confirmation or an error message. For the WCM solution, we are using an additional module to define and create the Web Services we need. If it works as documented, it won’t be nearly so painful.

The best part is that the client in question is thinking of changing RM systems in the near future. Know what that means? You got it. The interface into the existing RM will have to be re-written for the new one before long.

Now, if there was Standard defined, and widely implemented, that would allow me to send the content, metadata, and maybe even a flag indicating that it should be a record, that would be simple. The initiating system would send it and await an acknowledgment. It would eventually receive either a complete, a partial complete (say it couldn’t declare it as a record), or a failure. Life would be simple.

Then again, I wouldn’t get to make as much money consulting either.