In a recent Aberdeen research, they found that 95% of respondents (of 122 responses) replied on some level of manual processing in order to integration external data sources. Manual processing to integrate external data is time consuming, expensive and error prone so why do so many do it? Well, they often have little choice. If you look deeper, most of the time these data exchanges are with small partners and small partner enablement is a significant challenge for most organizations. For the most part, companies focus on automating exchanges with the top 20% of their vendors, supplier and customers which accounts for about 70-80% of most exchanges. We’ve heard from many customers that changes in business practices, an increase business demand for automation with external partners, and an increase in the variety data formats are starting to push the boundaries of how IT departments and business units manage exchanges with the remaining 80% or partners, vendors and suppliers. The first reaction can be to throw more people at the problem but increasingly customers are looking for a better way. After studying what a number of them have done and their resulting success, below some practices they have in common:
1. They exchange what their partners can exchange
- Excel, PDF, Word documents, email, text files, etc. The list is long but the point is they don’t try to force small partners to use data exchange methods reserved for larger partners. Mind you, this usually isn’t for lack of trying. All of them came to realize it costs more to rekey data or to partially automate than to find a way to automate and support these non-traditional B2B exchanges.
- Most use secondary B2B gateways to automate these non-traditional B2B exchanges. They do this for a couple of reasons: 1) typically the primary B2B gateways are managing order-to-cash processes and are therefore not optimized to handle unstructured data formats and; 2) often there is a desire to segregate data flows so access, on-boarding and data formats can be aligned to business purpose vs. technology.
2. They protect themselves from bad data
- There is no arguing that unstructured data formats, such as Excel, are more volatile and susceptible to unexpected changes that structured, defines formats. Most customers validate schema’s to make sure basic data quality checks are in place. For example
- Validate field formats (text, number)
- Validate column labels
- Validate mandatory fields are present
- Many also implement or extend Data Quality to further validate content before it loads into back office systems
3. They empower business users to manage on-boarding of small partners
- As these customers moved beyond their top 20%, the number of partners increases exponentially. The complexity of these partners is usually low so the critical factor for success here is reuse. All customers created one or two different workflows to support common inbound/outbound data flows, such as Excel or PDF’s. In defining these flows, they create configurable parameters that determine how a specific partner’s file is processed. With this in place, they can then allow non-developers, typically business analysts, to on-board new partners by simply selecting parameters.
- Transitioning who on-boards smaller partners is a key part of achieving ROI for these projects. The ability to respond quickly to new opportunities, on-board partners faster and gain benefits from automation are largely seen once this shift happens.
4. They use templates, templates and more templates
- Many customers create standard Excel templates that they use to exchange data with smaller partners where Excel is the best common denominator of technology.
- A few customers have created supersets and use templates to create partner views of data or reports. This gives business analysts the capability of customizing the templates to meet certain partner needs or special requests. Customization is definitely the exception but some found the flexibility was needed to avoid going back to development resources to create one-off reports. This way, it’s a configuration change instead of a code change, lowering maintenance costs considerably.
- Using templates allows for better data quality checking and faster on-boarding. Templates make it easier for small partners to comply and business analysts to explain what data is needed or what data will be sent.
If anyone else has additional tips or best practices, I’d love to hear them.