Today’s guest post is from Bernard Gunther of Lexington Analytics who recently brought you Buying Spend Analysis Systems: Taking a Test Drive. He can be reached at bgunther <at> lexingtonanalytics <dot> com.

A client who read my recent SI post, “Taking a Test Drive“, thought that relating the experience of their own test drive might help other readers who are investigating spend analysis approaches.

In this case, the company already had a spend analysis system, but the contract was about to expire. The test drive was intended either to provide ammunition for switching to another system that had been identified as a alternative, or justification for renewing the contract for the existing system. The company wanted to evaluate whether there were advantages with the alternative system, and whether or not the alternative system could improve performance for users whose buy-in was essential. A financial case either for making a change or for maintaining the status quo was also a deliverable.

As a result of the test drive, the company ended up changing systems, and believes that user needs are better met because of that decision. Data are cleaner, because vendor groupings and commodity mapping are more accurate, and analysis capability has improved greatly. The company reports spending less time supporting the new system. The company also added external consulting resources to work with their users each month to help extract additional value. Best of all, the monthly expenditure for the system — including the cost of the incremental external resources — dropped by more than 25%.

The Test Drive Process

To perform their test drive, the company focused on how each system would:

1. Meet the existing user needs: “must haves”
2. Deliver on known needs that users don’t have today: “wants”
3. Deliver additional value that may not be understood today: “didn’t know I wanted, but after seeing, can’t live without”

The business case needed to describe how each system would deliver on these three items at either a lower cost or, if the costs were higher, how the selected system would deliver an incremental return on investment.

The test drive for the new system occupied a few days over a three week period. Since the evaluation team understood their current system thoroughly, they focused on learning where additional value might be delivered, as follows:

1. Understand the current users of the system.
   The team interviewed users to see what they valued and what they were currently doing with the existing system, e.g. did they have features or data that they would like to see in the system, did they understand the value they were currently getting from the system, and did they know what they wanted the system to deliver in the future.
2. Understand the “non-users” of the system.
   Individuals were identified who were not current users of the system, but who the team felt could or should be users of the system. The team worked to understand what these potential users would need to see, and the value that they would receive.
3. Provided a sample of current data and reports to the supplier of the new system.
   Since the core data required for the demonstration was already available in the existing system, the supplier was able to produce a working spend cube for review with minimal effort.
4. Review with the suppliers how they would meet all the “must haves”, “wants”, and “future wants”.
   Evaluate the suppliers’ offerings to determine how each element generates savings and/or adds value. Users were involved with this part of the evaluation, as they were considered to be the best judges of how a new feature compared to an existing capability.
5. Put together the business case.
   The test drive showed that the new offering would both reduce costs and increase value, so it was not difficult to achieve internal agreement on a decision forward.

Survey Results

• User Must Haves
  • Users said that they obtained the most value from basic visibility to the spend data. However, other than the advantage of having all the AP spend data in one place, most users felt the existing system was just a “warehouse of data” that didn’t really help them do their job much better than data extracts directly from AP. They were unhappy with the vendor grouping and commodity mapping.
  • Users had the basic ability to filter data via point and click interfaces, but were unhappy with the speed and limited complexity supported.
• User Wants
  • Users expressed a desire for the ability to create and modify reports inside the system, without external support. The existing system had limited reporting, so in order to create all but basic data extracts, users had to dump raw data to their desktops and build custom reports and models outside the system.
  • Users were unhappy with vendor grouping and commodity mapping in the existing system. Getting changes made to groups and maps was awkward, required committee decisions, and took a long time. Users wanted to make changes to the commodity structure, commodity mapping, or vendor grouping, and immediately see the results.
• New Features
  • Users wanted the ability to make private and arbitrary changes to a spend dataset, to see if a change in data organization could improve their understanding of the data.
  • Users wanted the ability to build new data sets from scratch, on their own, as well as the ability to analyze many different kinds of data, such as commodity-specific invoice-level data.
  • Users wanted the ability to build complex reports inside the system.

In summary, the client believes that the test drive process was very useful. The value delivered by the spend analysis system has been increased, user satisfaction with the data and the system has gone up, and the cost of the system has gone down. The client also believes that if a decision had been taken to stay with the incumbent vendor, the test drive would have provided significant leverage for renegotiation.

Today’s guest post is from Bernard Gunther of Lexington Analytics.
He can be reached at bgunther <at> lexingtonanalytics <dot> com.

During the course of my work, I am always surprised at the large number of procurement organizations who still perform their analyses by dumping AP data into Excel or Access and developing their own reports. An analyst spends days or weeks each month maintaining this information. Even more troubling are those who have purchased an analysis tool, but still dump the transaction data into spreadsheets and spend hours or days creating their reports. The expensive analytics system they purchased is just serving as a repository for data. I suspect they didn’t take their analytic system for a serious test drive before buying. A test drive is a simple way to know exactly how the system is going to do what you need it to do.

To be useful to a procurement organization, a spend analysis system must be able to:

1. Load the data.
2. Transform the data. Make changes to hierarchies and create / modify rules to map the data.
3. Index in other information to further enhance the data; for example, identifying preferred vendors, vendors with contracts.
4. Create useful output — not just dump data – such as full reports, ready to send to users.
5. Update the data with the next month / quarter / year of information.

And, it must be able to do all these things without requiring significant support, either from the vendor or from IT, after the initial training. Most Procurement organizations want to be as self supporting as possible.

To get the most out of your test drive, have the vendor show you, step by step, how their tools work with your data. Before the meeting, create a text file with a reasonable sized segment of data, perhaps a year, in a single table with:

• Vendor name
• Cost Center
• GL Code
• Date
• Amount
• Description
• Other fields that are interesting to you, such as commodity or preferred vendor.

Select a reasonable size block of data so you understand the performance of the system under the load you expect to have. The goal is to have an interesting segment of data to work with. You want to understand how each step of the process works and delivers, not to build the full system.

After you are convinced that the vendor is capable of performing the essential tasks, you can focus on how the process will work on all your data and fully fleshing out your reports. Arrange a meeting with your vendor (in person or remotely) to have them show you how they do each step of their process with your data.

1. Load the data into their tool. Is it easy to load new data into the system? World class procurement functions build dozens of different cubes on different segments of their spend data, so this needs to be easy.
2. Look at the raw data. Can you see your top vendors, cost centers, GL codes?
3. Have them group some vendors and then look at your data. Can you group vendors as you want them?
4. Have them create a few nodes in a commodity structure and map some spending. Create some vendor rules, some GL rules and then some more complex rules. Modify a rule and see the results
5. Have them create a report. Print it out. They modify the report and see it again. Start with a simple report and then do something more complicated (for example, with pivot tables).
6. Perform a refresh. Add some more transactions to the data. See how all the existing vendor grouping and mapping are applied.

You should be able to do all these steps on your sample data in 2 to 4 hours. Once you’ve done this, you will understand how the system will work for you. If it takes hours to make simple changes in the demo, it will take hours in the final system. If six different people are involved in the demo, you’ll likely need this size team in production. If you can’t see your team doing the work shown in the demo, you’re going to need to rely on the vendor or some third party in the future. In the longer term, you may decide you want to leverage external labor to do the work, but to be able to be independent; you need to understand and be able to perform each part of the work yourself. If you can’t be independent, do you want to be dependent on this third party (vendor or IT group) for every single change you to make?

Spending a day or two test driving different spend analysis options is the easiest way to really know if the system the vendor is proposing will work for you.