Automating reference checks
Checking references is a time-consuming burden for many procurement entities. In Tucson, Ariz., for example, principal contract officer Nathan Daou says reference checks involve multiple members of the evaluation committee making numerous calls and often playing phone tag. Having multiple people do the calling increases the variability and makes comparisons difficult, but the burden is too large to assign to a single team member busy with other duties.
It's a thankless, but necessary, task. The best source of information about a supplier's current — and therefore future — performance is their customer base, as reflected by effective reference checks. To provide a new approach to collect reference information, NIGP and eVendorCheck have partnered to provide eVendorSelect, an automated web-based survey solution to assess and select reliable, effective suppliers at less cost, less risk and less effort than telephone reference checks and similar methods.
eVendorSelect allows procurement professionals to obtain real-time customer feedback gathered using web-based surveys of up to 25 references per supplier. The supplier company provides the reference contact information, and the system generates an automated email to each contact containing a link to an online survey. eVendorSelect converts confidential customer assessments into statistically meaningful predictions of supplier performance, according to the company. The data is collected by the third-party supplier, which minimizes variables and eliminates any perception of bias.
Another advantage of eVendorSelect is the practice of surveying multiple individuals at each reference entity, including a mix of project managers, procurement, finance managers, etc. Each reference provides multiple data points, which broadens the sampling of responses and increases the value of the resulting information. Survey questions provide a combination of numerical ratings and narrative responses that combine both a quantitative score and valuable verbatim comments.
eVendorSelect provides a documented, reviewable process and results to ensure transparency and integrity of referencing practice. Detailed and standardized ranking simplifies supplier reference ratings. The NIGP member fee is $99 per supplier evaluated.
Transparency is paramount
Reference checks can be critical in case of a protest; or worse, a lawsuit. A Massachusetts Supreme Judicial Court recently affirmed a public agency's right to conduct an investigation into whether the low bidder is a responsible bidder. The case centered on the Town of Holliston, which received several negative reference checks on the low bidder on a new police station construction project. When they concluded the low bidder was not a responsible bidder, the supplier sued the town arguing that the reference check was not authorized. The court ruled that it is up to each agency to determine responsibility, and that disappointed bidders have a right to challenge a decision in court.
"We really believe that our product fits in well with the legal and regulatory requirements of public procurement," said Joyce Herlihy, senior vice president of eVendorCheck, supplier of eVendorSelect. "In the non-public arena, there is no requirement to check references and no fear of being sued by the supplier or having to open your process to any outsider."
Herlihy says transparency is required in public procurement, and decisions are often guided by a fear of supplier protests. In the case of a multi-million dollar contract, for example, a supplier might hire lawyers to review every piece of paper, so the references have to be transcribed into documents that could be turned over to multiple parties in a protest. "Our system-generated reports fill the requirement of legible, non-biased documentation of what a customer's references said," according to Herlihy.
Tucson's Nathan Daou used eVendorSelect in a pilot program involving solicitation for hydraulic modeling software for the water department. The solicitation included language that any vendor who submitted agreed to work with eVendorSelect on the reference check. In the end, only one supplier bid on the contract, so the result of the reference check was moot. However, the process allowed Daou to see how the process works and to get a feel for the depth and quality of the resulting reference check information.
In the case of the Tucson solicitation, a single company provided 33 references (including multiple people at several entities), which Daou says reflects how easy it is for supplier companies to enter reference contacts. The 73 percent response rate suggests the survey is easy to use, said Daou.
Rules vary by state
Rules vary from state to state about reference checks, said Doug LaPasta, CEO/chairman of eVendorCheck. Generally, there is a dollar limit — usually around $40,000 or $50,000 — above which references must be checked. "Reference checks are often based on unstructured conversations and there is no consistency about what is said, which allows possible misinterpretation," said LaPasta. "The notes on a yellow pad are not archived, transparent or consistent, and may not answer the necessary questions."
In the case of eVendorSelect, because the questions are being asked by a third party, the answers are anonymous and references are more likely to be honest in their responses, said LaPasta. In contrast, phone calls also provide a smaller sample size.
Swimming pool contract in Tamarac
Keith Glatz, purchasing and contracts manager of Tamarac, Fla., recently used eVendorSelect when awarding a swimming pool management contract. After using the same supplier for years, the city had previously switched to a different supplier (based on a lower price), but was dissatisfied because the new supplier had not met the requirements. The city ended the contract. At the height of the summer swim season, the city had to bring back the previous supplier on an emergency basis and re-bid for a new supplier.
Unfortunately, the previous supplier, which the city had been happy with, had recently received bad publicity regarding an incident in which one of its lifeguards had been fired for leaving his zone to save a life. A city politician had publicly opposed awarding the new contract to the long-time supplier based on the incident and negative publicity.
The political aspects of the contract, and the recent bad experience with the newer supplier, made complete and impartial reference checks especially important to the city. "Using eVendorSelect ameliorated the issue of staff bias related to collecting references," said Glatz. "It took the political pressure off our staff, and provided a high-quality reference check with a depth of references we had never seen before."
Glatz said the references were provided within a matter of days and saved the eight to 10 staff hours it might have taken to check the references of the three companies that bid on the new contract. Glatz said the third-party reference taker was also more successful getting references to respond. Offering a combination of quantitative and qualitative (narrative) responses, the product turned a subjective process into an objective one. In the end, the city awarded the contract to the former supplier, despite the negative publicity, based on positive eVendorSelect ratings from references.