Organize Your Document Set
While there are tools that are able to sift through document sets and help in the identification of only relevant contractual documents, over the years my clients have experienced both successes and failures in leveraging this functionality.
Even though I’ve seen how technology can help to sort and filter duplicates, I strongly recommend conducting a manual clean-up exercise that ensures the completeness of your contract data. During clean-up, you can determine that there are no missing pages, attachments, or related documents, and remove irrelevant or out of scope documents. Further, this clean-up should ensure that all parent/child and family documents are complete and together (i.e. master agreements are associated with their sub agreements). To be clear, what I’m advocating here is a combination of human capital and technology in order to achieve a complete and accurate data set.
Once you have a clean set of data, uploading documents to an automated tool is as simple as ‘drag and drop’. Once uploaded, the tool can take over the task of processing the data. Leveraging built-in OCR technology, the documents will be machine readable, and the tool will capture the provisions you want to have analysed.
Establish the Right Level of Customization
Most contract analysis tools have their own set of built-in standard provisions that the tool has been trained to review and analyse, but you can also customize the tool to capture the provisions required in your analysis. I personally like and use Kira’s Quick Study module. This helps me and my team combine machine learning capabilities with human intelligence. The Quick Study module helps to customize the system to my client’s requirements, which can be dynamic in nature. More importantly, I don’t have to be a ‘tech geek’ to code the system. One important benefit of Kira’s Quick Study module is that it helps to measure the accuracy of the provisions that you teach the tool to abstract. Accuracy in the world of AI for contract analysis is calculated by combining recall and precision. For example, if the tool is trained to capture the term clause, the recall number would be the number of times the tool could capture the term clause. Whereas, precision is calculated based on the information captured in addition to the term clause, such as termination provision. A good accuracy percentage is achieved by getting the balance right between recall and precision.
Validate Using Human Eyes
Assuming you need an “almost perfect” degree of accuracy, the output from any tool will require review by a set of human eyes for quality control. In my experience, you can currently expect anywhere between 70-85% accuracy in the output from the tool’s built-in provision models alone; whereas, you can improve this level of accuracy with an additional manual review.
There will always be issues where dates are handwritten, or the tool sometimes captures excess information or misses out on capturing certain provisions all together. Most AI companies understand this, and they acknowledge that a level of human quality control is still needed for the most accurate results.
Find the Balance Between Speed and Cost
Some projects may cost less using manual review, particularly leveraging the labour arbitrage that offshore centres provide. However, if time is of the essence, then using an AI or another technology tool will certainly speed up your review process and help you complete the project on time. It is important to understand the drivers for your organization before making that decision.
Getting Started with Technology
By setting the right expectations and taking adequate steps, technology can assist you in managing large volumes of contracts efficiently and in a cost-effective manner. Aligning expectations with what technology can and cannot do is very important for a successful outcome. Technology will add to the financial investment, and you will want to make sure you are getting the required ROI.
For lawyers who are considering using technology in contract management, it is important to understand what you want the technology to do, and then to find the right partner to help you. Since most forms of technology still require human training and application, choosing the project team is as important as choosing the technology.
Integreon leverages several AI tools, including Kira Systems. I have personally been involved in a number of complex contract analysis engagements, including one for one of the world’s leading ride sharing companies. Click here to learn more about this engagement.