Managed Document Review Case Study: Legal 500 Law
Building a document review strategy to meet the tightest of deadlines. Client Legal 500 Law Firm Challenge The client was party to a large UK litigation relating to a breach of contract claim. The review was complex, and Integreon was instructed to complete the review of 23,000 documents in 18 working days. Additionally, the data had to remain in the UK due to data privacy restrictions. Following further collection of data, the review volume grew to almost 700,000 documents, and the deadline could only be extended by a few days. Solution Working closely with the selected technology partner and the client, the Integreon document review team recommended and implemented a prioritized active learning workflow to meet the deadline efficiently and cost-effectively. Integreon put in place a delivery model consisting of three shifts each day. Most likely, relevant and hot documents were reviewed during the day shift by Integreon’s core team of senior reviewers who had originally been put on the project. Other likely relevant documents were reviewed during the second shift by additional senior reviewers added to the team. Finally, the likely not-relevant documents were reviewed by more junior members of the team during the night shift. Results Using technology to bifurcate the workflow, alongside round-the-clock shift work, the Integreon team reviewed approximately 500,000 documents in just 9 days. Lower cost resources during the night shift coupled with bifurcation of the workflow resulted in cost savings for the client. The Integreon document review team’s recommended workflow enabled the client and its legal team to comfortably and confidently focus on more critical strategic areas of the litigation.
Demystifying DSARs: A Beginner’s Guide
This article originally appeared in The Legal Technologist November/December 2023 Issue here. Learn more about The Legal Technologist on their website https://www.legaltechnologist.co.uk As individuals, we have the legal right to access personal data held by an organisation, and an increasing number of requests are submitted each year. Recently, politician Nigel Farage and Caroline Green famously and publicly spoke of their own DSAR requests thrusting DSARs back into the spotlight. Consequently, organisations (particularly financial institutions), are bracing for an onslaught of customer DSAR requests. What is a DSAR? The 1998 Data Protection Act introduced the concept of individuals in the UK having a right of access to personal data held by their employers or other organisations. In May 2018, the General Data Protection Regulations (GDPR) legislation came into force, making the DSAR process easier and more accessible for individuals. The ability to access your personal information is empowering and is an essential tool in our data driven society. Conversely, it is a costly disrupting exercise for organisations, particularly when it is weaponised as an act of defiance or retaliation. The deadline for response is only one calendar month following receipt (extensions are possible under limited circumstances). How can organisations ease the burden of processing DSARs whilst also ensuring individual rights are upheld within the necessary timeline? Effective Data Governance There is more to GDPR than just DSARs. The GDPR drives accountability for the way in which organisations control and process personal data. If an organisation doesn’t have robust policies/ procedures, they risk fines and penalties for violation. Effective data governance is critical. To establish a strong data governance policy, organisations need to understand: • What data is collected? • Who does it belong to? • Why was it collected? • How/where is it stored? • Who has access to it? • How is it shared/used? • How long is it being retained and is it purged when retention is no longer required? Maintaining compliance requires an ongoing process of monitoring business practices. Policies and controls must be regularly assessed. Key documents such as staff and public privacy notices must be accurate. Continuing compliance activities may include conducting impact assessments for new systems, reassessing security procedures, and analysing new and existing contracts and agreements to ensure that data processing meets the requirements of GDPR. Having a data governance policy in place and being fully GDPR complaint is critical. But what happens when those dreaded DSARs come through the door? DSAR Process and Procedure If you don’t have a clear and documented DSAR process, you need to design one as a priority! Your DSAR procedures should be governed by a process document or playbook. It should clearly map out who is part of the response team and the tasks they are responsible for, along with timelines. Your DSAR team may include external advisors such as lawyers, eDiscovery companies and/or document review expert service providers. If so, include their contact and process details. If you organisation expects both employee and customer DSARs, make sure your playbook accounts for that as the process will differ. For example, customer data may be more structured and templated in nature, but employee data will include information from multiple sources across the organisation in numerous formats (email, HR files, financial etc.) This means there will be differences with respect to collecting, processing and reviewing the data. What stages should I include in my DSARs process? Stage 1: Acknowledge and Validate Acknowledge receipt and confirm the timeline Ask questions if needed to fully understand the scope and avoid time wasted down the line Validate the identity of the requester. Disclosing personal data to the wrong person comes with serious and costly consequences! Stage 2: Collect and Process the Data Don’t make assumptions, ensure you conduct a thorough check of all systems and servers. Depending on volume, technology tools can be utilised to effectively cull the data, saving time at the next stage. If an eDiscovery provider is engaged, they will advise on analytics and use of Al. For example, deduplication, email threading and the application of search terms (variations of the requesters name for example) will all be beneficial. Stage 3: Review and Redact the Data Review the data to ensure it is responsive to the request, that no exemptions apply and redact personal data that is not that of the requestor. This step can take time and is costly to an organisation as generally there is a lack of capacity to take on a full review and redaction exercise. Many organisations elect to use a specialist document review services company for this phase. In doing so, the organisation gains not only a cost and time saving but benefits from project management skills and subject matter expertise, both help to significantly reduce the burden. Input from internal or external legal advisors is crucial. Exemptions to disclosing personal data exist within the GDPR but the application of those exemptions can be complex. Once the data has been fully reviewed and redacted, the next step is to share the data with the requester. Other Best Practices What else can an organisation do to ease the burden? Record Keeping and Communication Keeping a record of all communications and actions undertaken for each DSAR processed is key. Good record keeping provides the organisation with evidence of compliance with both internal processes and legal requirements. Also, keep the channel of communication with the requester open. Providing clear and timely updates on progress is good practice and can save you unnecessary headaches. Lessons Learned and Continuous Improvement GDPR compliance requires ongoing monitoring, assessment and updating. This practice should also apply to your DSAR process and playbook. If there are lessons learned along the way, ensure your playbook takes these into account and is updated. Education and Training Finally, there is no point in having robust data governance policies and a watertight DSAR process if both are not bolstered with training and education. At a minimum, training should take place during onboarding with regular refreshers thereafter.
NetDiligence Cyber Risk Summit Miami
Join Integreon at NetDiligence Miami! Integreon is excited to sponsor the NetDiligence Cyber Risk Summit in Miami Beach on February 12-14, 2024. Join us in sunny Florida for unparalleled programming and three days of not-to-be-missed industry networking. Connect with Integreon Integreon will be on site all week and would welcome the opportunity to connect. Reach out to our cyber experts ahead of the event to set up a meeting, or stop by our table and connect with us in person. To set up a meeting, please reach out to Brian Duke SCHEDULE A MEETING LEARN MORE
Integreon Looks to Disrupt $20 billion Industry and Becomes the First Generative AI-Led ALSP with Leah Legal Copilot by ContractPodAi
Integreon will utilize ContractPodAi’s Leah Legal Copilot to power its offerings, adopting a fully generative AI-led approach for both its outsourced legal and business services delivery. LONDON, UK – December 5, 2023 – Leah, a leading and innovative provider of solutions in the contract lifecycle management and legal technology market, announced today that Integreon, a leading provider of tech-enabled legal, creative and business outsourced solutions, has selected Leah to power its suite of services. As a result, Integreon is now the first alternative legal services provider (ALSP) to adopt a fully generative AI-led approach. Over the last two years, ALSPs have grown by 45% into a $20.6 billion industry, primed for innovation. For 25 years, Integreon has been dedicated to supporting clients across a wide range of services while integrating new technologies and automated programs as they become more prevalent in the market. With Leah, Integreon clients will benefit from generative AI and its ability to streamline services and deliver even more efficiency. This is especially the case in areas of high-volume work with low or medium complexity, where Integreon will directly impact ROI with better scalability and productivity gains. “For more than two decades, ALSPs have relied on the same delivery method. Today, with the help of Leah by ContractPodAi, Integreon is more than just a market leader – we are a market disruptor,” said Subroto Mukerji, Integreon CEO. “We are proud to be the first provider to embrace a generative AI delivery model designed to provide value-based and scalable solutions in the next evolution of legal services.” On top of fueling its current legal and compliance solutions, Leah will also allow Integreon to support its enterprise clients to implement a generative AI strategy identifying use cases and building pilot programs. Integreon will also integrate generative AI across its corporate functions to reap the benefits of enhanced processes and increased productivity. “We are thrilled to be a part of ushering in this fundamental transformation with Integreon,” said Sarvarth Misra, co-founder and CEO of ContractPodAi. “This is a game-changer, and it represents that Leah and Integreon are taking the ALSP delivery model to the future. The ALSP market has seen some generative AI pilot projects, but nothing close to this degree of enterprise-wide business transformation. Our goal with Leah has been to reimagine the way legal services are delivered, which is exactly what we’re seeing Integreon do with this technology. The future is here and it’s legal, reimagined.” Leah is a cutting-edge AI solution designed to empower legal teams and law firms to enhance their work with productivity gains. Enabled by generative AI and multiple, best-in-class LLMs such as GPT-4 Turbo and Google, it revolutionizes the way legal documents are reviewed, analyzed, summarized, created, and interconnected. ContractPodAi is accelerating the future of digital transformation in the legal space by providing a full, enterprise-grade tech stack of legal work solutions, powered by generative AI, including Leah, its Legal Intake solution, and its all-in-one CLM platform. About ContractPodAi Recognized as a Visionary by Gartner in 2021, 2022, and 2023, ContractPodAi is on a mission to transform the way the world makes agreements. Lawyer-led and AI-powered, our ‘One Legal Platform’ was developed by attorneys as a solution for legal teams. Now, more than ever, it’s clear legal teams aren’t the only ones struggling with insurmountable workloads and complex contracts. That’s why we’ve spent over a decade evolving our AI and CLM to offer an all-in-one platform where teams take charge of their contract management process and our new generative AI-powered Leah Legal Copilot to provide super-powered support for Legal Teams. ContractPodAi amplifies your business’s readiness through our partnerships with complementary technology providers including Microsoft, OpenAi, DocuSign and Salesforce. ContractPodAi® is headquartered in London and has global offices in New York, Glasgow, Sydney, Mumbai, and Toronto. For more information about Leah Legal Copilot’s generative AI capabilities, visit https://leahai.com/. About Integreon Integreon is the trusted, global provider of legal, creative and business outsourced services to corporations and law firms seeking to expand their capabilities and transform their performance. The company’s 3,500+ professionals provide expert support across a range of managed services—from creative design, content delivery and administrative support to legal and compliance. With global delivery centers on three continents, Integreon delivers round-the-clock service in 50+ languages and is deeply committed to client success, consistently delivering innovative, tech-enabled solutions that improve agility and efficiency to drive business performance. Integreon is owned by EagleTree Capital, a leading New York-based middle-market private equity firm with over $5 billion of assets under management. For more information about Integreon’s range of services, email [email protected], visit www.integreon.com and follow Integreon at LinkedIn, Twitter and Facebook.
Managed Document Review Case Study: Fortune 100 financial services firm
Fortune 100 financial services firm reduced risk and costs with a streamlined document review process. The client A Fortune 100 global financial services firm that helps people, institutions, and governments raise, manage, and distribute capital. Challenge Ensuring Compliance and Responding to a Highly Regulated Industry Financial services is one of the most heavily regulated industries, creating tremendous pressure to respond quickly to ensure compliance. Meeting these demands, along with handling unforeseen litigation and investigations, placed a significant strain on the client’s in-house legal department. In addition, their global footprint raises challenges relating to data privacy, data security, cross-board data transfer, and related regulations such as General Data Protection Regulation (GDPR). In response, the client launched a search for an outsourcing partner with deep financial services compliance and litigation experience, as well as a proven secure approach for streamlining related processes. Solution The client selected Integreon’s Managed Document Review (MDR) Services because of the strength of our processes and the quality of our people. The client’s dedicated team of highly experienced review professionals became fully immersed in their business, operations, and preferences. Having such a deep familiarity with the client allows Integreon to quickly deploy additional reviewers from one case to another and scale the team when there are spikes in work with no quality degradation or loss of time for training. With a dedicated team in place, Integreon took on the Herculean task of expediting an average of 5.2 million documents per year, with the total count of documents reviewed from the beginning of the partnership in 2009 to 2022 standing at 75 million. Integreon’s follow-the-sun model and global network of legal experts across three continents allow us to review documents in 55+ languages and handle challenging turnaround times. Types of Cases Handled • Antitrust litigation • Arbitration • Bankruptcy • CFTC investigation • Class action lawsuit • Derivative lawsuit • Employment • European Commission investigation • FED investigation • FINRA investigation • FTC investigation • Indenture litigation • Internal investigation • IPOs • IRS information request • ISDAFIX matters • Libel • Regulatory request • RMBS • SEC investigation • Tax Managed Document Review Services Provided • First-level review • Second-level quality check (QC) • Privilege review and privilege log creation • Redactions (such as Personal Identifiable Information) • Deposition summary support • Transcription • Personal device review • Foreign language review and translation (55+ languages supported) Results For over 14 years, Integreon has consistently exceeded expectations and service level agreement (SLA) requirements for speed and accuracy. And having a managed services approach has enabled Integreon to develop a deep understanding of the client’s business and how they prefer to work and communicate. Since the start of the engagement, the Integreon client team has grown from 10 attorneys in Manila to 30 and from 2 attorneys to 12 in Fargo. Introduction of a Continuous Learning AI Tool One project initially had a population of 100,000 documents to be reviewed from the priority set. An early case assessment was conducted to cull non-responsive documents, and we suggested the removal and mass coding of approximately 30,000 of these documents. These exclusions were then applied to additional datasets. Seeing the low relevance rate results, Integreon suggested the introduction of an AI-based continuous active learning tool into the review process. This allowed the team to streamline further and complete the review process even faster, thus delivering additional time and cost savings to the client. Integreon’s ability to leverage technology, improve workflows, reduce documents that require review, and scale resources up or down to respond to client needs has built an enduring 14-year partnership. Recently, Integreon was selected by the client’s outside counsel to assist with the firm’s own ongoing investigations.
GenAI works best when there is an actual Human in the Loop
Almost daily, the power of generative artificial intelligence (genAI) reveals itself through new use cases, applications, and platforms. No one can argue that this technology is here to stay and the impact on speed and efficiency is unprecedented. And like genAI, the more we use it, the more we learn. Perhaps the most significant learning is that genAI cannot or more specifically should not be permitted to stand on its own, especially when supporting the legal arena. Optimal output is not simply a question or keystroke away. Enter the human into the loop. Anyone can tell you a process is only as good as its quality control, typically overseen by subject matter experts there to ensure guidelines are met, consistent quality is maintained, and continuous improvement is achieved. For genAI specifically, human supervision addresses: Quality: GenAI can be fallible producing errors, hallucinations, and even inappropriate responses. The human in the loop is there to ensure output meets established standards of accuracy, completeness, and integrity. Ethics: This is an area that clearly requires human review to flag content that may be perceived as biased, discriminatory, or offensive in nature. The system will not have a sufficiently nuanced sensitivity and understanding for what is and is not acceptable. Interpretation: While genAI is impressive in some settings, understanding the context of a word, situation, email, even an emoji can be critical and not always assured when using genAI. A human with knowledge of the variables of a situation or environment is invaluable in reviewing genAI generated content prior to use. Refinement: Having a human-in-the-loop QC not only addresses the quality of the machine output but allows for adjustments, adaptation, and more particular alignment with company objectives and requirements. Iterative human feedback can also help the system to learn, perform better and improve over time. Compliance: When leveraged, genAI can dramatically speed up tasks and gain efficiency. It is important to bear in mind, however, that in any circumstance where absolute accuracy, completeness, or sensitivity is required, human oversight and judgement is a must have part of the process. Creativity: Certain tasks require human creativity, judgment, and intuition that AI may not currently possess. Human intervention is necessary for tasks where subjective judgment plays a crucial role. Querying: GenAI only answers what is asked. Therefore, it is humans that need to develop and pose the questions. Humans choose what and how to make inquiries, along with which AI models to use to create the response and which data sources to leverage to limit the frame of reference and optimize accuracy and efficiency of use. Unknown: The AI system is trained on a finite (albeit very large) data set and can only respond with what it knows and has learned. Mere mortals, on the other hand, are far better equipped to address unpredictable and unforeseen scenarios. If organizations have learned anything as it relates to introducing new technology into their business and processes, it is that it requires an investment in oversight, input, and control conducted by humans with relevant expertise. The absence of a balance between automation and human processes and intervention can produce disappointing quality and even high-risk exposure. Many organizations are ensuring that balance by leveraging the technical, process, and subject matter expertise of outsourced providers who are skilling up to support genAI environments and tools to deliver their services. In doing so, in-house teams can focus on more strategic higher complexity work, assured that they are reaping the benefits of genAI while mitigating their risk and maintaining their quality standards. Go ahead and embrace genAI technology, but always keep a human in the loop!
Metrics You Should Expect from Your Outsourced Creative Services Partner
Outsourcing creative services is an effective model for marketing operations teams looking to extend their capabilities, streamline their processes, and tap into innovation. However, to ensure a successful partnership, it is essential for marketing operations professionals to establish clear metrics and expectations with their outsourced creative services partner. This blog post will explore key metrics marketing operations should ask for from their outsourced creative services partner to drive effective collaboration measure performance, and results. On-Time Confirmation: On-Time Confirmation is a critical practice that involves verifying and validating key milestones, tasks, or events to ensure they occur as scheduled. This practice promotes accountability, transparency, effective communication, and timely adjustments if necessary. By implementing On-Time Confirmation as a practice, businesses and individuals can enhance efficiency, build trust, and achieve successful outcomes. On-Time Delivery: Timely delivery of creative assets is fundamental for marketing operations to meet their campaign timelines, respond to internal stakeholders, and remain competitive. Make sure to request metrics from your creative services partner that track their ability to deliver projects on time. These can include measuring the percentage of projects completed within the agreed-upon deadlines or the average turnaround time for different deliverables. This will also help establish baseline turnaround times for different types of deliverables so that expectations can be set. Turnaround Time: The Turnaround Time Metric in marketing operations refers to the defined time frame within which specific marketing tasks or deliverables are expected to be completed. This metric measures the maximum duration it takes for a service provider to respond, fulfill, or deliver marketing requests. By establishing clear and measurable turnaround time expectations, marketing operations can ensure efficient execution of campaigns, content creation, and other time-sensitive activities, resulting in improved productivity and customer satisfaction. Quality of Creative Work: The quality of creative work directly impacts the effectiveness of marketing campaigns. Request metrics that assess the quality of creative assets produced by your outsourced partner. This can be done through evaluations or feedback surveys conducted by internal stakeholders or customers. Look for metrics that measure satisfaction levels, adherence to brand guidelines, and overall impact on campaign performance. Having this type of metric also supports internal “selling” of the model. Cost Efficiency: Understanding the cost-effectiveness of your outsourced creative services is crucial for managing budgets and maximizing return on investment. Request metrics that provide insights into the cost efficiency of the partnership. This can include metrics such as cost per deliverable, savings achieved compared to in-house production, or cost comparisons with alternative service providers. Communication and Collaboration: Consistent communication and effective collaboration are essential for a successful partnership with an outsourced creative services provider. Request metrics that evaluate communication and collaboration processes. This can involve measuring response times, feedback incorporation rates, or the number of revisions required to finalize a project. Such metrics help gauge the efficiency and effectiveness of communication channels and workflow management. Flexibility and Scalability: Marketing operations often require flexibility and scalability from their creative services partner to meet changing demands. Request metrics that assess the partner’s ability to scale resources or adapt to evolving project requirements. This can include metrics such as response time to resource allocation requests, the ability to handle peak workloads, or the success rate of resource adjustments. Campaign Performance Impact: Ultimately, marketing operations seek results from their creative services partner. Request metrics that demonstrate the impact of the creative work on campaign performance. This can involve tracking key performance indicators (KPIs) such as click-through rates, conversion rates, or engagement metrics associated with the creative assets produced. By analyzing these metrics, you can assess the effectiveness of the creative services partner in driving desired outcomes. Overflow support: Overflow support is a specialized model tailored for marketing operations, enabling businesses to manage increased workloads during peak periods effectively. This model provides an additional workforce to handle surges in customer inquiries, campaign management, content creation, and other marketing tasks. Overflow support ensures seamless operations, timely response to customer needs, and the ability to maintain high-quality service even during periods of high demand. By leveraging overflow support, marketing teams can optimize efficiency, meet deadlines, and deliver exceptional results to drive business growth. Establishing clear metrics for your outsourced creative services partner is the foundation for effective collaboration and ensuring you get the desired results. By focusing on metrics such as on-time delivery, quality of creative work, cost efficiency, communication, flexibility, and campaign performance impact, marketing operations professionals can ensure that their outsourced partner meets expectations and contributes to the overall success of marketing initiatives. Regularly reviewing these metrics, providing feedback, and fostering a collaborative relationship will help optimize the partnership and drive continuous improvement in creative services delivery. Metrics will also be a powerful tool internally, especially during budget and planning cycles. Transform Your Marketing Operations with Integreon Integreon brings tech-enabled processes and a team of tenured researchers to consistently deliver the insights you need to grow and protect your business. Based on the way you like to work and your specific needs, results are packaged for ease of use and clarity. Integreon works with clients across a wide range of sectors and industries to collect data, identify and track KPIs, and generate valuable insights. Our clients range from law firms to investment banks and financial institutions to large, diversified conglomerates. Integreon’s team of sector specialists ensures that no industry-specific nuance is missed. Learn more about Integreon’s Business Intelligence and Research services here: https://www.integreon.com/what-we-do/creative-and-business-solutions/business-intelligence-and-research/
Top Five Considerations When Responding to a Cyber Breach
When a cyber breach occurs, cooler heads need to prevail. This can be a highly emotional and stressful occurrence. Being prepared and having a clear plan of action will help you stay focused, meet your compliance requirements, and above all, minimize the fallout and risk. In building your response plan, here are five things to consider: 1. Contain First and foremost, isolate the affected systems or networks to prevent the threat actor from spreading or causing more damage. This may involve your IT and security teams shutting down impacted servers, partitioning networks, or disconnecting affected devices from the internet. 2. Notify Dictated by the type of breach and jurisdictional requirements; you may be legally obligated to notify impacted or potentially impacted parties, including customers, partners, employees, and regulatory bodies. If you are obligated to notify, you will need to provide information about the breach, including the extent, along with what you are doing to address it and steps individuals can take to protect themselves. 3. Investigate Don’t go it alone! Bring in professionals experienced in conducting post-breach forensics to gain a full understanding of the breach and the threat actor, including the extent of the breach, where and when it started, techniques used to access your environment, and what specific systems and data were compromised. Additionally, make sure you preserve evidence and logs that will assist in the breach analysis, and apply AI for greater efficiencies and accuracy for first-pass review. 4. Message Having a well-conceived communications strategy for handling internal and external messaging will serve you well in the near and long term. If you have a PR agency, engage with them early and inquire about crisis management services. Your communications plan should include a consistent set of messages crafted for each specific audience, such as employees, customers, and the media. 5. Recover Remember, you can overcome a breach! Once the breach has been contained and you have identified the entry points and tactics, focus on locking down the system and removing any vulnerabilities that allowed the incident to happen or could expose you in the future. This might include patching systems, changing passwords, strengthening security protocols, and implementing additional security measures. You will also want to have a plan to quickly and, above all, safely get affected systems and services back online to minimize downtime and mitigate the chance of another breach. CyberHawk-AI by IntegreonThe first cyber incident response machine learning application As the only end-to-end cyber incident response (CIR) solution downstream of forensics, Integreon understands the role of speed and accuracy during first-pass CIR reviews. Historically, this has been a highly manual process that could result in errors, inaccuracies, and bottlenecks. Integreon knew we could do more and do it better for our clients. Leveraging deep CIR expertise honed over five years working across industries, jurisdictions, and document types, Integreon’s center for innovation, i-Lab team developed CyberHawk-AI, the first CIR ML application. Learn more about CyberHawk-AI » Technology and deep expertise power scalable cyber-specific data mining and review platform Integreon is a pioneer in cyber incident response (CIR) with a highly evolved built-for-purpose platform and processes that enable handling of large, complex, multi-lingual projects. Our proprietary tech stack addresses niche document types and bespoke workflows and is the product of a team of experienced technologists and data scientists dedicated to CIR innovation. Learn more about Integreon’s Cyber Incident Response services here: https://www.integreon.com/what-we-do/legal-and-compliance-solutions/cyber-incident-response/