Contents: Introduction | Definition and context | Our ambitious goals | Organizational readiness | The key features | The innovative integrated solution | In closing | More information
Welcome to our second article in our ‘innovation through integration’ series
In this article we share our approach and solution for automating the writing of complex decisions. Our world is getting more complex every day, and inevitably, our disputes and the information it takes to achieve their resolution is getting more complex too.
For most organizations this relentless increase in dispute complexity is addressed with the power of the human mind (i.e. increased staff experience and skill), but a continued reliance on the human mind has significant consequences; it results in work that is increasingly harder to learn, harder to complete, and harder to do efficiently and consistently. But how do you find solutions in a world of ever-increasing complexity?
Alfred Whitehead said it well, “the only simplicity to be trusted is the simplicity found on the other side of complexity”
For a problem like automating complex decision writing, I couldn’t agree more. As humans, we have a self-defeating tendency to 'jump to solutions' before adequately working through the complexity of the problem. For example, you might like these two ideas for automating decision generation:
“Let’s fully replace complex human decision writing with A.I.” - Spoiler alert, this is vastly underestimating the complexity of writing decisions AND the complexity of training A.I.
“Let’s build simple templates with basic data pre-population” - Spoiler alert, this is mostly avoiding the complexity of writing decisions (staying on this side of complexity).
I find it useful to imagine this 'jumping to solutions' like jumping against a complex thicket of overgrown and interconnected branches and thinking the right leap is somehow going to get you through it. Sure, jumping at the thicket and bouncing off feels like productivity, and everyone appears to be working hard trying different jumping techniques, but it's really wasted time and energy - forward progress is being blocked by a complex barrier that cannot be penetrated by the simple act of jumping.
My advice with complex problems?
Never jump to solutions, put first things first, and start with dissecting a path through the specific complexity you are dealing with. Then, and only then, can you design effective solutions that can navigate the complexity.
How did we arrive at the decision generation solution in this article? We worked through the complexity of defining all the dispute issues (distinct claims people make in their disputes) and possible resolution outcomes (1000+ combinations) to establish the underlying pattern for complex decisions and their content. We then built DMS features and processes that gather the dispute file information necessary for the automation - and paired this with a dynamic content generation engine that populates a significant amount of the complex decision content that a human would have to create, but instantly.
I want to point out that this is not a theoretical solution. It's an innovation based on features being used for complex dispute files in a high-volume tribunal. As DMS features are very advanced themselves and we want to keep these articles focused, the supporting features are only briefly described.
What do we mean by automating complex decision writing?
A system that uses dispute file information to automatically select and generate a core document with a title, date, and necessary sections
A system that automatically generates and inserts section content including; relevant file information, party information, disclosure and service information, party attendance and participation information, the matters in the dispute (issues and claims), relevant evidence and testimony, and the preliminary matters – all with reference to their associated sections of rules, acts or legislation.
A system that uses the resolution outcome information for each issue (i.e. granted, severed, dismissed) to automatically generate and insert the analysis, conclusions, and order information - with highlighted areas that require human validation and placeholders where custom human content is inserted to support the issue outcomes.
A system that automatically dates and signs the decision, and generates a final PDF for delivery and an anonymized PDF version for publicly available search from the DMS decision search site or directly from any DMS dispute (to find exactly related decisions).
Authors Note: This article uses the word “decisions” to keep the article focused, but this approach and solution applies to any resolution document including; default decisions, settlement agreements, correction decisions, clarification decisions, review request decisions, substituted service decisions, adjournment decisions, interim decisions, etc.
Significant innovations require ambitious goals, these were ours:
To greatly reduce the required skill, experience and effort of resolution staff writing decisions, while greatly increasing the consistency, and quality of their decisions
To improve the completeness of dispute files and greatly reduce unnecessary resolution staff information requests, note taking, and the manual work of entering this missing information into decisions.
To create a system where everyone (parties and staff) can instantly see past decisions (anonymized) that match a dispute file they are looking at so that:
External parties can access previous decisions with the exact matters of their file so that they can make informed decisions based on real-world outcomes and can improve the completeness and relevance of the information they submit based on similar historical disputes.
Staff can instantly access previous decisions exactly like the file they are working on so that they can get examples and language to support decision writing and leverage previous decisions as a community of practice.
To enable powerful management reporting for decisions including; detailed resolution outcomes (comparative staff decision making), total resolution effort (comparative staff performance), and requests for corrections, clarifications, and reviews associated to decisions (comparative staff decision quality).
What were the organizational readiness requirements and how did we achieve them?
Three pillars of information are required for complex decision generation; “dispute file information”, “resolution information”, and “decision content”. You can imagine this as a 3-legged stool – if any leg is short or missing, the stool won’t stand up to real world use. Note: each unique document you are generating is its own 3-legged stool.
The first leg of the stool, complete “dispute file information" was achieved through the following approach and work:
We poured over the act, legislation, dispute files, and decisions to separate generic issues and claims into a clearly defined and complete list of specific issues and claims – each with their unique information requirements necessary for their resolution.
We analyzed historical decisions and defined all of the dispute file data that was being gathered in resolution sessions and being inserted manually into the decisions and created systems and processes to gather that information into the dispute file.
The second leg of the stool, complete "resolution information” was achieved by taking our defined list of issues and claims and creating a list of all of the possible outcomes (e.g. granted amounts, granted possession, settled, dismissed with/without leave to reapply, severed, removed by amendment, etc.). We then took this list of outcomes, issue by issue, and validated all of the use cases for each issue against the list of outcomes until we were sure it was complete, and we could describe each outcome with associated decision resolution information.
The third leg of the stool, complete "decision content” was achieved through the following approach and work:
We examined the common documents used by resolution staff and looked at where their editing of the template signified a core difference in content, and used that to create an expanded list of unique templates that respect the real variations in the documents that were being hidden by training and human modification.
We analyzed all of the sections in all of the documents and defined common sections that are shared across these documents – while noting any special use cases for specific documents.
We defined the repeatable content that is inserted into these shared sections of each document and validated that this content covered all identified document use cases for the sections.
We creating a matrix of issue and outcome based content (language) that includes conversational and act based titles, conversational act descriptions, strict act descriptions, conversational outcome text (i.e. if-granted text, if-dismissed text, if-settled text), conclusion text, and order text.
Through a series of sequential DMS releases that established and refined the systems and processes, we achieved foundational maturity in the following areas:
A comprehensive and proven set of all unique issues (claims) that can be included in a dispute, each with their own information, testimony and evidence (required and optional)
A complete and proven set of all possible resolution outcomes for each issue (claim)
A complete and proven data collection system that gathers all necessary data for automated population of decision content from parties and/or organizational staff.
A complete and proven set of resolution documents used by the organization, with validated rules and conditions under which they are used.
A complete and proven library of decision content that addresses all of the section content and outcome information with full reference to specific rules, sections of the act and legislation.
What are the individual features that, when combined, enable the final innovation?
A long list of individual DMS features were combined (integrated) to achieve the final innovation. Before I list these features, I’d like to re-iterate two important points that I covered in my fully automating complex applications and initial submissions article.
First, integrating poorly designed or buggy features will just create a bigger mess. You should start with getting core features released and proven before you combine them (or at minimum be very certain they are of very high suitability and quality).
Second, you are much more likely to achieve a better solution (all things considered) by having everything you are integrating within one system. Its for this reason that every feature and capability mentioned in this solution was built inside of the DMS system.
With those key considerations out of the way, the following is a list of the DMS features and capabilities that enabled the final innovation:
Guided intelligent intake(s)
An intuitive application information gathering systems that collect all of the general dispute information required for an associated decision to be completed without additional information requests or note taking by resolution staff.
A guided issue and claim selection systems that allow the applicant to define their exact issues and provide the information, evidence and testimony necessary for their effective resolution (e.g. a damages issue requires a description of the damages and how they were caused, proof of the damages, proof the damages were caused by the respondent, and proof of the cost of repairing the damages).
An automated algorithm that sets the dispute file urgency (priority), resolution process (oral mediation, oral hearing, written hearing), and associated acts and legislation so that the associated decision templates can be defined.
A disclosure and service management system that creates a 'submission package' and 'opposing party service records' for all notice, evidence and testimony added to the dispute file - with each opposing party service record allowing the dates (served, deemed served), method of service, and proof of service to be recorded.
A participation management system that automatically creates participation records for all parties and scheduled events on the dispute file, and that allows the parties or resolution staff to indicate participation and attendance (including the attendance of others not named in the dispute file).
A multi-channel submission system (office and online) that includes a conditional access engine that shows and hides submission channels based on the characteristics and status of a dispute file, party roles, and completed party actions. Specific to this innovation this system provides:
A guided process for all parties to submit positions, evidence and testimony on the specific claims and issues in the dispute file.
A guided process for disclosing the service of dispute notice, evidence and testimony to opposing parties.
A guided process for indicating intention to participate in resolution processes and attend scheduled events.
An amendment system that detects and records all changes to the core dispute file information after the original notice is provided. This includes:
An automated system that tracks and records the issues and parties that have been removed through amendment prior to the decision.
An automated system that updates issue and claim positions and information as new information is provided (including changed monetary compensation amounts).
An automated system that tracks and records the issues and parties that have been removed by resolution staff during the resolution session.
An outcome management system that resolution staff use to indicate the specific resolution outcome of reach issue in the dispute file (i.e. settlement, granted monetary award, granted orders of possession, removal through amendment, removal through severing, removal through non-jurisdiction, removal through dismissal with/without leave to re-apply).
An outcome document management system for resolution staff that allows specific document placeholders to added to a dispute file and used for the generation, storage, and delivery of decisions.
A posted decision system that, when an anonymized PDF decision is uploaded to a dispute file, automatically collects specific metadata from the dispute file and the text from the decision and allows it to be instantly available for searches by both external citizens and resolution staff - directly from DMS.
An automated document delivery system that combines decisions with a matching delivery email template (and optional additional attachments) and automatically emails documents to parties that have set a preference of email delivery.
What is the final solution and how does it work?
I’ll describe the final solution as a happy-path sequence of user and automated system actions. As many of the individual DMS features here are quite advanced and the process includes some complex edge case handling, I tried to balance "making it easy to understand" (without oversimplification) and "describing it completely" (without getting overly technical).
The applicant is guided through an intuitive online wizard-based intake that uses questions to tailor the application process to their specific needs (for something similar, think TurboTax). It then helps them to file a complete and clean application that contains all of the information and specific issues necessary to populate the associated decision.
Behind the scenes and unseen by the user:
The system automatically detects and assigns the dispute priority, process, acts and legislation, and other attributes that define the assocaited decision(s).
A dispute initiation notice package is created and provided to the applicant
Behind the scenes and unseen by the user:
The system automatically creates an empty record for the service of the notice to each responding party.
The system automatically creates scheduled event participation record for the participation and attendance of all parties and named supporters, advocates, agents or lawyers on the dispute file.
The applicant uses guided online (or office) processes to indicate that the dispute notice was served to each respondent
All parties use unique codes and links in the notice and follow guided online (or office) processes to provide their positions on each issue in the dispute file (information, testimony and supporting evidence). With each submission they receives a detailed submission receipt and instructions for serving the opposing parties.
Behind the scenes and unseen by the user:
The system automatically creates a service package for the submission and creates an empty record for the service of the submitted information to each opposing/responding party.
The system creates individual evidence records that resolution staff can mark as not accepted or referenced in the decision (with notes that get inserted)
The system sends email reminders of upcoming submission and service deadlines,
Depending on rules, the system automatically redirects the dispute into other processes or can abandon the file depending on the submissions and participation of the parties.
The applicant uses guided online (or office) processes to submit amendments to their dispute file
Behind the scenes and unseen by the user:
The system automatically creates an empty record for the service of the amended information to each responding party.
The system records the specifics of each amendment, including the automated handling of amend removed parties, or amend removed issues so that they do not appear in decisions.
The system records amendments that remove parties or issues that are made by resolution staff in a special way so that these changes are included in the decision.
All parties use guided online (or office) processes to indicate that the dispute notice and their associated submissions were served to the opposing parties.
Behind the scenes and unseen by the user:
The system records the details of service so that service can be included in the decision with the precision necessary under rules, acts or legislation
All parties use guided online (or office) processes to indicate and confirm their intention of participating upcoming scheduled events (if the dispute is assocaited to an oral process)
Behind the scenes and unseen by the user:
The system records the details of service so that service can be included in the decision with the precision necessary under rules, acts or legislation
The resolution event (e.g. hearing) is conducted and the following activities are completed in DMS by the resolution staff:
They confirm service of notice, amendments and supporting testimony and evidence.
They confirm the attendance of the resolution event (if a scheduled oral process).
They mark any testimony and evidence that they either want referenced in the decision (with an optional note)
They conduct the resolution session and record the specific outcomes of each issue or claim on the dispute file
They create an outcome document set on the dispute file, and select the specific document(s) they will be writing
They click a "generate" button on the specific document they want generated
They select the type of decision language they want generated "conversational", "strict act", or "basic"
Behind the scenes and unseen by the user:
The system creates an initial template based on the characteristics of the dispute file and the assocaited document
The system populates the repeatable sections of the document including; relevant file information, party information, disclosure and service information, party attendance and participation information, the matters in the dispute (issues and claims), relevant evidence and testimony, and preliminary matters.
The system uses the issue outcomes entered by resolution staff for each issue (i.e. granted, severed, dismissed, etc.) to automatically generate and insert analysis, conclusions, and order information - with highlighted areas that require human validation and placeholders where custom human generated content needs to be inserted.
The system generates a preview of the decision in DMS for the staff member to validate, including clear highlighting where any dispute file information was missing and the content could not be generated so that they can close the preview and add the missing information so that the entire decision is generated.
The resolution staff member validates the document preview and accepts the document to start editing in Microsoft Word.
The resolution staff use content highlighting and custom placeholders in the generated document to complete the human written elements of the decision.
When the decision is completed, the resolution staff initiate a provided script (in the ribbon of Word) that, in one click, saves the editable version, creates a PDF version for delivery, and creates an anonymized version for public posting.
The resolution staff opens a bulk upload view in DMS, and drags and drops each decision onto the appropriate upload slot to adds the documents to the dispute file.
The staff optionally marks the decision as "noteworthy" or "materially different" if required to have it highlighted for the community of practice.
Behind the scenes and unseen by the user:
The system automatically gathers the associated metadata from the dispute file and the text from the decision and creates a searchable decision
Where email delivery was requested by the participants in the dispute, the system generates a delivery email template, attaches the decision(s) and any instruction files, and automatically emails the associated decision(s)
The anonymized decision is now available for search in DMS.
External parties with similar dispute files can now access this decision and make better informed decisions on their issues and the associated information they submit to support their positions.
Staff working on similar dispute files can now access this decision instantly as an example for their own decision writing and/or learn from this decision as a community of practice if it is noteworthy or materially different.
In closing
I hope this article has provided you with new insights and approach ideas for automatically generating your own complex decisions and resolution documents. If you enjoyed this article, you can see all of our articles in our designing the future blog. Until the next article, keep on innovating and sharing your working solutions!
Mike Harlow
Solution Architect
Hive One Justice Systems, Hive One Collaborative Systems
©2022
Where can you get more information?
If you have questions specific to this article or would like to share your thoughts or ideas with us, you can reach us through the contact form on our web site.
If you are interested in the open source feature-rich DMS system that is the basis of the innovation in this article, visit the DMS section of our web site.
If you want to learn more about this solution, we offer live sessions and lectures that provide much greater detail (including diagrams, statistics and feature demonstrations) and allow breakout discussion around key areas of audience interest. To learn more about booking a live session, please contact us through our web site.
If you are an organization that is seeking information to support your own transformation, we provide the following services:
Current state analysis: where we evaluate your existing organizational processes and systems and provide a list of priority gaps, recommendations, opportunities and solutions for real and achievable organizational improvement.
Future state design: where we leverage our extensive technology, design, creativity and sector experience to engage in the "what is" and "what if" discussions that will provide you with a clear, compelling and achievable future state that you can use to plan your transformation.
Comments