Audit Orchestrator
User research | Design | Evaluation
Bringing intelligence, speed, and delight to a traditionally chaotic and dry process
DURATION
October 2019 - May 2020
MY ROLE
I was one of two designers during the conceptual work, and later the lead designer for the detailed research and design work.
TEAM
• Product owners
• Designers (2)
• Design sprint facilitator
• Sponsor users
• Business analysts
• Developers
CLIENT
Large, globally distributed IT organization
Highlights
• Used extensive co-creation with users to understand their needs and iteratively refine the design.
• Solved complex architectural challenges to provide consistency of experience across the audit and minimize development effort
• Created a prioritized roadmap for building out the tool and promote adoption.
Problem
Audits were difficult to coordinate, manually intensive, and lacking in transparency.
Project goals
• Eliminate reliance on team rooms and email for coordinating audit tasks.
• Provide a transparent, consolidated view of audit work.
• Make it easier to quickly identify the appropriate teams to participate.
• Leverage existing capabilities of a related sales transaction support platform, where it made sense to do so.
Challenges
• The audit data structure is complex and varies by phase.
• The solution was to be flexible enough to support other similar business processes.
• The UI design components for this project were also to be adaptable for improving an existing sales transaction support platform.
• The initial release was to take one quarter to build yet be capable enough to support a full audit.
• As a member of a small UX team, I was able to devote less than 50% of my time to the project, amidst other parallel projects.
To comply with NDA, I have intentionally hidden and replaced content in this case study.
Process
A collaborative design sprint resulted in a validated design concept, which I later extended and refined with strong involvement of users.
Discovery
We listened to users outline the challenges they faced with managing the many moving parts of an audit. Based on what we heard, we identified ‘how might we’ statements to create ideas around.
How might we:
• Allow the users an expert level of control?
• Ensure that the platform involves the right teams and focals based on the audit?
Methods
• Lightning talks / Interviews
• Competitive research
• Create tracking and communication that feels relevant and streamlined?
• Ensure clear communication of status and teams?
• Ensure speed and accuracy throughout the process steps?
• Automate and streamline steps needed to gather the required information?
Sprint
The team came together for a 4 day modified design sprint.
Lightning talks
We listened to the users describe their process, painpoints and ideas, while translating these into 'How might we' questions to innovate on.
User needs that we identified
Audits are complex undertakings, involving multiple teams, extensive gathering of documentation, and tight timelines.
Communication
Communication via team rooms and email among chains of participants is inefficient and difficult to follow.
Tracking
High volume of email difficult to monitor. Multiple audits need to be tracked concurrently and events responded to quickly.
Easy evidence sharing
Evidence stored in multiple disconnected repositories.
Speed
Searching for evidence to provide to auditors is slow.
Structure
Audit content is disconnected from the transaction data being audited.
Automation
A lot of manual effort is involved to copy auditor supplied information into the audit team room and to send notifications to obtain audit participants.
Compilation
Assembling a clear response to an audit question using information from multiple respondents is manually intensive.
Re-use
Copy-and-paste is extensive when setting up audits. Past audit responses are not easy to re-use.
Easy identification of participants
Its difficult to quickly identify who can best respond to each audit question and how to involve them.
Quality and accuracy
Manual effort is needed to check responses for quality. Respondents need support to convey an accurate, well organized story of what occurred within the audited business process..
User roles
Audit support
Coordinator for the audit, acting as focal point between the audit team and the quote-to-cash teams participating in the audit.
Goals
• Set up the audit with limited effort
• Involve the right participants
• Consolidate responses easily
Business operations
Back-office personnel that support quote-to-cash sales transactions based on established business processes, providing support for sales personnel.
Goals
• Provide clear responses to audit
• Track and respond to audit tasks
• Easily retrieve information requested by auditors
• Ensure quality responses
• Complete audit on time
• Share responses with auditors
Key painpoints
• Extensive manual and repetitive tasks
• Inefficient communication
• Difficult to know who to involve
Key painpoints
• Loss of visibility to audit tasks while managing their other work
• Evidence required for audit scattered among multiple tools
• May be wrongly assigned to audit tasks
Competitive research
We looked at a number of related tools, both commercial and internal, and identified ideas that might inform our solution.
Intelligent data mining
Dashboard of open/closed items
Composite overview of a task
Providing intelligent recommendations for who should participate in an audit by leveraging the inherent relationships between participants and business transaction being audited.
Highlight the events that need attention right now in a simple, clear manner.
View a comprehensive picture of a task in one place, including steps, status, documents, conversation and participants.
Design
We envisioned how the audit experience could be made simpler and more efficient.
Exploration
We individually sketched ideas for the UI and then reviewed them as a team to highlight concepts to pursue further.
Ideas we considered
Tiered view of audits
• High level status of current audits
• Audit tasks with their status, owners and due dates
Templates
Re-use questions and data structure across audits.
Task guidance
Utilize wizards to guide users in assigning roles and tasks, as well as gathering data.
• Single task details
Intelligent automation
Leverage AI and/or intelligent searches to help identify appropriate participants, gather data and answer repetitive questions.
My concept sketches
Leverage reusable templates
Auditor completes a form to filter and select from available templates that contain standard questions for the business (quote-to-cash support teams) to answer as part of the audit.
Early flow chart
Our early flow chart showed a wizard driven approach to guide the audit support role through the automated import of questions received from the auditors, followed later by the eventual extraction of the responses from the business to share back with the auditors. How to handle the variation in data and process that existed between phases of an audit would be addressed as part of the detailed design work.
Provide automated task support
On the left is a cognitive search that automatically gathers suggested matching documentation for answering each audit question . These can be added to a ‘shopping cart’ to build the response. User can also navigate to other related information.
Collaboration at the task level
A quote-to-cash user views the responses and documentation that they are assembling to address the auditor’s questions. Each auditor question can have its own threaded conversation among all participants, with accompanying alerts to draw users in when needed.
Prototype
I assisted with rapid creation of the first prototype by creating the detailed view of an audit task.
Methods
User reviews
We reviewed the prototype with users and obtained feedback to refine the design, followed by additional working sessions with users.
Results of early user reviews
Audit task board
Audit support role
Finding
Would like to search for occurrences of a data sample (selected business transaction examples) within an audit
Response
Added this function for first release
Quote-to-cash role
Finding
Need way to import documents resident in a separate application
Response
Included on product roadmap for a later release
Finding
Line of business needs to be visible in card or in a side information panel., as it helps to determine who to assign the task to
Response
Line of business added as field within the audit summary section
Task detail
Audit support role
Finding
Response section is of the primary importance and should be given the visual focus.
Response
Audit questions and responses was later combined into one section at top of the task details screen.
Finding
User needs an additional field to provide more description about the task. This would be an optional field. One purpose this can serve is to allow audit support to provide their translation of what the task is.
Response
Added field for this purpose.
Finding
One task can get split out to multiple duplicate tasks assigned to different individuals.
Response
Added support for flexible task assignment.
Detailed design
Coming out of the sprint, the details of the process steps were yet to be uncovered. Working regularly and closely with end users, I gained greater understanding of the audit process and the users’ primary needs. I set about to tame the audit process variability and multiple data layers, while solving the key technical hurdles with the help of the team.
Findings
Users need strong support for context switching
Coordinating an audit requires a great deal of cognitive support to be able to quickly resume tasks, remember contacts, and spot bottlenecks, despite a great deal of interruption and multiple events competing for attention.
Need to quickly assemble participants and data
Audits require quick, knowledgeable assembly of appropriate participants and information from a vast, distributed space.
Audit tasks are interrelated
Responding to audit task often requires participants to easily refer back to previous related tasks or audits.
Methods
Co-creation
User reviews
6 end users
Task observation
Secondary research
I looked at how users currently perform tasks and leveraged artifacts that they shared with me as I gained understanding of their needs.
To-be scenario map
We mapped the to-be experience across the phases of an audit for the key user roles, while mindful of painpoints and opportunities we identified.
Hill statements were created (a technique from Design Thinking) in order to summarize the primary value that the first release should deliver.
Key design challenges
How to provide a consistent data structure
In each phase of the audit, the auditors provided information (ex. questions, findings) that the business had to respond to. The type of information provided and how the business needed to respond to it varied by phase. This presented challenges in terms of both keeping the user experience consistent and minimizing development effort needed to support this variation.
Solution
I arrived at a multi-tiered solution whereby the phases of an audit would follow a request-task-subtask hierarchy. Tasks within a phase would in turn be structured as a parent-child hierarchy.
Rationale
This provided users with a consistent structure, layout and interaction model across the audit, while still supporting how each phase was conducted.
Making it easy to identify appropriate participants and relevant documentation
There are many relationship inherent in the audit space, such as between participants, business process topics of interest and process documentation. These can be difficult to keep track of manually. By leveraging these relationships, users can be provided intelligent task support through helpful suggestions at key moments, as shown below.
Solution
Use AI to take advantage of important relationships and patterns between audit participants, knowledge areas, business process, audit questions, and documentation.
Rationale
Users are challenged to locate the right audit participants and documentation from a wide search space amidst tight timelines.
Providing information back to the auditors
After responses to audit questions are compiled and reviewed, they are shared with the auditors. However, access to the tool by the auditor role was not in the planned scope for first release. So, a method to share back the information had to be identified with technical feasibility as a consideration.
Solution
I proposed that we publish responses to audit as a read-only view within the application and provide access to the auditors.
Rationale
This would be a more streamlined process than exporting and sharing, would potentially save development effort, and set the stage for the auditors to become a fully supported audience in the future.
Supporting easy navigation
Audits had to be tracked and participated in at multiple levels of granularity.
Solution
I created an information architecture that clearly organizes the information, whether the focus of the user is across audits or within a single audit, allowing for easy tracking of status and prompt navigation to areas needing attention.
View any level of the audit
Users could track the audits they were managing or participating in, understand the tasks on their plate, and dive into perform specific tasks. Tasks were aligned to the audit requests that generated them and could themselves spawn supporting sub-tasks.
Rationale
It was apparent from the research that audits had multiple levels of activity, any one of which could be of focus at any given moment. Users needed support to navigate this complex space.
UI component identification
One of the goals of the project was to be a potential source of UI components for a new design system to benefit a large related platform for sales people that our department also supported. In this way, design and development could be performed efficiently across the suite of applications, while providing a more consistent and effective user experience.
Advocated for creating a component library
Shared an example of how UI components could be identified from this project to benefit both of the applications that we supported.
Supporting key tasks
Create an audit
This form allows users to set up new audits based on the announcement letter that they receive from the auditors.
Organized the form based on topic
This breaks up the form into manageable pieces that are easy to digest.
Included the most important fields
With the help of users, we identified the core set of fields that were most important for describing the nature of the audit and guiding its planning.
Manage an audit
This screen provides users with status view of the requests that have arrived from the auditors. From here users can access the details of a particular request, as well as the tasks that are being performed to fulfill the request.
Provided context and status
Working with users from the audit support role, I gained understanding of the challenges they face as they coordinate the many tasks and participants that are involved in an audit. I created this view to provide them a high level overview of the audit’s focus, as well as the progress being made in fulfilling the incoming requests. By switching tabs, they could also view who is participating, and log important notes about lessons learned or new questions encountered.
Create tasks
Each request received from the auditors describes specific types of information that the business team needs to provide in support of the audit. The audit support team that is coordinating the response to the audit can create tasks for the business operations teams to gather this information.
Supported flexible task creation
To fulfill a request received from the audit staff, the work may need to be divided among multiple business teams. Using object selection and a slide-in panel, I created a way for audit support to create and assign tasks in a flexible manner, while retaining visibility to the details of the original request.
Respond to audit tasks
Business teams use this screen to provide responses to the audit tasks to which they are assigned. In doing so, they can create sub-tasks to obtain help and engage in conversation with audit support and others when needed.
Featured key summary data
Users needed to track both the due date for delivery to the auditors, as well as internal due date for readying the response. A notes field was added to allow audit support to include additional context or instructions about the task.
Narrative responses supported
It was important that users could tell a story within each response by interweaving text with files and images. I worked with the developer to come up with a feasible technical solution to support this for the initial (mvp) release.
Provided a roadmap for the future
Time was limited to develop the first release, yet we needed to address the full end-to-end process and gain adoption. Set out a course for evolving the product, by first creating a strong foundation, both in the UI and in the data structure, upon which increasing levels of support could be added.
Results
Set to deploy the MVP release, learn more and grow from there.
• The application was scheduled to be deployed in October 2020.
• The mvp was well balanced, emphasizing the important early functions needed to save users time and effort, as well as gain adoption. It also provided a platform and roadmap for further extending the tool for even greater productivity.
• Although the role of auditor was not in scope as a supported user role for first release, the auditors liked the design enough to want to have direct access to the tool in a future release. Based on the potential future roadmap for this tool, auditors could eventually self-service the conducting of an audit with minimal support from the ‘audit support’ role.
• The information hierarchy that I created was key to the project during showcases with additional business units to which the tool was looking to expand.
• The business case estimate of savings by the application was 11,000 hours/year.
Next steps
Due to the impact of Covid-19 business interruption, my project role ended early before I could complete all of the activities. My next steps would include:
• Final refinements to the visual design layer
• Usability test of first release
• Identification of kpi’s to measure performance of the deployed application
What others say
“Very grateful to you for taking a leading role with design research, crafting prototypes and driving alignment sessions. You’re making a very big impact on the success of this project.”
Lee Duncan
Enterprise Design Sprint Leader
“Thank you Steve for your keen attention to the users and dedication to the [] initiative. This opportunity has enabled the team to fully embrace design methodologies and best practices. It is amazing to see our stakeholders’ growth mindset burst wide open as they thoroughly enjoy this co-creating experience with us… the deliverables are yielding a completely altered way of working, it is collaborative & effective!”
Patty Lally
Q2C Transformation Leader
Reflection
It was a great experience to work on this project. I am grateful for the business owner providing us this opportunity to lead with design from the start and the confidence they showed in us, as well as for the talented team that I collaborated with on this effort.
Here are some takeaways that I have:
Make more extensive use of visual process flows
Particularly with a complex process like that of an audit, where several roles are involved at specific points in the process flow, visualizing this interplay makes its easier for both the designer and the rest of the team to comprehend the flow of activity to be supported.
Seek to observe users in person
In the environment I was working in, budgets for travel were slim, and in person user observation was not the norm. While I had users step me through tasks on a remote basis, I could have learned even more through a wider scope observation of their work context, either in person or remote, using the methods of Contextual Inquiry. In the future, I will place more focus on this type of research.
Acknowledgements
Business owner
Patty Lally
Product manager
Francesca Zafferoni
Workshop facilitator
Lee Duncan
Co-designer (early phase)
Daniel Benedict
Business analyst
Maria Joao Pereira
Developer
Ernesto Islas
Developer
Francisco Javier Sainz Ozegueda