Advanced analytics is the examination of data or content autonomously or semi-autonomously using sophisticated tools and techniques, usually beyond the scope of traditional business intelligence tools to discover deeper insights, generate recommendations and make predictions. Advanced analytics include techniques for data or text mining, pattern matching, machine learning, visualization, forecasting, sentiment analysis, semantic analysis, network, and cluster analysis, graphic analysis, multivariate statistics, complex event processing, and neural networks. Thoughtful use of advanced analytics can transform the efficacy and efficiency of the critical work performed by public sector agencies. It can streamline processes and improve outcomes to make the best use of limited resources.
Where does the bias lie?
Fair treatment is the core of the mission of public agencies. However, it is hard to preserve when decisions are based on algorithms designed on biased datasets. A bias might creep in because of training data, which includes biased human decisions, social inequities, or historical decisions. It can even stem from a flawed data sampling. The under-representation or over-representation of certain groups of people can cause such flaws. Unless modules are developed with extreme care, an amplification of bias and disparity related to socioeconomic status, race, and ethnicity can happen.
How do you mitigate the risk of biases?
- Hold someone accountable –
A leader of the unit can be held responsible for model risk management. The chief risk officer or CRO carried such a responsibility in financial and other regulated industries. In the case of federal agencies, the lack of robust enterprise risk management structures causes the chief information officer or chief data officer to hold power. Senior agency leaders must translate agency values and missions like equity and diversity into goals for AI risk management leaders. It requires both units to have some in-depth discussions. For instance, a goal to avoid recruitment gender bias can be ensured when the algorithm selects an equivalent number of men and women based on various job parameters.
- Develop and communicate a clear set of standards –
Every agency must establish a clear set of analytical standards and practices that are codified, adhered to, and widely communicated. This includes having clarity about a specific problem that the model addresses or a rigorous peer review process or any other empirical review of outcomes to detect some unintended bias. A good practice to help counter bias is to make sure of having diversity in analytics teams. Bias in training data can be easier to spot when someone with relevant life experience looks at it.
- Build a model-risk-management infrastructure –
The best principle to follow while building a model-risk-management infrastructure is to challenge models as they are being built and implemented continuously. An effective model governance program must include the following –
- Clarify definitions and other model governance parameters for risk management.
- Create and maintain an inventory of models across the agency.
- Develop and maintain standard workflow and ensure best practices for bias reduction.
- Assess the materiality of the model to establish the level of oversight required. Some models might require an audit.
- Create algorithm review panels –
Many agencies use acquisition revenue boards to assess acquisitions over a certain threshold. Similarly, an algorithm review panel can be created to reduce the risk of AI and ML projects that carry high risks. The panel can include both technical and non-technical people having various roles to determine the model’s potential impact on equity. The collective decision of the model’s fairness and assurance of other stakeholders to view the actual outcome similarly can be the goal of such a review panel. For high stake applications, outside academic and industry experts can audit models for various biases. Such audits reveal hidden problems in the system, create transparency, and increase the trust of the public in the government.
- Appoint an ombudsman –
An ombudsman is an official that is appointed to investigate complaints by individuals against any company or any organization, especially a public authority. An analytical ombudsman can act as a point of contact for external stakeholders who wish to raise issues. This is inspired by tax monitoring organizations that hire an official to make sure that taxpayers are treated fairly and their appeals are handled by the ombudsman on their behalf and they have an easy route to report problems they encounter.
- Strategize at an enterprise level –
Advanced analytics is a lot more effective when embraced throughout the agency instead of being maintained by a handful of advocates. Having a centralized effort to identify and prioritize high-impact use cases systematically, support them with funding, and communicate their progress, the professional standards raises and lessons learned across the agency can improve effectiveness. Greater visibility can improve the peer review system, promote a widespread adaptation of best practices across the agency and build momentum and demand for advanced analytics solutions.
Thus, advanced analytics techniques can provide the public sector with the ability to transform itself through artificial intelligence. By adopting practices that minimize the risk of bias and other forms of unfair treatment, the leaders of the public sector can empower their institutions to inculcate mission-enhancing AI and ML approach by generating public confidence in the use of analytics to improve outcomes by the government.
