Utilizing the power of AI comes with great responsibility, especially when only 35% of global consumers trust how organizations implement AI technology.
As AI spreads across different sectors across the U.S., responsible practices are vital for building public trust and ensuring a positive societal impact.
In an interview with agency directory DesignRush, Head of Responsible AI at Fractal Analytics Sray Agarwal shares how organizations can responsibly use AI in their business operations.
Who Is Sray Agarwal?
Sray is an experienced AI and analytics professional who currently serves as the Head of Responsible AI at Fractal Analytics where he leads the charge in ethical AI initiatives. He has pioneered Responsible AI frameworks for major U.K. and U.S. banks. Recently, he authored a book on Responsible AI published by Springer and has taught AI at renowned institutions like NUS, ISB, and Jio Institute.
Like any other technological tool, people can also misuse or abuse AI tools.
In light of this, Sray highlights a few ways in which AI can be used irresponsibly or unethically by incorporating biases that result in unfair treatment of certain groups:
- Bias in loan approvals: AI models used by financial institutions might be biased against single women, leading to their loan applications being unfairly denied
- Discrimination in medication pricing: AI can be used in the healthcare industry to set prices. If the models are biased, they might set higher prices for medications for people of color
- Inequitable insurance services: Insurance companies might use AI to determine eligibility for services. Biased models could result in denying insurance to individuals from economically disadvantaged backgrounds
He shares real examples where AI has been abused, mentioning that automated systems have promoted products with political or malicious agendas, with companies being caught in such practices.
A tech company also faced a massive data breach, exposing millions of users' personal information, sparking public outrage and concerns about data privacy in AI systems.
“The incident served as a stark reminder of the dire need for Responsible AI practices to protect our sensitive information,” he notes.
According to him, another example is differential pricing based on ZIP codes, where areas predominantly inhabited by certain religious or ethnic communities face higher prices.
Since Fractal places a significant emphasis on ethical AI:
“When we began developing AI, we recognized the imperative for responsibility across all AI endeavors. Whether embarking on AI projects or creating AI-based accelerators, we insisted on a human-centric approach, integrating appropriate checks, balances, and guardrails."
He adds that the company started with small guardrails but quickly saw the need for comprehensive AI best practices to educate clients and the community about robust governance.
This focus grew with increasing regulations and government discussions on privacy and ethics, including strict guidelines from the White House amid rising concerns about AI bias.
Additionally, Sray emphasizes that a few more events connected to face recognition disparities and stereotypical biases in generative AI have driven ongoing efforts to enhance and strengthen responsible AI practices in recent years.
“Ethical AI functions like a seat belt in a car – it's essentially a safety feature. It ensures that AI doesn't discriminate among people and operates transparently and explainable," Sray tells DesignRush.
There's accountability for every action, and personal data remains secure with complete control over privacy. Users have authority over what information is shared. AI won't produce incorrect results, especially in Generative AI, and if it does, there are checks and balances in place.
Toxic prompts and answers that could harm sentiments or well-being aren't permitted. AI remains stable, providing consistent answers with repeated runs rather than unpredictable or unstable outcomes."
He adds that raising awareness is vital and the company does that through discussions, articles, videos, blogs, research papers, international forums, and client and colleague interactions.
AI can be a perfect brainstorming partner — Watch this episode of the DesignRush Podcast to learn more:
Fractal implements its Responsible Artificial Intelligence (RAI) framework with its projects across diverse sectors to promote ethical, inclusive, and transparent practices.
“The framework addresses sustainability, human-centric well-being, privacy, accountability, explainability, hallucination, and numerous other factors, comprising more than 20 parameters or principles,” Sray explains.
Fractal recommends a discovery session to understand specific requirements after which Responsible AI is integrated into the client's ML Ops or AI Ops workflow at different stages of development and deployment.
They also encourage clients to establish an RAI committee for manual or semi-automatic oversight, ensuring timely monitoring, audits, and sign-offs. This committee should oversee training programs and manage change to minimize workflow disruptions.
If you're looking for a professional agency to help improve your brand engagement and loyalty be sure to check DesignRush's listing of top AI companies.
Beware of Potential AI Dangers in FinTech
“The financial sector is heavily regulated, yet there have been instances where AI, intentionally or unintentionally, has produced discriminatory outcomes,” Sray explains.
He brings attention to a major bank that has been charging higher interest rates based on a person's ethnic background and a car engine size that was used as a proxy for determining gender, resulting in discriminatory insurance pricing.
“Given the significant impact of financial decisions on individuals, the sector must exercise heightened caution. This entails thorough scrutiny of every AI model using appropriate privacy algorithms, policies, and measures to detect and mitigate bias and discrimination.
Transparency, accountability, and the establishment of an Ethics Committee are essential to ensure fair decision-making. The consequences of biased decisions can be profound, potentially altering an individual's life irreversibly,” he adds.
According to Sray, Fractal uses the following metrics to assess whether opportunities are distributed fairly:
- Do all groups have equal opportunities?
- Is the likelihood of receiving a positive decision similar across groups?
- Are demographic factors such as gender or marital status influencing final decisions made by the model?
He notes that their clients have been supportive of addressing model bias and complying with regulations. With new laws like the EU AI Act, there's been a shift towards proactive checks instead of reactive measures.
“Regardless of legal mandates, we emphasize the importance of addressing bias ethically. Even if regulations don't explicitly require it, we provide clients with recommendations on how to mitigate bias and ensure fairness toward all individuals,” he adds.
To learn more about the challenges posed by generative AI, watch this episode of the DesignRush Podcast:
Ethical AI Opens Business Opportunities
Sray argues that implementing ethical AI practices has greatly improved transparency, compliance, and fairness in the financial services industry. The main advantages include:
- Increased adoption of AI models within companies. Employees trust these models more because they have a clear understanding of how they operate and what factors influence their decisions
- New business opportunities. With the removal of biases and the adoption of neutral perspectives, financial institutions can now serve a broader range of clients based on merit rather than demographic factors
“Ethical AI has not only improved trust in AI models but has also enabled financial institutions to make more informed and inclusive decisions, leading to better outcomes for both businesses and clients,” he adds.
For businesses across sectors looking to ethically implement AI in their operations, Sray advises the following:
- Conduct a discovery phase to understand the organization's needs, identify necessary principles, and establish an overseeing AI committee that will ensure all initiatives are properly documented, audited, and supervised by humans
- Develop comprehensive AI principles that apply across all use cases and departments
- Mitigate resistance from data scientists and developers by avoiding major changes, ensuring a smooth transition to ethical AI
Our discussion also covered how AI advancements impact the U.S. economy, particularly in terms of job creation, efficiency gains, and the potential for economic inequality.
“When considering the impact of advancements in AI technology on the global economy, it's essential to reflect on historical precedents," Sray notes.
"Just as the introduction of the steam engine and automation in the banking sector initially raised concerns about job displacement, we've seen that these innovations ultimately led to the creation of new jobs and sectors, fostering economic growth.
Similarly, advancements in AI are poised to create new job opportunities. By enabling faster decision-making processes, AI technologies can improve productivity and allow businesses to serve more customers.
This increased demand for services can stimulate economic growth and create a need for both new technologies and skilled workers to support them.
By streamlining processes and expanding access to goods and services, technology has facilitated economic empowerment and accelerated progress. While there may be apprehensions about job displacement, a broader perspective reveals that AI advancements have the potential to fuel economic growth and enhance prosperity.”
