Share

Consumers in many African countries are not lagging behind in adopting new AI technologies. Charles Asiegbu and Chinasa T. Okolo for Brookings write that existing data protection regulations can be integrated into the AI governance frameworks that African countries seek to develop (link).

In a new report titled “Driving U.S. Innovation in Artificial Intelligence: A Roadmap for Artificial Intelligence Policy in the United States Senate,” the Bipartisan Senate AI Working Group, led by U.S. Senator Chuck Schumer, recommends at least $32 billion per year for AI innovation. The report is a culmination of a series of AI “insight” forums involving AI experts from industry and academia (link).

In addition to funding to enable cross-agency R&D, semiconductor R&D, AI Grand Challenges, and the National AI Research Resource, the report emphasizes the need for workforce up-skilling and improved immigration for STEM talent while ensuring AI systems comply with existing laws, especially in consumer protection and civil rights.

Critics argue that the report prioritizes industry and defense interests over public benefit, civil rights, and corporate accountability, lacking concrete measures to address AI's potential harms, such as bias and discrimination. See Tech Policy article.

While the report advocates for a risk-based approach to AI, comprehensive data privacy laws, and measures to manage national security threats from AI, including export controls and safeguards against bioweapons, it is not legislation and thus not a concrete regulatory intervention. Personally, I found the written statements from the expert briefings to be the most informative.

Kate Bartlett writes for VOA News about South Africa’s investment in AI expertise through the establishment of sector-focused AI hubs. More broadly, the article highlights the increasing interest of the U.S. and China in investing in AI on the African continent (link).

An emerging theme of U.S.-China collaboration on AI is capacity development for developing countries. Xue Lan, director of Tsinghua University's Institute for AI International Governance, recently said, “Another potential area that the U.S. and China can work together on AI is how to increase, to improve the capacity of developing countries in accessing and using AI technologies.”

The UK AI Safety Institute released a new tool for evaluating large language models. I’ve written about the AI Safety Institute in previous newsletters, and the newly published tool, Inspect, is designed to enhance safety evaluations by providing a standardized, open-source platform for assessing AI models' capabilities and security (link).

The Council of Europe has been proactively addressing the challenges posed by advancements in AI, particularly concerning human rights, democracy, and the rule of law. This effort culminated in forming the Committee on Artificial Intelligence (CAI) to develop a Framework Convention to ensure AI activities comply with these fundamental principles.

The resulting Framework Convention is heralded as the “first-ever international legally binding treaty” aimed at ensuring the respect for human rights, the rule of law, and democracy standards in the use of AI systems. The treaty was recently adopted in Strasbourg and aligns with existing international and domestic legal obligations in Europe, aiming to mitigate risks such as discrimination, undermining democratic processes, and violations of human dignity. See the press release for more details.

Helen Toner and Tasha McCauley, former board members of OpenAI, in The Economist, argue that private companies cannot be expected to align their profit motives with the public good when developing technologies like AI, necessitating effective governmental regulation (link). They explain how OpenAI's experiment in self-governance, despite its innovative structure, ultimately failed to uphold its public-interest mission, highlighting the need for external oversight.

Benjamin Cedric Larsen and Sabrina Küspert for Brookings compare and contrast the EU and U.S. approaches to regulating AI (link). They argue that the EU's approach has been more comprehensive and impactful. For example, the EU AI Act imposes legally binding regulations on providers of general-purpose AI within the EU, whereas the U.S. executive order primarily sets guidelines for federal agencies and imposes limited regulations on private entities. They claim that the voluntary nature of such guidelines aligns with the U.S.'s preceding preference for industry self-governance. However, as international efforts come into focus, cooperation between the EU and the U.S., together with other G7 countries, could serve as a model for aligning international efforts.

A newly enacted law in Colorado mandates that developers and deployers of high-risk AI systems use reasonable care to avoid algorithmic discrimination, with a rebuttable presumption of compliance if they meet specific requirements. Developers must disclose information about their AI systems, provide documentation for impact assessments, publicly summarize risks, and inform the attorney general and deployers of any risks within 90 days of discovery (link, link).

Sony Music in a recent statement prohibited unauthorized text or data mining, web scraping, or similar uses of their content for AI training or commercialization (link).

At the heart of the current wave of innovation in generative AI is massive amounts of data collected from the web. In a recent article for the NY Times, Cade Metz discusses how companies like OpenAI, Google, and Meta have scoured every corner of the internet for reputable English-language text to train their AI models. As these companies face a growing data shortage, they resort to various methods to gather more data, including transcribing YouTube videos and exploring the use of synthetic data. Metz highlights the legal and ethical dilemmas many AI companies encounter as they push the boundaries of data usage and copyright law to stay ahead in the AI race (link).

Similarly, obtaining high-quality training data for AI models in non-English languages is incredibly challenging. A recent discovery following the release of a new model from OpenAI revealed that the model’s Chinese token library was polluted with phrases from spam websites. Zeyi Yang discussed this in more detail in MIT Tech Review.

U.S. Secretary of Commerce Gina Raimondo recently announced plans to launch a global scientific network for AI safety, as well as establish the U.S. AI Safety Institute (AISI). According to the vision statement,

AISI “will enable more rigorous assessment of AI risk; more effective internal and external safeguards for AI models, systems, and agents; greater public confidence; and ultimately wider and more responsible development and adoption of AI.”

Technology companies do not seem to be shying away from military applications of AI. Following the inaugural “AI Expo for National Competitiveness,” hosted by the think tank created by former Google CEO Eric Schmidt, Caroline Haskins writes for The Guardian about how tech companies are discussing the applications of AI in military settings (link).

The big AI risk, according to Tyler Austin Harper, isn't the sci-fi scenario of robots becoming smarter than us and taking over. It is something closer to home and happening right now: the way we're letting AI take over basic life skills. Harper points out that while AI tools like dating apps, meal-kit services, and personalized recommendations are sold to us as ways to make life easier, they actually make us more dependent on algorithms for things we used to do ourselves—like socializing, thinking, and creating. This, he says, could lead to a future where we struggle to have normal human interactions without relying on AI, which would fundamentally change what it means to be human. Link to article on The Atlantic.

In an insightful interview, Linda Moore, CEO of TechNet—a tech lobbying organization—highlights the importance of educating both the public and policymakers on AI while ensuring that new regulations do not stifle innovation or drive businesses away (link).

In a powerful personal account, former content moderator for Facebook and Instagram in Kenya, Jatani Hussein, shares the traumatic experiences of reviewing thousands of graphic and violent posts daily. Employed through outsourcing companies, Jatani and his fellow moderators faced severe mental health challenges while scrubbing harmful content off the internet. Last week, Kenyan President William Ruto visited the White House. Jatani urges consideration of the risks faced by content moderators—a role that continues to be in demand as more companies try to ensure their AI models are used safely.

Thanks for reading!

Share

Subscribe now

Key Reads

When a new technology emerges, there are opportunities and risks. Generative AI is no different. The National Institute of Standards and Technology (NIST) published a report describing a set of 12 distinct risks for the widespread use of generative AI technology, including the spread of misinformation, data privacy violations, and the potential for AI to generate harmful content or biased outputs (link).

The report is one of four “guidance” reports published by NIST. Another report focuses on guidelines for documenting the origin and history of content, using methods like digital watermarking and recording metadata (link). The third advocates for collaborative efforts with international partners to develop and implement AI-related consensus standards (link), and the final report is a specialized extension of the Secure Software Development Framework (SSDF) (link) tailored specifically for generative AI and foundation model development. This extension provides a set of augmented recommendations and considerations that align with established SSDF practices (link).

See Tech Policy for a summary of each report (link).

A new bill introduced by U.S. Senators Mark R. Warner and Thom Tillis, co-chairs of the Senate Cybersecurity Caucus, aims to improve the tracking and processing of security and safety incidents and risks associated with AI, including enhancing information sharing between the federal government and private companies (link, link).

A new Florida law mandates that political advertisements using generative AI to depict unreal events prominently display a disclaimer, with violations treated as first-degree misdemeanors, effective July 1, 2024 (link).

Jeremy Howard, CEO of Answer.AI, critiques California’s SB-1047 (link), arguing it could stifle AI innovation by imposing harsh regulations on open-source development and small businesses. He suggests focusing on regulating AI applications in high-risk areas rather than development (link).

A new Colorado law aims to protect the privacy of neural data, increasingly collected by consumer neuro-technologies (link).

Tina Stowell, a member of the House of Lords in the UK Parliament, expressed approval for the UK government’s commitment to AI safety and support for startups but criticized its handling of copyright issues and lack of clear policies on AI standards. She emphasized the need for more decisive actions to protect the integrity of British businesses and innovation in a recent letter. See longer discussion in The Register (link).

The non-profit privacy group noyb, led by activist Max Schrems, has filed a complaint with the Austrian data protection authority accusing OpenAI’s ChatGPT of violating the EU's General Data Protection Regulation (GDPR). The complaint centers on ChatGPT’s generation of inaccurate personal data, specifically incorrect birthdays, and its failure to correct or delete such data upon request (link). This issue is part of broader concerns in Europe, as evidenced by Italy’s temporary ban on ChatGPT and the formation of a task force by the European Data Protection Board to address these challenges.

At the AI safety summit at Bletchley Park, technology companies initially agreed to allow the UK’s AI Safety Institute (AISI) to conduct pre-release safety testing on new AI models. However, six months later, this agreement appears to have largely failed to materialize, according to a new article by Vincent Manancourt, Gian Volpicelli, and Mohar Chatterjee for Politico (link). Despite the UK government’s claims of commencing such testing, most new AI models, like Meta’s Llama-3, have not been subjected to pre-release checks by the AISI. This situation highlights the challenges of enforcing voluntary compliance in the absence of stringent legal requirements and the reluctance of AI firms to share sensitive technologies due to fears of competition and jurisdictional issues.

Nonprofits backed by technology billionaires, such as the Center for AI Policy and Center for AI Safety, have begun lobbying in Washington to address the existential risks of AI, writes Brendan Bordelon for Politico (link). According to Brendan, these nonprofits advocate for regulations that could impose severe liabilities on AI developers and potentially halt high-risk AI projects. However, critics argue that these efforts might stifle smaller AI companies by raising entry barriers and focusing too much on catastrophic scenarios.

The use of AI to create compelling propaganda is a key risk. A recent survey experiment conducted with US participants examined the persuasiveness of news articles produced by foreign propagandists compared to content generated by a large language model (GPT-3) (link). The study found that both the original propaganda articles and those generated by GPT-3 were highly persuasive. While GPT-3-generated content was slightly less persuasive on average than the original propaganda, human involvement through editing prompts and curating output improved its persuasiveness.

During the recent 2024 China-Africa Internet Development and Cooperation Forum, collaboration between African countries and China on AI governance was one of the key themes. The call to action includes strengthening policy dialogue, promoting technology research and application, fostering industrial cooperation, enhancing talent exchanges and capacity building, and bolstering safeguards for cyber and data security (link).

An article by Alex Hern for The Guardian discusses the emergence of distinct linguistic patterns, dubbed “AI-ese”, in responses generated by AI chatbots (link). Alex highlights the telltale signs of AI-generated content, such as exaggerated politeness and aversions to brevity, alongside idiosyncratic linguistic tendencies like the overuse of words like “delve” (a recent source of controversy when a famous venture capitalist, Paul Graham, alluded to ignoring a message that included the word “delve”).

Labelling for AI chatbots, including large language models is often outsourced to cheaper labor markets. This implies their linguistic patterns can mimic those of the labelers. For instance, the overuse of “delve” in ChatGPT’s responses may reflect the linguistic norms of Nigerian English, where the word is more prevalent. Alex’s article underscores how the blurred lines between AI-generated content and human speech pose implications for language dynamics and societal perceptions of linguistic authenticity. Shout out to my senior colleague Kush Varshney for sending this article my way.

A new analysis by the Brookings Institute shows a 1500% increase in the potential value of US Department of Defense contracts related to AI between August 2022 and August 2023 (link).

Belinda Cleeland, Maxime Stauffer, and Malou Estier provide a summary and review of the Global Digital Compact (GDC) Zero Draft. The Zero Draft of the GDC is a set of principles and proposed actions for global governance of emerging technologies, particularly AI, set to be adopted by member states of the United Nations (link).

Opportunities

A new program by NIST called the GenAI evaluation program is seeking approaches from academia and industry on ways to discriminate between synthetic and human-generated content in text-to-text and text-to-image modalities. Registration is currently open (link).

Can wisdom and philosophy be automated? AI Impacts announced an essay competition focusing on the automation of wisdom and philosophy. The aim is to explore the potential impact of advanced AI on decision-making processes and philosophical understanding. There is a total prize pool of $25,000 (link).

Thank you for reading!

Victor.

Share

Let's dive right into the key reads and news I've come across in the past two weeks.

Key Reads

A recent article in MIT Tech Review by Zeyi Yang argues that China's approach to AI regulation is currently in a phase of lax regulation, due to the Chinese government's interest in the development of China’s AI industry (link). The article posits that Chinese AI regulations are looser than those in the US and Europe when it comes to preventing AI from causing harm, except in areas related to freedom of speech and content control. However, if there is a significant misuse of AI that “threatens social stability,” Angela Huyue Zhang, a law professor at Hong Kong University who is quoted, predicts that the Chinese regulatory pendulum will swing to the harsh side again, and it will be quick and unexpected.

Elections are being held in at least 64 countries globally this year, and the use of AI for spreading misinformation is top of mind. Rest of World has launched an AI elections tracker to track incidents of AI being used in elections globally—for campaigning, spreading misinformation, and creating memes (link).

MLCommons is a consortium of organizations in the technology industry and academia, built on a philosophy of open collaboration to make AI systems better for everyone. The MLCommons AI Safety working group has published a new benchmark to evaluate the safety of large language models (LLMs) (link, link). The benchmark is designed to measure when LLMs respond affirmatively when prompted according to seven hazard categories, including crimes, child sexual exploitation, indiscriminate weapons, suicide, self-harm, and hate. This is an important milestone that coincides with the release of Llama-3, a new LLM by Meta, which is touted as the most capable openly available LLM to date.

In May, the South Korean capital, Seoul, will host the next AI Safety Summit, following the inaugural event that took place in the UK last November. The country recently announced a $6.8 billion commitment to boost AI chip development and production (link).

Acceptable use policies for a digital product or service are often used to articulate how the service provider may restrict what users can and cannot do with their products and services. A recent article by Kevin Klyman catalogs the acceptable use policies for large-scale AI models (foundation models) by 30 technology companies. The article highlights the differences in how developers articulate and enforce acceptable use policies, which affects what content is prohibited and how the use of foundation models may be restricted in certain industries (link).

A recent panel discussion on the newly published 500-page AI Index report highlights emerging topics around AI governance and policy. The AI Index is an independent initiative at the Stanford Institute for Human-Centered AI that tracks, collates, distills, and visualizes data related to AI. The discussion highlighted how the industry continues to dominate frontier AI research (over academia), the shifting geopolitical dynamics of AI governance towards constraining or controlling AI, AI's environmental impact, and the role of open-source development in countries seeking to maintain leadership in AI development (link).

The Trade Union Congress (TUC), a federation of trade unions in the UK, recently published a bill to encourage the UK government to take a firmer stance on regulating AI. Key provisions include mandatory transparency for AI systems, including the ability for them to explain their decision-making processes, along with detailed risk assessments before and after deployment (link).

The bill also seeks to enforce strong consultation rights for workers and unions, establish a register of AI systems, and guarantee the right to human review. Additionally, it addresses data rights, bans harmful emotion recognition technology, and includes anti-discrimination measures by reversing the burden of proof for AI-related discrimination claims.

A newly introduced bill by Rep. Adam Schiff of California would require AI companies to disclose which copyrighted works were used to train their models (link).

A new report authored by a collaboration of prominent national security and cybersecurity organizations from several countries provides “Best Practices for Deploying Secure and Resilient AI Systems.” Organizations involved in the report include the US Federal Bureau of Investigation (FBI), the Australian Signals Directorate’s Australian Cyber Security Centre (ACSC), the United Kingdom’s National Cyber Security Centre (NCSC-UK), and their counterparts in Canada and New Zealand. The guidelines include securing the deployment infrastructure for AI systems, monitoring models for unauthorized changes or unexpected modifications, and limiting access to the model weights, among other practices (link).

The Federal Office for Information Security in Germany also published a report titled “Generative AI Models: Opportunities and Risks for Industry and Authorities”. Opportunities include the use of language models to examine existing software for known security vulnerabilities, among others. The report also highlights 28 different risks, with examples including information extraction attacks—where a malicious user attempts to reconstruct an LLM or gain information about its training data, evasion attacks—where a malicious user modifies the input to an LLM in such a way that the LLM's behavior is deliberately manipulated, and poisoning attacks that aim to induce a malfunction or performance degradation by poisoning the targeted model (link).

Thank you for reading!

Victor.

Key Reads

Nigeria’s Ministry of Communications, Innovation and Digital Economy, is bringing together 120 AI researchers to co-create a national AI strategy (link). Mauritius was the first country in Africa to publish a National AI Strategy in 2018, and Rwanda and Egypt have followed suit. Several other countries, including Ghana and Tunisia, are in various stages of the process.

Microsoft recently published a report on “policy frameworks” for governing AI in Africa, which highlights several of these efforts and proposes a legal and regulatory blueprint for AI governance on the continent. Key themes of the proposal include: the need to define a class of high-risk AI systems based on their capability, autonomy, and potential for large-scale harm; the role of governments in mandating mechanisms in such systems designed to detect, avoid, and disengage from unintended consequences; and the proposal that such AI systems should only be deployed on licensed AI infrastructure, thereby offering an additional layer of human oversight (link).

Most generative AI models are trained on text, image, and video data aggregated from the web. Christo Buschek and Jer Thorp investigate the curation and labeling process of LAION-5B, one of the largest and most commonly used datasets, and reveal several critical issues. The presence of illegal material in LAION-5B highlights the legal and ethical challenges in managing large datasets. Due to its sheer size, human curation is impractical, leading to reliance on automated processes that often introduce biases. These biases can be amplified when algorithmic data filters, as is commonly the case, are used (link).

The European Parliament released a briefing titled “What Think Tanks are Thinking” that references several position articles on AI and AI governance from a broad range of think tanks (link). Here are a couple of them I found interesting:

Licensing AI is not the answer—but it contains the answers (Tom Wheeler, Brookings Institution): While licensing AI efforts above a certain scale of capabilities may address some concerns, it is insufficient for effective oversight; instead, establishing standards and a new federal agency to oversee and enforce these standards is essential for governing AI in the public interest, ensuring safety, and fostering innovation and competition across all scales of AI development (link).

The AI Election Year How to Counter the Impact of Artificial Intelligence (Katja Muñoz, German Council on Foreign Relations): The strategic manipulation of information through generative AI poses a significant threat to democracy, particularly in an “AI election year” like 2024 with over 70 elections worldwide. This necessitates an assertive defense strategy and collective efforts from tech companies (e.g via content watermarking), social media platforms (e.g via better identity verification), Germany, the EU, and NATO to implement solutions, legislative measures, and tactics to combat these threats (link).

The U.S. government is implementing new, binding requirements for U.S. agencies to prevent discriminatory use of AI, including measures such as allowing travelers to refuse facial recognition scans at airports, establishing transparency and oversight for AI tools used by the government, and aiming to influence the AI industry through federal procurement policies (link).

The U.S. National Telecommunications and Information Administration (NTIA) published a report and recommendations on AI Accountability Policy (link). The report addresses several topics regarding the safety and governance of AI. I’ll highlight a key section on AI evaluation, edited for clarity:

The use of audits within AI accountability structures should depend on the risk level, deployment sector, maturity of evaluation methodologies, and available resources. Audits are appropriate for high-risk applications or models, at least for validating claims about system performance, limitations, and governance controls. Audits should employ replicable, standardized, and transparent methods. We recommend that audits be required for high-risk AI systems and applications, with government support for an independent evaluation ecosystem. Audits should incorporate applicable standards recognized by federal agencies. Designating AI as high-risk outside of specific contexts is challenging; however, the Office of Management and Budget has provided guidance on rights-impacting and safety-impacting AI systems, with exemptions as necessary.

The Utah Senate Bill 149, known as the Artificial Intelligence Policy Act, has been signed into law. The new law introduces regulations on the use of generative AI in commercial communications and healthcare, requiring clear disclosures about AI use, establishing fines for violations, and creating a state Office of Artificial Intelligence Policy to oversee AI practices and innovation (link).

In The Economist, Madeleine I.G. Daepp and Robert Osazuwa Ness discuss how generative AI is being used to spread targeted disinformation in Taiwan, highlighting how fact-checkers and government agencies are struggling to keep up, and the need to develop new strategies for monitoring and countering disinformation at scale (link).

Most AI researchers and companies agree that the government should not impose limitations on open-source AI, including blocking access to the weights of large-scale models. The NTIA received over 300 comments from the public, which will help inform a report to the President containing the NTIA’s findings and policy recommendations (link). Meta, OpenAI, and a group of researchers affiliated with the Institute for Advanced Study in Princeton have published their comments.

The Emerging Technology Observatory, a project of the Center for Security and Emerging Technology at Georgetown University published an analysis that shows that AI safety research constitutes 2% of all AI research. Despite its smaller proportion, AI safety research is highly cited indicating its impact within the broader AI research community (link).

The U.S. and UK recently signed a Memorandum of Understanding to collaborate closely on AI safety research and testing, leveraging shared scientific approaches and expert exchanges to enhance global AI safety standards and address emerging risks (link).

Achieving global governance of AI is complicated due to differences in countries' approaches to development, safety, and application, including in military contexts. Despite these challenges, Xue Lan, Chinese public policy expert, Professor and the Dean of Schwarzman College at Tsinghua University advocates for inclusive dialogue, technology sharing to bridge the intelligence gap, and the formation of globally inclusive governance mechanisms to ensure the ethical and safe advancement of AI technology (link; article needs to be translated from Chinese).

Content moderation involves screening and monitoring user-generated content on online platforms to ensure it adheres to the platform's rules, guidelines or legal requirements. Alyssa Boicel discusses in Tech Policy Press the challenges and limitations of using large language models (LLMs) for content moderation, particularly in the context of their inability to fully grasp cultural nuances, evolving language, and the reappropriation of derogatory terms (link).

U.S. Congress members, led by Co-Chair of the House AI Caucus, Anna G. Eshoo, and Congressman Neal Dunn, have introduced the Protecting Consumers from Deceptive AI Act to combat the challenges posed by AI-generated deepfakes and other deceptive content. The legislation aims to develop standards for identifying and labeling AI-generated content, requiring disclosures from AI developers and online platforms (link).

The Canadian government plans to allocate $2.4 billion to enhance Canada's AI capabilities, and maintain its competitive edge globally. Key initiatives include investing in computing capabilities and technological infrastructure, supporting AI startups, aiding small and medium-sized businesses in integrating AI, and allocating funds to train workers in industries that could be potentially disrupted by AI. Additionally, Canada plans to establish a Canadian AI Safety Institute (link).

The use of automated systems, including AI, by private and public entities can potentially result in unlawful discrimination and violate federal laws, given issues like biased datasets, lack of transparency, and incorrect assumptions in their development. Various U.S. federal agencies, including the CFPB, DOJ, EEOC, FTC, and others, in a recent statement, assert their commitment to enforcing civil rights, non-discrimination, and consumer protection laws, emphasizing that these laws apply to automated systems just as they do to traditional practices (link).

Opportunities

USAID is requesting comments as it seeks to develop a global AI research agenda. The key questions focus on identifying effective practices, guidelines, and frameworks to ensure that AI research is conducted safely, ethically, and with consideration for global contexts (link).

More! by Damola

The /ai 'manifesto'

Large language models have become good at crafting texts, audio, still, and moving images. We are getting to a point where to retain the trust others have in you and your work, tell us how you made use of AI in your craft. Create a page on your website that reflects how you use generative AI. Let’s call it the /ai page. That is, yourwebsite/ai…

Read more

2 years ago · Damola Morenikeji

The AI page for this website is at https://www.safeai.news/ai

AI is not a new technology, but it is increasingly becoming an important part of the way many people around the world live. This newsletter is a curation of content and news pertaining to how technologists, policymakers, and everyone in between are working towards ensuring the impacts of AI are beneficial.

Key Reads

All 193 members of the United Nations General Assembly have unanimously adopted a resolution aimed at governing artificial intelligence (AI), marking a global consensus for the development of safe, secure, and trustworthy AI systems (link). The resolution emphasizes the need for capacity building to close digital divides, discourages the use of AI to undermine peace or repress human rights, and calls for the responsible development and deployment of AI technologies. It highlights the mutual reinforcement of innovation and regulation, and the importance of education, transparency, and knowledge sharing on a voluntary basis.

Although non-binding, the resolution is seen as a step toward using AI to tackle existential threats and seize opportunities across various sectors, including healthcare, education, agriculture, and democracy promotion. It sets the stage for future collaboration within the UN and beyond. Representatives from Japan, Morocco, Singapore, and the UK, in a subsequent media stakeout (link), expressed their support and outlined their views on the resolution's impact.

While there is no consensus on the likelihood and severity of risks from near-future AI systems, the White House's executive order on AI, issued on October 30, requires companies developing “dual-use foundation models” of AI—which could pose significant safety and security risks—to report various related information to the government. In a recent article, Thomas Woodside of The Center for Security and Emerging Technology at Georgetown University argues for the importance of the U.S. Congress focusing on establishing an early warning system for severe risks posed by advanced AI systems as part of their legislative agenda on AI (link). He asserts that legislative action is necessary to solidify and enhance these provisions beyond the limitations of an executive order, which relies on the older Defense Production Act and may be rescinded by future administrations.

The cost of developing cutting-edge AI technology is increasingly becoming prohibitive for academics, as large-scale compute clusters demand substantial financial resources that typical research labs may not be able to cover with research grants. Consequently, academics are becoming more dependent on major technology companies, thus pushing AI researchers to orient their work towards commercial applications. Fei-Fei Li of Stanford advocates for the creation of a national AI repository to democratize access to computing resources and data (link). This initiative aims to counter the dominance of tech giants and preserve the independence of AI research. Conversely, Meredith Whittaker, President of Signal and Chief Advisor to The AI Now Institute, recently stated in a tweet that “...proposals to alleviate this imbalance largely exacerbate it, insofar as they amount to the government paying Big Tech to provide academics access to Big Tech resources” (link).

In a recent panel discussion, experts from The RAND Corporation were unified in recognizing the importance of independent research to understand AI's risks and implications for public policy (link).  An analogy of AI as a slow-moving catastrophe, akin to climate change, was drawn, emphasizing incremental harms that could gradually erode the quality of life and essential societal structures. The discussion also touched upon existential threats, with varying opinions on whether AI poses an unrecoverable harm to humanity. While some panelists viewed AI as a magnifier of existing problems rather than an existential threat, others raised concerns about AI's potential to disrupt meaningful human activity and connections, positing a more profound existential risk.  Policy solutions, such as transparency, oversight, and diversity in AI development, were suggested as means to mitigate these risks.

A recently introduced bill in the UK’s House of Lords seeks to establish a comprehensive regulatory framework for AI in the UK (link). Among many important stipulations, the bill mandates that individuals involved in AI training must document and report the use of third-party data and intellectual property (IP) to an authority, ensuring all data and IP are utilized with informed consent and in compliance with applicable laws. Copyright and IP issues are increasingly becoming contentious within the AI industry, exemplified by several high-profile lawsuits, such as a recent case involving book authors and the software company Databricks.

A new series of articles published by the AI Now Institute “survey the nationalist narratives around AI and emergent industrial policies being proposed by governments with differing economic and geopolitical motivations”. I’ve only read the first article and it was deeply insightful.

The first article, written by Susannah Glickman, delves into the deep historical intertwining of the tech industry, government intervention, and the development of AI and semiconductors within the United States. It highlights how government policies and funding have played pivotal roles in the evolution of technology, dispelling the myth of the lone tech entrepreneur and emphasizing the importance of state support and intervention (link).

The narrative begins in the mid-1980s, tracing the emergence of 'tech' as a category and its subsequent growth, which was fueled by government intervention. It showcases how the tech industry has been closely linked with national security, government priorities, and economic policy, illustrating this through various administrations' approaches to industrial policy, such as the CHIPS Act and executive orders focusing on semiconductor manufacturing. The history of AI is deeply intertwined with government funding and its strategic importance to national security. The development of AI is shown to be reliant on advances in semiconductor technology, which, in turn, benefits from massive government support and global cooperation.

Some key highlights include the Cold War's influence on tech development, the strategic importance placed on semiconductors and AI during different administrations, the impact of global competition (notably with Japan), and the shift towards a fabless model in semiconductor manufacturing. The article also discusses the recent re-engagement of government with the semiconductor industry in response to global challenges and supply chain issues, underscoring the ongoing significance of government involvement in sustaining the tech industry.

The narrative culminates in a discussion on the current state and future prospects of AI and semiconductors, emphasizing how intertwined these fields remain with governmental policies and the broader political economy. It asserts that the history of government intervention in tech is not just a series of isolated events but a fundamental aspect of how the tech industry has evolved and operates within the global economy.

The other articles in the series examine the policies and narratives in other countries, including South Africa, the UK, India, and the UAE (link).

Former IBM CEO Ginni Rometty expressed strong support for up-skilling in a recent podcast, discussing how AI can assist in job performance and provide opportunities for those with less formal education or credentials due to rapidly changing skills. In the same podcast, Dragoș Tudorache, a member of the European Parliament, explores the potential impact of AI on the workforce (link).

A recent investigation conducted by the AI Democracy Projects tested five leading AI text models (OpenAI's GPT-4, Anthropic's Claude, Google's Gemini, Meta's Llama 2, and Mistral's Mixtral) for their ability to provide accurate and reliable election-related information. Despite AI companies' pledges to promote information integrity, the study found that the AI models often delivered responses that were inaccurate, misleading, incomplete, or harmful, with all models struggling to correctly answer queries about election rules, such as the prohibition of campaign-related apparel at Texas polling places. Among the models, GPT-4 performed best in terms of accuracy, but the overall findings highlighted significant gaps in the models' utility for providing public election information (link).

Melissa Heikkilä wrote a comprehensive summary in MIT Technology Review (link) about how the development and use of AI within the EU will be influenced by the recently enacted EU AI Act. 

The first global summit on deepfake sexual abuse, organized by My Image My Choice and partners, including Bumble, convened experts across fields to discuss the burgeoning issue of deepfake-generated pornography. Deepfakes are 'synthetic media that have been digitally manipulated to replace one person's likeness convincingly with that of another.' This form of abuse, notably targeting women, has seen a dramatic increase in instances due to easily accessible AI tools. The experts called for stronger enforcement of terms of service by tech giants, legal reforms including adjustments to Section 230 of the US Communications Decency Act, and cautioned against the inadequacy of AI watermarking (link).

In a recent article, Arvind Narayanan and Sayash Kapoor argue that AI safety largely depends on the context and environment in which the AI is deployed. The article posits that focusing solely on making AI models “safe” overlooks the potential for misuse in varying contexts, as models lack the necessary information to discern between harmful and benign uses. They recommend shifting the focus of safety efforts away from model alignment alone and advocate for the development of external defenses against misuse. They also suggest a more nuanced assessment of marginal risk - risks that would otherwise not be present, a reorientation of red teaming towards identifying early warnings of misuse, and third-party-led red teaming to ensure unbiased risk assessments (link).

Opportunities

The UN Secretary-General's AI Advisory Body is seeking input on its recently launched Interim Report on AI Governance. The key questions for further discussion in the next phase of work by the Advisory Body focus on creating data protection frameworks and common standards for data labeling, promoting shared compute resources and determining the financing and capacity-building mechanisms necessary for effective international governance, among others. There's an emphasis on enhancing AI talent through international exchanges, joint academic programs, and cross-domain development, alongside leveraging international collaboration to support scientific research. Key challenges include reaching consensus on AI risks, especially concerning autonomous weapons, and establishing enforceable guidelines for ethical boundaries.

Link: https://www.un.org/en/ai-advisory-body  

Key reads

The African Union Development Agency has published (link) an AI Continental Roadmap for the continent with proposals for measures African countries should adopt for an “inclusive and sustainable AI-enabled socioeconomic transformation”. The strategic pillars focus on developing human capital, leveraging infrastructure and data, creating a supportive environment, establishing an economic climate, building partnerships, and enhancing monitoring and evaluation capacities to strengthen AI systems and strategies in African countries.

Can AI be an inventor? In the US, the answer seems to be No (link), but in Brazil, a newly introduced Bill (link) seeks to recognize AI as patent inventors.

Several biology-technology (Bio-tech) researchers from around the world are articulating values and principles to guide the responsible development of AI technologies in the field of protein design. They emphasize safety, security, equity, international collaboration, openness, responsibility, and societal benefit. The commitments focus on conducting beneficial research, preparing for health emergencies, adhering to biosecurity in DNA synthesis, improving DNA screening, continuously evaluating software risks, promoting scientific openness, and updating practices as needed to ensure AI developments are safe, secure, and beneficial for all. See: https://responsiblebiodesign.ai

In a recent analysis (link) on Bloomberg, AI models from OpenAI were found to exhibit clear signs of name-based discrimination: resumes with names distinct to Black-Americans were the least likely to be ranked as the top candidate for a financial analyst role, compared to resumes with names associated with other races and ethnicities.

Valentin Hofmann tweets “when LLMs are asked to pass judgment on defendants who committed murder, they choose the death penalty more often when the defendants speak African American English rather than Standardized American English … without being overtly told that they are African American” (link). This is one of several analyses done in a recent paper on dialect prejudice when AI is used to assess employability, criminality and character. Paper link.

In a recent study (link), researchers from Brown, CMU, Data & Society and Mozilla Foundation survey the landscape of AI audit tooling to understand the challenges to accountability and opportunities for tool development and research. AI audits are independent evaluations of performance, risks and safety of AI systems. Standards and regulations of AI systems around the world are largely still in early stages. The study on AI audit tools reveals a rich landscape but with notable gaps particularly in harm discovery, audit communication, and advocacy, emphasizing the need for tools that support comprehensive evaluation and genuine accountability. The authors call for a focus on legal clarity, community engagement, and interdisciplinary standards to enhance AI accountability.

A team of ethical hackers identified critical vulnerabilities in Google's AI and Cloud services, including unauthorized access to user images via an Insecure direct object reference (IDOR) in Bard's Vision feature, a DoS vulnerability through GraphQL directive overloading, a security issue with a hardcoded signature key, and the most significant finding: exfiltrating sensitive Google Workspace data via Bard (now Gemini) by injecting custom markdown. Security challenges in AI systems are constantly evolving as evidenced by these discoveries for which they were awarded significant bounties (link).

The US Department of Veterans Affairs is leveraging AI to improve veterans' healthcare by enhancing diagnostic accuracy, predicting cancer risks, and optimizing treatment paths. Through an initiative launched in 2017, they identified over 117,000 veterans at high suicide risk, leading to increased outpatient attendance, new safety plans, and reduced mental health admissions and suicide attempts. In addition, a tool called STORM assists in managing opioid risk through predictive models. Another FDA-authorized system, GI Genius, has been deployed in 106 facilities, enhancing the detection of precancerous polyps during colonoscopies. Listen to a recent oversight hearing (link).

ASCII art, a technique that uses characters from the ASCII set to create images, has been identified as a vulnerability in the security of large language models (LLMs) like GPT-3.5, GPT-4, Gemini, Claude, and Llama2 (link). Researchers have discovered that these state-of-the-art models are susceptible to "jailbreaks" through ASCII art, allowing individuals to circumvent safety protocols. This vulnerability enables the elicitation of undesired behaviors, such as producing instructions for making bombs if the prompts are cleverly disguised as ASCII art. This security loophole arises because current safety measures for LLMs primarily focus on interpreting the semantic content of text, overlooking the potential for ASCII art to convey complex and potentially harmful instructions beyond simple semantics.

Compute governance is an emerging tool for AI governance. Researchers from a cross-section of institutions in a recent paper (link), emphasize that computing power (compute) is crucial for AI progress, with a 350 million-fold increase in compute used for training leading AI systems over thirteen years. Compute governance involves regulating the access and usage of computing power to achieve policy goals, enhance safety, and manage the development and deployment of AI technologies. The paper proposes that compute governance can serve three main purposes: enhancing visibility into AI development and usage, allocating resources to foster desirable AI projects, and enforcing rules to ensure safe and lawful AI deployment. It also explores innovative enforcement mechanisms, including physical limitations on hardware and multiparty control protocols to mitigate risks associated with large-scale AI systems.

In a recent paper (link), researchers from multiple institutions including CMU, MIT and Stanford, advocate for legal and technical safe harbors to protect researchers conducting good-faith evaluations of proprietary AI models, highlighting the importance of independent assessments in identifying vulnerabilities and ensuring AI safety. They propose voluntary commitments from AI companies to facilitate public interest research without fear of legal repercussions or account termination to create an equitable, transparent, and accountable AI research ecosystem. In the same light, calls for broader access to these AI models i.e access to their Weights, Activations, or Gradients were made by another group of researchers (link).

Anna Makanju, VP of Global Affairs at OpenAI, on a recent World Economic Forum Podcast (link), advocates for a global regime that includes every country to address catastrophic risks of AI, suggesting a model similar to the International Atomic Energy Agency (IAEA) for AI governance.

The EU AI Act (link), potentially the first comprehensive legal framework for AI globally, aims to ensure that AI systems are trustworthy by mandating that they respect fundamental rights, safety, and ethical principles. It introduces a risk-based approach to regulation, categorizing AI applications into four levels of risk and applying corresponding requirements, ranging from transparency obligations for low-risk AI to strict controls and obligations for high-risk applications. The European Parliament is scheduled to vote on the AI Act on 13 March. This vote is one of the last steps before the law is enacted, but enforcement will not begin until 2025.

Amba Kak, a co-director of the AI Now Institute writes in Foreign Policy (link) that we need to critically examine whether large, and increasingly consolidated AI companies have interests that align with broader society.  She writes  “In an overwhelming focus on AI-driven harms, we’ve missed a key piece of the puzzle: demanding that firms articulate, clearly and with evidence to back it, what the benefits of AI are to the public … we haven’t asked enough of AI firms to show us their homework, instead permitting them to coast on shallow assertions that AI will inevitably lead us down the path of technological innovation.”

Opportunities

The DARPA AI Cyber Challenge (AIxCC) is a competition to develop AI-enabled systems for real-time identification and remediation of software vulnerabilities. Registration is currently open and there is a $4 million prize pool.

https://www.darpa.mil/news-events/2023-12-14 

The data.org Privacy Enhancing Technologies (PETs) for Public Health Challenge is currently open. The challenge involves demonstrating how financial transaction data, while ensuring privacy, can be leveraged to inform and support epidemiological policy decisions and public health responses during pandemics.

https://data.org/initiatives/pets-challenge