Joina startup creating great language models to rival those of Open AI, today announced the launch of a non-profit research lab: Cohere For AI. Led by Google alumnus Sara Hooker, Cohere says Cohere Labs will strive to solve some of the industry’s toughest challenges by contributing “fundamental research” to the open source community.
“We are truly excited to lead a new non-profit AI research lab as we continue to expand how and where research is done. There’s so much to discover, and our goal will be to openly collaborate and contribute to basic research,” Hooker told TechCrunch via email. “At the same time, a key part of our work will be to expand the community and help train the next generation of talent, creating new entry points for working on basic research. “
The AI community has long been concerned about the lack of funding earmarked for AI research outside of wealthy corporations. A study found that corporate connections – whether funding or affiliation – in AI research increased significantly from 2008 to 2019. Another study showed that Google’s parent company Alphabet, along with Amazon and Microsoft, hired 52 tenure-track AI teachers between 2004 and 2018, removing these future teachers from academic and nonprofit work.
The concentration of power within corporations has a number of obvious drawbacks, but one of the most alarming is that it tends to underestimate values such as beneficence, justice and inclusion on the research side. A number of experts, Speaking to Wired for a 2020 article, point out that corporate AI projects have led to an “unscientific fixation” on projects only possible for people with access to powerful data centers. Regardless of the field, work within companies is often tightly guarded and takes years to see the light of day, if at all.
“Our program is centered on advancing progress on machine learning issues alongside community-driven research,” Hooker said.We also want to have a proactive research program so that we can identify key challenges before they become problems that we need to fix retroactively. We focus on a variety of different disciplines to work on bias mitigation, for example, and a very central piece of research is around the safety of AI and the robust use of models. »
Another core element that Cohere For AI hopes to develop is access to computational resources, Hooker said — specifically helping researchers make better use of “cutting edge” models to help grow their work.” access to computing is evolving, as illustrated by the language model trends (i.e. AI systems that understand and generate text). Only a few years ago, creating a highly sophisticated language model required massive computational resources. But now, thanks to academic breakthroughs and the work of the open source community, the barriers to entry are much lower than before.
Road to non-profit
Backed by AI luminaries, including UC Berkeley AI Lab Co-Director Pieter Abbeel, Cohere was founded in 2019 by a top team including Aidan Gomez, Ivan Zhang, and Nick Frosst. Gomez co-authored the academic paper “Attention is all you need” which introduced the world to a foundational AI model architecture called Transformer. (Among other high-level systems, OpenAI’s GPT-3 and Manuscript are based on the Transformer architecture.) Zhang, alongside Gomez, is a contributor to For.ai, an open AI research collective involving data scientists and engineers.
“For.ai was designed to help early-career enthusiasts interact better with more experienced researchers,” Hooker said. “Many of the founding members have pursued doctoral studies or worked in university or industrial laboratories. At the time, For.ai was one of the first community research groups to support independent researchers around the world. Now the Cohere team and its supporters are excited to reintroduce the original concept but with more resources built from Cohere.”
According to Hooker, Cohere For AI will provide ways for data scientists to “meet and collaborate” through mentorship research opportunities, engagement with traditional conferences, and contributions to research journals. This will in part involve promoting the management of open source scientific practices and the “responsible” publication of code, as well as the support efforts that encourage “science communication” through different mediums, such as blog posts.
“WWe really want to make Cohere For AI an ambitious research lab that contributes to the research community, but also seeks to better engage a diverse set of voices. We want to help change where, how and by whom research is done,” Hooker said.
Despite its ambitious goals, Cohere For AI — which Cohere itself will fund — is likely to invite skepticism from researchers wary of Cohere’s commercial ties. Cohere has raised $170 million to date from institutional venture capital firms, including Tiger Global Management and Index Ventures, and has several associations with Google. Fei-Fei Li, chief scientist of Google Cloud AI, and Geoffrey Hinton, a colleague at Google, were early supporters of Cohere, and Gomez and Frosst previously worked at Google Brain, one of the research divisions on the Google AI. Cohere also has a Partnership with Google to train large language models on the company’s dedicated hardware infrastructure.
infamous google dissolved an AI advisory board in 2019, just a week after it was formed. And in 2020, the company fired Timnit Gebru, a leading artificial intelligence researcher, in what she said was retaliation for sending colleagues an email criticizing Google’s management practices. Google then fired another ethicist, Margaret Mitchell, who had publicly denounced the company’s handling of the situation, and a third, Satrajit Chatterjee, after she co-wrote an article questioning Google’s work in security systems. AI-powered chip design.
Paved with good intentions
Generally speaking, nonprofit initiatives to fund AI research have been mixed.
Among the success stories is the Allen Institute for AI (AI2), founded by the late Microsoft co-founder Paul Allen, which seeks to achieve scientific breakthroughs by building AI systems with reasoning capabilities. Although not strictly nonprofit, Anthropic, started by former OpenAI execs, has raised more than half a billion dollars researching “trustworthy, interpretable, and steerable” AI systems. .
But for every AI2 and Anthropic, there is an OpenAI, which started as a non-profit before moving to capped profit and accepting $1 billion investment from Microsoft. Meanwhile, former Google chairman Eric Schmidt recently announced a $125 million fund for AI research sparked new controversy after Politico reported that Schmidt wields unusually heavy influence over the White House Office of Science and Technology Policy. (One of the first recipients, Berkeley professor Rediet Abebe, asked to have his name removed from the review.)
However, some newer collectives have shown promise, including Gebru’s Distributed AI Research, a global nonprofit organization for AI research. Projects like Hugging Face’s BigScience and EleutherAI are other strong examples of what can be achieved in AI beyond the confines of corporate influence.
“Ultimately, it’s up to us to prove that Cohere For AI won’t venture over time,” Hooker said. “Although Cohere For AI will rely on Cohere for resources and funding, a separate separation has been created between the two to preserve its independence as a research laboratory. This separation is crucial for it to continue contributing and to serve the community at large as an independent entity. Cohere For AI is structured as a non-profit organization and was intentionally designed to collaborate openly with many different organizations. His work will be open source to allow better access to the wider community.”