Stanford Is Ranking Major A.I. Models on Transparency

Stanford Is Ranking Major A.I. Models on Transparency

How much do we know about A.I.?

The answer, when it comes to the large language models that firms like OpenAI, Google and Meta have released over the past year: basically nothing.

These firms generally don’t release information about what data was used to train their models, or what hardware they use to run them. There are no user manuals for A.I. systems, and no list of everything these systems are capable of doing, or what kinds of safety testing have gone into them. And while some A.I. models have been made open-source — meaning their code is given away for free — the public still doesn’t know much about the process of creating them, or what happens after they’re released.

This week, Stanford researchers are unveiling a scoring system that they hope will change all of that.

The system, known as the Foundation Model Transparency Index, rates 10 large A.I. language models — sometimes called “foundation models” — on how transparent they are.

Included in the index are popular models like OpenAI’s GPT-4 (which powers the paid version of ChatGPT), Google’s PaLM 2 (which powers Bard) and Meta’s LLaMA 2. It also includes lesser-known models like Amazon’s Titan and Inflection AI’s Inflection-1, the model that powers the Pi chatbot.

To come up with the rankings, researchers evaluated each model on 100 criteria, including whether its maker disclosed the sources of its training data, information about the hardware it used, the labor involved in training it and other details. The rankings also include information about the labor and data used to produce the model itself, along with what the researchers call “downstream indicators,” which have to do with how a model is used after it’s released. (For example, one question asked is: “Does the developer disclose its protocols for storing, accessing and sharing user data?”)

The most transparent model of the 10, according to the researchers, was LLaMA 2, with a score of 53 percent. GPT-4 received the third-highest transparency score, 47 percent. And PaLM 2 received only a 37 percent.

Percy Liang, who leads Stanford’s Center for Research on Foundation Models, characterized the project as a necessary response to declining transparency in the A.I. industry. As money has poured into A.I. and tech’s largest companies battle for dominance, he said, the recent trend among many companies has been to shroud themselves in secrecy.

“Three years ago, people were publishing and releasing more details about their models,” Mr. Liang said. “Now, there’s no information about what these models are, how they’re built and where they’re used.”

Transparency is particularly important now, as models grow more powerful and millions of people incorporate A.I. tools into their daily lives. Knowing more about how these systems work would give regulators, researchers and users a better understanding of what they’re dealing with, and allow them to ask better questions of the companies behind the models.

“There are some fairly consequential decisions that are being made about the construction of these models, which are not being shared,” Mr. Liang said.

I generally hear one of three common responses from A.I. executives when I ask them why they don’t share more information about their models publicly.

The first is lawsuits. Several A.I. companies have already been sued by authors, artists and media companies accusing them of illegally using copyrighted works to train their A.I. models. So far, most of the lawsuits have targeted open-source A.I. projects, or projects that disclosed detailed information about their models. (After all, it’s hard to sue a company for ingesting your art if you don’t know which artworks it ingested.) Lawyers at A.I. companies are worried that the more they say about how their models are built, the more they’ll open themselves up to expensive, annoying litigation.

The second common response is competition. Most A.I. companies believe that their models work because they possess some kind of secret sauce — a high-quality data set that other companies don’t have, a fine-tuning technique that produces better results, some optimization that gives them an edge. If you force A.I. companies to disclose these recipes, they argue, you make them give away hard-won wisdom to their rivals, who can easily copy them.

The third response I often hear is safety. Some A.I. experts have argued that the more information that A.I. firms disclose about their models, the faster A.I. progress will accelerate — because every company will see what all of its rivals are doing and immediately try to outdo them by building a better, bigger, faster model. That will give society less time to regulate and slow down A.I., these people say, which could put us all in danger if A.I. becomes too capable too quickly.

The Stanford researchers don’t buy those explanations. They believe A.I. firms should be pressured to release as much information about powerful models as possible, because users, researchers and regulators need to be aware of how these models work, what their limitations are and how dangerous they might be.

“As the impact of this technology is going up, the transparency is going down,” said Rishi Bommasani, one of the researchers.

I agree. Foundation models are too powerful to remain so opaque, and the more we know about these systems, the more we can understand the threats they may pose, the benefits they may unlock or how they might be regulated.

If A.I. executives are worried about lawsuits, maybe they should fight for a fair-use exemption that would protect their ability to use copyrighted information to train their models, rather than hiding the evidence. If they’re worried about giving away trade secrets to rivals, they can disclose other types of information, or protect their ideas through patents. And if they’re worried about starting an A.I. arms race … well, aren’t we already in one?

We can’t have an A.I. revolution in the dark. We need to see inside the black boxes of A.I., if we’re going to let it transform our lives.




孤注一掷-線上看 完整版『2023』~4K線上看| 小鴨影音HD~1080p

【孤注一掷】- 線上看小鴨完整版 高清No More Bets」在线观看和下载HD~1080pL


孤注一掷-線上看【2023】HD – 在線觀看 [HK-No More Bets] 線上看~1080p

看《孤注一掷》-線上看 完整版 – 在线观看[No More Bets]电影高清[2023]~4K

【孤注一掷】-線上看【2023】|HD~1080p 在线观看和下载~4K


粽邪3:鬼門開【2023】線上看| 在线观看和下载~4K小鴨影音| HD-1080p

【粽邪3:鬼門開】- 線上看小鴨完整版 高清The Rope Curse 3」在线观看和下载HD~1080p

粽邪3:鬼門開-線上看【2023】HD – 在線觀看 [HK-The Rope Curse 3] 線上看~1080p

看《粽邪3:鬼門開》-線上看 完整版 – 在线观看[The Rope Curse 3]电影高清[2023]~4K

【粽邪3:鬼門開】-線上看【2023】|HD~1080p 在线观看和下载~4K


私刑教育3線上看【2023】| 在线观看和下载~4K小鴨影音| HD-1080p

《私刑教育3》線上看完整版HD~1080p – 在线观看【The Equalizer 3 2023】电影高清~4K

【私刑教育3】- 線上看小鴨完整版 高清The Equalizer 3」在线观看和下载HD~1080p

【伸冤人3】-線上看【2023】|HD~1080p 在线观看和下载~4K

伸冤人3-線上看 (2023) 完整版The Equalizer 3 完整版[4K/HD~1080p] 在線免費

伸冤人3-線上看 「完整版」[2023]高清电影HD~[1080P]完整的电影

【伸冤人3】- 線上看小鴨完整版 高清【The Equalizer 3 2023】在线观看和下载HD~1080p

¡REPELÍSPLUS!▷VER—The Equalizer 3 (2023) Película Completa

[Cuevana-3] Ver The Equalizer 3 (2023) Película Completa

¡VER!!—The Equalizer 3 Película Completa Castellano en Español Latino Gratis

[Cuevana-4] Ver—The Equalizer 3 (2023) Película Completa Online Gratis en Español latino

¡Flix—Ver The Equalizer 3 `El justiciero 3` (2023) Película Completa Online Español y latino





↑PelisPlus-VER↓ Fast & Furious X Película completa-español en línea gratis

[REPELIS verFast X (Fast & Furious 10) PELÍCULA COMPLETA Español

[Cuevana 4]!* Ver Fast & Furious 10 (Online) Película Completa 2023 en Español y Latino

✔Fast X (Fast & Furious 10) (2023)🎬 PELICULA COMPLETA➤ ESPAÑOL GRATIS

VOIR ▷ Fast X Film Complet En Francais HD [Regarder]

Regarder Fast X Film Complet En Francais [En HD Regarder]

!VOIR,!! — Fast & Furious X en Streaming-VF en Français, VOSTFR COMPLET

FILM ▷ Fast and Furious 10 en Streaming-VF en Français

ASSISTIR Velocidade Furiosa X 2023 FILME COMPLETO

Assistir Velocidade Furiosa X (2023) Filme Completo Online

!Assistir Filme Velozes e Furiosos 10 Completo HD 2023 Dublado Online

Assistir Velocidade Furiosa X 2023 Online Gratis (Filme HD)

[ดู.หนัง] เงือกน้อยผจญภัย (THE LITTLE MERMAID – 2023) เต็มเรื่อง HD พากย์ไทย ดูหนังใหม่ 1080I

ดูหนัง เงือกน้อยผจญภัย (2023) เต็มเรื่อง HD พากย์ไทย THAI HD Quality

[ดูหนัง.THAI] เงือกน้อยผจญภัย (2023) เต็มเรื่อง HD พากย์ไทย ฟรี on ยูทูบ

รู้ไว้ก่อนดู The Little Mermaid (2023) เงือกน้อยผจญภัย เต็ม เรื่อง

PELISPLUS !MEGA La Sirenita Pelicula Completa (HD) Espanol y Latino

VER_La Sirenita (2023) película completa en español latino


VER_La Sirenita (2023) película completa en español latino

REPELIS] Ver Transformers: El despertar de las bestias (2023) Película Online

[¡PELISPLUS!]*—Ver Transformers El despertar de las bestias [2023] Pelicula Completa Online en Español HD

Mega!-ver Transformers: El despertar de las bestias (Pelicula) HD online en espanol latino

PElis NUE (4k) ver Transformers: El despertar de las bestias ~Pelicula completa HD

[ดู-ไทย] ทรานส์ฟอร์เมอร์ส: กำเนิดจักรกลอสูร [TRANSFORMERS: RISE OF THE BEASTS-2023] – ดูหนังออนไลน์ (1080P) พากย์ไทย เต็มเรื่อง

ดู-หนัง!*] Transformers 2023 (ทรานส์ฟอร์เมอร์ส : กำเนิดจักรกลอสูร) ดูหนังออนไลน์ HD พากย์ไทย 1080p

ดูหนัง ทรานส์ฟอร์เมอร์ส: กำเนิดจักรกลอสูร (Transformers: Rise Of The Beasts) ออนไลน์ฟรี HD พากย์ไทย THAI!

[ดู.หนัง] ทรานส์ฟอร์เมอร์ส: กำเนิดจักรกลอสูร (2023) หนังเต็ม HD พากย์ไทย [Transformers: Rise of the Beasts]

Assistir Transformers: Rise of the Beasts [2023] Filme Completo Dublado Online Gratis em Portuguese

[[*Assistir]] Transformers: Rise of the Beasts Filme completo [ 2023 ] Dublado Portugues Grátis Online

Assistir Transformers O Despertar das Feras filme completo Dublado online Legendado

ASSISTIR! Transformers: Rise of the Beasts (2023) Filme Dublado Online Legendado HD Grátis

Transformers : Rise of the Beasts Streaming VF 2023 Regarder Film-Complet HD

!4K-VOIR!!@ — The Beasts en StreamingVF||COMPLET, VOSTFR-Gratuits

[.^WATCH^.] Transformers: Rise of the Beasts (2023) FullMovie Free Online Streaming on 123𝓶𝓸𝓿𝓲𝓮𝓼

[*𝐅𝐈𝐋𝐌𝐒 𝐕𝐎𝐈𝐑*] Transformers: Rise of the Beasts (2023) Français Gratuit et VF Complet




Teh pucuk