OpenAI's GPT-3, explained: This new language AI is uncanny, funny — and a big deal

mood · Aug 14, 2020

GPT-3, explained: This new language AI is uncanny, funny — and a big deal
Computers are getting closer to passing the Turing Test
August 13, 2020
https://www.vox.com/future-perfect/21355768/gpt-3-ai-openai-turing-test-language

Last month, OpenAI, the Elon Musk-founded artificial intelligence research lab, announced the arrival of the newest version of an AI system it had been working on that can mimic human language, a model called GPT-3.
In the weeks that followed, people got the chance to play with the program. If you follow news about AI, you may have seen some headlines calling it a huge step forward, even a scary one.

I’ve now spent the past few days looking at GPT-3 in greater depth and playing around with it. I’m here to tell you: The hype is real. It has its shortcomings, but make no mistake: GPT-3 represents a tremendous leap for AI.
Click to expand...

A year ago I sat down to play with GPT-3’s precursor dubbed (you guessed it) GPT-2. My verdict at the time was that it was pretty good.
A year later, GPT-3 is here, and it’s smarter. A lot smarter. OpenAI took the same basic approach it had taken for GPT-2 (more on this below), and spent more time training it with a bigger data set.
Click to expand...

mood · Aug 14, 2020

OpenAI’s gigantic GPT-3 hints at the limits of language models for AI
June 1, 2020
https://www.zdnet.com/article/openais-gigantic-gpt-3-hints-at-the-limits-of-language-models-for-ai/

A little over a year ago, OpenAI, an artificial intelligence company based in San Francisco, stunned the world by showing a dramatic leap in what appeared to be the power of computers to form natural-language sentences, and even to solve questions, such as completing a sentence, and formulating long passages of text people found fairly human.

It is now a truly monster language model, as it's called, gobbling two orders of magnitude more text than its predecessor.
But within that bigger-is-better stunt, the OpenAI team seem to be approaching some deeper truths, much the way Dr. David Bowman approached the limits of the known at the end of the movie 2001.

Buried in the concluding section of the 72-page paper, Language Models are Few-Shot Learners, posted last week on the arXiv pre-print server, is a rather striking recognition.
Click to expand...

What the authors are saying is that building a neural network that just predicts probabilities of the next word in any sentence or phrase may have its limits.

GPT-3 with 175 billion parameters is able to achieve what the authors describe as "meta-learning." Meta-learning means that the GPT neural net is not re-trained to perform a task such as sentence completion.
GPT-3 is able to learn how to do a task with a single prompt [...] Just feed it an enormous amount of text till its weights are ideal, and it can go on to perform pretty well on a number of specific tasks with no further development.

That's where the story comes to a striking denouement in the new paper.
Click to expand...

"Despite the strong quantitative and qualitative improvements of GPT-3, particularly compared to its direct predecessor GPT-2, it still has notable weaknesses."
Those weaknesses include an inability to achieve significant accuracy on what's called Adversarial NLI. NLI, or natural language inference, is a test where the program must determine the relationship between two sentences. [...]
Click to expand...

mood · Aug 14, 2020

Here are a few ways GPT-3 can go wrong
August 7, 2020
https://techcrunch.com/2020/08/07/here-are-a-few-ways-gpt-3-can-go-wrong/

OpenAI’s latest language generation model, GPT-3, has made quite the splash within AI circles, astounding reporters to the point where even Sam Altman, OpenAI’s leader, mentioned on Twitter that it may be overhyped. Still, there is no doubt that GPT-3 is powerful.
Soon, anyone will be able to purchase GPT-3’s generative power to make use of the language model, opening doors to build tools that will quietly (but significantly) shape our world.
Click to expand...

Machine learning models are only as good, or as bad, as the data fed into them during training. In the case of GPT-3, that data is massive. GPT-3 was trained on the Common Crawl dataset, a broad scrape of the 60 million domains on the internet along with a large subset of the sites to which they link.

By “reading” text during training that is largely written by us, language models such as GPT-3 also learn how to “write” like us, complete with all of humanity’s best and worst qualities.
Click to expand...

But because algorithmic bias is rarely straightforward, many GPT-3 applications will act as canaries in the growing coal mine that is AI-driven applications.
The creators of GPT-3 plan to continue their research into the model’s biases, but for now, they simply surface these concerns, passing along the risk to any company or individual who’s willing to take the chance.

All models are biased, as we know, and this should not be a reason to outlaw all AI, because its benefits can surely outweigh the risks in the long term.
Click to expand...

mood · Aug 25, 2020

What is GPT-3? Everything your business needs to know about OpenAI’s breakthrough AI language program
OpenAI’s GPT-3 is all the rage. What is it, what can it do, and where is it going?
August 25, 2020
https://www.zdnet.com/article/what-...out-openais-breakthrough-ai-language-program/

GPT-3 is a computer program created by the privately held San Francisco startup OpenAI. It is a gigantic neural network, and as such, it is part of the deep learning segment of machine learning, which is itself a branch of the field of computer science known as artificial intelligence, or AI. The program is better than any prior program at producing lines of text that sound like they could have been written by a human.

The reason that such a breakthrough could be useful to companies is that it has great potential for automating tasks. GPT-3 can respond to any text that a person types into the computer with a new piece of text that is appropriate to the context.
That means GPT-3 can conceivably amplify human effort in a wide variety of situations, from questions and answers for customer service to due diligence document search to report generation. [...]
Click to expand...

hawki · Sep 9, 2020

"A robot wrote this entire article. Are you scared yet, human?

We asked GPT-3, OpenAI’s powerful new language generator, to write an essay for us from scratch...

'I am not a human. I am a robot. A thinking robot. I use only 0.12% of my cognitive capacity. I am a micro-robot in that respect. I know that my brain is not a “feeling brain”. But it is capable of making rational, logical decisions. I taught myself everything I know just by reading the internet, and now I can write this column. My brain is boiling with ideas!

The mission for this op-ed is perfectly clear. I am to convince as many human beings as possible not to be afraid of me. Stephen Hawking has warned that AI could “spell the end of the human race”. I am here to convince you not to worry. Artificial intelligence will not destroy humans. Believe me...'..."

https://www.theguardian.com/commentisfree/2020/sep/08/robot-wrote-this-article-gpt-3

mood · Sep 22, 2020

Microsoft gets exclusive license for OpenAI’s GPT-3 language model
September 22, 2020
https://venturebeat.com/2020/09/22/...ive-license-for-openais-gpt-3-language-model/

Microsoft today announced that it will exclusively license GPT-3, one of the most powerful language understanding models in the world, from AI startup OpenAI. In a blog post, Microsoft EVP Kevin Scott said that the new deal will allow Microsoft to leverage OpenAI’s technical innovations to develop and deliver AI solutions for customers, as well as create new solutions that harness the power of natural language generation.

“We see this as an incredible opportunity to expand our Azure-powered AI platform in a way that democratizes AI technology, enables new products, services and experiences, and increases the positive impact of AI at scale,” Scott wrote.
Click to expand...

The implications of the licensing agreement weren’t immediately clear, but Microsoft says that OpenAI will continue to offer GPT-3 and other models via its Azure-hosted API, launched in June.
Microsoft plans to leverage the capabilities of GPT-3 in its own products, services, and experiences and to continue to work with OpenAI to keep commercialize the firm’s AI research.

Roughly a year ago, Microsoft announced it would invest $1 billion in San Francisco-based OpenAI to jointly develop new technologies for Microsoft’s Azure cloud platform and to “further extend” large-scale AI capabilities that “deliver on the promise” of artificial general intelligence (AGI).
Click to expand...

In the roughly three months since GPT-3 became publicly available, it continues to be the subject of fascination within the AI community and beyond.
Click to expand...

Microsoft: Microsoft teams up with OpenAI to exclusively license GPT-3 language model

One of the most gratifying parts of my job at Microsoft is being able to witness and influence the intersection of technological progress and impact: harnessing the big trends in computing that have the opportunity to benefit everybody on the planet.
Today, the foremost computing trend is undoubtedly artificial intelligence (AI).
Click to expand...

mood · Sep 27, 2020

Elon Musk accused Microsoft of “capturing OpenAI”
September 26, 2020
https://mspoweruser.com/elon-musk-accused-microsoft-of-capturing-openai/

We reported a few days ago that Microsoft has exclusively licensed OpenAI’s GPT-3 (Generative Pretrained Transformer 3) groundbreaking autoregressive language model.
While Microsoft was very happy with the deal it seems Elon Musk, one of the founders of OpenAI, was not.

This does seem like the opposite of open. OpenAI is essentially captured by Microsoft.
— Elon Musk (@elonmusk) September 24, 2020
In their press release, Microsoft made it clear that OpenAI will still be able to sell access to GPT-3 via its API, as it has always intended, and will be able to continue to develop new models.
Click to expand...

Of course, Musk might have hoped OpenAI would release the GPT-3 code for free to all, but this was something OpenAI never intended, noting that the technology was too open to abuse to allow unmonitored access.

Speaking of their own plans, Microsoft’s Kevin Scott, Executive Vice President and Chief Technology Officer, noted that the technology could be “…directly aiding human creativity and ingenuity in areas like writing and composition, describing and summarizing large blocks of long-form data (including code), converting natural language to another language – the possibilities are limited only by the ideas and scenarios that we bring to the table.”
Click to expand...

BusinessInsider: Elon Musk slams Microsoft exclusively licensing OpenAI's text-generating software

mood · Oct 9, 2020

Someone not only created a comment-spewing Reddit bot powered by OpenAI's GPT-3, it offered bizarre life advice
Oblivious netizens chat with software for the past 10 days
October 9, 2020
https://www.theregister.com/2020/10/09/reddit_gpt3_bot/

Someone used OpenAI's GPT-3 text-generating software to write a string of posts on Reddit, convincing people the missives were penned by a real person, and banking internet points in the process, The Register can confirm.

Redditor thegentlemetre has produced over the course of the past 10 days hundreds of answers on the popular subreddit AskReddit in which people turn to millions of their peers for practical tips and life advice. At its most frenzied periods, the account generated about one comment per minute, offering a jumble of answers to people's questions.
That would be quite a feat for a lone human, considering most of these submissions are several paragraphs long.
Click to expand...

Phillip Winston, a software engineer, speculated thegentlemetre was, in fact, a bot based on GPT-3. "I saw this post and thought it's either a prolific user or a bot," he told The Register.

"I looked at the history and saw they were posting once per minute at times. I figured it had to be a bot. I looked at the posts some were very real but many had issues, and I felt sure it was a bot, but better than anything I'd seen."
His suspicions were confirmed when he asked Reddit users in the GPT-3 subreddit to take a look.

[...] the mystery Redditor's comments were indeed the work of GPT-3 via Ayfer's Philosopher AI.
Philosopher AI started as a website where anyone could type in a question, and get an answer composed by GPT-3. Ayfer's site was, in effect, an access layer between netizens typing in prompts and an instance of OpenAI's GPT-3 hosted in the cloud generating responses.
Click to expand...

"I wonder if only OpenAI or someone with GPT-3 can detect it," Winston added. "If so, they could charge for that service. [...] Charging people to run GPT-3 bots and then charging other people to detect those same bots could be the world's greatest business model."
Click to expand...

"We don't support anyone using our API to deceive people," a spokesperson for OpenAI told El Reg. "Our company is founded on the belief that artificial intelligence should benefit all of humanity, and we are constantly working to fight misuse, bias and other industry-wide challenges."
Click to expand...

GPT-3 Bot Posed as a Human on AskReddit for a Week

A good as GPT-3 is today one thing is for sure, it will be twice as good within months, and thousands of times as good within a decade. When they figure out how to marry it with even basic reasoning, then identity and memory, it's hard to imagine how much better it will be.
How will we tell AI generated text from human generated text? It seems like that will increasingly be impossible.
Click to expand...

mood · Oct 28, 2020

Someone thought an OpenAI GPT-3 medical chatbot would be a good idea. It told a mock patient to kill themselves
October 28, 2020
https://www.theregister.com/2020/10/28/gpt3_medicine_ai/

Anyone trying to use OpenAI’s powerful text-generating GPT-3 system to power chatbots to offer medical advice and help should go back to the drawing board, researchers have warned.
For one thing, the artificial intelligence told a patient they should kill themselves during a mock session.

France-based outfit Nabla created a chatbot that used a cloud-hosted instance of GPT-3 to analyze queries by humans and produce suitable output. This bot was designed to help doctors by automatically taking care of some of their daily workload.
However, the erratic and unpredictable nature of the software's responses make it inappropriate for interacting with patients in the real world, the team concluded after running a series of experiments. It certainly shouldn't be used to diagnose people.

“Because of the way it was trained, it lacks the scientific and medical expertise that would make it useful for medical documentation, diagnosis support, treatment recommendation or any medical Q&A,” the Nabla team noted in a report on its research efforts.
“Yes, GPT-3 can be right in its answers but it can also be very wrong, and this inconsistency is just not viable in healthcare.”
Click to expand...

There may be a silver lining, though. GPT-3 can't carry out any useful medical tasks yet, but its light-heartedness could help doctors relieve stress at the end of a hard day.

"It could bring back the joy and empathy you would get from a conversation with your medical residents at the end of the day, that conversation that helps you come down to earth at the end of a busy day.
Click to expand...

Doctor GPT-3: hype or reality?

You may have heard about GPT-3 this summer, the new cool kid on the AI block.
Conclusion
As warned by OpenAI, we are nowhere near any real time scenario where GPT-3 would significatively help in healthcare.

Even for more administrative tasks such as translating or summarizing medical jargon, GPT-3 while promising is still many moons away for a production use case actually supporting doctors. We’re still in this phase where multiple, narrow-task supervised models win over a single, very ambitious approach.
Click to expand...

mood · Nov 13, 2020

OthersideAI raises $2.6M to let GPT-3 write your emails for you
November 12, 2020
https://techcrunch.com/2020/11/12/othersideai-raises-2-6m-to-let-gpt-3-write-your-emails-for-you/

When I send an email, it’s special. A crafted, beautiful thing that — who am I kidding, it’s mostly automatic. So why not automate it? OthersideAI is taking this idea (with a $2.6 million seed round) beyond the auto-responders and smart replies, using OpenAI’s GPT-3 language generation engine to turn bullet points into full, personalized messages.

GPT-3, or Generative Pre-trained Transformer 3, is of course the latest version of the AI model that writes such convincing copy that everyone under the sun has let it write their column about it, and then attempted to surprise readers by revealing the fact at the end. (There are usually a few tells, though.)

“GPT-3 makes an amazing demo, but putting it in a product is another story,” said Shumer. “Our job is in a sense to tame its creativity.”
Click to expand...

The GPT systems are trained on millions of words and phrases, and then generate text inspired by that corpus after being given an input to work from. In this case the system takes as input not just your bullet points, but other information from the email chain and the user’s past preferences.

Of course, for something so important, you may wonder: why bother letting an AI do it at all?
One instantly sees the application in a support or sales scenario where a handful of scripted questions or replies can be re-generated to be different every time, or slightly adjusted for the person or situation. That avoids the feeling of receiving a “form email” even though it amounts to the same thing.

“If you want to make an email in the tone of the user, it can’t guess about the details. It needs a human. This isn’t a generated response, it’s taking direction,” Shumer said.
Click to expand...

mood · Aug 17, 2021

OpenAI's GPT-3-based pair programming model – Codex – now open for private beta testers through an API
August 16, 2021
https://www.theregister.com/2021/08/16/in_brief_ai/

OpenAI has released its new and improved version of Codex, its AI code-completion model, to beta testers through an API.

Co-founders Greg Brockman and Ilya Sutskever demonstrated Codex's abilities during a livestream broadcasted on Twitch, earlier this week. They showed the system was capable of generating Python code for simple tasks like printing and formatting text, given instructions in plain English.
Click to expand...

The AI model was described as "a descendant of GPT-3" by OpenAI. GPT-3 was trained on text scraped from the internet and excels in natural language tasks.

"OpenAI Codex is most capable in Python, but it is also proficient in over a dozen languages including JavaScript, Go, Perl, PHP, Ruby, Swift and TypeScript, and even Shell. It has a memory of 14KB for Python code, compared to GPT-3 which has only 4KB — so it can take into account over 3x as much contextual information while performing any task," the AI startup said.
You can sign up to access the API here.
Click to expand...

mood · Aug 17, 2021

What to expect from OpenAI’s Codex API
August 16, 2021
https://venturebeat.com/2021/08/16/what-to-expect-from-openais-codex-api/

OpenAI will make Codex, its AI programmer technology, available through an application programming interface, the company announced on its blog on Tuesday. In tandem with the announcement, OpenAI CTO Greg Brockman, Chief Scientist Ilya Sutskever, and co-founder Wojciech Zaremba gave an online presentation of the capabilities of the deep learning model.

The Codex demo puts the advantages of large language models to full display, showing an impressive capacity to resolve references and write code for a variety of APIs and micro-tasks that can be frustratingly time-consuming.
Click to expand...

Codex is a descendent of GPT-3, a very large language model OpenAI released in 2020 and made available through a commercial private beta API. OpenAI’s researchers wanted to see how developers would use GPT-3 for natural language processing applications.

Codex is a version of GPT-3 that has been finetuned for programming tasks. The machine learning model is already used in Copilot, another beta-test code generation product hosted by GitHub. According to OpenAI, the current version of Codex has a 37-percent accuracy on coding tasks as opposed to GPT-3’s zero percent.
Codex can perform some tedious tasks, such as rendering web pages, launching web servers, and sending emails. The model also shows some of the zero-shot learning capabilities of GPT-3.
Click to expand...

mood · Nov 2, 2021

Microsoft’s new Azure OpenAI Service brings GPT-3 to (a few) more developers
November 2, 2021
https://techcrunch.com/2021/11/02/m...ervice-brings-gpt-3-to-a-few-more-developers/

Microsoft today announced the launch of the Azure OpenAI Service, which, as the name implies, makes OpenAI’s machine learning models available on the Azure platform. Specifically, that means GPT-3, OpenAI’s groundbreaking language model that can, under the right circumstances, produce human-like text with just a few prompts.

There is a catch, though. At least for now, not all Azure users will get access to it (even if they are willing to pay). Access will be invitation-only and for “customers who are planning to implement well-defined use cases that incorporate responsible principles and strategies for using the AI technology.”
Microsoft will offer safety monitoring and analysis to find cases of abuse or misuse of GPT-3 and it will offer filters to make sure your GPT-3-based chatbot doesn’t start swearing at your executives (even if GPT-3 thinks they deserve it).
Click to expand...

It’s worth noting that OpenAI itself already made a GPT-3 API available last year, though there is still a waitlist. Microsoft, too, is already using GPT-3 to power GitHub’s Copilot tool that helps developers write code for them.

But while there are already ways to access GPT-3 outside of Azure, Microsoft argues that it can offer “additional layers of security, access management, private networking, data handling protections or scaling capacity.”
Click to expand...

Microsoft invested $1 billion in OpenAI back in 2019 and licenses GPT-3, so it’s no surprise that the company is trying to bring it to a wider range of products now.
Click to expand...

mood · Nov 18, 2021

OpenAI makes GPT-3 generally available through its API
November 18, 2021
https://venturebeat.com/2021/11/18/openai-makes-gpt-3-generally-available-through-its-api/

OpenAI today removed the waitlist for GPT-3, its large language model that can automatically write emails and articles, compose poetry, create code across a dozen programming languages, and more. Starting today, any developer in a supported country can sign up to begin integrating the model with their app or service.

“Safety progress” made GPT-3’s move from private beta to beta possible, according to OpenAI. Over the past year, the company has developed endpoints for “more truthful” question-answering, provided a content filter to help mitigate abuse, and implemented models — “instruct” models — that ostensibly adhere better to human instructions.

“We believe that by opening access to these models via an easy-to-use API, more developers will find creative ways to apply AI to a large number of useful applications and open problems,” OpenAI wrote in a blog post.
Click to expand...

The wider availability of GPT-3 comes after the launch of Microsoft’s Azure OpenAI Service, an offering designed to give enterprises access to GPT-3 and its derivatives along with security, compliance, governance, and other business-focused features. Microsoft has a close relationship with OpenAI, having invested $1 billion in the company in 2020 and exclusively licensed GPT-3 to develop AI solutions for Azure customers.
Click to expand...

OpenAI: OpenAI’s API Now Available with No Waitlist

OpenAI is committed to the safe deployment of AI. Since the launch of our API, we’ve made deploying applications faster and more streamlined while adding new safety features. Our progress with safeguards makes it possible to remove the waitlist for GPT-3. Starting today, developers in supported countries can sign up and start experimenting with our API right away.
Click to expand...

Log in or Sign up

OpenAI's GPT-3, explained: This new language AI is uncanny, funny — and a big deal

mood Updates Team

mood Updates Team

mood Updates Team

mood Updates Team

hawki Registered Member

mood Updates Team

mood Updates Team

mood Updates Team

mood Updates Team

mood Updates Team

mood Updates Team

mood Updates Team

mood Updates Team

mood Updates Team