By | August 16, 2023
menu icon

Sakana AI founders Llion Jones, left, and David Ha meet at a rooftop bar in Tokyo. Jones, a co-author of Google’s seminal Transformers research paper, is the chief technology officer of the generative AI research laboratory, and Ha, a former Google researcher, is its CEO.

David Ha

Llion Jones had a big role on Google, where he worked for almost 12 years. He was one of eight authors of the central Transformers research paper, which is central to the state of the art in generative artificial intelligence.

But like all his co-authors, Jones has now left Google. He joins former Google researcher David Ha to build a generative AI research lab in Tokyo called Sakana AI. Jones said that while he holds no ill will toward Google, he recognized that the size of the company prevented him from doing the kind of work he wanted to pursue.

“It’s just a side effect of big business-itis,” Jones told CNBC in an interview. “I think the bureaucracy had built up to the point where I just felt like I couldn’t get anything done.”

Jones, who studied AI in college and has a master’s degree in advanced computer science from the University of Birmingham, takes center stage. The 2017 paper he helped write at Google and came up with innovations that played into OpenAI’s creation of the viral chatbot ChatGPT. T stands for Transformers, an architecture behind much of today’s frantic generative AI activity.

“We’re kind of crazy,” Jones said. “We’re looking at nature-inspired methods to see if we can find a different way of doing things, rather than making a huge, huge model.” Sakana does not announce any investors.

Jones became a software engineer at Google’s YouTube in 2012. According to his LinkedIn profilehe began “researching machine intelligence and natural language understanding” at Google in 2015.

Google is one of a number of big tech companies that have hired hordes of researchers in recent years, some straight out of universities, to construct AI models aimed at enriching their products. Over time, Jones said, he encountered questions about why the software didn’t work and whose fault it was. He thought the whole thing was a distraction from the research.

“Every day I would spend my time trying to access resources, trying to access data,” Jones said.

Now, after years of building products in labs, Google is rushing to incorporate generative AI, including large language models (LLMs), into its search engine, Youtube and other products. The models can summarize information and come up with human-like answers to written questions.

In Jones’s view, Google is focusing “the whole company around this technology,” and innovation is more challenging “because it’s a pretty restrictive framework,” he said.

Ha said he and Jones have talked to others who want to work on the LLM, but they haven’t finalized their plans.

“I’d be surprised if language models weren’t part of the future,” said Ha, who left Google last year to be head of research at startup Stability AI. He said he doesn’t want Sakana to be just another company with an LLM.

Both Jones and Ha have unflattering things to say about OpenAI, which has brought the concept of generative AI into the mainstream but raised billions dollars from Microsoft and other investors to do so. Ha described it as “getting so big and a bit bureaucratic”, not really different from groups within Google.

Jones said he doesn’t think OpenAI is that innovative. He said that for OpenAI’s two biggest successes, ChatGPT and the DALL-E service for creating images with a few words of text, the startup took research he conducted at Google and applied it at scale, making improvements along the way but waiting for to share the development with society. Although OpenAI has not released any of the technologies under an open source license, it has published papers on some of the underlying systems.

Representatives from Google and OpenAI did not respond to requests for comment.

Ha said Sakana has hired a part-time researcher from academia, and the company will eventually hire more people. Asked if they have added any other Googlers, Ha replied, “Not yet.”

LOOK AT: It is now time to shift attention from AI hardware to AI software: ARK Invest

#Google #researcher #left #build #startup #encountering #big #businessitis

Leave a Reply

Your email address will not be published. Required fields are marked *