I got access to Gemini Pro 1.5 this week, a new private beta LLM from Google that is significantly better than previous models the company has released. (This is not the same as the publicly available version of Gemini that made headlines for refusing to create pictures of white people. That will be forgotten in a week; this will be relevant for months and years to come.)
Somehow, Google figured out how to build an AI model that can comfortably accept up to 1 million tokens with each prompt. For context, you could fit all of Eliezer Yudkowsky’s 1,967-page opus Harry Potter and the Methods of Rationality into every message you send to Gemini.
1 million tokens is insane (tokens = words (kind of)). For context, OpenAI’s GPT-4 Turbo can accept 32,000 tokens.
To be fair, these days I run into GPT-4’s token limit rarely. My prompts aren’t that big. But a 1 million tokens opens up a new world. The author of the article says you could send Gemini a 2,000 page book as a prompt. And I could see myself using it for that use-case. Often I remember I read something but can’t find the passage. I could copy and paste the full book text from a .mobi file and ask Gemini for help.
I think it would also be useful for my notes. They’re all individual .md text files. But I’m sure there’s a tool out there that could combine them into one big file. And then I could send it to Gemini and ask questions.