Worth It? Wednesday: Gemini Advanced Edition

ai app gemini google llm worth it Dec 18, 2024

Gemini Advanced is like that kid in school who was always behind the smart kids, but then suddenly blossomed in their senior year. It's got potential, but it's playing catch-up. And it’s still rough around the edges. I’ve paid for Gemini Advanced for a total of five months so you didn’t have to. Is it worth it over the free version of Gemini? Is it better than ChatGPT? Is it THE best LLM? Here’s my take

Google isn’t just a search engine. Now they have built their own LLM, as well as other generative AI products. It was late to the game, has been rebranded, and had some major hiccups along the way. But, that doesn’t matter today. Is it worth learning and paying for? Here’s my take: 

Free vs Pro Features

Is there a compelling reason to pay for Gemini Advanced? At the time of publishing this article, the short answer is no. While there are some unique perks here such as 2 TB of Google One storage, this won’t be useful to everyone. Unless you plan on deep integrations with Google Drive and Gmail and need the impressive context window of Gemini Advanced, the paid version isn’t bringing a ton of features over the free version. 

But, is Gemini Advanced bringing anything spectacular to the table that the same $20 per month gets you from ChatGPT Plus, Claude Pro, or Perplexity Pro? Let’s look at the unique features of Gemini Advanced:

Unique Features of Gemini Advanced:

  • The newest models are both excellent AND fast! Gemini 2.0 Flash has a great default tone (comparable if not better than Claude 3.5 Sonnet) and it’s blazing fast. It also has access to the internet, something Claude lacks and ChatGPT still struggles to do well. Gemini has rolled out two different versions of 2.0 that are both excellent, and compared to ChatGPT & Claude’s flagship models they are every bit as good, if not better!
  • Voice mode is superb. You’ve seen the commercials, right? I have used Gemini’s “Live” feature more than the traditional text chat interface. It has ten different voices to choose from, and it nails a feature of live voice mode that ChatGPT struggles with in comparison: interruptions. Gemini “Live” is markedly better at NOT interrupting you mid prompt, while taking a break, or a brief pause. It can’t do some of the more advanced things like accents and emotions, and it doesn’t have a mute button. As a pure conversation tool and live translator though, it is one of the best I have used.

  • Gems are basically Gemini’s version of Claude Projects or CustomGPTs. I have only built one, and it worked well with a few paragraphs of prompting and about five PDFs of documentation in its knowledge base. These allow you to customize the output of your LLM without needing to re-prompt for context or content that you’ve already given it. But, are they better than ChatGPT & Claude’s equivalent? At this point, Gems can’t be shared with other users, but neither can Claude Projects. I don’t see any reason to use Gems over CustomGPTs that many of us have been building and refining for a year at this point. If Gems could be shared with other users or even accessed via a “Gem Marketplace” type interface, I might consider investing in them. But they lack access to the newest models. For example, I built a Gem but it could only use Gemini 1.5 Pro or Flash, not 2.0 or Deep Research. Gems also can’t generate images.
  • Deep Research is a new feature, but after using it for a week it clearly has potential. One of the issues that keeps me from going all in on Claude is its lack of internet search. ChatGPT’s newly released to free users “SearchGPT” is still rough. Perplexity.ai has been the best search-based LLM, and I’ve had it as my default browser search engine for six months (I rarely use Google Search and don’t miss it!). Deep Research is a model that you prompt with a search term or topic, then it comes up with a sequence of tasks. You can accept or reject these, then initiate the research. Using Google Search, Gemini will index dozens of sites (I’m seeing 20-38 in my experiments with Deep Research). 

What is it good for? It uses these sites to create a document answering your question. This isn’t anything new, but clicking the “Open in Docs” button to export it as a Google Doc is. The Doc it creates has in-text citations, headers, and even a Works Cited Page at the bottom. 

Anyone wanting to combine LLM based research with document editing in Google Docs will quickly see the utility here. One glaring issue I haven’t resolved: you can’t edit out sources from Deep Research. For example: I had a search run on a historical topic, but one of the sources it indexed was a historical fiction book listed on Ebay. Users should be able to edit sources like these out of the results.

  • Gemini’s most unique offering? That has to be Gemini 2.0 Multimodal Live API

What the heck is that? Basically, users can enable Gemini to “look” through their webcam or cast their screen live. This is only available via the Google AI Studio on desktop. This lets you combine their LLM model 2.0 Flash with live video and audio. What is it good for? This type of tool is so new it’s hard to say, but ChatGPT released something similar within 24 hours and it’s only available on mobile. One use case I’ve been waiting to try it on: visually impaired students having a live “view” of class materials or presentations. I experimented with it as an online shopping assistant, as well as having it narrate content from a slide show. It was a lot better at narrating the slide deck content, but I had frequent crashes and it seems unreliable at the time of writing this review. I saw this error every time I tried using it, and always within ten minutes of launching a live session.

  • Gemini also has “extensions”. These are accessed in a chat by using the @ followed by the tool you want to help with the query. I used @youtube multiple times, and it mostly worked. The other extensions are listed below. While this is an intriguing feature, I haven’t found anything exceptional it allows for.

What’s Missing From Gemini Advanced?

  • Account access is currently terrible. Accounts on Google Enterprise or Google Apps for Education that have been enabled by their admin can access Gemini, or you are stuck with a personal @gmail account. But Aaron, I pay for my own Google Workspace. Surely I can access Google Gemini on that account…right? Nope! This is a huge roadblock for me and hard to accept for a tool that costs $20 + the $15 per month I pay for my Workspace.
  • Integration with other Google generative AI tools is non-existent. Is NotebookLM part of Gemini? Not exactly, even though it runs under the hood. If users are paying for Gemini Advanced, they should be able to interface with NotebookLM. Google says Gemini Advanced subscribers will get access to NotebookLM Plus in early 2025, but I’m not holding my breath.
  • There isn’t truly a Gemini Advanced subscription! What, I thought that’s what this whole article is about? In classic Google style, you actually have to subscribe to Google One and get their cloud storage which happens to grant users access to Gemini Advanced as a result. Is this Google pushing their cloud storage subscriptions, or just being lazy about how they integrate their LLM into their product suite? I’m not sure, but it doesn’t make sense compared to what the other frontier models offer.

The Verdict ⚖️

Worth It For:

  • People deep in the Google ecosystem on a GAFE or Google Enterprise account

Not Worth It For:

  • Everyone else

 

This might seem harsh, but I’ve used Gemini Advanced all the way back to the early Bard days (silly Google, always changing the names of their products!). It really boils down to this: if you are living in all things Google: Gmail, Drive, Docs, etc. AND your main account can enable Gemini, you might consider investing your time and $20 per month in Gemini Advanced. I have a real love-hate relationship with Google, but my “school” account doesn’t have access to Gemini, and my “work” account can’t access it. My “personal” @gmail account can, but it doesn’t have any of the content I’d actually want to integrate into Gemini.

Even if my preferred accounts could access Gemini, I don’t think I’d use it regularly. Which is a shame, but Google has really improved their models and rolled out unique features like Multimodal Live API, Deep Research & 2.0 Flash which all have great potential.

Perplexity is still better for most search purposes, ChatGPT & Claude both integrate my Google Drive files better than…Google’s own AI. I don’t see a reason to move CustomGPTs to Gems, and Claude's writing styles and artifacts are both great features Gemini lacks.

What did I miss? Do you pay for Gemini Advanced? 

Do you think it’s better for your purposes than ChatGPT, Perplexity, Poe, Llama, etc? 

Tag or forward this to that tech nerd in your life who will want to tell me what I got wrong!

The AI On-Ramp Course is Coming Soon

Are you a college student who wants to learn how you can get started using AI tools to raise your grades & lower your stress without cheating?

Join the Waitlist