I have a 2018-era MacMini with 64Gb of RAM + negligible GPU, running a 35 billion parameter model and I’m getting 4 tokens per second out of it; tell me again how we aren’t going to be running AIs locally at home?

Models are simply fuzzy compressions of all of the pages in the internet & elsewhere, starting with Wikipedia.

Back in the 1980s and 1990s, JPEGs were exotic, took ages to create and they looked like shit. Nowadays they are baseline. Who here remembers* FITS, TGA & TIFF?

This is where we are now with AI. This is also why we must stop the government trying to restrict access to such technologies.


[*] …I’m sure that somebody here uses TIF on occasion, and will want to say so, but the point is: you’re not anywhere close to the mean, median or mode anymore.

Fediverse reactions

Comments

One response to “I have a 2018-era MacMini with 64Gb of RAM + negligible GPU, running a 35 billion parameter model and I’m getting 4 tokens per second out of it; tell me again how we aren’t going to be running AIs locally at home?”

  1. Drew

    I’ve been inspired to experiment. I have something resembling that machine.

    Care to share? What’s the name of the software? What OS is it running on?

Leave a Reply

Your email address will not be published. Required fields are marked *