Running LLMs Locally
This is a short post on how I've setup llms to run locally.
Hello - I’m Tony🔗︎
Professionally, a Network Engineer with a passion for Python, open source stuff, indie games, boardgames, and radio. This site is mostly a reference for myself to remember how I got things to work. There's a chance it will help you too! (1)
The views, opinions, and positions expressed on this blog are my own and do not necessarily reflect the views, opinions, or positions of my employer or any other organization with which I am affiliated. All content provided on this blog is for informational purposes only and is not intended as professional advice.
Using UV
This post covers how I configure and use UV appropriately. Some of these tips are specifically useful within a corporate network.
MakeMKV and FFmpeg
All of the physical media I own gets backed up to a NAS which is also a media server running Jellyfin. Recently, I upgraded my version of The Lord of the Rings. Here are some of my notes on ripping that media, and how I used FFmpeg to combine the multi-part films.
Getting Started with Google AI Studio
Google was a bit later than expected with widely available GenAI. They've rapidly gained momentum, and have several compelling models that lend themselves to interesting use cases. Here's a recent example
I, Napoleon
This isn't a review, or a post about my thoughts on the game. It's more a players guide. Something that's digital, mobile friendly, and a shortcut to many of the things I consistently look up in the official rulebook.
Advanced RAG
First things first... This is not advanced RAG. But it does mark an evolution in my ability, and feels advanced to 2024 Tony.
The other day, I came across this post. It's a video of El Risitas guy, and the subtitles (included below) convey all the laughably ridiculous things people writing RAG apps today are doing.
My Plans for the 2024 Solar Eclipse
This post is mostly a collection of my notes from here with a little more context throughout.
If you haven't heard, there's going to be a Total Solar Eclipse across North America on April 8th Get Hyped! 🌑
Speech to Text using Whisper
This year, I've had fun finding ways to use OpenAI tooling in everyday life. For most folks, this means interacting with ChatGPT through the app, or in a web browser at https://chat.openai.com. For me, it's been learning how to effectively use the tools programmatically. One of those tools, is Whisper.
People I Follow
This list ended up being way longer than I anticipated. If I wrote about someone here, there's a good chance I've sponsored them in some way. Via Patreon, or buying merch, or books they've authored, etc... I would advocate that you do the same for interesting people that in some way, provide value to you.
Basic Linux Server Setup & Hardening
Here are initial steps I perform on any fresh Linux install. My preferred distro is Debian, so anything below is within that context.