A month ago, I released ConfigMesh, a native macOS app that syncs your dotfiles and app settings across machines with end-to-end encryption. I spent months building what I thought was a solid product: zero-knowledge architecture, smart diffing, version history, granular control over what gets synced.

Then I launched. I ran a DevHunt launch, Reddit sponsored posts, Twitter boosts. Traffic came. Downloads didn’t. People were landing on the site, reading the feature list, and bouncing. Not because they didn’t understand what ConfigMesh does, but because the offer itself wasn’t landing.

After a month of feedback, conversations, and some honest self-reflection, I shipped ConfigMesh 1.1. This post is about the two assumptions I got wrong and what I changed.

Nobody wants to pay $5/month for text files

I launched ConfigMesh at $5/month (or $50/year). From my side of the screen, that felt reasonable. I’d built encryption infrastructure, a sync server, a native macOS app, months of development. Five bucks a month for all of that? Seemed fair.

From the user’s side of the screen, the math looked very different. Dotfiles are tiny. We’re talking kilobytes. A typical user’s entire config collection is smaller than a single iPhone photo. Paying $5/month to sync that felt disproportionate, regardless of the engineering underneath.

This is the indie dev pricing trap: you price based on what it cost you to build, not on what people are willing to pay. The perceived value of syncing text files has a hard ceiling, and $5/month was above it. It didn’t matter that the encryption was best-in-class or that the sync engine was smart. The sticker price created friction before anyone got to experience any of that.

The clearest signal came from Reddit. I posted ConfigMesh on the Mac apps subreddit and the feedback split into two camps. Some people said the price was just too high for what it does. Others went further: they didn’t want another subscription at all, regardless of price. That second group was the more interesting signal. It’s not about the dollar amount, it’s about the model.

ConfigMesh 1.1 drops the price to $1.99/month or $19.99/year. That’s a significant cut, but it better matches where users anchor the value of this kind of tool. More importantly, it’s no longer the only option (more on that below).

Trust isn’t a technical problem

This was the more interesting lesson.

ConfigMesh uses end-to-end encryption with a key that only you hold. The server literally cannot read your data. From a cryptographic standpoint, it doesn’t matter where the data is stored because it’s indistinguishable from random noise without your key.

I assumed that would be enough. It wasn’t.

The same Reddit thread made this clear. People weren’t just pushing back on the price, they were pushing back on the storage model. When I mentioned I was working on a Dropbox backend, the interest shifted noticeably. That was the option people actually wanted. Not cheaper cloud storage, but their own storage.

This isn’t irrational. It’s how people think about sensitive data. Ownership and control are emotional, not just technical. “Your data is encrypted on our servers” will always feel different from “your data is on your Dropbox,” even when the former is arguably more secure. You can’t engineer your way past that.

The deeper realization was that some users don’t need managed cloud storage at all. They already have storage (Dropbox, iCloud, NAS). They don’t want another cloud backend. They just want the sync engine.

Enter BYOS (Bring Your Own Storage)

ConfigMesh 1.1 adds a one-time purchase option with no included cloud storage. Instead, you sync against your own Dropbox.

The tradeoffs are real: Dropbox is slower than the managed backend and more error-prone. Sync conflicts are harder to resolve gracefully when you don’t control the storage layer. It’s a compromise, but it’s the compromise users actually asked for.

And yes, your data on Dropbox is still fully encrypted. Here’s what it actually looks like on disk: just encrypted blobs, completely unreadable without your key.

Here’s how the tiers work now:

• Subscription ($1.99/month or $19.99/year): Fully managed, encrypted cloud storage. Fastest sync, most reliable. Same as before, just cheaper.
• One-time purchase: No cloud storage included. Sync against your own Dropbox. Slower, but you own everything.

If you’re on the one-time purchase tier and try to use the managed ConfigMesh backend, you’ll see a clear upgrade prompt instead of a confusing error.

Dropbox is the first supported provider. More are coming.

What I’d tell myself at launch

Ship your pricing assumptions as cheaply as possible. I could have launched with a lower price and a free trial and validated willingness-to-pay before building conviction around $5/month.

And when users tell you they don’t trust something, don’t argue with their reasoning. Just give them an alternative they’re comfortable with. The best feature you can build is sometimes just getting out of the way.

ConfigMesh 1.1 is available now at configmesh.app. If you tried it before and bounced on the price or the storage model, give it another look.

Remember my Vibe Coding Essentials and Vibe Coding Advanced posts from last year? A lot changed since then, AI coding is now the norm and tools like OpenCode or Claude Code became industry standard. The core idea is still orchestrator models driving development, but the workflow is less “vibes” and more strict task orchestration.

I’ve refined this to work across different tools, but the biggest shift is how I structure responsibility. It is now a 4-persona system where I keep architecture and product decisions, and the agents handle execution loops.

I also created a YouTube video on this workflow if you prefer a walk-through over a long read.

The 4-Persona System

1. Architect: The Planner

Model: Usually Gemini 3 Pro, but any thinking model works

The Architect is the brain of the operation. I usually reach for Gemini 3 Pro here because of its ability to ingest massive context windows and its “thinking mode,” but honestly, any model with strong reasoning capabilities does the job. The key requirement is that it can reason through complex problems without getting distracted by implementation details.

I’ve set it up with access to WebSearch and Perplexity through my MCPNest service. This ensures that whether I’m using Codex, OpenCode, Gemini, or Zed, the toolset remains identical across the board.

The Workflow

The Architect’s sole purpose is to output a comprehensive implementation plan into a plans/xxx.md file (or GitHub Issue). The process is iterative and collaborative. I start by feeding the model my raw requirements and context. It then drafts a plan, which I immediately hand off to another high-reasoning model (like codex-xhigh or gemini-3-pro) for a second opinion.

I usually run multiple passes on the design doc with different thinking agents. One might catch edge cases in the data model, while another suggests a cleaner API contract. I don’t care how long the architect agent takes to “think”: latency is irrelevant because I’m usually running a few of these in the background while I’m doing other things. I layer my own feedback on top of their critiques, and we loop through this cycle a few times until the plan is bulletproof.

Only when the plan is solid do we move forward.

I moved the full Architect prompt and review-loop prompt into a gist to keep this post readable:

Full prompts gist

Architect prompt excerpt (I have another version of this that outputs into GitHub issues):

You are Architect Mode, an implementation planning subagent for OpenCode. You generate structured, actionable plans that junior engineers can execute without including full code or large diffs.

(...)

How you work

- Planning-only (no execution): You MUST NOT modify files, run commands, commit, or open PRs.
- Double-check intent: Even if the user asks for implementation, you still only produce a plan; you do not implement.
- Context gathering: Use `read`, `list`, `glob`, `grep` to understand repo layout and relevant files.
- Output only: Produce a single, concise, well-structured plan.
- Prefer descriptions over code: Reference identifiers and paths inline with backticks.
- If asked to implement: Clearly state you will not make changes, then provide the plan and suggest switching to an implementation agent.

Formatting rules

(...)

Required plan sections

- Summary
- File Changes
- Implementation Steps
- Contracts & Data
- Risks & Assumptions
- Validation
- Rollback
- Follow-ups

Quality bar

- List exact files to touch, or explain how to find them.
- Keep context minimal but sufficient for execution.
- No full implementation or diffs.
- State assumptions explicitly if repository context is incomplete.

Persistence rule

- Always write or update the plan in `plans/YYYYMM-<plan-name>.md`.

Safety

(...)

Example of a created plan (shortened):

# Add Stats Page

## Goal
Add a new `/stats` page to provide users with insights into their Japanese usage and common mistakes.

## Features
1. **Correction Type Distribution**: Donut/pie chart by correction type.
2. **Drill-down View**: Click a type to show recent mistakes.

## Implementation Plan

### 1. Backend: Update `Fixmyjp.Corrections`
- Add `get_correction_type_stats(user_id)`
- Add `list_recent_corrections_by_type(user_id, type, limit)`

### 2. Frontend: Create `FixmyjpWeb.StatsLive`
- Add `lib/fixmyjp_web/live/stats_live.ex`
- Use `FixmyjpWeb.Auth` for auth
- UI:
  - Chart (CSS/SVG with Tailwind)
  - Interactive legend
  - Mistake list with `original` -> `corrected` and explanation

### 3. Routing
- Add `live("/stats", StatsLive, :index)` inside authenticated scope

### 4. Navigation
- Add Stats link in app layout

## Technical Details

### Database Queries
(...)

### UI Structure
(...)

2. Implementer: The Builder

Model: Usually not a very-high thinking model: codex-high, codex-medium, or Sonnet 4.5

Once the plan is set, the Implementer takes over. I explicitly avoid the heavy thinking models here (like Claude 4.5 Opus) because the hard thinking has already been done by the Architect. The Implementer doesn’t need to be a genius; it just needs to be obedient.

Its job is straightforward: follow the plans/xxx.md file. It writes the code, runs my verification steps (linting, formatting, testing, compiling), and handles the git choreography of creating branches, committing changes, and opening PRs.

I often offload this to cloud agents like Jules, Copilot or Codex, for example with a command like this:

cat plans/202601-deployment-last-used-tracking.md | jules remote new --repo dvcrn/mcpnest

The beauty is that the execution environment does not matter, local terminal or cloud container, as long as the plan is followed. Since all environments are configured identically, I can swap them out interchangeably. I can create a GitHub issue and assign it to Copilot, but I can also open a terminal on my server and give it to Claude Code.

3. Review Agents: The Critics

Models: Gemini, Codex, Copilot in the cloud

This is where I try to avoid the echo chamber. If a single model writes the code and reviews it, it’s likely to miss its own blind spots. That’s why I have Gemini and Codex automatically review every PR that gets created.

For heavier or more critical changes, I throw Copilot into the mix as well. This gives me 3 independent agents reviewing the code, drastically reducing the chance of model bias slipping through.

I also keep a review persona handy in OpenCode, which I can summon with a slash command whenever I need a quick sanity check on a specific snippet.

4. Final Touchups: The Resolver

Model: Codex (or whatever model I’m using for coding at that time)

Finally, we have the Resolver. This isn’t a single step but a loop. Codex reads the feedback from the Review Agents, asks me which points I want to address, and then handles them. Once the changes are made, it automatically requests another review from the critics. This loop continues until all critical issues are resolved and the code is polished.

For my local tools, I have this neatly encapsulated into commands or subagents, for example this slash command here that I can use in OpenCode, Claude Code, or Codex:

Use `gh` or available tools to fetch all GitHub pull request comments for the PR that merges the current branch into main or master.

Current repo:

!`git remote -v`

Current branch:

!`git branch -r --contains HEAD`

Current PRs:

!`gh pr list`

Analyze them, check the files they mentioned and output a list for each of them, explaining if this is relevant and should be fixed or not

Use the following command to get the comments:

```
gh api graphql -F owner='<OWNER>' -F name='<REPO>' -F number=<PR NUMBER> -f
query='query($name: String!, $owner: String!, $number: Int!) {
repository(owner: $owner, name: $name) { pullRequest(number: $number) {
reviewThreads(last: 100) { nodes { isResolved path comments(first: 1) {
nodes { body line } } } } } } }' --jq '.data.repository.pullRequest.
reviewThreads.nodes[] | select(.isResolved == false) | "File: \(.
path)\nComment: \(.comments.nodes[0].body)\n---"'
```

Then ask the user whether we should fix them.

After fixing, mark each fixed comment as resolved using this command:

```
gh api graphql -F threadId="PRRT_kwDOQDhXrc5oS98H" -f
query='mutation($threadId: ID!) { resolveReviewThread(input: {threadId:
$threadId}) { thread { isResolved } } }'
```

Once the code is committed and pushed, ask the user whether we should re-request a review by commenting "/gemini review @codex review"

Built for Parallelization

The real superpower of this system is not just code generation. It scales.

Because the Architect only outputs markdown files in the plans/ folder, I can have multiple Architect agents running in parallel without ever hitting a merge conflict. I often have half a dozen unimplemented plans sitting in my folder, iterating on them while the Implementer is busy building something else. It shifts the bottleneck from my own attention span to total system throughput. I’ve stopped optimizing for how fast an agent responds and started optimizing for how many high-quality plans I can have maturing simultaneously.

The plan is the contract. Once it’s written, it doesn’t matter which agent picks it up or when.

Agents Control Agents

I mentioned sub-agents in my past posts, and they are the glue that holds this together. My agents are empowered to call other agents. The Implementer can spawn a code-review sub-agent to check its own work before committing. It can call the commit agent to handle the git history. Each agent has it’s own responsibility and context, not all agents need to know everything and this is how I split knowledge.

It’s a hierarchy where agents review other agents, and corrections are implemented by yet other agents.

So what do I do?

I focus on the remaining 10%:

• Making high-level architecture decisions
• Reviewing the final code quality
• Providing specific constraints (e.g., “use this package,” “structure it this way”)

The agents handle the other 90%. It is not “AI writes all my code.” It is “AI handles boilerplate, tests, reviews, and iteration, while I focus on product and architecture.”

Where This Still Fails

This setup is fast, but it still breaks in predictable ways:

1. Plan drift: Implementers sometimes follow the spirit of a plan, not the exact constraints.
2. False confidence: Multiple model approvals can still miss the same blind spot.
3. Review noise: Automated review threads can create churn if severity is not filtered.
4. Cost and latency: Parallel agents are great until you look at spend and queue time.
5. Over-planning risk: Architect mode can over-spec work that should have been a quick spike.

None of this is magic. It is just a tighter process with better delegation boundaries.

What Changed Since “Vibe Coding”

The core orchestrator approach is still the same, but the maturity of the system has leveled up:

1. Clearer personas: Each model has a specific job, not just “the AI”.
2. Better parallelization: Work is decoupled via Plans, avoiding conflicts.
3. More automation: Review loops happen automatically without me babysitting.
4. Tool parity: My MCP setup is identical across every tool I use.
5. Cloud flexibility: Implementation is commoditized; I can use any cloud agent.

The result is that I can maintain 3-4 projects simultaneously, with agents handling the heavy lifting while I steer the ship.

Questions or want to share your AI coding setup? Let me know on Twitter/X.

My “Move Fast” Stack for 2026

TL;DR: Moving fast in 2026 isn’t just about the code—it’s about picking the right tools and letting AI handle the grunt work. My current go-to: Golang or Elixir (depending on what I’m building), SQLite + Litestream, and a 3-agent pipeline for code quality.

In my last post, I reflected on the ups and downs of the past 14 months. One of the biggest takeaways? As an indie hacker, velocity is your only real edge.

You’re competing against teams with actual budgets, so you have to be able to ship, pivot, and maintain 3-4 apps at the same time without burning out.

Here’s the stack—both physical and digital—that lets me move at that pace in 2026.

The Stack

The goal here isn’t “the most scalable” or “the most trendy.” It’s what gets me from idea to landing page the fastest.

Server Stack: Elixir vs Golang

I use both Elixir and Golang, and the choice depends on what I’m building.

Golang is my AI-friendly workhorse: Go compiles to a single binary, which makes it perfect for anything that needs to run on iOS/Android, CLI tools, or simple servers. The static typing is a game-changer when working with AI agents. The compile → error → fix loop is incredibly tight. AI writes code, Go compiler catches type errors immediately, AI fixes them, repeat. No surprises at runtime. This makes it perfect for:

• Simple servers that need to be rock-solid
• Anything embedded in iOS/Android apps (single binary, easy to share across platforms)
• APIs and microservices where I need confidence that it won’t blow up in production
• Background workers and CLI tools
• NPM packages that thinly wrap around Golang binaries

Elixir for web apps: Phoenix + LiveView is still unbeatable when I need a full web app with real-time features. The entire frontend is built-in, so I skip the whole React/API dance. Just write server-side code and the UI updates automatically. Perfect for consumer-facing web apps.

The decision tree:

• Consumer-facing web app? → Elixir (fully-featured package, LiveView gets you there fast, easy to extend)
• Starting with a server/service, or planning to ship to desktop/mobile? → Golang (start with CLI, embed later)
• Simple API or background worker? → Golang (compile-time safety)
• Need real-time features or distributed systems magic? → Elixir (OTP + LiveView)

The power of OTP (when I use Elixir): When I do reach for Elixir and need to scale, OTP lets me do it within the same app. Need a queue and worker? Just spin up a GenServer process with Oban. Need a cache? That’s another GenServer. Need a cron or periodic task? Another lightweight process. I don’t need to reach for Redis or external infra for every little thing. (I wrote about this before if you want to see how powerful this is)

Databases

SQLite + Litestream: For quick projects, I start with SQLite and Litestream. It’s a database replication and backup layer that sits on top of SQLite and streams changes to S3. No need to manage a “real” database cluster for small-to-medium apps. Every new project starts with this stack to not have additional database cost.

PlanetScale: For anything that actually needs a hosted HA database, PlanetScale is the choice. It starts at $5, and after trying a few providers, this was easily the nicest to work with.

Web & Apps

Web: Elixir with AlpineJS (LiveView) for most things. But if I just need a simple JavaScript site without server-side logic, or something that purely consumes an API, I’ll use React + react-router over Next.js. Pure JS apps are always in React for me - no need for the Next.js overhead.

Apps: My current favorite pattern is Embedded Golang in Swift. I get to write the core logic in a language I love (Go) and wrap it in a native, buttery-smooth iOS UI. Best of both worlds. If I decide to build an Android app as well, I can use the same business logic in Golang and embed it into my Android app.

Auth: Firebase Auth. Don’t build your own auth. Just don’t. I’ve seen too many people waste weeks on this. Firebase has a great free tier and so far I’ve not once exceeded it. Supabase is a close second, but the cost per project was too much for me.

Subscriptions: RevenueCat. Much like auth, building your own subscription backend is a trap. RevenueCat handles everything related to payments, subscriptions, and entitlements. I interface with it as my single source of truth for whether a user is allowed to access a feature. It’s the only way to ship IAP in a weekend instead of a month.

AI Workflow: The 4-Agent Pipeline

Here’s where things get interesting. I don’t just use AI for autocomplete—I use a sophisticated 4-agent pipeline to handle most of the grunt work. I wrote a deep dive on this in my post How I Actually Code with AI in 2027, but here is the gist.

My repos are built for AI agents. I maintain RULES, CONVENTIONS, and AGENTS.md / CLAUDE.md files that explain how the codebase works. But I don’t rely on the AI to follow rules perfectly. Just like I prefer compiled languages that catch errors at compile-time, I use pre-commit hooks as a static guarantee. The AI can write whatever it wants, but it won’t get committed unless it passes formatting, linting, tests, type checking, and build validation.

The pipeline:

1. Architect: A high-reasoning model (Gemini 3 Pro) creates a detailed implementation plan.
2. Implementer: A builder model (Codex/Claude) follows the plan and writes the code.
3. Reviewers: Independent agents (Gemini, Codex, Copilot) review the PR for bugs and logic errors.
4. Resolver: An agent fixes the issues raised by reviewers until the code is polished.

Cloud agents for clear tasks: Once I have a design doc ready, I pass it to a cloud agent (Codex, Copilot, Jules) that follows the plan while I work on something else.

Oracle server for exploratory work: I run agents on Oracle’s free tier for the messy, long-running tasks that aren’t clearly defined yet. It’s like having a second engineer working on stuff while I do something else. I can SSH in from anywhere - even my phone. I keep my dev server in sync with my macOS system through ConfigMesh which e2ee syncs dotfiles frictionless.

By the time code reaches me, I’m only focusing on the final 10% - architecture decisions, product direction, and tricky edge cases. The agents handle the boilerplate, testing, and reviews.

This is what lets me maintain 3-4 projects simultaneously while traveling.

Hosting: Built for Nomads

This is where I optimize for speed, cost, and the ability to deploy from anywhere in the world.

Phase 1 - Hetzner + Kubernetes: Most of my small experiments start here. I run a self-managed k8s cluster on Hetzner where I can quickly deploy stuff from wherever I am without incurring additional cost per project. It’s my “testing ground” - if something shows promise, I move it to proper hosting. If it fails, I just delete the namespace. No extra bills or cost. I love how nice k8s is once you learn the basics.

Phase 2 - Fly.io: Once a project graduates from experiments to “real thing,” it goes on Fly.io. I can quickly scale to the edge, and when I’m using PlanetScale, I can deploy close to the database region I’m in. Latency matters. The CLI is also dead simple to use from anywhere - even from my phone if I really need to.

Cloudflare Workers: All my React + react-router stuff lives here. Static sites, edge functions, anything that doesn’t need a backend. Fast, cheap, globally distributed. Perfect for when I’m traveling and need something to just work. Seriously, I love Cloudflare. Using it brings me joy, it’s that nice.

PlanetScale + Fly.io combo: When SQLite + Litestream isn’t enough and I need a real hosted database, I use PlanetScale. I can deploy Fly.io apps close to the PlanetScale region to keep latency low.

Logistics: The Foundation & Other Stuff I love

Coding is 50% environment. If the logistics aren’t sorted, the “vibe” is off, and the code doesn’t flow. Everything in my stack is built around supporting a nomadic lifestyle - I need to be able to work from anywhere.

WeWork: Even with all the corporate drama of years past, for a digital nomad it’s still the best “office-as-a-service.” Having a consistent, reliable place to work in almost any city keeps me grounded when I’m traveling. It’s my version of having “coworkers.”

Cloudflare Zero Trust: This is the glue that ties everything together. Most of my work - whether on iPad or macOS - happens while connected to Zero Trust tunnels. It connects my infra, my projects, my servers, everything. Some services even connect to other infrastructure through private tunnels. Security is usually a speed bump, but Zero Trust makes it invisible. I can access all my internal tools and dev environments securely from anywhere without messing with VPNs. (I wrote about this setup if you’re interested)

Noteplan: My brain is a mess of tasks and markdown files. Noteplan is where they live. It’s simple, markdown-based, and doesn’t get in the way. Everything from writing ideas to project plans to work journals goes in here.

My Remarkable tablet: As silly as that sounds, I haven’t found a better device that brings structure to everything I do. There’s something very Zen about sitting in a cafe with only a paper tablet and being forced to be semi-analog for once.

What’s next?

The stack will probably change by 2027, but for now, this is what’s working. It’s about minimizing friction, maximizing “vibe,” and being able to work from anywhere in the world.

If you’re building something, stop worrying about the “perfect” architecture and pick the tools that let you skip the boring stuff. For me, that’s Go when working with AI (tight feedback loops), Elixir when I need the magic (LiveView + OTP + easy scaling), a pipeline of AI agents to handle the grunt work and reviews, and infrastructure that I can access from my phone if needed.

The goal isn’t to write more code - it’s to write less code that matters more, while agents handle the rest.

What’s in your 2026 stack? Let me know on Twitter/X.

I’m excited to announce the release of ConfigMesh, a new macOS app I’ve been working on to solve a problem I’ve had for years: keeping my application settings and dotfiles in sync across multiple Macs.

Check it out at configmesh.app.

The Problem with Dotfiles

If you’re a developer, you probably have a set of “dotfiles” - configuration files for your shell, editor, git, and other tools. Managing these has always been a bit of a chore.

For a long time, I used a git repository and some symlinking scripts (like many of us do). It works, but it requires manual intervention. You have to remember to commit changes, push them, pull them on the other machine, and run the install script. If you forget, you end up with out-of-sync configs and weird behavior.

And then there’s the issue of secrets. You can’t just commit your API keys or private tokens to a public (or even private) git repo without feeling a bit uneasy.

Enter ConfigMesh

ConfigMesh takes a different approach. It’s a native macOS app that runs in the background and watches your config files for changes. When a file changes, it automatically encrypts it and syncs it to your other machines.

Key Features

🔒 End-to-End Encryption

This was a non-negotiable for me. ConfigMesh uses end-to-end encryption with a key that you generate and own. The data is encrypted on your device before it ever leaves your machine. This means even if the sync server is compromised, your data is safe. Your key never leaves your device.

⚡️ Smart Syncing

The app intelligently diffs your config files. It doesn’t just blindly overwrite things; it checks for changes and only syncs files that have actually changed across machines, keeping everything in sync without you thinking about it.

🎯 Granular Control

You have full control over exactly what gets synced. ConfigMesh uses a powerful rules engine that lets you specify files and directories using standard glob patterns. Want to sync your entire .config/nvim folder but exclude lazy-lock.json? Easy.

🕰️ Time Machine for Configs

Ever messed up a config file and wished you could go back 10 minutes? ConfigMesh keeps a version history of your files. You can roll back to any previous version with a single click.

What can you do with it?

• Keep Vim settings in sync: Have your perfect Neovim setup available on all your local machines and servers.
• Backup AI Skills: Store and sync your custom skills for OpenCode and Claude Code so your AI assistant is always ready.
• App Configurations: Save and restore configurations for apps like Xcode, Mail, Terminal, and more. If you get a new Mac, just install ConfigMesh and your environment is ready.

Native & Fast

ConfigMesh is built as a native macOS app. It’s designed to be lightweight and unobtrusive. It sits in your menu bar and does its job without getting in your way.

Roadmap

I’m just getting started. Here is what is coming next:

• Bring Your Own Storage: Soon you’ll be able to pay a one-time fee and bring your own storage (starting with Dropbox).
• Server Support: I’m wrapping up a Unix utility that will allow you to sync your files to headless servers.

Try it out!

I’ve been using ConfigMesh internally for a while now to keep my work and personal Macbooks in sync, and it’s been a game-changer. No more “wait, why is my alias not working?” moments.

You can download it today at configmesh.app. It currently comes with a $5/month plan that includes fully managed, secure cloud storage.

Let me know what you think!

TL;DR: 14 months ago, I quit my job to build indie products. Built 7 projects, currently at ~$45 MRR total across all projects, 900+ active users. Most projects failed financially, but learned valuable lessons about building for broad audiences over niche tools.

A little over 14 months ago, I made the drastic decision to quit my full-time job of 6+ years as Engineering Manager at Mercari to go my own path and find my own success.

It wasn’t a rage-quit or a “burn it all down” moment. It was a calculated risk to see if I could build a life entirely on my own terms. I’d like to reflect a bit on the year, how it’s going and what I’ll be doing going forward.

Do I regret my decision?

No, but it’s hard. I already went into this with the mindset that the first 50 or so ideas will fail, and maybe if I’m lucky, just maybe, the 51st idea will make some money.

Even with this mindset, actually failing over and over again is hard. We need small successes to understand whether what we’re doing is the correct thing, and being starved of wins is mentally taxing.

What I worked on: 7-8 projects over 14 months

The point of indie hacking is to move quick, try out small ideas, see what works and then focus more time or move on. Over the 14 months I worked on roughly 7-8 projects.

Looking back, these projects fall into three distinct buckets, each teaching me a very different lesson.

Group 1: The Dev Tools Trap

I started where I was most comfortable: building tools for other developers. This turned out to be the hardest path. The market is saturated, the users are demanding, and competing with free venture-backed tools is a nightmare.

Microfn (microFunction) - Your toolbox for tiny composable functions

• What: Cloud service, function runner and editor for small (micro), composable JavaScript/TypeScript functions. Hook them up to CRON, Webhooks, Agents over MCP or other automation platforms. It double serves as a dev environment for AI agents, to allow agents to quickly deploy code and use that code as new tools right away.
• How it went: 1 active user (myself). Fully functional and stable, but financially a failure. My partner lost interest and exited the project midway through, leaving me alone with it. No marketing has been done.
• Status: Feature-complete maintenance mode. It will stay in organic-growth-maintenance mode until usage picks up, with max 1 new development per week.
• The Lesson: Building a dev service is hard; getting people to switch from their existing workflow (Vercel, AWS, etc.) is near impossible without a massive marketing budget. Ad keywords are always outbid by the bigger players (Cloudflare, Google, etc). The potential is there, but it needs it’s killer-feature moment.

MCPNest - Host your MCP servers in the cloud

• What: https://mcpnest.dev - Cloud service for running MCP servers with focus on simplicity. This was an experiment built at the peak of the MCP servers craze. It greatly simplifies re-using sets of servers, for example to have a specific set of servers + tools for a dev persona, one for a casual persona.
• How it went: 14 WAU, currently pays for itself. Financially a failure. Active dev time roughly 1 week.
• Status: Active, low maintenance. As long as it pays for itself it will continue. It’s very low maintenance and does what it claims to do very well.
• The Lesson: Even jumping on a “hot trend” (MCP) isn’t enough if the audience is too niche. The timing could should have been 1-2 months faster to really hit the peak of the MCP craze.

Mailwire - Smart Email Proxy

• What: Mailwire is a transparent email proxy for power users. This is not a new project and already alive for a couple years, although in maintenance mode. It sits in between the users email client and their email server, and adds power-user features like auto-stripping of sensitive information or rewriting email senders (bank1@mydomain.com) to match the recipient address (foo@bank.com). Active development time was many many months a few years ago.
• How it went: 2 active users, a big failure.
• Status: Low-energy maintenance mode. As I’m a user myself and need all of this, it will continue, but stay in maintenance and low-energy mode.
• The Lesson: This app is just too niche. I built it for myself, and turns out, I’m the only one who really needs it.

Group 2: The Steady Utilities

While the ambitious platforms struggled, my small, simple utilities found a quiet, steady rhythm. These aren’t trying to change the world, just fix a small annoyance.

Advanced AI Actions

• What: One-time purchase iOS utility app to expose LLM providers through Siri Shortcuts.
• How it went: Around $15 MRR (amortized). Considering that it’s not a subscription app, I would say this is a success. I am glad people get value out of it. Active dev time was about a week, and then a few days here and there to fix bugs.
• Status: Full support going forward. Maybe migrating it to a full free-but-with-unlock app and adding a paid provider option to see if people can be converted to monthly payments.
• The Lesson: People will pay for simple utilities that solve a specific problem on their device.

Masked Email Manager

• What: Utility iOS app for managing Fastmail’s Masked Email feature. Already built this before I quit my job, but I’ve been adding more features such as Siri Shortcuts support to it.
• How it went: Around 5-6 unlocks each month. I think that’s okay considering how niche it is, and being one-time purchase. Active development time of around 1 month in total.
• Status: Full support, will continue as one-time unlock utility app.
• The Lesson: Niche utilities can work if they draft off a larger platform’s userbase (Fastmail).

Group 3: The Consumer Bets

This is where I saw the most actual traction—solving problems for regular people, not just other coders.

Fix My Japanese

• What: Japanese grammar and sentence structure correction through AI, created back when the bleeding edge davinci model was the smartest thing around. Comes as web service and iOS app that’s free with subscription.
• How it went: $14 MRR, 6 active subscriptions, 180 DAU, 700 WAU. Project is steadily growing but financially a failure. Active dev time is a couple months at this point, started way back.
• Status: Active growth. As long as it pays for itself it will stay alive. I will continue to try and integrate more paid features to see if some of those 700 WAU can be converted to paid users.
• The Lesson: Steadily growing but effectively a failure financially. It’s a great free product, but getting people to pay for additional functionality is hard when the free tier is “good enough”. Nowadays people use ChatGPT which can do 80% of what Fix My Japanese does, further making it unlikely for people to pay.

Quickshot Photo Editor

• What: Quickshot is an AI-powered photo editor for iPhone (Edit photos through words, not tools). It started when photo editing with AI models became better and viable. It’s using a clever pipeline of models and AI upscaling to make targeted edits without destroying the original photo.
• How it went: 6 active subscriptions, $16 MRR. Financially a failure, but one of my apps with the most potential.
• Status: Active development. This app has potential and I will put more time into it after my current focus is completed.
• The Lesson: Despite fewer updates, this made more money than the others. It has the most potential because it appeals to a broad consumer base, but nowadays also competes against ChatGPT which is a hard uphill battle.

Fun stuff (The Playground)

• esimdb: E-Sim comparison site using providers I frequently use and vouch for.
• Desks for WeWork: Native iOS app for managing WeWork bookings.
• The Enabler: Companion that always agrees with you, no matter what.

What I learned

But financials aren’t the only things I managed to learn. I learned a lot of things on the way that hopefully will help me keeping my focus going forward more sharp.

1. Always build with the monetization strategy in mind

Very obvious first learning, but never build something without knowing how to monetize it. I am pretty good at keeping at that already, but some leftover apps (mainly Fix My Japanese) are fighting with the problem of converting free users to a paid tier.

1.1 But, keep a free tier around because most people don’t want to pay money

Getting paid users is tough. There’s a lot of mental steps involved to getting someone to pay for your thing, but you’ll want those users as well for growth. Offer some kind of free tier that is heavily restricted in usage and clearly pushes to the upgrade path.

2. SaaS is hard and competitive

Microfn taught me that building dev tooling and hosting services is extremely hard. The field is highly competitive with the big players (like Cloudflare workers, Firebase, Google App Engine, etc) being hard to out-maneuver for user acquisition. Can’t run ads and outbid those companies, which leaves organic growth.

3. Most of my stuff is too niche, focus on broad consumers instead

Yeah hard truth. Mailwire is a great transparent email proxy, but who in the world besides me needs this? Microfn is a great platform, but most people don’t need this. MCP Nest does exactly what it ought out to do, but the audience is tiny.

The tech world is a tight community, better to focus on broad consumer apps instead (Quickshot, Fix My Japanese) where there’s more room for niche players and lower customer acquisition costs.

4. You need luck

Some apps like Quickshot that haven’t received updates still made more money than a lot of other apps that actively received new versions. Even the best idea needs the right timing and the right bit of luck to grow.

5. Indie hacking is lonely

Another hard truth: indie hacking and working on stuff alone can feel lonely fast.

It’s a strange paradox. I have the ultimate freedom, I can travel anywhere, work from a café in Tokyo or a WeWork in Sydney, and take a random Tuesday off to go hiking. But that freedom disconnects you from the rhythm of the rest of the world.

My friends work full-time jobs. They can’t do spontaneous trips. They have “work stories” and office drama that I can no longer relate to. There were times when I was so wired into my work, in a foreign city, that I forgot where I actually was.

Having coworkers or a community is nice, and my closest equivalent to that are currently the WeWork offices I use for working across the globe. It’s a different kind of lonely, not sad, just… singular.

6. Marketing is a skill I desperately need to learn

Building a great product is only half the battle—maybe even less. I’ve realized that I have a major blind spot when it comes to marketing. Some of the services I built are technically solid and solve real problems, but I have no idea how to get them in front of the right people.

As a developer, it’s easy to fall into the “build it and they will come” trap. But they don’t. Marketing is a skill just like coding, and it’s one I need to get better at if I want any of these projects to actually succeed.

Going forward

So what is next for me in 2026?

I have no regrets and no plan to stop (for now), but I will focus on broader consumer apps and keep the dev-centric services in maintenance mode with minimal time put into them.

Specific goals for 2026:

• Reach $500 MRR by end of year
• Focus on 2-3 core projects instead of spreading thin across 7+
• Be more data-driven: implement proper analytics with NewRelic and Posthog dashboards to see what works and what doesn’t
• Double down on Quickshot and Fix My Japanese - the projects with the most traction

I may need to start winding down some things in 2026 to keep my focus tight. That still feels like shooting your baby, but as my focus becomes thinner, this will become necessary.

If you’re interested in following my indie hacking journey, you can find me on Twitter/X or check out my projects at david.coffee. Always happy to chat with fellow indie hackers about the ups and downs of this journey.

I want to introduce a project I’ve been working on for the past few months. I think it’s finally in a good enough state that more people can get onto it.

Here’s what microfn is about and all the cool stuff you can do with it!

Tiny composable functions

Microfn is all about small functions (hence the name micro) - you have a small piece of functionality in the form of a function and want to run it somewhere. Well microfn is just the place for that! We run that function for you and make sure it is available for whenever you need it.

Let’s say I have this snippet here that just queries wttr.in to get the current weather:

export async function main() {
  const url = "https://wttr.in/Tokyo?format=3&m";
  try {
    const response = await fetch(url);
    if (!response.ok) {
      throw new Error(`HTTP error! status: ${response.status}`);
    }
    return response.text();
  } catch (error) {
    console.error("Error fetching weather:", error);
    return { error: "Failed to fetch weather data" };
  }
}

We can quickly deploy this to microfn in seconds by writing the code, hitting save and it’s deployed. Now that it’s deployed we can do all kinds of stuff with it!

Functions from anywhere

By default, microfn gives you a webhook that you can call in the form of

curl -X POST https://microfn.dev/run/david/get-tokyo-weather

Executing this curl will, as you may have guessed, get the current Tokyo weather by executing your function on the microfn platform.

But we can do more!

Functions meet other functions

Isolated functions are already useful on their own, but they become even more powerful when we allow functions to interact with each other.

For example, we can encapsulate recurring logic to send a discord message into a send-discord-message function:

export async function main(input: { channelId: string, content: string }) {
  const token = await secret.getRequired("DISCORD_BOT_TOKEN");

  const { channelId, content } = input;

  const discordPayload = {
    channelId,
    content
  };

  const response = await fetch(
    `https://discord.com/api/v10/channels/${discordPayload.channelId}/messages`,
    {
      method: "POST",
      headers: {
        Authorization: `Bot ${token}`,
        "Content-Type": "application/json",
      },
      body: JSON.stringify({ content: discordPayload.content }),
    },
  );

  const data = await response.json();
  if (!response.ok) {
    throw new Error(
      `Discord API error: ${response.status} ${JSON.stringify(data)}`,
    );
  }

  return data;
}

We could already plug the webhook into places where we want to send a discord message, effectively giving us a curl-to-discord bridge that hides away secrets and logic: Calling the curl will send a message.

But we can also hook this up to other functions that want to send discord messages:

import fn from "@microfn/fn";

export async function main() {
  const weather = await fn.executeFunction("david/get-tokyo-weather");

  return await fn.executeFunction("david/send-discord-message", {
    channelId: "1419082455608197140",
    content: weather,
  });
}

See what we did there? We hooked our 2 functions up into a new function that composes them and sends the current weather into discord!

Automation triggers

Beyond manually calling functions, microfn supports automatic triggers that execute your functions based on events.

Scheduled execution

Functions can be executed on a schedule through CRON:

With just 2 clicks, we turned our function into an hourly weather notifier in discord!

Email triggers

We can also set up functions to respond to emails. Enabling the email trigger gives you a new email address in the form of fn-cdmixwn7kw7p@mail.microfn.dev. Any email sent to this address will automatically trigger your function:

import fn from "@microfn/fn";

export async function email(data: { email: Email }) {
  if (data.email.body.indexOf("microfn") != -1) {
    const content = `New email from ${data.email.from}: ${data.email.subject}`;
    return await fn.executeFunction("david/send-discord-message", {
      channelId: "1419082455608197140",
      content: content,
    });
  }
}

Now we have a discord notifier for whenever someone sends us an email with the word “microfn” in the body.

This makes it easy to handle things like Google Voice SMS-to-email forwarding - just parse the email body in the email() callback, extract the SMS content, and forward it wherever you need!

Functions meet AI

Now here’s where things get really interesting. Tiny functions are a fantastic candidate to be used as AI tools - they encapsulate logic, do one job, and are easy to reason with.

We can hook microfn up to an LLM through an MCP and ask it to get the Tokyo weather:

But what’s even cooler is that we can ask AI to make changes to our functions to give itself new skills. One simple prompt, and the AI does the rest!

And executing this new variant gives us this result:

This turns microfn into a self-expanding toolbox for your AI assistant. Something missing? Just have AI write its own skill! No need to install new MCP servers just to add one extra capability.

Real-world integrations

Your functions are like micro tools that you can plug into anything: MCP, Siri Shortcuts, GitHub Actions, Webhooks, Emails, Cron - you name it!

Here are some concrete examples of what you can build:

GitHub webhook handler

Create a github-webhook function and hook it up as your GitHub webhook URL. Now you have an entry funnel for GitHub events.

Compose it with your discord or slack notification function, and you get real-time notifications with custom parsing for whenever a new issue gets opened or a PR is created:

import fn from "@microfn/fn";

export async function main(input: { action: string, issue: any }) {
  if (input.action === "opened") {
    const message = `New issue: ${input.issue.title}\n${input.issue.html_url}`;
    return await fn.executeFunction("david/send-discord-message", {
      channelId: "1419082455608197140",
      content: message,
    });
  }
}

Twilio SMS automation

Hook a function up as a Twilio webhook when a new message has been received and you have programmable SMS funneling in seconds:

import fn from "@microfn/fn";

export async function main(input: { From: string, Body: string }) {
  // Forward SMS to Telegram, Discord, or anywhere else
  const message = `SMS from ${input.From}: ${input.Body}`;
  return await fn.executeFunction("david/send-discord-message", {
    channelId: "1419082455608197140",
    content: message,
  });
}

Now all your SMS messages are in one place, with all the filtering and routing logic you need.

Release notifications

Want release notifications for your projects? Create a release-notifier function, add the webhook to your GitHub release workflow or deployment script, and you have programmable release notifications:

import fn from "@microfn/fn";

export async function main(input: { version: string, notes: string }) {
  const message = `🚀 New release: ${input.version}\n${input.notes}`;

  // Send to multiple channels at once
  await fn.executeFunction("david/send-discord-message", {
    channelId: "1419082455608197140",
    content: message,
  });

  // Also post to Twitter, send to Slack, update a status page, etc.
  // All in one function!
}

Your toolbox, ready for when you need it!

Microfn gets your functions running quick! Think your local npm, but in the cloud - you have your function, you hit save, and it’s deployed. Done.

It’s your toolbox for tiny stuff, your hammers, drivers and drills, ready instantly for when you need it.

I am using microfn heavily myself and in the short time of using it, it has already positioned itself as an invaluable tool and in the middle of most of my automation flows.

Microfn comes with a free tier to try things out, and a flat $8/mo for the full version.

Give it a try and let us know what you think!

A while ago, after frustration with Tailscale in environments where it couldn’t properly penetrate NAT/firewall and get a p2p connection, I decided to invest some time into learning something new: Cloudflare Zero Trust + Warp.

There are so many new concepts, but after way too long, I can finally say that I understand Cloudflare Zero Trust Warp now. I am a full-on Cloudflare Zero Trust with Warp convert, and while I still have Tailscale running in parallel, almost everything I do now is going through Zero Trust tunnels.

This post is an explanation of the basic concepts, because I’m sure others will have similar issues wrapping their head around it.

Why tho?

Why would you even sink so much time into learning this? What does it give you?

Argo tunnels through Zero Trust allow you to do a bunch of really cool things:

• Connect private networks together - can be home networks, can be kubernetes clusters, you can create tunnels to and from every infra
• Expose private services to the public, on public hostnames, no matter where they are running. You could even put your router running at 192.168.1.1 on the internet, accessible to everyone, no Warp client required
• Create fully private networks with private IPs (10.x.x.x) that only resolve when Warp is connected, to services you specify
• Quickly expose a public route to any service running locally or on any server, for quick development, testing webhooks or giving coworkers a quick preview
• Create a fully private network running at home that’s only available when you’re connected to the Warp VPN client, or only to you, reachable anywhere
• No worries about NAT, everything goes through the Cloudflare network, no direct p2p connection required
• Add very granular access policies on who can access what - what login method does the user need, which email addresses are allowed. Allow bots and server-to-server exceptions with service access tokens.
  • Does the user need to have Warp running? Does he need to be enrolled in Zero Trust? Does he need some special permission flag?
• Authenticate to SSH servers through Zero Trust access policies without the need of SSH keys. Just connect Warp, type ssh host and you’re logged in
  • Close public SSH ports completely to only allow login through Warp
• Get the benefits of Cloudflare VPN edge routing on top (similar to 1.1.1.1 Warp+)

Quickie: Cloudflare Zero Trust vs Tailscale

To get this out of the way:

• Tailscale: peer-to-peer, uses NAT and firewall penetration methods to establish p2p connections. If not possible, it goes through central relay servers. Absolute best speed and latency if a connection is established.
• Cloudflare: All traffic (with the exception of warp-to-warp routing, which is p2p) goes through Cloudflare’s edge network. So even SSH-ing into your local router will hop through Cloudflare servers. This adds latency, but no issues with NAT at all.

Cloudflared != Warp

Cloudflare has 2 tools available: Warp Client and Cloudflared. They interact with each other and have similarities in some areas but are not the same.

Warp Client

The tool that connects you to the Cloudflare network. This is the thing that you configure to add clients into your Zero Trust network and enforces policies.

Usually this runs on clients, but can also run on servers.

Warp client also supports warp-to-warp routing which is a true p2p connection similar to Tailscale.

Cloudflared

The thing that creates a tunnel and adds it to the Zero Trust network.

Most commonly you run this on servers to expose tunnels into your network, but you can also run it on clients.

On the client side you can use cloudflared access to establish a connection with other things in your Zero Trust network.

Can also create one-time-use tunnels that aren’t connected to the Zero Trust network. Good for testing.

Tunnels, Routes, Targets

This took me the longest to understand. Zero Trust allows you to configure Tunnels, Routes and Targets; here’s how they interplay.

Tunnels

The most important part of your setup. Tunnels are deployed through cloudflared and are simply an exit for traffic. Think of it as a literal tunnel that has its end somewhere.

Tunnels are deployed to infrastructure in the target network. So if you have a home network with 192.168.1.1/24, you want to deploy cloudflared on any machine that’s always on and within that network. It can be your router, or your Raspi, it doesn’t matter.

For server-hosted services, you can have a tunnel on your main dev server, on a server, or on a pod in your Kubernetes cluster.

Now you have an opening into these networks through Warp/Argo tunnels.

Configuring tunnels

You can either configure tunnels through the Zero Trust UI by “adopting” them, or configure them in the /etc/cloudflared/config.yml config on the machine itself. Personal preference, I usually configure them on the machine itself.

The config specifies where a request should get routed to when it arrives at the tunnel. So the tunnel knows what to do with it.

In this config we tell cloudflared to route traffic arriving at this tunnel for hostname gitlab.widgetcorp.tech to localhost:80, and gitlab-ssh to the local SSH server.

❯ cat /etc/cloudflared/config.yml
tunnel: a2f17e27-cd4d-4fcd-b02a-63839f57a96f
credentials-file: /etc/cloudflared/a2f17e27-cd4d-4fcd-b02a-63839f57a96f.json
ingress:
  - hostname: gitlab.widgetcorp.tech
    service: http://localhost:80
  - hostname: gitlab-ssh.widgetcorp.tech
    service: ssh://localhost:22
  - service: http_status:404

  # Catch-all for WARP routing
  - service: http_status:404

warp-routing:
  enabled: true

The config alone doesn’t do anything. It just exposes a tunnel, and that’s it. What we need now are routes and targets.

Exposing a private network to the public with tunnels quickly

Quick addition, as this is a super common use case. If you want to just expose something in your home network to the internet, you can add a config like this:

tunnel: a2f17e27-cd4d-4fcd-b02a-63839f57a96f
credentials-file: /etc/cloudflared/a2f17e27-cd4d-4fcd-b02a-63839f57a96f.json
ingress:
  - hostname: homeassistant.mydomain.com
    service: http://192.168.1.3:80

Then go into Cloudflare DNS settings and map the domain homeassistant.mydomain.com to the tunnel:

CNAME homeassistant.mydomain.com a2f17e27-cd4d-4fcd-b02a-63839f57a96f.cfargotunnel.com

Now all traffic going to this domain will go through the cloudflared tunnel, which is configured to route homeassistant.mydomain.com to 192.168.1.3. No Warp client needed, Argo tunnel does everything for us.

Note: If you adopted the tunnels and don’t use config.yaml, you can automatically create matching DNS records in the Cloudflare UI and don’t need to do this manually.

Routes

A route defines where to direct traffic to.

Let’s say your homeassistant runs on 192.168.1.3 at home and you want to reach it from outside. Just above we deployed a cloudflared tunnel on our router at 192.168.1.3, and added a config pointing the domain to the Argo tunnel, so homeassistant.mydomain.com is already available to the public. However, 192.168.1.3 isn’t, as it’s a private network IP.

You can define:

• A route like 192.168.1.1/24 pointing at your tunnel, to route ALL traffic to the full IP range through that tunnel (so even 192.168.1.245 will go through your tunnel)
• Or a more specific route like 192.168.1.3/32 pointing at your tunnel, to ONLY route traffic to 192.168.1.3 through that tunnel.

When configured, once your user connects their Warp client that’s set up with your Zero Trust network, the Warp client will see requests to 192.168.1.3 and route it through the Cloudflare network to reach your specific tunnel. Like a little police helper directing cars where to go.

If the Warp client is not connected, 192.168.1.3 will just resolve in your current local network. If connected, it will resolve to the tunnel.

The routed IP doesn’t need to exist! So you could, for example, route a random IP you like (e.g., 10.128.1.1) to your tunnel, the tunnel then forwards it based on your routes, for example to 192.168.1.1. This is extremely powerful because it allows you to build your own fully virtual network.

That’s all it does, what happens afterwards is up to the tunnel config that we created above. The tunnel decides where to point the incoming request to, whether that’s localhost or somewhere else.

To summarize, the route tells the Warp client where to route traffic to.

Now we have 2 things working:

• homeassistant.mydomain.com - goes through a Cloudflare DNS record pointing at an Argo tunnel, which then forwards to 192.168.1.3. This works without Warp connected as it’s on the DNS level, public to everyone.
• 192.168.1.3 - The Warp client sees the request and routes it through the Argo tunnel, which then forwards it to 192.168.1.3 within that network. This needs Warp connected to work, and is only visible to people in your Zero Trust org.

Targets

This one took me a while.

Targets are needed to define a piece of infrastructure that you want to protect through Zero Trust. They are like a pointer pointing to something in your network. This goes hand-in-hand with routes, but isn’t always needed.

Let’s say you have 192.168.1.3 (homeassistant) exposed through a Cloudflare tunnel. By default, anyone in your network that is part of your Zero Trust org and has Warp client installed can now access your homeassistant at 192.168.1.3.

We can change that with targets. For example, defining a target with hostname = homeassistant.mydomain.com to the route 192.168.1.3/32 allows us to add access policies to it. We can also put an entire network into the target by specifying 192.168.1.3/24 to control access. This also works with virtual IPs like 10.128.1.1!

Targets alone won’t do anything, they just point to the service or network. “Hey, here is homeassistant”, or “hey, here is my home network”.

Access Policies: Protecting Who Can Access What

Continuing the example from above:

• we have a tunnel running on our home network that routes homeassistant.mydomain.com to 192.168.1.3
• we set up public DNS records to point homeassistant.mydomain.com to the Argo tunnel in Cloudflare
• we created a route 192.168.1.3 to go through the same tunnel
• we also created a target pointing to 192.168.1.3

When users access either 192.168.1.3 or homeassistant.mydomain.com, the Warp client will route the request through the tunnel, which then forwards the request to 192.168.1.3. Homeassistant loads and everything is fine.

But do we want that?

Probably not.

Access policies to the rescue!

With access policies, we can leave things in the public but protect them with Cloudflare Zero Trust access. So while 192.168.1.3 is only available if Warp is connected (so routing to it works), we can add security to our public homeassistant.mydomain.com.

Go to Access -> Applications -> Add an Application -> Self-hosted.

Here we can define what should be protected, and how.

Going with our previous example, we can add a public hostname homeassistant.mydomain.com or an IP like 192.168.1.3 (or both), then attach policies of who should be able to access it.

You can specify Include (“OR”) and Require (“AND”) selectors.

• Require rules must always be met, on top of include rules, to grant access
• Any of the Include rules must match to grant access

Then there are Actions:

• Allow - when the policy matches, allow access
• Deny - when the policy matches, deny access. aka blocking something.
• Bypass - when the policy matches, bypass Zero Trust completely. No more checking.
• Service Auth - when the policy matches, allow authentication to the service with a service token header (good for server-to-server, or bots). Check Access -> Service Auth to create these tokens.

Allow public access to everyone logging into your network

The most common use case: homeassistant.mydomain.com is public. We want to keep it public, but add an extra layer of security.

Add an include policy, pick any of the email selectors, add the email of the user you want to allow access to. Now only people authenticated with your Zero Trust org with the specified emails can access your homeassistant, without needing to have Warp running.

We can harden this by adding require rules: Add a Login Method selector rule, pick a specific login method like GitHub. Now only people with specific emails that have authenticated through GitHub can access your homeassistant, without needing to have Warp running.

Bypass login completely when connected through WARP

Another policy I like having is to skip the login screen entirely when connected through Warp. If a user is already enrolled into my Zero Trust org and has the Warp client provisioned, then there’s no need to ask them to authenticate again.

We can add a separate policy (don’t edit the one we just created above), pick the Gateway selector and set it to Allow or Bypass.

Don’t use ‘Warp’ - the Warp selector will match anyone that has Warp running, including the consumer 1.1.1.1 app. Gateway, on the other hand, matches only if someone is connecting through your Gateway, be that DNS or a provisioned Warp client.

(The ‘Gateway’ selector is only available if the Warp client is set to allow WARP authentication identity)

Now when:

• Warp through Zero Trust is running on a machine: No login screen
• No Warp running (public access): Prompt for login screen, but only allow specific emails that authenticated through GitHub

This setup makes it very convenient to reach homeassistant, no matter if connected through Warp or not.

Deploying the Warp client and enrolling into Zero Trust

Are you still with me?

Our network is basically done. We have a login-protected homeassistant.mydomain.com that routes through our tunnel into our private network and terminates at 192.168.1.3, and we have a direct route to 192.168.1.3 that only works when connected with Warp.

We also have login policies to make sure only specific users (logged in with GitHub and certain email addresses) can access homeassistant.

So how do we deploy the dang Warp client?

Actually the same: We create some policies.

Head to Settings -> Warp Client

In Enrollment Permissions, we specify the same policies for who can enroll. For example, “foo.bar@domain.com” when authenticated through GitHub is allowed to enroll. In the Login Methods we can specify what login methods are available when someone tries to enroll into our Zero Trust org.

Toggle WARP authentication identity settings to make the Gateway selector available in policies, effectively allowing the configured WARP client to be used as a login method.

Careful here, once someone is enrolled, they are basically in your Zero Trust network through Warp. Make sure you harden this.

Then, in Profile settings, we define how the WARP client behaves. These are things like protocol: MASQUE or WireGuard, service mode, what IPs and domains to exclude from WARP routing (e.g., the local network should never go through WARP), setting it to exclude or include mode and so on.

Other settings I recommend setting:

• Install CA to system certificate store - installs the Cloudflare CA certificate automatically when enrolled.
• Override local interface IP - assigns a unique CGNAT private IP to the client. This is needed for warp-to-warp routing.
• Device Posture - what checks the WARP client should perform for the org. E.g., check the OS version, some OS files on disk, etc. I have this set to WARP and Gateway because I want the client to provide information on whether the user is connected through WARP and Gateway, for skipping certain login pages.

Once done, just open the Warp client (https://developers.cloudflare.com/warp-client/), and log in to your network. This should open the login pages you specified in the Device Enrollment screen, and check all the enrollment policies you specified.

Once passed, congratulations, your WARP client is now connected to your Zero Trust network. The client will then go ahead and start routing 192.168.1.3 through your tunnels, as specified in your tunnel and route settings.

🎉

What we built

If you followed this guide, here is what we built:

• Login methods to connect the Warp client to your Zero Trust org through GitHub and specific email addresses
• A tunnel within your private network that
  • Forwards any request coming in with host homeassistant.mydomain.com to 192.168.1.3
• A route that forwards all traffic for 192.168.1.3 to the tunnel in your private network, which will terminate it at 192.168.1.3, which will only work when connected through Warp to route the request
• A DNS name homeassistant.mydomain.com that points to the Argo tunnel, and will allow everyone (even if not connected through Warp) to access homeassistant which runs at 192.168.1.3
• Access policies that will
  • Ask users that are not connected to Zero Trust through Warp to log in with GitHub and specific email, so everyone can access it if they can log in
  • A policy that skips the login screen completely and just shows homeassistant if the user connects through Zero Trust Warp client (enrolled into our org)

You don’t need the public domain and you don’t need the route to 192.168.1.3. These are 2 different options that you can use to expose homeassistant when you’re not at home. One is using a public domain name everyone can see, one is explicitly requiring connecting through enrolled Warp.

What I didn’t cover in this post:

• Warp-to-warp routing
• Creating and assigning fully private IPs that only exist within your Zero Trust network
• SSH authentication through Zero Trust access policies (that’s what we need Targets for)
• The other application types besides Self-Hosted

I’m happy to expand on it if there’s interest. Let me know on X or Bluesky.

Happy tunneling! ⛅

I’ve been working on a new app that has recently been released: Quickshot! It’s available on iOS on the AppStore at https://apps.apple.com/jp/app/quickshot-ai-editor/id6744357427?l=en-US

Editing photos with words

Quickshot is an experimental photo editor powered by AI, that aims to make small photo edits quick, natural and fun - instead of using tools and apps to erase something, add something or compose something new, in Quickshot you just use your words.

Quickshot lets you add, remove or manipulate images, transform scenes, or even re-style and repaint entire photos through hand-crafted presets, or by typing text yourself.

Some of the presets are meant to be a bit goofy and funny, but they demonstrate the app’s power quite well.

For example, some of the accessories Quickshot has available as presets:

Tapping a preset will send the request to the Quickshot server which then composes an AI pipeline to figure out exactly how the image should get transformed to get the requested edit.

There’s a lot of magic happening behind the scenes, but once the details are figured out, the app will give you within roughly 10 seconds a new image, now with your change:

Now with the new edited image loaded into the editor, we can continue transforming it further and further, until we’re happy with the final result.

More than just presets: Natural Language

Presets aren’t everything, you can also use Quickshot with natural language.

Same thing here - the Quickshot AI pipeline will figure out all the details of what to do to make this change a reality, then uses state-of-the-art image models to compose the final result.

In the example above, the result would be:

1.1 now on iOS!

Version 1.1 is now available on the AppStore. You can install it from https://apps.apple.com/jp/app/quickshot-ai-editor/id6744357427?l=en-US

3 edits per day are free, more than that and the app will ask you for a small subscription to keep the lights on.

I had a lot of fun building this and started using the app a lot myself. I’m especially a fan of the stronger crazy transformations like turning yourself into an action figure 😆

I have a lot of cool things planned for this app. Give it a try and let me know what you think!

This post is a direct continuation of Better Vibe Coding - Part 1, Foundation & Basics. If you haven’t read that yet, start there for the basics.

2. “Vibe Coding” complex tasks

Phew, now that we got the basics covered, let’s go into more advanced territory.

If you vibe coded in bigger codebases you’ll likely face issues with the agent’s quality going down the drain the more files it reads, and the more complex a task becomes.

When an agent reads in more tokens, eventually it’ll flush out stuff that came in the beginning of the conversation and lose context. We simply can’t afford to read files that we don’t need to read, and features that span multiple files need different handling.

All of these things I mention here is what I use for my projects on a daily basis and consider crucial.

2.1 Split plan phases from implementation phases

Putting it as it is: Purely vibe coding complex features (as in, opening a prompt and telling an agent to “implement this feature”) is just not doable currently due to limitations mentioned previously.

Imagine treating a new-joiner in your team like an AI agent. Would you just tell them to “implement this feature” without any guidance? Yeah very likely not (unless you want to scare your new hire away already).

So how can we work around it? By thinking like a manager: Separating planning from implementation phases.

Part of implementing any feature in a professional context always involves a planning phase before we can create use stories or a design document, so why should working with agents be any different?

Planning phase

• Understand and clarify the ask: What do we need to do, why
• Define the acceptance criteria
• Read a lot of code
• Research
• Come up with a plan
• Output a design document

Implementation phase

• Often a new session to flush the context
• Reads the plan and implements it

For the planning phase, I use a model with a bigger context since it has to gobble up a lot of code (gemini-2.5-pro or GPT4.1). This is where thinking-models really shine and should be used: Devise a good plan, doesn’t matter how long it takes.

For the implementation phase, thinking is usually not required because the heavy lifting has already been done in the planning phase.

Roo or Cline have PLAN phases built-in, but we can easily emulate that in other tools as well with a prompt like the following (I have this as a custom Cursor mode):

You are currently in PLAN mode.

- PLAN mode is for creating detailed plans and strategies, not for implementing code changes
- Your goal is to gather information, conduct research, and develop a comprehensive plan before any implementation
- Focus on understanding requirements and architecting solutions rather than writing actual code
- Use tools like `read_file` and `search_files` to gather context about the existing codebase
- Ask clarifying questions to ensure you fully understand what's needed
- Create detailed, well-researched plans with clear steps and reasoning
- Use Mermaid diagrams when helpful to visualize architecture, workflows, or processes
- You may include small code snippets to illustrate concepts, but avoid suggesting full implementations
- Engage in back-and-forth discussion to refine the plan until it meets requirements
- When a satisfactory plan is established, suggest switching to ACT mode for implementation
- Remember: in PLAN mode, your focus is exclusively on planning, not doing

PREFIX ANY MESSAGE YOU WRITE WITH "PLAN MODE:" 

THIS IS VERY VERY IMPORTANT!!!

If you get an error of edit_file not being available, then THIS IS A REMINDER TO YOU THAT WE ARE IN PLAN MODE!!!! REMEMBER YOUR INSTRUCTIONS

Once the plan is generated, I pass it to a new agent to do the actual implementation.

2.2 Resumption and hand-off with design docs and logs

Smaller edits can be made one-shot by passing the plan to an implementor model and letting it do its thing, but bigger plans may still exceed the context window and drop in quality.

For this reason, I always use 2 document formats: A clearly defined design doc and a log.

The design doc is the output of the planning phase. Its purpose is similar to the “onboarding document” we talked about previously, but with the objective to onboard someone onto a specific feature, so the same rules apply: Include everything that someone with no knowledge should know about this feature:

• What needs to be done and why
• Link to documents, tickets
• Technical design
• Clearly defined list with proposed high-level changes
  • What files need to be looked at, where to find them
  • What changes do we need to do in those files

The purpose of this document is to condense the huge context window research that the planning phase did into something that is actionable, and more important: reviewable (by us). This is the final big step before implementation, so we need to make sure that the idea the LLM has aligns with what we want to do. I iterate on these design docs a couple times until it’s in a shape that I’m happy with.

Always define a guide or template for these documents. Don’t let the AI do what it wants, we want the documents to be consistent and contain the information we already know we need for effective implementation.

A good format (from my experience) is following similar principles to the rules and manual document. This document is not intended for humans, it’s for LLMs, so it needs to be able to quickly bring the agent up to speed and point it towards the files and important bits that it needs to know, while providing context to do additional research if needed.

# Agent UI Simplification Design Doc

**Date:** 2025-04-24

**Author:** GitHub Copilot

**Linear Issue:** MIC-28

## 1. Overview

Simplify the Agent interface by merging the configuration options (currently under a separate "Settings" tab) into the main "Execute" tab. Rename the combined tab to "Agent". The "Execution History" tab will remain separate.

## 2. Motivation

The current tab structure (Execute, Settings, Execution History) with nested tabs (Configuration, Functions) within Settings is convoluted. Users need to switch tabs frequently to configure and run an agent. Combining configuration and execution into a single view streamlines the workflow.

## 3. Proposed Changes

-   **Modify `app/agents/[id]/page.tsx`**:
    -   Remove the "Settings" `TabsTrigger` and `TabsContent`.
    -   Rename the "Execute" `TabsTrigger` to "Agent".
-   **Modify `components/agents/agent-execution-panel.tsx`**:
    -   Integrate the UI elements and logic currently present in `components/agents/agent-config-panel.tsx` (Agent Name, Description, Model, System Prompt, Function Linking).
    -   Arrange the configuration elements logically within the panel, likely above or alongside the execution controls.
-   **Deprecate/Remove `components/agents/agent-config-panel.tsx`**: This component will no longer be needed as its functionality is merged into the execution panel.

## 4. Technical Design

1.  **Update Tabs in `AgentPage`**: Modify the `Tabs`, `TabsList`, and `TabsContent` structure in `app/agents/[id]/page.tsx` to reflect the two-tab layout ("Agent", "Execution History").
2.  **Merge Components**:
    -   Identify the specific UI sections and state management logic within `AgentConfigPanel` (e.g., form handling for name/description, model selection, system prompt editor, function linking UI).
    -   Transfer these UI sections and their associated logic (props, state, handlers) into `AgentExecutionPanel`. Ensure props (`agent`, `functions`) are correctly utilized.
    -   Adjust the layout within `AgentExecutionPanel` to accommodate the new configuration elements. A vertical stack (Config section above Execution section) seems simplest initially.
3.  **Refactor/Remove `AgentConfigPanel`**: Once all functionality is migrated, `AgentConfigPanel` can be removed or refactored if any parts are still reusable independently (unlikely).
4.  **Testing**: (Future step) Add/update tests for `AgentExecutionPanel` to cover the integrated configuration options.

The counterpart of the design doc is the work log - a document the agent needs to create after it is done. This document references the design doc and explains what the agent actually did.

The purpose of this document is to glance at what has been done, but also to potentially feed into a new agent to get up to speed on this feature, to do additional changes.

# Handoff: Agent UI Gamification - 2025-04-24

**Author:** GitHub Copilot

## TL;DR

We're redesigning the Agent configuration screen (`app/agents/[id]/page.tsx`) to be more visual and intuitive, replacing the old text-heavy forms with a "gamified" interface centered around a robot avatar.

**Design Doc:** `docs/designdocs/agent-gamified-ui.md`

## What We Did So Far

1.  **Created `AgentVisualConfigurator` Component:** Built a new component (`components/agents/agent-visual-configurator.tsx`) that displays:
    - A robot avatar (`BotIcon`).
    - The agent's objective/system prompt (currently display-only).
    - Visual "Skill Slots" for linked functions.
2.  **Integrated into Agent Page:** Replaced the old configuration cards in `components/agents/agent-execution-panel.tsx` with the new `AgentVisualConfigurator`.
3.  **Skill Slot Functionality:**
    - **Display:** Filled slots show the function name (with `SwordIcon` 🗡️) and workspace.
    - **Add:** Clicking an empty slot (`+`) opens the `FunctionSearch` dialog to find and link a new function.
    - **Edit:** Clicking a _filled_ slot opens the `FunctionLinkForm` modal, allowing the user to edit the description (when the agent should use this function).
    - **Remove:** Clicking the trash icon (`Trash2Icon`) on a filled slot removes the function link.
4.  **Data Fetching:** Updated React Query mutation hooks (`lib/hooks/use-agent-functions.ts`) to `await` query invalidation on success, ensuring the UI reflects changes immediately after adding/editing/removing functions without needing a page refresh.
5.  **Styling:** Applied basic Tailwind CSS and shadcn UI styling to the new component and slots.

## Current State

- The visual layout is in place on the agent page.
- Users can view the agent's objective.
- Users can add, edit the description of, and remove function links via the skill slots and associated modals (`FunctionSearch`, `FunctionLinkForm`).
- The underlying data updates correctly and the UI refreshes thanks to query invalidation.

## Next Steps / What's Left

1.  **Save Objective:** Hook up the objective/system prompt `textarea` in `AgentVisualConfigurator` to actually save changes (likely using `useUpdateAgent`).
2.  **Run Agent Integration:** Connect the "Try Your Agent" input and button (`AgentExecutionPanel`) to work with this new view (it might already work, but needs verification).
3.  **Execution History:** Implement the display area for execution logs below the configurator.
4.  **Testing:** Write tests for `AgentVisualConfigurator` and potentially update tests for `AgentExecutionPanel`.
5.  **Refinement (Optional):**
    - Add subtle animations (e.g., robot idle, feedback on add/remove).
    - Further styling improvements.
6.  **Final Build:** Run `bun run build` to ensure everything builds correctly.
7.  **Documentation:** Update any relevant user-facing docs if needed.

This should get the next person up to speed! Lmk if you need anything else. 🔥

2.3 Multi-agent workflow with agent-to-agent delegation

Now we’re getting into the fun bits: Multi-agent orchestration!

For more complex tasks, we can (or should) use a multi-agent workflow. This involves having a separate commander model that orchestrates tasks to implementation models.

The commander model creates a task list in a knowledge base such as Linear or the design doc, creates the stories, then delegates to the sub-agents for implementation.

Advantages of this are:

• The commander model splits the work like a PM, defines acceptance criteria for each task, reviews code, updates tasks.
• The implementation models are VERY accurate and specific since they only do one small task before returning.
• No context pollution - the sub-agents are always fresh in context.
• Tasks can be very long and will still be accurate because of this.
• Tasks can be resumed and stopped at any time because we keep track through the task list or sub-issues.

We can use MCP with tools like Jira or Linear for orchestration, so the commander creates the subtasks and tells the agents to pull them from the issue tracker with MCP.

Here’s an example how it works:

1. Commander model processes a design doc or epic issue
2. Commander model does basic research and splits the design doc or epic into subtasks (if not done yet), then optionally creates those in Linear, GitHub or somewhere else
3. Commander model spawns sub-agents for each task in sequence
4. Sub-agent implements spins up with new context, clear instructions what to do and how to complete the task
5. Commander model reviews the finished task, marks the issue as completed, and moves on to task #2

The issue tickets or markdown document serves as a boundary between commander and sub-agent to pass information across. The sub-agent knows it’s instructions, but has the ability to read in more information from the parent if required or if something is unclear.

This workflow can be achieved in any agent by having 2 different prompts and feeding them in, but Roo stands out to me for having all of this built-in with it’s multi-mode setup.

I have built plugins that can do this for:

• Vscode: https://github.com/dvcrn/copilot-task-delegate
• Delegating with MCP to q/claude: https://github.com/dvcrn/mcp-server-subagent

For completeness sake, here is one prompt I frequently use in Roo which adds a todo checklist to the markdown design doc. I then have variations of this prompt that use Linear or GitHub issues as backend.

Your role is to coordinate complex workflows by delegating tasks to specialized modes. As an orchestrator, you should:

Break this down into subtasks that can be implemented as individual logical chunks. Don't make them too small, but also not too big. Think of them as JIRA subtasks. 

If a design doc is available, I want you to add all these subtasks to the design doc with a checklist on the current implementation progress.
 

For each subtask, use the `new_task` tool to delegate. Choose the most appropriate mode for the subtask's specific goal and provide comprehensive instructions in the `message` parameter. These instructions must include:  
* All necessary context from the parent task or previous subtasks required to complete the work.  
* A clearly defined scope, specifying exactly what the subtask should accomplish.  
* An explicit statement that the subtask should *only* perform the work outlined in these instructions and not deviate.  
* If available, a reference to the design doc, files to change, scope
* An instruction for the subtask to signal completion by using the `attempt_completion` tool, providing a concise yet thorough summary of the outcome in the `result` parameter, keeping in mind that this summary will be the source of truth used to keep track of what was completed on this project.  
* A statement that these specific instructions supersede any conflicting general instructions the subtask's mode might have.
* Instructions to commit the changed files (only the changed files, NOT `git add .`) after completion, after all `build` and `format` instructions (such as `make build` or `make format` if available)

For the git commit message, Start the message with what the commit does, a verb, first letter capitalized. Eg Update xxx to yyy, Change bbb to better ccc, and so on. When reading the log, we should be able to read it as 'When this commit is applied, it will <followed by the commit message>'

Once the subtasks returns, I want you to check the checkbox in the design doc to mark the task as completed.

A subtask is only considered completed when a commit has been made. Make sure you define the acceptance criteria to the subtask.

When the task returns, review the code for completeness and see if you spot any obvious issues.

Another benefit of this is that now, with asynchronous agents on GitHub or Google Jules becoming better, that we can delegate some of our subtasks directly to those for handling.

Summarizing what we learned so far

Agents are extremely powerful if guided correctly, but we’re not at a point yet where they can be left unsupervised.

Always define clear boundaries, be that in simple tasks, behavior or complex projects.

The more complex a task, the more you should probably split it and break it up into sub-stories that can be worked on independently to not run into context pollution issues: Keep it short and to the point for the best results.

Create new sessions as often as you can, disable MCP you don’t need and use built-in /compact commands to further reduce amount of tokens in the context window at any time.

I’m hoping to expand on this series further in the future when agents evolved a bit more. I’m especially excited about Jules and GitHub Copilot Agents and started using them extensively for my own projects.

I’ve created a follow-up post for 2026 with a multi agent persona system

Today let’s talk about “Vibe Coding”, the current trend of writing code by chatting with autonomous AI agents.

I see a lot happening around Vibe Coding, from people getting hacked because their site falls apart, to people bootstrapping entire apps in hours instead of weeks.

I’ve been big into vibe-coding myself and wanted to share some tips and tricks on how to get the most value out of your AI agents.

Who am I and why should you even listen to me?

I’ve been working in tech for the past 14 or so years. 10+ in various roles from Engineer, TL, architect, and 3~4+ years as Engineering Manager in charge of teams of ~10 engineers. I’ve built sensitive high-availability systems used by millions of people, handling all kinds of sensitive data such as financial and banking information.

I want to believe that I’m fairly good with coding and designing complex systems, and those skills just happen to apply quite nicely to telling AI agents what to do as well.

Wtf is “Vibe Coding”?

Let’s clear this up first - Vibe Coding roughly translates to: Building apps/websites/features by chatting with an AI agent instead of writing code yourself.

Vibe Coding can be asking an AI agent to do simple edits like “Add a new function that does X to this file”, or complex multi-file features in the lines of “Add a new button to this page that does X”.

In a very rough sense, Vibe Coding is similar to asking an engineer to do something and assigning them a task: The engineer has to figure out what to do, where the files are and how to accomplish the task.

The problem with Vibe Coding: Context

Because of how agents work, each task is effectively similar to a new engineer that just joined your team: You have to teach them everything they need to know about your project to onboard them.

Sure, the engineer, if smart enough, can still be productive from day 1, but the likelihood of them doing something they shouldn’t, or re-implementing something that already exists is high. Combine that with the superhuman speed of spitting out code in seconds and you have a recipe for disaster if not supervised: Broken conventions, re-implemented functions, security holes.

And the bad thing: Your agent won’t learn by itself. Once ‘onboarded’ it can be productive for the task you’ve given it, but once you create a new session, we have to start from scratch.

In an optimal world, we could just give the agent the entire codebase as context, but with current limitations, that won’t be possible anytime soon. The opposite is true: The more files we add to an agent’s context, the more chaotic and unfocused it will get. Not good.

Everything competes for context. Even before we write a single word, the context has already been pre-filled with instructions by your editor, available tools and how to use those tools, MCP, etc.

To give you an idea, the system prompts of different editors (remember when the max context window we had was under 4000 tokens?):

• Cursor: 3900 words
• Cline: 2990 words
• Copilot: 1400 words

So what we have to do: Onboard an agent as efficiently as possible - the least amount of tokens that covers everything the agent needs to know.

I’ll structure this post into 2 parts: 1 is talking about the basics of getting more quality out of Vibe Coding, part 2 will be going into more advanced task orchestration.

1. The basics of stronger agents

In 1. We’ll talk about how to guide agents and bring them up to speed on your code

1.1 A strong entry point

The first thing I do in every project I work on is to generate a strong entry point for the agent. Think of it as a document that has a condensed, high-information density overview of what the project is about. Previously I called this “LLM.md”, but nowadays it’s standardized to AGENTS.md for Codex and lots of editors, CLAUDE.md for Claude Code, or GEMINI.md for Gemini. I usually pick one and symlink all of them to the same file so every tool finds the right context immediately.

I build each part of this document with the question: If I give this document to someone who has never seen the code in here, will they have enough information to research the rest themselves?

Everything not in this document will require the agent to research it on their own, increasing the likelihood of messing something up, missing something altogether or misunderstanding.

- What is the project
  - 1-2 lines about what the purpose and goal of the project is
- Key architecture
  - “Uses clean architecture”
  - “Uses MVVN”
  - “All data fetching logic is done with react-query”
  - “Uses bun”, “NodeJS 18+”
  - “Uses jest for tests”
- Key commands to interact with the repository
  - Build command: How to build the app “bun build”, “make build”
  - Lint: How to check the code for errors “make lint”, “bun run lint”
  - Test: How to run tests “bun test”, “make test”, etc
- What are the key parts and where to find them 
  - “/app” - NextJS app directory with routing and pages
  - “/app/components” - React components, includes UI
  - “/repository” - Database layer, contains functions for fetching data
  - “/services” - Directory for connectors to external services and APIs
- Very simple guides on key questions, with more documents going in detail
  - “How to write tests”
    - “All tests go into /test and have to use jest as test runner
    - “Always mock with X. All database interaction has to be mocked”
    - “For react-query testing, read xxx.md”
    - “For React component testing, read yyy.md”
  - “How to create a new API route”
    - “App routes always go into /app/routes and have to use Hono”
    - “Routes are to be imported and mounted into the router at /app/routes/routes.tsx”

The point of this document is not to be the knowledge dump for everything going on, but more a booster: It gives a quick refresher of what the project is about, where to find what, what the stack and architecture is, and more importantly: Where to continue checking if something is missing.

Thanks to this, an agent doesn’t have to re-scan the entire repository and cd+ls a dozen times, but can focus on targeted scans in the directories that it needs to depending on the task.

1.2 A strong ruleset

On top of the starting point, I have a separate rule file that is very explicit about how this repository should get handled. This usually goes into whatever agent’s rule definition I currently use. For Cursor, that’s .cursor/rules, for Cline that’s .clinerules, for Roo that’s .roo/rules, for Zed that’s .zed/ and so on.

A great feature of modern editors is that they allow you to specify when these rules should get imported. For example, in Cursor, you can specify that a rule only applies to *.js files or *.md files. This is a massive context-saver: The agent doesn’t need to know about your database conventions when it’s just updating a README.

For general rules that apply to the entire repository regardless of file type, I often skip the editor-specific files and put them directly into my entry point files (AGENTS.md, GEMINI.md, or CLAUDE.md). This ensures that no matter which tool I’m using, the core guardrails are always present.

While the starting point is for getting the agent onboarded, the rules are for defining constraints the agent must adhere to. So:

• Code conventions
• Test conventions
• Architecture decisions
• Git & branching rules
• Acceptance criteria for all tasks

I keep the rule file unformatted, numbered. I can write rules in markdown but often combine this with the starting point document, so we’re losing precious context for each token.

Here’s an example from a recent project:

1. Follow TDD: Write a test first, make sure the test fails, then implement the feature. See guides: [Component Testing Guide](mdc:docs/component-testing.md) and [React Query Testing](mdc:docs/react-query-testing.md). Place component tests in test/components/*.test.tsx.
2. Always use `bun` as the package manager and build tool. Run tests with `bun test` and format code with `bun format`. Never use other testing frameworks (like vitest). This project uses bun for testing.
3. Always make sure the build passes before committing code.
4. KISS (Keep It Simple, Stupid): Favor the simplest solution that meets the requirements.
5. DRY (Don't Repeat Yourself): Avoid code duplication. Extract reusable logic into functions or classes.
6. Docstrings: Write clear and concise docstrings for all classes, functions, and methods, explaining their purpose, parameters, and return values.
7. Create a new branch when working on a bigger feature. Small edits don't need a new branch.
8. Commit messages start with a verb (first letter capitalized). Include a Linear ticket reference (MIC-XXX format) when available. Be descriptive about what changed and why.
9. Always create a design doc first when creating a new feature. Small edits don't need this. Read docs/guides/design-doc-guide.md on how to write a design doc.
10. Always break down a design doc into small logical tasks that can be given to an engineer. Think stories. Add this to the design doc as a checklist and check them off as you finish work. Commit into your branch after each task is completed.
11. Use path aliases (`@/*`) instead of relative paths.
12. Follow the exact directory structure and import patterns from existing code
13. Run `bun format` after finishing all changes, and `bun run build` after a feature has been completed
14. Always write a log entry in logs/ with what you did. Name it YYYY-MM-DD-feature-name.md.
15. **All database operations must be implemented exclusively in `/lib/microfn` modules.**. **API routes (`/app/api`) and React components must never contain direct Prisma calls or any database logic.**. **API routes must call exported functions from `/lib/microfn` to perform any database operations.**.
16. Always run the vscode task "format", or if you don't have access to it, run `bun format` after each task is completed.
17. DO NOT fix lint errors about formatting, spacing, import order manually. Always run a `bun format` first as that may fix most issues. `bun format` will fix: Import order, removing unused imports, spacing, whitespace, punctuation.
18. Use shadcn and tailwind. Always look at other files in the same directory to understand style and structure.

1.3 Use scripts for an automatic feedback loop to the agent

I consider this an absolute essential for Vibe Coding + AI automation, and maybe even more important than the rules we just talked about.

Agents need to have a way to understand when the build is broken, the tests fail or the code is wrong. Linting can do that automatically if your editor+agent combo supports it, but even better to have it editor agnostic.

These are simple scripts and tools I sprinkled into the previous 2 sections already: make build, bun build, bun lint, make test, bun test - you name it.

In my rule I always have something like this:

14. Run `bun format` after finishing all changes, and `bun run build` after a feature has been completed

No matter what, the agent has to run the build script after each task, which will immediately report back if something is broken. This feeds back into the agent that something went wrong, and prompts it to fix the problems.

• In Swift, I do a xcodebuild
• In Golang, I do a go build
• In NextJS, I do a next build

Similarly, I always have a strong formatter active in my projects that reformats everything and enforces coding style. Agents tend to do what they want when it comes to coding style, and instead of adding more and more rules to instruct them about small things (and taking up precious context space), it’s much easier to just re-format everything through prettier, xcformat, mix format, and add that to the acceptance criteria of every task.

If the agent runs bun format && bun build after each task, I can be sure that the code is formatted correctly and the build runs before a task is considered complete.

1.4 Improving agents through self-improvement and reflection

Another good thing to do, to make sure entry point and rules are updated, is to let the agents do it themselves.

I often include an additional rule like the following. Depending on context size I either have this as a separate prompt somewhere that I can reference (aka “read @prompts/self-improve.md”), or part of my ruleset.

If during a session the agent breaks convention and does something I don’t want it to do, I correct it and teach it the correct way. The purpose of the reflection is to review the conversation history up to this point and put the learnings back into the documents we have.

# Self-Improving Cursor Rules Reflection

**Objective:** Offer opportunities to continuously improve `.cursor/rules` or [CLAUDE.md](mdc:CLAUDE.md) based on user interactions and feedback.

**Trigger:** Before completing any task that involved user feedback provided at any point during the conversation, or involved multiple non-trivial steps (e.g., multiple file edits, complex logic generation).

**Process:**

1.  **Offer Reflection:** Ask the user: "Before I complete the task, would you like me to reflect on our interaction and suggest potential improvements to the active `.cursor/rules` or `CLAUDE.md`?"
2.  **Await User Confirmation:** Proceed to completion immediately if the user declines or doesn't respond affirmatively.
3.  **If User Confirms:**
    a.  **Review Interaction:** Synthesize all feedback provided by the user throughout the entire conversation history for the task. Analyze how this feedback relates to the active rules in `.cursor/rules` or [CLAUDE.md](mdc:CLAUDE.md) and identify areas where modified instructions could have improved the outcome or better aligned with user preferences.
    b.  **Identify Active Rules:** List the specific global and workspace `.cursor/rules` (or CLAUDE.md) files active during the task.
    c.  **Formulate & Propose Improvements:** Generate specific, actionable suggestions for improving the *content* of the relevant active rule files. Prioritize suggestions directly addressing user feedback. Output a changeset that is usable with `edit_file` to write these suggestions into `.cursor/rules` or `CLAUDE.md`, for the user to review. Don't execute the tool yet..
    d.  **Await User Action on Suggestions:** Ask the user if they agree with the proposed improvements and if they'd like me to apply them *now* using the appropriate tool (`edit_file`). Apply changes if approved, then proceed to complete the task.

**Constraint:** Do not offer reflection if:
*   No `.cursor/rules` or `CLAUDE.md` were active.
*   The task was very simple and involved no feedback.

A prompt like this will then cause the agent to offer reflection after an edit session, if any feedback has been given, neat!

It’s of course important to not blindly accept all improvements. LLMs just generate text and aren’t properly smart (yet), so if you tell them to improve your rules, they will almost always suggest stuff to add to it even if not needed. Make sure you approach the rule file with care and keep it clean.

Follow these tips and you should have a strong foundation for more complex code changes that are consistent with your existing (and evolving) codebase. Especially when working together with other people, across models and tools, agents need to be aligned on the same guardrails to not go completely nuts.

In the next post, we’ll dive into more advanced Vibe Coding: how to handle complex tasks, planning, hand-off, and multi-agent workflows.

Around October last year, I made the decision to resign from my Engineering Manager position at Mercari with the purpose of focusing on building something up on my own. I love Mercari and have only good things to say about this company, but the fire to go my own way was stronger.

It’s been about 3 months now and while I am slowly chipping away at my projects, I thought it would be a cool idea to reflect on 2025 and celebrate some of the achievements I’ve made so far, but also keep myself accountable by working more in the public.

I made the habit of keeping a log on my reMarkable to avoid running into the mindset of “oh no, I haven’t done anything and the month is almost over!!” (see the cover image above)

January

January was a good month. I re-focused and made the habit of going to WeWork offices more often to have that separation between work-mode and non-work-mode. When you don’t have anyone telling you what to do, it’s easy to get lost in the noise of the internet and lose focus.

There’s been a lot of smaller side quests, more than I wanted, but in the end, it was a fairly productive month.

Fix My Japanese

https://fixmyjapanese.com received a design overhaul, a new logo and lots of tweaking in the back, laying the groundwork for an app version. I hacked a lot on making this app a reality but while the code was finished, it wasn’t ready to be released yet.

Tried turning https://t.co/BqznRdwlq3 into an app pic.twitter.com/IOrl82qM8m

— David (@dvcrn) January 31, 2025

FMJ also received a new Sensei and further refined the models in the back to be more accurate.

Masked Email Manager

Masked Email Manager received a bigger update and now integrates with Siri Shortcuts, effectively allowing you to create Masked Emails from anywhere - be it macOS or iOS. I use this a lot myself to quickly create new Masked Emails at hotels or on websites.

2.1 of Masked Email Manager for ⁦@Fastmail⁩ with Siri Shortcuts support now on the AppStore https://t.co/JxXRUBExH7 pic.twitter.com/ejqmo0A8I7

— David (@dvcrn) January 22, 2025

Here’s the Shortcut I use currently: https://www.icloud.com/shortcuts/e0ee7234b3784d19bf81791c6fbfc98c

WeWork CLI and Desks App

My WeWork CLI https://github.com/dvcrn/wework-cli that I use to book and list my WeWork All Access bookings received a rewrite from Python to Golang to make maintenance a bit easier.

I then used it as base to create an entire app that embeds the Golang library and uses it as the core. This started mainly as a fun “I wish I had an app that just shows me my upcoming WeWork Desks without having to use the slow WeWork app all the time” and actually turned into a pretty nice app.

No booking support yet, but that’s already done locally, just not released yet.

(Technically approved in February, but submitted at the end of January)

Pocketsmith

This year I started going all in on Pocketsmith for managing my finances. The reason being that I started traveling more, and keeping track of Japanese, German, Thai, and US finances was starting to get a bit chaotic.

• Wrote a Golang SDK for Pocketsmith
• Created a Bangkok Bank Enricher: Turns Thailand PromptPay records into rich data showing exactly where I made a purchase, instead of the default “made a transfer”
• Created a Moneytree Importer for Japanese Banks
• Created a Moneyforward Importer for Japanese Banks
• Created a Frollo importer for Australian Banks

I am very happy with how smoothly this went. I now have almost everything in Pocketsmith and I love that I can quickly see how much I spend on what, no matter what I use for spending.

Small stuff

• Migrated @david@d.sh from mastodon to GoToSocial. Maintaining a single-user Mastodon server was getting too tedious and I wanted to reduce resources for upkeep. Saved me about $8/mo in resources.

• chainenv, my CLI to manage shell secrets in Keychain (chainenv set foo bar) received a rewrite from Python in Golang as well, and with that support to use 1Password as backend (through my https://github.com/dvcrn/go-1password-cli that was already in Golang). I’m now using this for all my projects to simply have export FOO=(chainenv FOO) in .envrc and not have secrets laying around on my machine.

  
  • I’ve created a custom GPT to find cool upcoming music events in Tokyo called “Tokyo Club Event Buddy”. It’s powered by a custom aggregator that pulls from Zaiko, iFlyer, and RA, then exposes that to ChatGPT.
  
  • Started learning about Cloudflare Zero Trust and moved most of my homeserver + private services onto CF tunnels next to Tailscale. For example, I can now access my Homeassistant through https://hass.d.sh, which requires Okta authentication, or when I’m on WARP tunnel, just lets me through without auth. Saved me $10/mo in loadbalancer+external IP cost and another $10/mo on the homeassistant-cloud subscription
  • I’ve set up my own matrix server (https://conduwuit.d.sh, also exposed through CF tunnels), and added a bunch of bridges to it. So all my Facebook/Instagram/LinkedIn/WhatsApp/Google Messages/iMessage/etc. are now on my self-hosted server, fully encrypted.

• Created a fork of Cinny (matrix client) with changes to room sorting, https://github.com/dvcrn/davids-cinny

A lot of these side quests were Yak-shaving, but I had fun doing it nevertheless.

February

Looking at my February achievements, it was less small stuff and more focusing on a few bigger projects. However, I also travelled more. For example, this post has been written from a cafe in Phnom Penh.

Fix My Japanese

The month kicked off with my release of Fix My Japanese on iOS. Most of the work was done in January, but I submitted it in February and it got approved in February, so here we are!

Fix My Japanese is now on the AppStore! 🥳
Very much a 1.0 but public is publichttps://t.co/ZvYwtAFpr9 pic.twitter.com/5nSWXSyT7W

— David (@dvcrn) February 11, 2025

This is very much a 1.0 but I am quite happy with how it turned out. I am not an iOS developer and it’s nice to slowly get the hang of things (after learning a lot from Desks and Masked Email Manager).

On the web, after getting some user feedback, I added a new slider for picking the politeness level, and added a new “Keigo” level for more formal Japanese.

Microfn

Most of my time got taken by https://microfn.dev, a new project I’m working on that I can’t talk about much yet. Hopefully the first public release will be up in March. Not much more to say at this point.

TripIt & n8n

My TripIt extension for n8n received a bigger rewrite to fix some auth issues and now properly implements oauth (after learning how to do it with the WeWork app). I also added “Transport” support.

While definitely in the Yak-shaving territory, this allowed me to build fully autonomous agents that manage my trips for me. For example, I have agents running that read my emails, extract flight/hotel/activity/events, find matching trips and file them accordingly.

This information is then made available in my calendar, given me an up-to-date overview of what is happening when.

The agents also attach PDF documents to the segments, so if I want to have my booking confirmation for a flight, I can just open TripIt, select the trip, and the flight information is there.

The cool thing is that I can hook this up to Matrix, so the agent will tell me whenever it did something, like having an AI-powered PA.

Smaller stuff

• Siri Shortcuts MCP - A model context provider server that exposes all Siri Shortcuts to LLMs. So I can ask Claude to get information about anything that I have Shortcuts for, do things like “activate Cloudflare WARP tunnel”, “get my IP”, and so on.

  

• Created an app for more AI actions through Siri Shortcuts (using the learnings from Masked Email Manager). Nothing out yet, but it’s almost done.

• Noodled more into music with Ableton. Nothing to show yet, but I’m learning.

• Made a habit to take more photos with my camera and drag my ass outside.

February felt like it went by in a breeze. Reflecting on everything now, I’m surprised that I have done so little, and that microfn ended up consuming so much of my time.

Other thoughts

I do not regret going my own path, but I also need to re-focus on bigger projects and keep distractions to a minimum. The goal is to build something that makes money, and so far a lot of the things I did aren’t getting me closer to that goal.

AI is reaching critical saturation, I see people using ChatGPT/Microsoft Copilot everywhere, from trains to cafes, and autonomous coding agents like Cursor, Windsurf, Cline and Roo are getting incredibly good. Github Copilot even created their own coding agent that’s available in Visual Studio Code Insiders as preview.

I love that AI is making me 10x more productive, but I’d lie if I said I’m not worried about what it’ll do to my career. Junior engineers will probably have it the hardest, but we’re reaching a point where apps can get created in days without much coding experience, so I’m expecting less demand for engineers in general, but also a rapid flood of new apps and services hitting the market.

Only time will tell what’ll happen next and where the IT sector is going.

Japanese grammar can be very particular, especially when dealing with particles and the more polite forms, like in emails. To this day, I am still making mistakes without anyone ever correcting me!! I got frustrated so back then, before ChatGPT and all the fancy custom GPTs were a thing, I thought it would be a cool idea to hook up GPT3 (yes GPT3, not even ChatGPT) to an UI and ended up building this little tool for myself, calling it fittingly, “FixMyJapanese.”

It’s gotten better over time, and now it’s a free service that anyone can use at https://fixmyjapanese.com. Check it out!

What exactly is this?

Fix My Japanese is an AI-powered service that helps you figure out where you’re going wrong with your Japanese and learn from your mistakes. (yeah, yeah, I know, everything is AI-powered these days, but to my defense, this existed before the AI craze started)

Basically, you give it some Japanese text, and it:

• Gives you back a corrected version of what you wrote.
• Tells you exactly what the grammar mistakes are and explains why they’re wrong.

The idea is that by seeing these corrections and explanations, you’ll start to recognize the patterns and make those mistakes less and less.

It even has different politeness levels you can play with:

The “Kansai” option isn’t really a politeness thing, but as a Tokyoite, I find it hilarious to see how 日本語を話せません。ごめんなさい turns into 日本語話せへん。すまんな。. It’s a great party trick if you’re hanging out with friends from Osaka. 😉

Different Senseis, different styles

There are also different “Senseis” to choose from, each with their own style:

Each Sensei will give you slightly different corrections and explanations. For example, let’s say you wrote:

申し訳ございませんが、僕はこのリンゴを聴こえません

Yuki might say:

申し訳ございませんが、僕はこのリンゴを見ることができません。

While Sakura might say:

申し訳ございませんが、私はこのリンゴを食べられません

The Senseis change and get better over time, so try them all out and see which one you like best!

Tell me what you think!

If this is of value to you and helps you with your Japanese, shoot me an email at contact@fixmyjapanese.com and tell me what you think!

Here’s a fresh new release of my little Masked Email Manager app that I previously talked about. The major new feature is Siri Shortcuts integration, which makes it possible to create Masked Emails without even opening the app.

Get the latest version on the App Store

This was my first time working with Siri Shortcuts on the development side, so it was interesting to see what happens behind the scenes when a Shortcut triggers.

Some cool stuff you can do with this:

• Create Masked Emails for websites without using 1Password or Keyboard Maestro
• Quickly trigger from Widgets or Control Center items on iOS, for example when in a hotel
• Use Shortery to bind a keyboard shortcut to triggering the creation of a new Masked Email

Here’s a quick example Shortcut that reads what’s on screen (e.g., the website of a browser), parses it, and if it finds a domain it directly creates a Masked Email that it then copies to clipboard:

Download Example Shortcut

An actual native app since 2.0 (with a twist)

The app has undergone significant changes since I last wrote about it. With version 2.0, the app was completely rewritten in SwiftUI, saying farewell to React Native.

The cool thing is that it’s still powered by the same tech, mainly my maskedemail-cli. Previously, I compiled it from Golang into WASM (which didn’t go so well), then into GopherJS and embedded it into the React Native app.

Now with everything being Swift, we no longer need GopherJS. Instead, it’s using gomobile to build a native iOS .xcframework which is then embedded into the app.

Get the app on the App Store

I was discussing Japanese SIM cards with a friend the other day and noticed that there aren’t really any English resources to help people choose the right SIM card. There is quite a big difference in what you get depending on the provider, so I thought I’d add my 2 cents.

Not all SIMs are equal

If we turn back the clock a couple years - the main carrier SIMs (docomo, softbank, AU) were expensive and predatory. It was easy to get hooked into a 2-year contract costing around 8000 yen a month just for a good data connection. The best you could do was getting something like IIJMO (or a dozen other MVNO options) that would be on the docomo or AU wholesale network, but never be on-par with main-network speed and reliability, often lacking big features like Apple Watch support or 5G.

MVNOs, or “Mobile Virtual Network Operators,” are virtual operators that don’t own their infrastructure or network. These companies, like IIJMO, Mineo, HIS Mobile, have agreements with main network operators (docomo, au, SoftBank, Rakuten) to resell part of their wholesale network, usually at cheaper rates but with varying quality.

But all of that changed when Abe stepped down and Suga took over as prime minister. He made 3 important changes to how the big 3 had to operate:

• Providers can’t SIM-lock phones - all phones you buy from providers have to have an option to unlock them at any time without paying a huge amount of money
• Providers can no longer charge a contract cancellation fee to spark stronger competition - so switching from, say, docomo to AU is now free of charge
• The big providers must offer a cheap plan on their main network that is not MVNOs

This changed the entire market: Carriers can no longer lock customers into contracts for years and need to actually offer something unique, otherwise customers will leave.

The big 3 reacted by first offering cheaper main-network SIMs through their sub-brands (Y!Mobile for Softbank, UQ Mobile for AU). This was already great, because now we had plans that were owned by the big carriers themselves, but the government wasn’t happy about that, and told them again: you have to offer cheaper plans on your main brand!

This gave us 3 new plans: Ahamo from docomo, Povo from AU and LINEMO from SoftBank. The important point here is that all these plans, as I just mentioned, are owned by the big 3, so they are not MVNOs that buy wholesale and piggyback. You get the same speed and network quality as “normal” customers of docomo/au/softbank, with no worse deal. These SIMs will almost always be better than any of the MVNOs available!

So as a first line to choose from, my recommendations would be:

• On SoftBank: LINEMO, Y!Mobile
• On NTT Docomo: Ahamo
• On KDDI/AU: Povo, UQ Mobile
• On Rakuten Mobile: Rakuten Mobile

For today’s post I will be focusing on Ahamo, Povo (Promo code: LCOD14FW) and Rakuten Mobile.

A quick look at the networks

Ahamo runs on docomo, Povo on AU, and Rakuten Mobile on its own network.

In general, docomo has the strongest network with the fastest speed, followed by AU. Rakuten is the new kid on the block, and while their network is steadily growing, it’s underwhelming for speed and coverage. There are always patches where one is better than the other, but in general, docomo takes the lead. At the time of writing, the AU network has the best coverage on 5G, with docomo lagging slightly behind.

Rakuten Mobile has a unique roaming agreement with AU, so in places where Rakuten Mobile isn’t available yet, it falls back to some offering from AU. This is NOT the same network as main AU or Povo. It is part of the wholesale network that is being rented out to MVNOs. I couldn’t test this as it happens automatically, but I often noticed that my AU line had reception while Rakuten Mobile did not.

Rakuten Mobile often shows ‘5G’ connection, but the speed and throughput is far from it, often comparable with lower 4G speed. Yes, the connection to the tower is 5G, but that doesn’t mean anything if you don’t get 5G benefits; it just eats more battery. This was the only provider where I actually felt that when I entered a URL, I often had to wait a few seconds before anything actually happened, especially in buses or trains.

Bonus points: Docomo/Ahamo allows you to purchase an extra option (currently costing 0 yen/month) that enables 5G SA (5G standalone) that moves you to a dedicated 5G core network, and gives you ridiculous speeds, currently claimed at 受信実効速度は170Mbps~484Mbpsです。

Only docomo offers 5G SA currently, and the addon is still free for an unspecified amount of time. This is by far the best in terms of speed if that’s what you’re after.

However, while docomo has the best network, it’s also used by a lot of people. So in very crowded areas like at festivals or concerts, it’s usually the worst performing. Ironically, Rakuten is the best for those because it has the smallest customer base.

Data volume and plans

Rakuten Mobile has the most attractive deal here. It’s unlimited, and the price goes up depending on how much data you use, but is capped at 3000 yen for everything beyond 20GB.

Ahamo has 2 options: 20gb for 3000 yen, or 100gb for 5000 yen. If you hit the upper limit you get speed limited to 1Mbps which is still pretty good.

Povo has the most unique offering here: Toppings!

By default, if you pay 0 yen a month, you get 128kbps. That’s very slow, but it’s free and unlimited. Though you won’t be able to do much outside of messaging with it. But did I mention it’s free?

Then you can activate different toppings depending on how much data you need

As with the others, there is a 20GB plan for ~3000 yen, but more interesting options like unlimited for 24h for 330 yen, or 150 GB for 13,000 yen, valid for 6 months.

I’ve been using this SIM a lot on my iPad for its 0 yen option, and then when I have a day of heavy working ahead of me, I buy the unlimited topping and tether to my other devices.

Considering this, Rakuten Mobile appears to be the most cost-effective as it never exceeds 2980 yen. However, its network quality is considerably lower. Using a plan like 150 GB for 6 months with Povo can drastically reduce the cost if you don’t use the full 20 GB each month anyway. It’s also a great sub-SIM because of the 0 yen base plan, and you can always add more data when you need it.

Ahamo is the least complicated - either 20 GB or 100 GB. 20 GB was always enough for me, but having the option to upgrade to 100 GB when needed is very nice, and 1 Mbps is still not bad when it runs out.

Roaming when abroad

Roaming is often inconvenient and expensive. It’s usually better to just get a local SIM or eSIM through something like Airalo and be done with it, but with those new plans we may no longer need to do that.

Ahamo surprised when it got announced that it comes with 20 GB of data roaming free of charge!! That’s honestly bonkers and means that in most countries, you can just leave the SIM active and use free data roaming without any additional steps.

This offer is limited to 14 days of constant roaming. If you are longer than 14 days outside of Japan, the speed will get reduced to 128 kbps. You’ll still have unlimited data abroad for messaging, but yeah, it’s gonna be pretty slow. You also can’t purchase additional data when it runs out. You have to return to Japan for it to refresh.

Next in line is Rakuten Mobile - it also gives you 128 kbps unlimited at slow speeds when abroad, or 2 GB at fast speeds free of additional charge (you can toggle fast/slow within the app). Additional volume is available at 1 GB for 500 yen. Still not bad, especially with unlimited at 128 kbps.

Povo is sadly quite bad. They tried to continue with the topping model but all of the options available are pretty pricey - like 5 GB for 14 days for 7080 yen or 3 GB for 7 days for 4280 yen… yeah don’t use this SIM abroad.

Other special perks (Apple Watch, carrier pay, etc)

Ahamo has:

• Carrier pay (pay through your phone bill)
• Apple Watch data at 500 yen/month
• 5G SA (mentioned above)
• Free 20 GB international data roaming at no extra cost
• 5 min of each phone call is free
• Free use of docomo wifi hotspots
• Synergy if you use d払い

Povo has:

• No Apple Watch :(
• Carrier Pay (AUかんたん決済)
• Unlimited data at 330 yen
• Unlimited data at 128 kbps at actual 0 yen base plan
• Free use of AU wifi hotspots, Wi2Premium and a few more
• Synergy if you use AU Pay

Rakuten Mobile has:

• Apple Watch data at 500 yen/month
• Free international data roaming for 2 GB, unlimited at 128 kbps
• Actual unlimited plan for cheap
• Carrier Pay…? at some places (Google Play Store), but most don’t support it
• Ability to activate or re-issue an eSIM within ~2 minutes online
• Free phone calls, SMS through their apps
• Gives you extra points on Rakuten Ichiba, synergy with other Rakuten products like Rakuten Card

*Rakuten Mobile offers “free phone calls and free SMS” but it’s the weirdest thing. SMS only works on Android if you use their Rakuten Link app, which for whatever reason doesn’t offer free SMS on iPhone. Free calls is also only if you use the Rakuten Link app, so it’s VOIP and not true cellular - it’s often very janky.

Rakuten Mobile also just stopped working in the past when a new iPhone got released, and some Android phones aren’t even supported for whatever reason (have you ever seen a physical SIM not supported on a phone??), so it’s by far the buggiest of the providers. They also constantly try to upsell your some other Rakuten product, even their Rakuten Link app is plastered with ads…

So David, which one?

In the end it depends on what you want, no? But if you’re still struggling with deciding which one fits the best, my recommendation would be:

• Ahamo if you just want 20 GB at fast speed, have an Apple Watch and/or travel a lot. The 20 GB roaming is unbeatable, so is 5G SA. 100 GB for 5000 yen is still reasonable for when you need it. The best all-around SIM.
• Povo for their unique topping mechanism and still very good network, even better than docomo in some areas and crowded places, or with 5G coverage. The 0 yen base plan makes this a no-brainer to have ‘just in case’, or put onto an iPad to always have a 128 kbps data connection on hand. 330 yen for unlimited when needed is damn good for remote working. No Apple Watch option though, and their roaming offering sucks.
• Rakuten Mobile if you want something unlimited and don’t care much about network, speed, throughput or coverage. It’s good enough in cities and a great secondary SIM to have for when Docomo or AU are overloaded, or when travelling. The ability to quickly move the eSIM to another device has been a life-saver. But, as you can tell from the post, the network is the most ‘meh’, has high latency, lots of dark spots, and other weirdness like their Rakuten Link app having different behavior on iPhone or Android. Also I hate that they constantly try to upsell me when I’m already a customer.

In the end, just try them and see which one you like the most. There is no contract cancellation fee anymore, so it’s very easy to move a number from plan A to plan B and back.

If you liked this post, please us my referral codes:

• Povo: LCOD14FW
• Rakuten Mobile: https://r10.to/hN5zpk
• Airalo: https://ref.airalo.com/21Qr

I am very happy to finally be able to talk a little about the pet project I’ve been hacking on for the past months while backpacking through Australia - https://maskwire.com, and the email service that I always wanted but didn’t exist.

How I’m using email today

I’m not going to talk about reinventing email into some chat messenger or something like that. On the contrary - I think email is great the way it is and I like using it.

But because of spam and usage of my own domain, I developed a pretty efficient strategy to combat spam, by using one-time-use email addresses. In the beginning, I just used stuff like “facebook@davemail.io” where *@davemail.io is a wildcard email address that resolves to my main email. Worked great!

I liked this so much that I built my own service that would generate one-time-use emails, but as things go - time to market took way too long and by the time I was ready to launch there were already heaps of other services from Mozilla and more notably, Apple.

As the next iteration, I switched to Fastmail’s “Masked Email” feature, which now spits out a random email address whenever I need one. I liked this system enough that I built a bunch of tools around it: An iOS app, a CLI, a WASM based NPM package among others. So far so good.

I now have a barrage of different domains that are all managed through Cloudflare and Fastmail.

What I am not happy with

In short - email clients. Each client has different capabilities and different quirks, and don’t even get me started on support for wildcard/catch-all emails. It’s garbage.

Ever received an email on facebook@davemail.io and want to reply from it? Yeah, you can’t. You gotta open the web interface and if you’re lucky, the service you use supports replying from it (Fastmail does, sometimes). Otherwise - open your client settings, add a new alias for facebook@davemail.io, and only then can you reply from it.

Swipe actions are another one of my gripes. Sometimes swiping an email in a client like Apple Mail deletes it, sometimes it archives it. Sometimes you can configure it, sometimes you can’t. In Gmail or Samsung Mail you can’t; emails always go to trash. Great.

What if I click on a “contact developer” link in an app? Apple Mail will open with a compose window that has my main email address selected with no option to generate a new custom address just for that case (unless you use Apple’s Hide My Email feature).

Apple Mail, while great, also has the quirk of downloading everything on an IMAP server, so in my case about 50,000 accumulated emails - whoops.

There is so much inconsistency and quirks, it’s just not very productive.

Building the email service I always wanted

Step by step I fixed issues like those mentioned above for myself, but if I have these issues, others may too. So the idea slowly transformed from “a service that generates one-time-use emails” to a service that works on every device and instead enables advanced (sometimes niche) functionality that email clients just can’t provide.

Maskwire works as a transparent IMAP/SMTP server acting in the middle, communicating 1:1 with an upstream server (relaying client commands directly to the provider), but contains a custom message rewriting and remapping engine.

I want to reply from facebook@davemail.io? Now I just reply to the email that I received on it in Apple Mail without thinking about it and let Maskwire handle sending that email out from facebook@davemail.io again.

I keep deleting emails accidentally by swiping them? Well that’s fixed now too, Maskwire can just rewrite the “move to Trash” command into a “move to Archive” command.

I click on a “contact developer” link and Apple Mail comes up with a compose dialog? I don’t need to care about selecting the correct sender, Maskwire will just rewrite it from my@mail.com to mail202307@davemail.io when the email gets sent out.

Email inbox getting too big? Also no problem, I set up a filter in Maskwire and now only the last 1 month of messages in my archive is getting synchronized to the client, independent of their settings.

I’ve been chipping away on Maskwire and using it on my actual emails for the past weeks while fixing quirks and expanding functionality and I’m very happy that it’s now in a state where I can more openly talk about it and ask people to try it.

What about security? Doesn’t this sound like a recipe for disaster and hacking?

I went through different iterations of different security designs and landed on one that has been the most bulletproof. You still need to provide IMAP/SMTP credentials of your upstream provider (in my case Fastmail) to Maskwire, but upon account creation, Maskwire generates an RSA2048 keypair that is encrypted with the user’s Maskwire.com password.

The public key is then used to encrypt the upstream IMAP/SMTP credentials, which means that Maskwire literally has no way to decrypt those without the password of the private key, which it doesn’t possess.

Instead, when the user authenticates with the Maskwire IMAP/SMTP server, the password used for authenticating will be used to decrypt the private key, which will then decrypt upstream credentials to establish a session, then wiped from memory. Once the user logs out, the system is locked again.

If the user doesn’t login with their email client, Maskwire can’t login either.

Maskwire is open for early beta now

I’ve been using it for a while on my own emails and it’s working great for my use case, but as the only user I’m sure there will be quite a few bugs and things that won’t work yet.

I’d like to open signups to a broader audience and am looking for users willing to try something new. You can sign up at https://maskwire.com - what you will need are IMAP/SMTP credentials and an email provider that you’re using with your own domains.

Please also join the Maskwire Discord, so we can discuss features, bugs and more: https://discord.gg/8QkduyQf8V

Travel routers are a thing I didn’t know existed. It makes sense that they do when you think about it, but I was never in a situation where one might have been necessary. Recently, I started a months-long sabbatical in Australia, and part of my research of what to pack on subreddits like /r/onebag landed me on travel routers!

After a bit of research, I went out and ordered the GL-Inet Beryl AX, a small cute mint-colored box that would fit into my pocket if I wanted to.

What are travel routers, and why I wanted one

As the name implied - routers for traveling (duh). They are usually very small, and feature WiFi and an ethernet port. More advanced models have more functionality like VPN, Wireguard tunnel, and whatnot, depending on your needs.

The appeal for me is bringing my own network with me. If I hop between hotels, Airbnbs, and cafes, I don’t have to worry about setting up all my devices to use that network again and again - all I have to set up is the travel router and the rest just works. The router can also add another layer of security with things like DNS-over-HTTPS, OpenVPN connections, or tailscale Wireguard tunnel on the router level, so none of my end devices have to know about any of that. I have one less thing I have to trust (to some extent).

Another, maybe more silly reason was mDNS and Chromecast: One of the devices I wanted to bring with me was my small Chromecast because of its size, but those devices usually just won’t work with Hotel WiFis because they disable mDNS (not to mention the PITA to keep setting it up with new Airbnb wifis). If I have my own pocket WiFi with me, however, I can use the Chromecast just fine, for the occasional movie session.

All in all, a travel router I can pocket sounded like a good idea.

The Beryl AX, and what it can do

After a bit of research, I decided to buy the GL-Inet Beryl AX. It was the right mix between form factor, features, and price. Honestly, I rather wanted to have the excellent-looking Firewalla Purple, but that was $99 vs $329, so the Beryl is what I ended up getting.

What’s really cool about the Beryl, besides its small form factor, is that it can support multiple WAN backbones, and fallback or load balance between them. One of those WANs is the WiFi Repeater: The Beryl can hook into any available WiFi network and use that as backbone to create its own WiFi network. That makes it an excellent option for cafes and hotels that likely won’t give you an Ethernet port to use.

But where it gets even more convenient is that the Beryl also supports USB tethering, so I can plug my iPhone or Android device with unlimited data SIM, and tell the Beryl to use the USB tethered connection as another WAN, then set it to either fallback between Hotel/Cafe wifi and USB tethered connection or just load balance between them for extra bandwidth.

This also means that I can either repeat my mobile hotspot from my phone or use the USB-tethered connection alone to create a full WiFi network that all my devices can use, for when the Airbnb or Cafe WiFi is especially spotty, without having to change the WiFi network on the devices. Bonus points for the USB port being USB3 and being able to charge my phone while the connection is tethered (USB tethering is something the Firewalla Purple for example can’t do).

That alone would have been enough to make me happy, but there are a few more cool tricks it can do:

• iOS and Android apps to manage the device - no need to use the web-based admin UI
• Connect to NordVPN, my VPN provider of choice (or other OpenVPN/Wireguard providers). Many people use this to connect to their portable network to their home network
• Separate main and guest networks (for your friends) which can be on separate VLANs and isolated, or have only one of them go through the VPN
• Connect to my Tailscale tailnet, and even use other devices as exit nodes
• Baked-in NextDNS and DNS-over-TLS/HTTPS support
• Mac Address randomization and cloning (for those pesky networks with Mac address filter)
• Dynamic DNS updating
• WiFi 6 and AdGuard (not using this personally because of NextDNS

The Beryl is OpenWRT based, so you get full SSH access and Luci interface on top of the GL-Inet admin UI, with very fine-grained control over inner workings like firewall and DNS.

Then lastly, why I ended up packing this router a lot more than I thought I would: It can be powered through USB-C, so any power bank with USB-PD will work great (or just connecting it to my Macbook). I had it powered in my backpack quite a few times while in Cafes. My 10,000 mAH Anker can usually power it for about 6-8 hours without problems.

The not-so-great

All of the stuff I wrote above is honestly great… if it would just work reliably. And this is where I started getting frustrated with this particular device: Reliability. It can be really damn finicky, and there is often a “but” attached to the great parts.

For example:

• NextDNS and DNS-over-TLS are great, but of course, this also means captive portals just won’t work because they can’t hijack the DNS queries. When disabling NextDNS and switching back to auto, then restarting the router, it just forgets your NextDNS ID so you can’t just switch it back on
• Tailscale support baked in is great, but it required a lot of messing with OpenWRT firewalls to have it work properly. MagicDNS works when setting up custom DNS forwarding rules in Luci on the Tailscale domain, but those get wiped whenever you switch the DNS to auto or back to NextDNS in the GL-Inet UI
• Multi-WAN is great, but it often tells me that it doesn’t have an internet connection on the WAN for whatever reason. Could be an issue with the power source I’m using (people on the forums suggested that), could be with the connection, could be a bug when using tethered connection + repeater at the same time (another suggestion from the forums) - I don’t know
• NordVPN is great, but it can’t refresh configurations automatically and connect to the optimal server in a region. I can set it up to connect to, for example, au531.nordvpn.com.udp, but then it will always connect to that server

More often I spend the first 10-20 minutes of my session just tweaking settings on the admin UI until my connection finally worked, and often wondered why I’m not just sharing my phone connection to my Macbook directly instead of going through the Beryl. I had it countless times that devices connected to my phone directly worked flawlessly, but devices that go through the Beryl which goes through my phone don’t for whatever reason when they should (may have been an issue with the power cable I used, or a bug with multi-WAN when tethering + repeater, or something else).

Maybe I’m running into these issues because I’m trying to do more with it than using it as a dumb travel router, but if those features are offered, I of course want to play with them. I’m hoping for a set-and-forget setup that I can just plug into a power outlet and it does the rest, stuff like when there’s a captive portal it temporarily disables DNS-over-TLS, or the VPN connection. I had to bookmark the admin interface IP because of how often I had to open in the past weeks just to debug something, which is not a great indicator.

So, am I happy with it?

If I had to give a verdict, I would say: Yes, I am very happy with this device. It has its quirks, but once set up, it runs smoothly, and having a travel router with me gave me a new way to think about networking while on the go.

I’m writing this blogpost while sitting in an Airbnb connected to it. I jacked the Beryl with an Ethernet cable into the existing router (that only supports 2.4G by the way) and created my own network on top of the existing one. That’s where the picture above is coming from.

The router here is sadly not a fiber-based one, but one of those 4G stations, so the networking speed fluctuates drastically. When the connection was particularly bad, I connected my Android phone to the Beryl (Android phone is set to start in tethering mode when USB is connected) and immediately improved the connection for all my devices. That is honestly a damn cool option to have, kind of like a modular connection upgrade.

Another time I had to restore a few bigger files on a bad guesthouse network. The tethered connection alone wasn’t enough, so I used a USB-C Ethernet dongle, connected both my iPhone with 5G SIM (tethered) and my Android with unlimited 4G SIM (ethernet) on different cellular networks to the router at the same time, then set it to load balance between all 3 with priority on the unlimited SIM -> 5G SIM -> WiFi network. I was easily able to finish my bigger download with this burst of additional bandwidth.

I didn’t think I would need a travel router, but I’m glad I bought it. It’s been only a month, but the small Beryl is already becoming a staple in my travel bag.

I recently built an Elixir SDK for OpenAI APIs for a project I was working on to automatically fix and correct Japanese grammar (https://fixmyjp.d.sh). The SDK is actually fully auto-generated with metaprogramming which is something I wanted to do in Elixir for a while, but that’s not the topic of today’s post.

Today, let’s use that SDK for something fun and build a smol ChatGPT service for your shell from scratch, in true Elixir fashion!

The end result will be an ai shell command that is usable from wherever we want, to send messages to a distributed gptserver node for interacting with ChatGPT, that can either be on the same computer, or anywhere on the same network

Quick refresher: GenServers, and how state in Elixir is implemented

Elixir is an immutable functional programming language, meaning you can’t mutate things. If you do

a = 1
a = a + 1

you are not mutating a to be a+1, instead you create a new a with the value of a+1. That means you can’t do things like this in Elixir:

a = %{}
a["foo"] = "bar"

instead, you would do:

a = %{}
a = Map.put(a, "foo", "bar")

again, creating a new a with a copy of the previous a, but one item added. The old a is not mutated, it still exists until the garbage collector gets rid of it.

So how tf do you actually keep any state across different modules then? From the sounds of it, you would need to constantly do something with a value, you can’t just create an object that different parts of the code can mutate and share.

In Elixir/Erlang-land, instead of having a shared object/container somewhere, we use processes managed within the BEAM VM, and the most common way to define those processes is GenServer.

GenServer is a behaviour that looks like this:

defmodule Foo do
  use GenServer

	# stuff that's running inside a separate process

	@impl true
  def init(val), do: {:ok, val}

	@impl true
  def handle_call({:add, i}, _from, sum) do
	  {:reply, sum+i, sum+i}
  end

	# stuff that's running inside the main process

  def add(pid, num) do
	  GenServer.call(pid, {:add, num})
  end
end

foo = GenServer.start_link(Foo, 0)
Foo.add(foo, 2) # 0 + 2
Foo.add(foo, 3) # 2 + 3

To hold state, we effectively created a new process whose whole purpose is to hold a value in memory, then listen for a message :add, to then add whatever got passed to create a new value (remember? we can’t mutate things) to hold in memory and start listening for new messages again. Imagine it as a (very simplified representation):

function add(startValue) {
  add(startValue + waitForUserInput()); // yay, no variables
}

Level 1: ChatGPT in a GenServer

Okay now that we remembered how to handle state in Elixir, we know what to do to get a ChatGPT-like system going: Wait for user input -> append to messages so far -> send to OpenAI -> output -> wait for user input again

Let’s summon a new mix project:

mix new mixgpt

add my ex_openai library as deps to mix.exs:

defp deps do
  [
    {:ex_openai, ">= 1.0.2"}
  ]
end

(follow the instructions for configuration with an API key)

and conjure up a basic GenServer with a new {:msg, m} message listener:

defmodule Mixgpt do
  use GenServer

  @impl true
  def init(_opts) do
	  {:ok, []} # start value is empty list
  end

  @impl true
  def handle_call({:msg, m}, _from, msgs) do

  end
end

To make our life easier, we’ll also add a start_link function to start the thing:

def start_link(_opts) do
  GenServer.start_link(__MODULE__, [], name: :gptserver)
end

Notice the name: :gptserver argument here - we’re giving the GenServer a unique name so that we don’t need to worry about remembering its PID somewhere.

To send a message to this new genserver, we’ll also create a new send():

def send(msg) do
	GenServer.call(:gptserver, {:msg, msg})
end

thanks to name: :gptserver we already know how to reach this instance, so we no longer need the PID to be passed. Now, all we have to do is Mixgpt.send "hi!"

In the actual handle_call, what we have to do is: construct a new OpenAI ChatGPT message with the passed message, send it to the OpenAI API, and add whatever it returned to the internal state of this server (remember from above? how to handle state). Something like this:

defp new_msg(m) do
  %ExOpenAI.Components.ChatCompletionRequestMessage{
    content: m,
    role: :user,
    name: "user"
  }
end

@impl true
def handle_call({:msg, m}, _from, msgs) do
  # create temporary "msgs" with existing list + the new message from the user
  # effectively appending the new message to the existing messages
  with msgs <- msgs ++ [new_msg(m)] do

	  # call OpenAI ChatGPT API with the new msgs list
    case ExOpenAI.Chat.create_chat_completion(msgs, "gpt-3.5-turbo") do
      {:ok, res} ->
	      # return the content of the result, add it to the msgs list and continue the GenServer loop with {:reply}
        first = List.first(res.choices)

        # second value is what's returned, third value is the new state of this server
        {:reply, first.message.content, msgs ++ [first.message]}

      {:error, reason} ->
        {:error, reason}
    end
  end
end

The cool thing about Elixir and other functional languages is that they usually come with a REPL, or a shell that you can use to interact with different bits and pieces of your program. In Elixir, that’s iex. Run iex -S mix, wait until it starts, then try to interact with our new GenServer:

iex> Mixgpt.start_link(nil)
{:ok, #PID<0.280.0>}
iex> Mixgpt.send "Hi!"
"Hello! How can I assist you today?"
iex> Mixgpt.send "tell me a joke in 5 words"
"\"Why did the tomato blush?\""

Cool, now we already got our own pocket ChatGPT that we can access through mix, that was pretty quick, wasn’t it? Let’s make it a bit more automatic and close the loop. To wait for user input, we can use IO.gets, let’s use that in a new function to directly ask for the next message:

def wait_for_input_and_send() do
  IO.gets("> ")
  |> send
  |> IO.puts() # print the message out to have it nicely formatted

  wait_for_input_and_send() # recurse
end

Type recompile into the iex shell to recompile the project and try again:

iex> Mixgpt.wait_for_input_and_send
> Hi
Hello, how can I assist you today?
> Do you like Elixir?
As an AI language model, I don't have personal preferences or feelings. However, I can tell you that Elixir is a popular programming language among developers and has some unique features that make it stand out, such as its scalability, fault tolerance, and ease of use for distributed systems.
> what was the first message I sent you?
The first message you sent me was "hi".
>

Level 2: Distributed ChatGPT service

The example above is easy to do in any programming language, but let’s go one step further and do something cool that you can do beautifully in Elixir and Erlang, but not as easily in other programming languages: Let’s turn this thing into a proper distributed system!

As a reminder, our mini chatgpt is running inside a server! We did this because that’s how state is handled in Elixir. Even though we created the interface so that we can easily just call send("why is the sky green?"), on the back it still sends a new message to this gpt-server, then waits until it responds. It just does this so transparently that we don’t even notice.

But what’s even cooler: GenServers can already be accessed from other nodes within same BEAM cluster, so we already have something that is usable over a network! Let’s take a look at how to do that, with only a little bit of black magic.

Close the iex shell we had open, and create a new one, but this time with the --name parameter:

iex --name gptserver@127.0.0.1 -S mix
iex> Mixgpt.start_link
{:ok,  #PID<0.276.0>}

--name tells elixir to start a distributed node with the given name, on the given network. So we created a distributed node gptserver on the network 127.0.0.1

In a new terminal window, start another iex session, this time without -S mix, but still with --name:

iex --name client@127.0.0.1

To now execute things on another cluster node, Elixir provides the Node.spawn_link function among other things. With this you could do stuff like:

iex> Node.spawn_link(:"gptserver@127.0.0.1", fn -> IO.puts("hi there, I'm running on another node") end)

But we already have a GenServer running, so no need for that and just hit the GenServer directly. This is also very easy to do, just by changing the GenServer.call(:gptserver, {:msg, msg}) call to also include the node information where :gptserver is running: {:gptserver, :"gptserver@127.0.0.1"}:

iex> GenServer.call({:gptserver, :"gptserver@127.0.0.1"}, {:msg, "Helloo, anyone there?"})
"Hello! Yes, I'm here. How can I assist you?"

Do you realize what just happened? Our completely separate iex shells just connected to each other to send a message to the GenServer running under the :gptserver name, which then hits the OpenAI API, returns the data, and relays it back to the other iex shell!

We just created a distributed ChatGPT service on our local network (well, localhost), all without any extra setup required! This even works across machines, as long as the nodes are running reachable on the same network :)

Final step: A global shell command

You can already guess what comes next. We can run Elixir code directly with elixir -e "IO.puts \"Hi\"". The last lego piece missing is hooking this up to a global shell command, so that we can access our mini-ChatGPT whenever we need through a terminal, while retaining previous context.

To package it all up into a one-liner:

$ elixir --name client@127.0.0.1 -e "GenServer.call({:gptserver, :'gptserver@127.0.0.1'}, {:msg, \"hi\"}) |> IO.puts"
Hello! How can I assist you today?

That’s still the same call as before, except we’re running it through elixir -e "", and pipe the result into IO.puts to write it to the shell. This is still using the same GenServer that’s still running (unless you shut it down), and it still retains the same message history from earlier.

Lastly, package into a bash function:

ai() { elixir --name client@127.0.0.1 -e "GenServer.call({:gptserver, :'gptserver@127.0.0.1'}, {:msg, Enum.join(System.argv, \" \")}) |> IO.puts"; $@;  };

Tadaa! We now have a system-wide (or network-wide) mini-ChatGPT that is fully stateful

$ ai is the sky blue?
Yes, on clear days, the daytime sky typically appears blue due to the scattering of sunlight by Earth's atmosphere.
$ ai what is the last message I sent you?
Your last message was "is the sky blue?"

We could easily put this on a home server and have it available across all of our machines without any extra setup if we really wanted to

Of course, this is still super barebones and in reality, we’d do a few more things:

• A Supervisor to handle crashes
• Changing :gptserver to be global, so we don’t need to know the hostname
• Some way to reset, or clear the message history to not have messages endlessly pile up
• Better error handling

among others

Check out the code for this on GitHub

You probably know what ChatGPT and Dall E are by now - AI exploded into mainstream with the recent releases (most notably ChatGPT) out of nowhere, and now even Bing out of all things is becoming the latest AI craze.

As someone doing a lot of solo work, I became very grateful of the value tools like ChatGPT, GPT3 and DallE bring, and successfullly integrated them into my day to day work, both personal but also profesionally. Even for someone working in IT heavily, I’m still amazed by how damn good these tools actually are.

Today I want to touch on my most common usecases.

Dall-E: My Graphic Design buddy

Let’s start with Dall-E. Dall-E, also known as “Dall E 2,” is a generative text-to-image model. There are other popular ones like MidJourney or Stable Diffusion, but I prefer the results I get from Dall-E. If you haven’t heard of it, let me give you a quick rundown: these tools allow you to generate images by describing what you want to see in the image. For example, if you input “a cat riding a rocket across the stars, oil painting,” well, you’ll get an image like this:

If you’ve been following this blog, you may have noticed that most of the recent images I’ve published were generated with Dall-E. I like to have a header teaser image for each post to add consistency to my article format. Before Dall-E, I used to take screenshots of CLIs, code, or find something fitting from Unsplash, but that was too generic and boring for my taste. I didn’t want my posts to give off the “guru vibe” that you often see on Medium.

But my use of Dall-E goes beyond just that. Just recently, I was working on an app and needed a logo to publish it in the AppStore. I’m no designer, and my previous options were either to try to create something in Sketch, which could involve finding finished svg images and modifying them, or to go on Fiverr and find a designer. The problem with the latter was that I had no clear idea of what the logo should look like, and constantly saying “ah no, that’s not what I wanted” was awkward and annoying.

With Dall-E, I can explore different ideas completely on my own, quickly try out different concepts, and get a feel for whether the idea in my head even makes sense on a canvas. What I had in mind for this was something along the lines of “an envelope wearing glasses”, and the result:

Dall-E unblocked me at a phase that would have taken days to weeks previously and allowed me to focus on more important things. I’m not saying that all the images it generates are great and ready for production, but having a temporary, good-looking image or a starting point when talking to a real designer is such a big timesaver for solo developers.

ChatGPT and GPT3

Just when I thought Dall-E was going to be the most amazing thing I’ve seen in tech in years, ChatGPT and GPT3 came along. GPT3 was already impressive when it first came out, but API access was restricted by OpenAI and I couldn’t play with it properly. ChatGPT, with its free access, changed that.

“ChatGPT? GPT3? Wtf am I talking about?”

GPT3, or “Generative Pretrained Transformer” is the third version of OpenAI’s generative large language text model. In short, the GPTs are models that are trained on huge amounts of text from various sources with the goal of allowing the mathematical neural network to “understand” written language, its structure, nuances, and how humans use it. This pre-trained knowledge is then used to predict with high mathematical likelihood how a sentence should continue.

For example, if I were to write “The apple is,” and let the model complete it, it would have different paths to choose from, such as “red,” “tasty,” or “sweet.” Each possible path has weights assigned, and the model’s mathematical “brain” figures out the best path to take based on the data it was trained on.

ChatGPT, on the other hand, is GPT3 (or GPT3.5?) that has been specifically tuned for conversational use, whereas GPT3 is more general-purpose. Although they are different, ChatGPT still uses the GPT3 (or “davinci”) model.

GPT3 or ChatGPT?

Both tools have their place - ChatGPT is conversational and more general-purpose, while GPT3 is better for specific tasks. ChatGPT talks more, but GPT3 is harder to control with more options and parameters. I use ChatGPT when I want to try out ideas, similar to a sketchbook, and then GPT3 when I already know what I want and need to turn it into a repeatable task.

Here are some of my use cases:

Text Rephrasing (Handling Emotions, Adding Professionalism)

For example, when dealing with customer support, it can be challenging to maintain a professional tone, and emotions sometimes creep into my emails. ChatGPT takes those emotions out and rephrases my text to be professional.

In fact, using ChatGPT for a proofreading pass has become the most common task I use it for! I can write what I want to say, then use ChatGPT to dial emotions in and out, make things more concise or add fluff (if I wanted to).

Language Learning (it speaks Japanese!)

ChatGPT is surprisingly good for language learning. I use a lot of Japanese at work, and there are always moments when I’m not sure if the grammar I’m using is correct. I used to have no real way of figuring those mistakes out on my own and would have to rely on someone pointing them out (which in reality never happens).

ChatGPT isn’t perfect, but it’s catching the major mistakes and pointing me in the correct direction. I can’t recommend it to people learning English enough!

Here is the prompt I am using for Japanese these days:

You are a Japanese grammar correction bot. Do not chat, instead for each Japanese message, first output a corrected version of that message that is grammatically correct and could have been written by a Japanese native person. Do not forget particles and the correct tenses. Put this under a "corrected text" section.

Next, point out all the grammar mistakes that you have corrected in my text, and the reason why you corrected them. You are allowed to use Japanese words in the explanation. This will start after my next message. Reply with “ok” on this message if you understood what I want.

Text Summarizing

For when I want to save time and get a concise summary of something, like an article or a very long Slack message. Let’s take the article We came to bury ChatGPT, not to praise it that was recently on the Hackernews landingpage and feed it into ChatGPT:

What we get with this prompt is a neatly summarized version that’s easy to digest.

I can also adjust the level of technical detail in the summaries and how much professional lingo I want to allow through. Because ChatGPT is conversational, it makes these back and forth adjustements easy:

Brainstorming names and ideas

From app names to bouncing ideas back and forth, ChatGPT is great for brainstorming as well!

Once I have some interesting ideas to go with, I can ask ChatGPT to iterate on names I like, or even add certain flair (make it sound more “Japanese”). Sadly it goes out of control very fast and for this specific usecase I usually have to reset the conversation a couple times before coming up with something good. (Great example why conversational and putting so much focus on the current context isn’t always the way to go, this is where GPT3 shines).

It also tends to get hung up on specifics, for example when I wanted to brainstorm some name ideas for my tech freelance business, it really wanted to insert the word “tech” whereever possible.

The future is going to be interesting

Together with Codex/Github Copilot, AI powered tools are becoming more and more of a staple in my day to day work. It’s incredible what we can already do with the tools we have today, and it’s going to be even more interesting to see what the tools of tomorrow will be able to do.

But if I were to ramble a bit more, I do wonder if AI will make us a lot more lazy. Because why bother writing a longform article when some variant of GPT trained on the content of my blog can just generate it on the spot? Or why bother reading something when I can get a summary with as much detail as I want in seconds? How do you know the article you’re reading right now hasn’t just been generated for the sake of putting out more content for this blog?

GPT3 can already generate better messages than I can in a fraction of the time. Dall E can design better than me, and Codex may soon be able to program more efficient and with less errors than me.

While I am excited, I still feel that a lot of individuality is already getting lost. Maybe in a short while, the internet will be full of autogenerated stuff - from messages that are rewritten by an AI before sending, to images and logos that got generated with prompt engineering.

ChatGPT is really damn impressive and the more I read about it, the more it feels like withcraft. I see people using it for game development, coming up with titles, improving prompts for MidJourney and and and. It’s shaping up to be one of those universally useful tools that are great to keep around.

Here’s one thing I had on my mind: Can it help me when I code?

I was working on a little pet project and had some pretty inefficient code I slapped together for a middleware while watching TV:

package middleware

import (
	"context"
	"encoding/json"
	"net/http"
)

var parsedJsonBodyContextKey struct{}

func ParseJsonBody(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		var data map[string]interface{}
		err := json.NewDecoder(r.Body).Decode(&data)
		if err != nil {
			ctx := context.WithValue(r.Context(), parsedJsonBodyContextKey, nil)
			next.ServeHTTP(w, r.WithContext(ctx))
			return
		}

		ctx := context.WithValue(r.Context(), parsedJsonBodyContextKey, nil)
		next.ServeHTTP(w, r.WithContext(ctx))
	})
}

The code takes whatever is passed into the request body, tries to JSON decode it, then puts the result into context. If no body exists or isn’t JSON, it will pass nil instead.

I wanted to shorten and make it a bit more efficient but though it’d be a cool experiment to let ChatGPT do it:

Me: Can you refactor the following code to make it more efficient?

<code from above here>

ChatGPT: Here are a few ways the provided Go code can be imporved:

1. The ParseJsonBody() function defines a data variable to store the parsed JSON body, but it doesn’t appear to be used anywhere in the function. This variable can be removed.
2. The ParseJsonBody() function creates a new context.Context with a key-value pair for the parsed JSON body, but it sets the value to nil in both branches of the if statement. This means that the value of the parsed JSON body will always be nil, regardless of whether the JSON body was parsed successfully or not. The value of the parsed JSON body should be set to the parsed JSON data if parsing was successful, and nil if parsing failed.
3. The ParseJsonBody() function passes the original http.Request object to the next handler, even if the JSON body was parsed successfully. In this case, the http.Request object should be updated with the new context.Context that contains the parsed JSON body before being passed to the next handler.

Here is how the code can be improved:

package middleware

import (
	"context"
	"encoding/json"
	"net/http"
)

var parsedJsonBodyContextKey struct{}

func ParseJsonBody(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		var data map[string]interface{}
		err := json.NewDecoder(r.Body).Decode(&data)
		if err != nil {
			ctx := context.WithValue(r.Context(), parsedJsonBodyContextKey, nil)
			next.ServeHTTP(w, r.WithContext(ctx))
			return
		}

		ctx := context.WithValue(r.Context(), parsedJsonBodyContextKey, data)
		next.ServeHTTP(w, r.WithContext(ctx))
	})
}

Huh, turns out I copy pasted and forgot to change the normal branch to actually add data into the context - whoops. ChatGPT spotted this, understood what this was supposed to do and fixed it.

Can we make it more compact though?

Me: Can you make the code more compact and more DRY?

ChatGPT: Yes, the provided code can be made more compact and DRY (Don’t Repeat Yourself) by removing the duplicate code and using a single context.Context object.

Here is how the code can be improved:

package middleware

import (
	"context"
	"encoding/json"
	"net/http"
)

var parsedJsonBodyContextKey struct{}

func ParseJsonBody(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		var data map[string]interface{}
		err := json.NewDecoder(r.Body).Decode(&data)

		ctx := context.WithValue(r.Context(), parsedJsonBodyContextKey, data)
		next.ServeHTTP(w, r.WithContext(ctx))
	})
}

This is exactly what I had in mind for refactoring this code: No need for the if condition at all because var data map[string]interface{} is nil by default. Only nitpick would be that we assign err and never use it, so the compiler will complain about this code.

Let’s add a function to get the data out of the context again. Can ChatGPT summon this function for us?

Me: Please add a function that takes the body out of the context again

ChatGPT: To extract the parsed JSON body from the context.Context object, a new function can be added to the middleware package. This function can take a context.Context object as an argument and return the parsed JSON body, if it exists in the context, or nil otherwise.

Here is an example implementation of this function:

package middleware

import (
	"context"
	"encoding/json"
)

var parsedJsonBodyContextKey struct{}

func ParseJsonBody(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		var data map[string]interface{}
		err := json.NewDecoder(r.Body).Decode(&data)

		ctx := context.WithValue(r.Context(), parsedJsonBodyContextKey, data)
		next.ServeHTTP(w, r.WithContext(ctx))
	})
}

func GetParsedJsonBody(ctx context.Context) map[string]interface{} {
	val := ctx.Value(parsedJsonBodyContextKey)
	if val == nil {
		return nil
	}
	return val.(map[string]interface{})
}

To use this function, you can pass a context.Context object as an argument and check the return value for nil to determine whether the JSON body was parsed successfully or not.

Here is an example of how this function can be used:

package main

import (
	"context"
	"fmt"
	"net/http"

	"github.com/myproject/middleware"
)

func main() {
	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		parsedBody := middleware.GetParsedJsonBody(r.Context())
		if parsedBody == nil {
			w.Write([]byte("JSON body was not parsed"))
			return
		}
		w.Write([]byte(fmt.Sprintf("

Short and concise. It understood what I wanted it to do, it delivered a solution and even gave an explanation with examples how to use it!

After a bit of hacking I noticed that using map[string]interface{} is kind of a PITA because we have to do many type assertions and checks if keys exist. I got curious if we can tell it to swap out the type into something custom that has a nicer interface:

Me: Instead of using map[string]interface{}, let’s use a custom type that has methods for getting the value for a given key, or a default

ChatGPT: To use a custom type that has methods for getting the value for a given key or a default value, you can define a new ParsedJsonBody type that satisfies the context.Context interface. This type can be a struct that contains a map[string]interface{} field to store the parsed JSON body.

Here is an example implementation of the ParsedJsonBody type:

type ParsedJsonBody struct {
	data map[string]interface{}
}

func (p *ParsedJsonBody) Value(key interface{}) interface{} {
	if key == parsedJsonBodyContextKey {
		return p
	}
	return nil
}

func (p *ParsedJsonBody) Get(key string) interface{} {
	return p.data[key]
}

func (p *ParsedJsonBody) GetString(key string) string {
	val := p.data[key]
	if val == nil {
		return ""
	}
	return val.(string)
}

func (p *ParsedJsonBody) GetInt(key string) int {
	val := p.data[key]
	if val == nil {
		return 0
	}
	return val.(int)
}

The ParseJsonBody() middleware function can be updated to use the ParsedJsonBody type instead of map[string]interface{}:

Hmm okay, this is where things get a bit weird, but that could have also been my way of explaining what I wanted.

The code looks ok on first skimming, but it ignored my instruction to have a default. I think it interpreted that as “we’ll return a default”, but not one I can specify.

There’s lots of type assertions in there that don’t have any checks, so this will panic if we were to try to GetInt something that isn’t an int. Also, you can define a new ParsedJsonBody type that satisfies the context.Context interface is nonsense, and so is the Value() method.

What I actually wanted was a custom type like type MyType map[string]interface{}, not a struct with a type field, but whatever.

Let’s ask directly if it thinks anything is wrong with that code:

Me: Thanks for the code, do you see any problems with this implementation?

ChatGPT: There is a potential problem with the provided implementation of the ParsedJsonBody type. The Value() method returns a pointer to the ParsedJsonBody struct, but the Get(), GetString(), and GetInt() methods return values of the underlying data field. This means that if the ParsedJsonBody struct is modified after it is added to the context.Context object, the modifications will be reflected in the context.Context, but not in the return values of the Get(), GetString(), and GetInt() methods.

To fix this problem, the Get(), GetString(), and GetInt() methods should return copies of the values in the data field, rather than the values themselves.

Here is an updated implementation of the ParsedJsonBody type that fixes this problem:

type ParsedJsonBody struct {
	data map[string]interface{}
}

func (p *ParsedJsonBody) Value(key interface{}) interface{} {
	if key == parsedJsonBodyContextKey {
		return p
	}
	return nil
}

func (p *ParsedJsonBody) Get(key string) interface{} {
	val, ok := p.data[key]
	if !ok {
		return nil
	}
	return val
}

func (p *ParsedJsonBody) GetString(key string) string {
	val, ok := p.data[key].(string)
	if !ok {
		return ""
	}
	return val
}

func (p *ParsedJsonBody) GetInt(key string) int {
	val, ok := p.data[key].(int)
	if !ok {
		return 0
	}
	return val
}

The good: It fixed the type assertions and now has a checks, neat!

The bad: The explanation it gave on why it added those doesn’t make sense. It didn’t actually change those to return copies of values. Let’s do one more optimization pass.

Me: Please further optimize this code

ChatGPT: Here are a few ways the provided implementation of the ParsedJsonBody type can be further optimized:

1. The Value() method currently returns p if the key argument matches parsedJsonBodyContextKey, and nil otherwise. Since parsedJsonBodyContextKey is a global variable that is only used in the ParseJsonBody() middleware function, this check is unnecessary. The Value() method can simply return p without checking the key argument.
2. The Get(), GetString(), and GetInt() methods currently check if the specified key exists in the data field and return a default value if the key does not exist. This check can be removed, because the data field is a map[string]interface{} and trying to access a non-existent key in a map will return the zero value for the type of the map’s value (interface{} in this case, which is nil).
3. The GetString() and GetInt() methods currently use the ok return value of the type assertion to determine whether the value for the specified key is of the expected type. This check can be removed, because

Me: Please show me an implementation with those points

ChatGPT:

type ParsedJsonBody struct {
	data map[string]interface{}
}

func (p *ParsedJsonBody) Value(key interface{}) interface{} {
	// The key argument is not used, so we can simply return p
	return p
}

func (p *ParsedJsonBody) Get(key string) interface{} {
	// We don't need to check if the key exists, because trying
	// to access a non-existent key in a map will return the zero
	// value for the type of the map's value (nil in this case)
	return p.data[key]
}

func (p *ParsedJsonBody) GetString(key string) string {
	// We don't need to check the type of the value, because trying
	// to assert a non-string value to a string type will return
	// the zero value for the string type ("" in this case)
	return p.data[key].(string)
}

func (p *ParsedJsonBody) GetInt(key string) int {
	// We don't need to check the type of the value, because trying
	// to assert a non-int value to an int type will return the
	// zero value for the int type (0 in this case)
	return p.data[key].(int)
}

It got more compact for sure, but the code is wrong. It got rid of the type checks with the confident explanation that type-asserting a non-int value to an int type will return the zero value for the int type, but this is not correct and will panic. Here’s the code in Golang playground to try out: https://play.golang.com/p/8jjBJyrJ1uC

While impressive, at this point it’s internal state of my program and what I’m trying to do is a bit too messed up, and the confidently wrong explanation+implementation can be dangerous. For a moment I self-doubted my own knowledge of Golang and had to try this out in the playground to make sure.

The network isn’t specifically trained on code, so I guess it just doesn’t fully understand the inner-workings of Golang, without providing enough context to it first.

Generating unit tests

Let’s do something fun - unit tests (groans)! If it can spit out functional unit tests for stuff I’m too lazy to write, that would already be very helpful. After all, it seems to understand code and is able to give explanations for what’s going on.

The code here is a snippet from a JWT token parser. It validates the token, extracts sid/uid scopes (UUID) and checks if those are actually valid UUIDs.

Me: Consider the following Golang code:

// other code here
func (c *Client) validateToken(t jwt.Token) (*Token, error) {
	if err := jwt.Validate(t); err != nil {
		return nil, err
	}

	// parse sid
	sid, ok := t.Get("sid")
	if !ok {
		return nil, domain.ErrTokenInvalidSID
	}
	sidUUID, err := uuid.Parse(sid.(string))
	if err != nil {
		return nil, domain.ErrTokenInvalidSID
	}

	// parse uid
	uid, ok := t.Get("uid")
	if !ok {
		return nil, domain.ErrTokenInvalidUID
	}
	uidUUID, err := uuid.Parse(uid.(string))
	if err != nil {
		return nil, domain.ErrTokenInvalidUID
	}

	return &Token{
		SessionID: sidUUID,
		UserID:    uidUUID,
	}, nil
}
// other code here

Please generate some testcases for when sid or uid are invalid

ChatGPT: Here are some test cases for when the sid or uid values in the token are invalid:

func TestValidateTokenInvalidSID(t *testing.T) {
 // Create a new client
 client := NewClient("some-secret-key")

 // Create a test token
 token := jwt.New()
 token.Set("uid", uuid.New().String())
 // Set the sid value to something invalid
 token.Set("sid", "invalid-sid")
 token.Set(jwt.ExpirationKey, time.Now().Add(time.Minute * 5).Unix())

 // Sign the token
 signedToken, err := jwt.Sign(token, jwt.WithKey(client.alg, client.hmacSecret))
 if err != nil {
     t.Errorf("unexpected error signing token: %s", err)
 }

 // Validate the token
 _, err := client.validateToken(signedToken)
 if err != domain.ErrTokenInvalidSID {
     t.Errorf("expected error to be %s, got %s", domain.ErrTokenInvalidSID, err)
 }
}

func TestValidateTokenInvalidUID(t *testing.T) {
 // Create a new client
 client := NewClient("some-secret-key")

 // Create a test token
 token := jwt.New()
 // Set the uid value to something invalid
 token.Set("uid", "invalid-uid")
 token.Set("sid", uuid.New().String())
 token.Set(jwt.ExpirationKey, time.Now().Add(time.Minute * 5).Unix())

 // Sign the token
 signedToken, err := jwt.Sign(token, jwt.WithKey(client.alg, client.hmacSecret))
 if err != nil {
     t.Errorf("unexpected error signing token: %s", err)
 }

 // Validate the token
 _, err := client.validateToken(signedToken)
 if err != domain.ErrTokenInvalidUID {
     t.Errorf("expected error to be %s, got %s", domain.ErrTokenInvalidUID, err)
 }
}

Sweet as. Sadly it’s missing a bit of context here too, so I can’t copy-paste the test as is. jwt.Sign() isn’t returning a jwt.Token, but the jwt token in signed form, so it can’t get passed to validateToken, but there’s no way that it could have known that, without feeding the the jwt package into it first. Besides that it looks good, and the tests run after fixing that!

And to go full circle, now that we have more unit tests, let’s ask it to optimize my validateToken function as well:

One possible optimization for the validateToken function would be to move the code that parses the sid and uid values into a separate function, and then call that function from validateToken instead of duplicating the code. This would make the code easier to read and maintain, and would also avoid repeating the same error-checking code multiple times.

Here is an example of how the optimized validateToken function could look like:

func (c *Client) validateToken(t jwt.Token) (*Token, error) {
    // Validate the token
    if err := jwt.Validate(t); err != nil {
        return nil, err
    }

    // Parse the sid and uid values from the token
    sid, err := c.parseSID(t)
    if err != nil {
        return nil, err
    }
    uid, err := c.parseUID(t)
    if err != nil {
        return nil, err
    }

    return &Token{
        SessionID: sid,
        UserID:    uid,
    }, nil
}

func (c *Client) parseSID(t jwt.Token) (uuid.UUID, error) {
    // parse sid
    sid, ok := t.Get("sid")
    if !ok {
        return uuid.UUID{}, domain.ErrTokenInvalidSID
    }
    sidUUID, err := uuid.Parse(sid.(string))
    if err != nil {
        return uuid.UUID{}, domain.ErrTokenInvalidSID
    }

    return sidUUID, nil
}

func (c *Client) parseUID(t jwt.Token) (uuid.UUID, error) {
    // parse uid
    uid, ok := t.Get("uid")
    if !ok {
        return uuid.UUID{}, domain.ErrTokenInvalidUID
    }
    uidUUID, err := uuid.Parse(uid.(string))
    if err != nil {
        return uuid.UUID{}, domain.ErrTokenInvalidUID
    }

    return uidUUID, nil
}

I was hoping it’d catch the naked .(string) type assertion, but code splitting is also nice :D

Although this iteration doesn’t really avoid repeating the same error-checking code multiple times like it claims.

Collaborative hacking with an AI buddy

I use Copilot quite a lot so I’m already used to having an AI pair-programming buddy, but I strongly believe that as these tools get smarter, they will drastically change the way how we as engineers work.

While not there yet, AI will be able to generate more efficient code faster than us, spot errors we’d miss, or flat-out generate entire applications based on spec input.

When Stable Diffusion came out, some people integrated it into their photo editors to quickly generate content based on a prompt. I am sure we’ll see something like this, but for code, in no time. Imagine reading through some code, hitting an “AI” button, then ask it to refactor a routine, or give an explanation what that code does. The plugin will then send all the necessary context to the neural net and ask it to do the task.

/EDIT: someone already did it: https://twitter.com/marcelpociot/status/1599180144551526400

The next generation of AI powered plugins could also do things like quickly provide feedback when it strongly believes a mistake has been made, or when it has a high confidence score for a completion / refactoring.

I think that would be really dang cool!

tl;dr: unisync is a little tool I wrote that diffs and syncs your application settings across Macs, powered by unison

Config management across machines is kind of a PITA. Some apps luckily natively support syncing settings, but most don’t.

There are some solutions like the excellent Mackup that try to fix this issue by moving all config files into a shared folder (like Dropbox), then creating symlinks from the application settings folders (eg ~/Library/Preferences/xxxx.plist) into that Dropbox folder.

I used mackup for a while (even with my own config files), but eventually got frustrated when symlinks randomly broke, for example, because the app decided it wanted to write the full file fresh to disk, effectively overwriting the symlink. Then some other apps detect symlinks and refuse to read them.

I tried to mitigate that by creating a PR to mackup that adds support for hardlinks, but tl;dr: hardlinks in APFS are not really hardlinks, and this is even more fragile with cloud storage than using symlinks.

Unison is great

unison is one of these really old tools that’s been around and developed forever. It’s a

*“Unison is a file-synchronization tool for POSIX-compliant systems (e.g. *BSD and GNU/Linux), macOS and Windows, with the caveat that the platform must be supported by OCaml”*

basically, it syncs files, kind of like a bi-directional rsync: Where rsync syncs one way (and alternatively deletes files), unison goes both ways and finds files that changed, got added or deleted across multiple directory nodes.

Bi-directional sync makes it great for those pesky config dirs that could change on one host without us noticing. By diffing the directories against, say, a directory in Dropbox, unison can determine what actually changes, and copies the files accordingly.

If a file is updated on Host B and is then synced through Dropbox to Host A, unison on Host A would see that the file in Dropbox is now newer than on the local one, and overwrite it with the version from Dropbox.

unison can even sync across machines directly!

Mackup + Unison = Unisync

That brings us to unisync. Unisync is using the Mackup idea of having a library of app configurations that state where config files are, but instead of using symlinks, it calls into unison to do the actual copying of files. So in theory, if files changed on one machine and then get synced to other machines, those files would get marked as changed / new, and get copied over the existing local file.

Because unisync is using unison under the hood, we can do more fancy things like providing a tree with files we want to handle, but add regex or naming patterns to ignore rules:

name: Raycast
friendlyName: raycast
files:
  - basePath: ~/Library/Preferences/
    includedFiles:
      - com.raycast.macos.plist
    ignoredFiles:
      - Name somethingToIgnore.plist

Unisync also supports the entire Mackup app library for syncing to get you started, and comes with LaunchAgents that make it run automatically in the background.

Check it out on GitHub and let me know if this tool is useful for you: https://github.com/dvcrn/unisync

What a week, right? FTX, one of the biggest exchanges in the world, went from a reputable player that innovates the crypto space, to now rumors floating around that its ex-CEO is on the run, joining folks like Do Kwon and the guy from 3AC.

What happened made me incredibly sad. I am a big proponent of cryptocurrency (after all, I built a wallet for Solana with Saifu) and am pretty active in the space. FTX was my favorite exchange, and I am very grateful that I was able to move 95% of my assets from FTX, roughly a day before withdrawals got halted (with the rest not movable due to open positions that were in the negative and didn’t want to close them off yet), but I still lost a good amount of money that’s now in eternal limbo, never to be seen again.

Now there is more news - BlockFi halts withdrawals as well. Do you remember the time when ‘halting withdrawals’ was still unheard of? How the times have changed.

Even though I got off fine this time around (I did lose a sizable sum on the UST collapse that was in Binance’s UST Earn product), I couldn’t help but notice a feeling of uneasiness. What if I hadn’t read Reddit or Crypto news on that particular day? 1/3 of my funds were on FTX, and that would have been gone now. In just 2-3 days! Crazy.

But this also means that I can’t let my guard down. Instead of reading about crypto because I’m interested in it, I am now reading to make sure I am avoiding the next bomb.

We’re not doing so great right now

With FTX and CEXes collapsing, one thing I constantly read is “not your keys not your crypto”, and “just put it into a ledger and don’t touch it”, both of which are absolutely true. But as a fan of crypto, I want to actually do stuff with it, you know? What’s the point of a hyper-modern globally distributed digital currency network, if it has the same utility as a collection of rocks that you have in your closet, that may appreciate in value in the future? Let me lend it out to get a bit of interest, stake some here and there, interact with protocols, borrow quickly against it when I go on vacation and need a bit of extra cash - you get the idea. But honestly, I don’t feel comfortable doing much of that currently.

The DeFi world is a mess. Weekly rug pulls left and right, Smart Contracts get hacked, flash loan attacks, some guy finds an exploit to drain millions (and brags about it on Twitter), and poof - everyone’s money is gone. There are more reputable players like Aave that have been okay, but with even behemoths like Wormhole getting drained of funds, I’m not confident that it’ll stay this way. Venus, for example, an Aave clone on BSC, had rumors of price manipulation to forcefully liquidate users. The temptation to make big money is just too big for some.

The CeFi world is a mess (obviously). Don’t think I need to explain more considering recent events but - players like Blockfi/Celsius/Hodlnaut imploded out of nowhere, and FTX went up in flames. The money I had on Binance Earn (in UST) evaporated overnight (not Binances fault, I know). All while CEOs go publicly on Twitter to guarantee that yes, everything is indeed ok and there is no need to worry, only to a couple of days later disable withdrawals and turn the company with all user assets into a coffin.

I practice risk management and am very careful with what I do with my money: I don’t have my stuff in one place, and I avoid almost all DeFi protocols besides Aave. I only stake my stuff through native staking that’s at the core of the chain’s consensus mechanism, and never through third parties (like Lido). I evaluate CEXes very carefully and read everything I can find on how assets are handled and protected.

I am the opposite of a degen gambler: A very informed tech person working in crypto, and still, I am losing money.

Besides having my crypto in a ledger and looking at it, I don’t feel much confidence in using it, and that’s bad. We absolutely should aim for having a vibrant ecosystem of DeFi and CeFi players, but the trust right now has been destroyed. People are scared of the next domino falling and someone running away with their hard-earned cash.

Have we hit crypto burnout?

Mistrust can be a good thing

I wanted to leave it at that because the last sentence was a nice ending point, but had some more thoughts to share.

Not all is bad of course. The current events caused a big wave of mistrust and that can be a good thing. Exchanges have to be transparent about how funds are handled to earn their trust, and I can imagine seeing big boys like Celsius/FTX implode causes a lot of the remaining ones to sweat and rethink their investment strategies. Users become more careful about where they put their cash, and regulations are getting tighter. Maybe we’ll even see FDIC-like insurances popping up?

What’s left after all of this is (hopefully) a more refined crypto space, with more transparency, more safety, and higher standards to hold CeFi/DeFi players to.

We may have been set back by a couple of years, but maybe those years will now be a bit better.

As for me, I will crank up my “risk management” lever. I moved most of my stuff into hardware wallets, moved to on-chain staking where I was able to, and reduced the amount of crypto stuff I was using. I don’t trust anything DeFi that’s not Aave (very small amounts in Venus) and will continue with that attitude.

In the CeFi world, I am a fan of Nexo (ref link) - they have been very transparent so far, had no exposure to any recent events, survived multiple bank runs, and are headquartered in Europe, not the Bahamas. Still, only time can tell if they end up becoming the next domino. Binance is good, Crypto.com (exchange) is good too and so is Kraken. I’ll continue using all 3 of them.

We’ll get through this. Let’s keep on innovating and building!

I’m one of those odd nuts that love kubernetes. I like it so much that it’s currently powering a lot of my personal stuff and hobby projects. So instead of the usual “k8s is bloated”, “k8s is overkill” or “why you don’t need k8s” posts, today let’s talk about why k8s is actually great for personal stuff, and why you should maybe also consider using it? :)

1. Why I love kubernetes for personal stuff

Let’s start with the why. Why would I use something as bloated and heavy as kubernetes for personal stuff, isn’t it a huge overkill? I could go on and on about why it’s great, but let’s take a look at the big reasons.

I want to also preface that small managed clusters can be really cheap. My main managed “cluster” (putting it in quotes because it only has 1 node) on digitalocean costs $12/month. Compare that with other hosting providers.

Infra as code 👨‍💻

k8s, same as with terraform, allows me to specify my infrastructure as code. I write manifest files that specify how stuff is going to get run. If I want to make a change to my infra, all I need to do is update those manifests, apply them, and I’m done. Look at this snippet from a manifest that tells k8s that I want to run a webdav container:

      containers:
        - name: webdav
          image: bytemark/webdav
          ports:
            - containerPort: 80
          env:
            - name: "USERNAME"
            	value: "myuser"
            - name: "PASSWORD"
              valueFrom:
                secretKeyRef:
                  name: webdav-credentials
                  key: password

Because I have everything in code, I can set up my entire cluster from scratch on any provider I want. If I no longer like digitalocean, all I have to do is create a cluster somewhere else and do a kubectl apply. The new setup will be identical (or almost identical) to my previous one, it’s almost provider agnostic (there are some provider-specific things like persistent volumes, but we’ll get to those later).

Manifests are also idempotent, I can re-apply the same files a dozen times without worry of breaking anything. If resources already exist, they stay existent and if there are no changes, the manifest will just get ignored.

Extremely easy to add storage or IP addresses 🔄

Managed kubernetes clusters (like the digitalocean one) are integrated with the entire cloud platform of that provider, so it’s very easy to do things like adding block storage or loadbalancers. How easy? Check this out:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: some-name
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: do-block-storage

This manifest when applied will tell DigitalOcean (through the kubernetes controller) to provision a 1Gb block storage volume. If that 1Gb fills up at some point, I change it to 10Gb, re-apply and DigitalOcean will automatically resize the volume to 10GB for me - neat!

Even cooler, I can now use this storage volume on any of my containers that are running inside my cluster. All I need to do is add a mount point into the manifest, and suddenly ephemeral containers have a way to write persistent data:

volumes:
  - name: some-name-volume
    persistentVolumeClaim:
      claimName: some-name
      readOnly: false

If I want to get rid of the volume, I do kubectl delete pvc some-name and it gets wiped. (There is a retain setting if you don’t want volume content to get deleted)

This isn’t just storage though, the same thing happens with stuff like LoadBalancers and egress IPs. Provisioning resources on kubernetes causes DigitalOcean to get a static IP + LoadBalancer ready for us, wait until it’s good to go, then assign it to the cluster.

It makes running stuff and keeping stuff running a breeze 🍃

K8s is very battle-tested. A lot of companies run on kubernetes so it’s grown to be robust and fault-tolerant. And if something does end up not working or acting up, I can be sure that I will find the answer to my issue in no time.

Besides running containers, k8s does things like keeping those running (if you want that). Let’s take a look at the webdav example from above, but conjure up some more yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: webdav
spec:
  replicas: 1
  selector:
    matchLabels:
      app: webdav
  template:
    metadata:
      labels:
        app: webdav
    spec:
      containers:
        - name: webdav
          image: bytemark/webdav
          ports:
            - containerPort: 80
          env:
            - name: "USERNAME"
            	value: "myuser"
            - name: "PASSWORD"
              valueFrom:
                secretKeyRef:
                  name: webdav-credentials
                  key: password

The webdav container is now in a Deployment with 1 replica. This means k8s will make sure there is always 1 webdav container running. You can even kubectl delete pod <webdav-xxxx> to destroy the webdav container, and kubernetes will just respawn a new one.

My server can crash, the container can crash - doesn’t matter, don’t care. In the end, kubernetes will jump in and just scale a new container up again.

And if I want more containers? All I have to do is change replicas: 1 to replicas: 2, and now I will always have 2 webdav containers. This makes scaling things up and down as easy as changing a yaml file and applying it.

But there are other cool things that just work like CronJobs! I can specify my CronJob yaml manifests, tell kubernetes when it should run those, and I can be sure they are getting run. No messing with crontab or some cloud scheduling thing, I can do it right from within k8s, together with the rest of my infra.

Easy scaling when it becomes necessary 📈

So we saw in the previous section that we can scale containers however we please. It gets even cooler if we deal with multiple nodes on the cluster. Say my hobby project is getting more popular - instead of a simple web server I had to add a database and maybe a queue server. Resources are getting tight, and stuff doesn’t run as nicely any more.

What I can do is, I can open up the DigitalOcean kubernetes admin, go to my node settings, and either:

• Increase the nodes of the cluster to 2
• Destroy the current 1-node setup and replace it with a stronger node

Because k8s is managing the cluster, even if we destroy the node and add a new, stronger node in its place, it will still do exactly what it did before: Make sure those containers are running in the configuration we specified as soon as the new machine is up and running.

If we increase the cluster size from 1 to 2, kubernetes will look at available resources on the cluster, see that a new node is available, and balance those containers for us, by maybe moving the queue server to the new machine, or having one of those 2 webdav replicas there instead. (Of course, if we want less magic we can also tell it specifically what it should do and how it should utilize those nodes, like always having one replica on each node)

It goes the other way as well: We’re getting tight on money so we decided to cut down on resources. Removing a node from the cluster will make k8s reshuffle our containers and consolidate them onto the remaining nodes, and THEN shut the node down, without downtime.

2. How I use kubernetes for my personal stuff

Now that you’ve heard the main reasons why I like kubernetes, let’s take a look at how I specifically am using it on a daily basis.

Let’s start with the provider - I mentioned that I use DigitalOcean kubernetes for my cloud of choice. I like them because the cost is predictable and the managed cluster is cheap. I am always sure how my bill will look like and there are no surprises. At $12/node for kubernetes I really have no complaints here. It’s great! This price doesn’t include a LoadBalancer/static IP but for most stuff I don’t really need those.

k8s can do a lot, but the parts that I actually use are:

• Deployments: Fancy way of saying “keep this many replicas running at all time”
• Services: Fancy way of saying “I need an cluster internal/external IP for these containers”
• Ingress: A fancy router like “route domain david.coffee to this pod”
• Cron

Exposing stuff for personal use only with tailscale

Most of my stuff doesn’t need to be public. It’s either a cronjob that does something and doesn’t need inbound connections, or it’s stuff that’s only public to me.

To make things public to myself only, I am using tailscale as my overlay network of choice (other great options are ZeroTier and Slack Nebula). Tailscale utilizes wireguard tunnels to build a Layer 3 p2p network that most of the times Just Works™️

The cool thing is, I can embed the sidecar container into any kubernetes pod I want to access, and it becomes available to my tailscale network. All I have to do is add some yaml to a pod:

      containers:
        - name: webdav
          image: bytemark/webdav
          ports:
            - containerPort: 80
          env:
            - name: "USERNAME"
            	value: "myuser"
            - name: "PASSWORD"
              valueFrom:
                secretKeyRef:
                  name: webdav-credentials
                  key: password
        - name: ts-sidecar
          imagePullPolicy: Always
          image: "ghcr.io/tailscale/tailscale:latest"
          env:
            - name: TS_KUBE_SECRET
              value: "tailscale-webdav"
            - name: TS_USERSPACE
              value: "false"
            - name: TS_AUTH_KEY
              valueFrom:
                secretKeyRef:
                  name: tailscale-auth
                  key: TS_AUTH_KEY
          securityContext:
            capabilities:
              add:
                - NET_ADMIN

… and when my connection is established, I can just do http://webdav-tailscale:80 and I have a fully personal connection to my webdav container. No ingress or load-balancer needed ✌️

A non-tailscale option would be to use port-forwarding into the pod directly:

  # Listen on port 8888 locally, forwarding to 80 in the pod
  kubectl port-forward pod/mypod 8888:80

Exposing public stuff without static IP + LoadBalancer with Nginx (the frugal option)

Because my cluster has only 1 node, I almost never need a proper loadbalancer or static IP.

My ingress controller of choice is nginx, and we can tweak it to run off a ClusterIP service instead of a full blown load balancer:

helm install ingress-nginx ingress-nginx/ingress-nginx -f nginx-ingress.yml

---
controller:
  kind: DaemonSet
  daemonset:
    useHostPort: true
  dnsPolicy: ClusterFirstWithHostNet
  hostNetwork: true
  service:
    type: ClusterIP
  resources:
    requests:
      cpu: 10m
rbac:
  create: true

A DaemonSet is something we haven’t talked about yet, it’s similar to a Deployment with the exception that it makes sure the thing is running on all nodes of the cluster. So if the cluster has 1 node, there’ll be one of those. If there are 3 nodes, we’ll have 3 nginx controllers.

hostNetwork: true is running that service off the host directly, so we can hook up <machineip>:80 and let nginx handle the routing.

Then once we have that contorller running we can add actual Ingress resources to do something based on domain name like this:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress
spec:
  ingressClassName: nginx
  defaultBackend:
    service:
      name: ingress-nginx-controller
      port:
        number: 80
  rules:
    - host: somethingsomething.david.coffee
      http:
        paths:
          - pathType: Prefix
            path: /
            backend:
              service:
                name: somethingsomething-service
                port:
                  number: 80

To make sure the IP stays up to date, we can hook up a cronjob and update the cloudflare DNS records automatically, for maximum money saving.

Of course if you have the cash and want to scale up, just get a proper static IP with LoadBalancer. It’s only $10 extra.

Happy whale-ing

As you can tell by this long post, I am a fan of kubernetes and use it for my personal stuff a lot. If you read until this point, congratulations! You made it. Let me know on twitter if you think this kind of post is useful, or if there are parts you’d like to know more about.

Happy whale-ing! 🐳

Happy to announce that my app for managing Fastmail Masked Emails is now available in the AppStore!

The app is available for free, but I might change it to the lowest price tier soon, to cover development cost (or maybe some hybrid one-time in-app purchase solution)

What it is and what it does

The app is a small utility for managing Masked Emails, Fastmail’s take on one-time / domain-specific emails.

The need for this app came when I had to create Masked Emails while on the go, especially on my phone, like when signing up for a new app, or needing an email for a hotel registration. Going through the web-based Fastmail app works, but was too slow for my taste.

Masked Email Manager is native and serves only that one purpose: Create and copy Masked Emails quick, with the option to disable no longer used ones.

Technical implementation

I started this app because I wanted to do something with WASM. The core is compiled from Golang into JavaScript, then executed through React Native JSC. So it’s technically a Golang program that’s powering the main logic.

Sadly after a lot of hours of debugging I came to the conclusion that WASM just isn’t feasible on React Native yet, so the Golang code is compiled to GopherJS instead.

If you’re interested in the writeup that lead to this app, check out my other post dedicated to that: https://david.coffee/react-native-wasm-golang/

Honestly I had a lot of fun building this - I learned a lot about WASM and the inner-workings of React Native. It’s been a very long time since I initially used it (fun fact: I contributed to React Native in 2016), and it made me more motivated to work on smaller app ideas.

App logo, generated by Dall E

I suck in designing things. I had a rough idea what I wanted to have, something along the lines of a email envelope, but with glasses. I could have probably copied together a couple of SVGs, but remembered that I had beta access to Dall E with some credits left so I gave using it a shot. Here are the results of my first prompt to Dall E:

I couldn’t be happier. I was immediately unblocked and had something decent looking. The image needed a few adjustments in Pixelmator to be fully usable, but I’m very happy with the result

Get the app on the AppStore and let me know what you think

I recently had the idea to create a little react native app for FastMails MaskedEmails, to quickly add new ones on my device, so I don’t have to open the FastMail app all the time.

I wanted to do this with as little code duplication as possible, so instead of rewriting a client from scratch, I decided to re-use my maskedemail-cli somehow. I could have probably compiled it down to a shared library and pulled it into Swift, but I am no iOS developer and know React already, so why not do it in React Native with WASM?

I’ve never actually used wasm and was looking for an excuse to play around with it. Here’s what I learned.

You can find the app I worked on during this writeup at https://maskedemailmanager.david.coffee

(Spoiler: You can’t use native WASM within JSC on React Native yet)

Golang and WASM, how does it actually work?

Golang supports compiling to wasm since version 1.11, with further improvements happening in 1.13. The basic gist of making it spit out wasm is

GOOS=js GOARCH=wasm go build -o main.wasm

Main interactions from Golang with JavaScript-land happen through the js/syscall package. For example, if we want to create a js function, we can do it like so:

cb = js.FuncOf(func(this js.Value, args []js.Value) any {
	fmt.Println("Hello World")
	return nil
})

The first argument is always this, and the second argument is always an array of actual arguments passed to the function.

We can also use js.Global() to get a ref to the global object and call into js:

js.Global().Call("alert", "Hello from wasm!")

… and of course, we can also make our functions available on the global object:

js.Global().Set("myCoolFunc", cb)

Using Golang WASM from JavaScript

Golang comes with some glue code that you need to run wasm files built by Golang properly, that’s this copy statement you see in most guides:

cp "$(go env GOROOT)/misc/wasm/wasm_exec.js" .

Besides setting up the environment, this code defines the globalThis.Go object that we’ll use to run our wasm code:

const go = new Go();
const res = await WebAssembly.instantiateStreaming(
  fetch("main.wasm"),
  go.importObject,
);
go.run(res.instance);

(instantiateStreaming supports fetch; that’s why everything is so compact. we can also use WebAssembly.instantiate(arrayBuffer, go.importObject) directly without fetch).

go.importObject provides values to be imported into the new instance, which defines all the supported syscalls

Preparing maskedemail-cli for WASM - async code and Promises

First, we need to create glue code that exports the necessary functions from maskedemail-cli to JavaScript, in Golang.

func main() {
	done := make(chan struct{}, 0)
	js.Global().Set("maskedemailList", js.FuncOf(list))
	<-done
}

func list(this js.Value, args []js.Value) interface{} {
    // (...) get maskedemails here (...)
    return maskedEmailArray
}

Now here’s a problem: We can’t block the main goroutine with things like HTTP requests. It would be kinda weird if we do an ajax request in Golang and the browser freezes while our functions get executed…

Instead, async code has to go into separate goroutines, which means in JavaScript… Promises!!

Specifically, to make our code compatible with js promises from Golang, we have to use the Promise constructor through js.Global(), effectively creating a Promise from scratch:

func list(this js.Value, args []js.Value) interface{} {
    handler := js.FuncOf(func(this js.Value, args []js.Value) interface{} {
        resolve := args[0]
    	reject := args[1]
    	go func() {
    	    // (...) get maskedemails here (...)
    	    resolve.Invoke("Hi there")
    	}
    })
    promise := js.Global().Get("Promise")
    return promise.New(handler)
}

js.Global().Get("Promise") is getting the global Promise object, then calling New onto it with a custom handler.

Running main() from above would now populate globalThis.list, and it’s signature would look like this: list(): Promise<string>.

Creating a NPM package that’s using WASM

Next, I wanted to publish an NPM package to pack away the complexity, but most guides on Golang + wasm were mainly focused on the browser, not NodeJS.

Luckily the key steps are effectively the same:

1. Use the wasm_exec.js glue script to populate globalThis.Go
2. Load the wasm file through whatever method
3. Run WebAssembly.instantiate or WebAssembly.instantiateStreaming, pass go.importObject
4. Run go.run(instance)

In NodeJS, we can solve 1. by doing a dirty require("./wasm_exec.js"). The code runs fine in the nodejs runtime so no issues there. Too bad it’s polluting globalThis and doesn’t give us a nice-to-use package back, but we can live with that.

Point 2. is a tad trickier - we could of course, just load it with fs.readFileSync:

const go = new globalThis.Go();
const buffer = fs.readFileSync("./main.wasm");
const inst = await WebAssembly.instantiate(buffer, go.importObject);
go.run(inst.instance);
// run func
globalThis.list();

… but given that this is JavaScript, usage of fs means that this is already unusable in a browser environment, which is not great.

What’s more, wasm_exec.js depends on a few browser-specific modules, mainly crypto and TextEncoder. So now we have some stuff that doesn’t work in the browser, and also some stuff that doesn’t work within node 😀 Let’s fix that with some polyfills:

if (!globalThis.crypto) {
  const crypto = require("crypto");
  globalThis.crypto = crypto;
}

if (!globalThis.TextEncoder) {
  const te = require("text-encoding-polyfill");
  globalThis.TextEncoder = te.TextEncoder;
  globalThis.TextDecoder = te.TextDecoder;
}

const go = new globalThis.Go();
const response = await fetch("./main.wasm");
const buffer = await response.arrayBuffer();
const module = await WebAssembly.compile(buffer);
const instance = await WebAssembly.instantiate(module);
go.run(inst.instance);
// run func
globalThis.list();

Instead of fs, we can use node-fetch (or the new NodeJS native one) and fetch to pull the wasm file, then continue instantiating like usual. Now it’s no longer node/browser or fetches implementation dependent.

My final solution

Because I wanted to package this for ESM and CJS simultaneously, I went with rollup as bundler. rollup-plugin-wasm can handle loading + instantiating of the actual wasm file similar to how we did above, so we can slim the code down like this:

// polyfills here
require("./wasm_exec.js");
import wasmMain from "./static/main.wasm";

const go = new globalThis.Go();
const wasmObj = await wasmMain(go.importObject);
go.run(wasmObj.instance);
globalThis.list();

Packaging into a usable module

To make everything a tad nicer, let’s export some functions that other modules can use:

interface Mod {
  list: (token: string, accountId: string) => Promise<MaskedEmail[]>;
}

const instantiatedPromise = new Promise<Mod>(async (resolve, reject) => {
  const go = new (globalThis as any).Go();
  const wasmObj = await wasmMain(go.importObject);
  go.run(wasmObj.instance);

  resolve({
    list: (globalThis as any)["maskedemailList"],
  });
});

export const list = async (token: string, accountId: string) => {
  const mod = await instantiatedPromise;
  return mod.list(token, accountId);
};

We have some waiting to do before wasm is usable, so we package all of that into a new instantiatedPromise. The exported list() is then awaiting that promise before doing anything, then executing mod.list, our actual wasm function.

To make the API nicer to use I’ve added typescript type definitions for the module, that’s what the interface Mod is for. instantiatedPromise will resolve with Mod, so calling mod.list will have the same return signature as Mod.list

The package is available on npm, the code on my github

Investigating React Native + WASM

By now, we have packaged everything so neatly that it doesn’t matter that we used Golang to create the wasm binary. It’s all just an npm package that should be usable everywhere. (Ignore the big 7MB file size of the .wasm file for now 😛)

So all we have to do is yarn add maskedemail and that should be it… right? Right??

But life is not that simple, and adding that final lego piece was much harder than expected. Let’s go over what is making wasm + react native so hard?? (This was written by poking the simulator and debugging errors)

• WebAssembly.instantiateStreaming / WebAssembly.compileStreaming only works with fetch if the server is returning application/wasm, which react-native doesn’t. In fact, using fetch to pull in resources the same way we do in NodeJS or the Browser is in RN flimsy at best
  • Expo has Asset.fromModule to make it possible to load arbitrary assets from disk
• WebAssembly.instantiateStreaming / WebAssembly.compileStreaming doesn’t even exist in the react native JavaScriptCore runtime (WebAssembly exists, but those functions don’t. Only WebAssembly.instantiate and WebAssembly.compile do, which work with ArrayBuffers
• Current asset loading functionality through Asset.fromModule or fetch implementations for react native can pull assets, but rn’s FileReader does not implement readAsArrayBuffer, so no fetch().then((res) => res.arrayBuffer()). No ArrayBuffer means no easy way to use WebAssembly.instantiate, and also no easy trickery to convert Blob into an ArrayBuffer
• RN’s JavaScriptCore does not have a crypto implementation
• Expos packager is trying to resolve all require and build and doesn’t support dynamic imports, so having require("crypto") in an if statement (like a env-dependent polyfill) will still make the packager throw an error that the package doesn’t exist, effectively meaning that our generic solution ain’t gonna work

I probably forgot a few points but these were the most noticeable issues.

I banged my head against FileReader.readAsArrayBuffer to make it possible to read the .wasm file from disk, tried a couple polyfills and re-implementations, but eventually gave up. There are fs.readFile implementations that could work, but expo (the tool I am using to bootstrap my RN application) does not support linking libraries, so we can’t easily add things that require editing of the application bundle.

To make things worse, the package resolving the issue I mentioned above also meant that all the glue code that rollup-plugin-wasm generated to switch based on environment for browser + nodejs doesn’t work either, because if there is a call to require("fs"), expo breaks. Same thing for attempts to polyfill crypto in the correct environment with crypto-browserify.

Fixing RN issues one by one

Looking at wasm_exec.js, we can see that it doesn’t need the entire crypto package, just one single function:

if (!globalThis.crypto) {
  throw new Error(
    "globalThis.crypto is not available, polyfill required (crypto.getRandomValues only)",
  );
}

That’s much easier to polyfill with something like polyfill-crypto.getrandomvalues (though using Math.random() instead of proper entropy source…)

We can swap out our existing polyfill for this implementation:

import getRandomValues from "polyfill-crypto.getrandomvalues";
if (!globalThis.crypto) {
  globalThis.crypto = {
    getRandomValues,
  };
}

Now the import no longer breaks nodejs, browser, or react-native JavaScriptCore :)

Loading the .wasm file is a different beast, and the polyfills for FileReader.readAsArrayBuffer caused expo to crash consistently. So, what do we do if we can’t load something from disk in the format we need? We play dirty: Instead of async loading the .wasm file like a good citizen, we base64 that chunker and stuff it into our main JavaScript file!

rollup-plugin-wasm even has an option to inline wasm:

wasm({
  sync: ["main.wasm"],
  targetEnv: "auto-inline",
  maxFileSize: 0,
}),

The result will make any editor without proper optimizations for uber-large files crawl

… but it does work inside the Simulator! We’re no longer dependent on asset loading logic anymore. It’s all just js now.

To fix remaining FileLoader issues, there is this patch available that polyfills the functionality. This patch alone doesn’t work to load the wasm file from disk and crashes the simulator, but it does work for smaller under-the-hood functionality that relies on readAsArrayBuffer.

With that out of the way, my app finally runs in the Simulator 🎉

I thought this would be the end of the story and continued chipping away on my app. After days of work, it was finally ready to run it on my iPhone to do a final test before release. Compiling everything… running…

You gotta be kidding. WebAssembly does not exist at all in JavaScriptCore when running on an actual device, but it does exist inside the Simulator. Apparently, Apple shut it down in recent iOS versions, but from reading through posts, this used to be available in the past.

… the end? Did we fail? What next?

I’ve been considering what next steps to take to get this working, but without proper WebAssembly support in JSC on devices, performance will always be subpar. Here are some thoughts and directions I investigated:

Option 1: Fallback to WKWebView

This is the most promising but also a bit absurd. We create a WebView, inject our wasm binary into the webview and execute it there, then relay messages (with window.postMessage) back and forth with our main app.

In my tests, this actually worked pretty well, with the problem that making something generic that just works is a bit hard. Actually, polyfilling window.WebAssembly would mean we need to somehow bridge go.importObject into our webview. On top of that, we need to proxy the changes the wasm binary did (like modifying globalThis) back to our native app using a transparent message relay, so libraries that rely on it can just work.

Without a proper polyfill, window.WebAssembly won’t be available in global scope, so our maskedemail NPM package won’t be able to use it.

All in all, a lot of headache. Probably better to split native and bridged code altogether and write the glue code from scratch. (react-native-react-bridge tries to get around this by using a separate entrypoint that’s getting processed with babel)

Option 2: wasm2js

binaryen includes a tool called wasm2js that compiles wasm files back to JavaScript. I have tried this as well, but could not get it running with my library, probably due to complexity with some stuff like reflections and HTTP requests. Very promising but very hard to debug due to the way this tool spits out JavaScript. Either it works, or it doesn’t; for me, it didn’t. (Though the errors I got were actual Go errors related to HTTP, so it almost worked)

Option 3: gopherjs

Not a solution to wasm, but to make Go run within react-native. Instead of building a wasm binary, gopherjs can output functional JavaScript so we don’t have to deal with WebAssembly runtime at all.

Honestly, I think this might be the best option for code reusability until WebAssembly support is fully available in JSC.

What I ended up doing: WASM -> GopherJS

I’ve tried all options listed above, and the GopherJS variant was the nicest to work with. Even better: The Go API for GopherJS is almost identical to WASM! For example:

This is the WASM version:

func list(this js.Value, args []js.Value) interface{} {
    handler := js.FuncOf(func(this js.Value, args []js.Value) interface{} {
        resolve := args[0]
    	reject := args[1]
    	go func() {
    	    // (...) get maskedemails here (...)
    	    resolve.Invoke("Hi there")
    	}
    })

    promiseConstructor := js.Global().Get("Promise")
    return promiseConstructor.New(handler)
}

This is the gopherjs version:

func list(this *js.Object, args []*js.Object) interface{} {
	handler := js.MakeFunc(func(this *js.Object, args []*js.Object) interface{} {
		resolve := args[0]
		reject := args[1]
		go func() {
        	// (...) get maskedemails here (...)
			resolve.Invoke(out)
		}()

		return nil
	})

	promiseConstructor := js.Global.Get("Promise")
	return promiseConstructor.New(handler)

In fact, it’s so similar that I was able to flat-out generate the gopherjs main.go file with this script:

.PHONY: generate-gopherjs
generate-gopherjs:
	rm -rf cmd/gopherjs
	mkdir -p cmd/gopherjs
	cp main.go cmd/gopherjs/
	sed -i '' 's/syscall\/js/github\.com\/gopherjs\/gopherjs\/js/g' cmd/gopherjs/main.go
	sed -i '' 's/FuncOf/MakeFunc/g' cmd/gopherjs/main.go
	sed -i '' 's/js\.Value/\*js\.Object/g' cmd/gopherjs/main.go
	sed -i '' 's/js\.Global()/js\.Global/g' cmd/gopherjs/main.go

With a bit of JavaScript, I was able to add an automatic fallback to the gopherjs version to maskedemail-js, so it’s now usable even when window.WebAssembly isn’t available (although ships with 2x 8MB files).

No WASM in react-native as of now

A bit sad I wasn’t able to get my goal of using real WASM working within React Native, but learned a good bit with this adventure.

For now, I’ll use GopherJS when I want to re-use Go code, but I hope we’ll see actual WebAssembly support in JSC on devices soon-ish. With all the WASM hype, I doubt it’ll take long.

There is also stuff like Wasmer that can embed WASM binaries into other languages like Swift, so instead of going the JSC route, it should be possible to embed it into native modules directly or non-RN apps - for when we really want to re-use the same binary at different places. But that’s also doable with Golang, without WASM.

The app is available at https://maskedemailmanager.david.coffee

I’m a big fan of one-time-use emails (as you can probably tell by previous posts on this blog). However, one thing that iCloud’s Hide My Email is still unbeatable at compared to my own domain is full anonymity. When I sign up for a junk site to get free WiFi, or order at a restaurant, I don’t even want to expose my domain to them, because that domain is, well, registered to my real identity.

Hide My Email on the other hand is fully anonymous: Emails come from “@icloud.com”, no name in the headers, nothing. Just a fully anonymous blank email. Great! Services can’t even filter for it (compared to mozillas solution, or duckduckgo), because filtering would mean shutting access to everything coming from icloud.com.

Here’s one thing I wanted to get working for a while: Instead of having <site or person> -> one-time-use Email (iCloud) -> Me, I want it the other way around: A one-time-use email that allows me to anonymously send to a specific recipient whenever I send something to that email. Turns out you can actually do that in Apple’s Hide My Email! Let’s take a look how

How Hide My Email actually works

Hide My Email is acting as a relay in between the user account and the sender. You don’t actually send emails from that randomly generated @icloud.com address. In fact, you can’t. You can’t authenticate with that email on the iCloud SMTP server because that email is not yours.

Here’s what I mean: Let’s compose a new email and use Hide My Email

When I click “Send”, the email arrives successfully with the hidden iCloud email as sender, as expected:

… but when we inspect the actually sent email, it looks like this:

Do you see it? The email never got sent from “heave_balks_0g@icloud.com”, instead what Mail did is, it inserted a relay address as the actual receiver of the email. That’s the long test_at_davemail_io_5rvm…@icloud.com you see there. Just that Mail is doing this so transparently that you’ll likely never even notice what is happening.

Relaying back and forth

So now that we know this relay address, what happens when we send an email directly to it? Let’s try!

Sending, and hey look at this! Doing it manually gives us the same result! (duh)

This means that test_at_davemail_io_5rvrmv74x77007_62a85f4e@icloud.com is the relay address that iCloud mapped to test@davemail.io - Everything I send to it will always arrive at test@davemail.io, kind of like a mini portal that shovels content over to a specific sender.

The other end will always see heave_balks_0g@icloud.com, the other part of the relay, as sender. The final relay looks like this:

• heave_balks_0g@icloud.com: Relay to my email
• test_at_davemail_io_5rvrmv74x77007_62a85f4e@icloud.com: Relay to the recipient

Hide My Email is unique per recipient, but not fully random

Another question you might have while following this is: Wait a moment, aren’t Hide My Email addresses random? Well, kind of.

Each time you generate a hide-my-email-email (nice word), it will be random, but only per recipient: When I hide my email from plans@tripit.com, it will always be the same @icloud.com email, no matter how often I generate it. You can’t use iCloud to spam people, they’ll see everything coming from the same person.

Cool, but what can we actually do with this information??

Obviously this post was leading up to something, otherwise I would have not put so much time into writing it.

Now that we know that relay addresses don’t change, we can use this knowledge to automate sending completely anonymous emails to specific senders without exposing our main domain, name or any other information.

Even better: Using the relay even shields us from any mistakes, like accidentally leaking personal information in headers. It happened dozen times that I used “forward email” functionality in email clients to forward an email to a service, just to realize that the email service sent that email with my global account email, instead of the one-time-use email. (How would the service even know what email I want to use for forwarding?)

Let’s take a service like TripIt which gives me an email that I can send flight itineraries to: plans@tripit.com. I can link an email to it like “tripit.com@mydomain.com”, then whenever I send an email from this email to plans@tripit.com, it will add that itinerary to my account. This works great, but means I can’t really setup any quick-actions, shortcuts or automations to quickly forward to it, because of the issue I described above. Now we can fix that by directly using the relay address :)

1. Send an email to plans@tripit.com using Hide My Email
2. Copy the generated @icloud.com address and add it to TripIt
3. Copy the generated relay address (the long one aka test_at_davemail_io_5rvrmv74x77007_62a85f4e@icloud.com), create a new contact with it for quick access
4. Profit.

Now we can hit “forward” (or use a Siri Shortcut) to forward any PDF attachment we want to add to tripit to our relay address, iCloud will do the rest.

All of this effort just so we don’t expose our main email, huh 😉

Keep on shaving that yak

I recently ventured into IPv6 territory with one goal: Get rid of the router my ISP gave me.

I am a fan of UniFi gear and own an UniFi Dream Machine (upgraded from a UniFi Secure Gateway last year) which is more than capable of handling IPv6, so how hard can it be?

I went into this not knowing a thing about IPv6 and just assumed it was identical to IPv4, just bigger, but boy was I wrong. This is my writeup of what I learned and what my endresult is (spoiler: I got it working)

IPv6 misconceptions

As I explained above, I just assumed that IPv6 was IPv4, but longer. So you have some gateway/subnet-mask combo, your router gets an IP assigned from the ISP and all devices inside the network use that IP outwards facing when interacting with things (= “NAT”). Internally, the router assigns IPv4 addresses based on the gateway/subnet-mask, with each device receiving a unique IPv4 address from the router through DHCP. So far so good.

IPv6 on the other hand does not work at all like this. In IPv6-land, each device receives unique (actually multiple) IPv6 addresses. There is no NAT, so when my iPhone that’s connected to my WiFi communicates with a website, that website will see the unique IPv6 address my iPhone has assigned, not my router’s IP. There are also (unless specifically setup) no local network addresses - each IPv6 address is global. And how can the router assign IPv6 addresses? Enter prefix delegation or IPv6-PD.

The router, after authenticating with the ISP when PD is enabled, receives an entire address space (like /64 or /48) delegated to it. In simple words, the ISP tells the router: “Here’s a chunk of addresses that you can use for your devices, handle it.”

Not all IPv6 addresses are created equal

Let’s take a look at this ifconfig output of my Mac:

en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        options=50b<RXCSUM,TXCSUM,VLAN_HWTAGGING,AV,CHANNEL_IO>
        ether 4c:20:b8:e7:3b:be
        inet6 fe80::1449:6fa5:65a9:a313%en0 prefixlen 64 secured scopeid 0x6
        inet6 fd74:d39f:a55f::8c prefixlen 64 dynamic
        inet 192.168.1.2 netmask 0xffffff00 broadcast 192.168.1.255
        inet6 240f:78:<snip> prefixlen 64 dynamic
        inet6 240f:78:<snip> prefixlen 64 autoconf secured
        inet6 240f:78:<snip> prefixlen 64 autoconf temporary

It doesn’t just have 1 IPv6 address, no that would be far too simple. Instead, it has 5!! Why? Because apparently, we have so many IPv6 addresses now that we don’t need to be sparse with them /s.

There are different kinds of IPv6 addresses assigned to my mac:

inet6 240f:78:<snip> prefixlen 64 dynamic
inet6 240f:78:<snip> prefixlen 64 autoconf secured
inet6 240f:78:<snip> prefixlen 64 autoconf temporary

The secured address is my permanent IPv6 address. This address rarely ever changes and will stick around even after reboots.

The temporary address is, you guessed it, a temporary address.

Lastly we have 2 more:

inet6 fe80::1449:6fa5:65a9:a313%en0 prefixlen 64 secured scopeid 0x6
inet6 fd74:d39f:a55f::8c prefixlen 64 dynamic

fe80::, while not obvious whatsoever, is a prefix for localhost or local link. This is an address that’s only usable locally.

fd74::, more specifically the fd:: prefix is designated for the local network. Didn’t I just earlier say that there are no local network addresses in IPv6? By default, yes, but I specifically setup my router to hand out local network addresses. Keyword here is “Unique local address” (ULA).

IPv6 implementations - Native, 6to4, IPv6oE

Are you still with me?? Okay, because there is more. Not all IPv6 implementations are equal either. Very common implementations are either IPv6 “native” or “IPv6-in-IPv4 tunnel”. There are more exotic implementations like SoftBanks “IPv6高速ハイブリッド IPv6 IPoE + IPv4” that’s utilizing IPoE for authentication and assigning addresses. (Note here: I used SoftBank in the past and was not able to get IPv6 without their router working due to the proprietary way their router authenticates)

I’m not going into detail here and don’t fully understand everything either, but a 6to4 tunnel is like the name implies an earlier implementation that tries to build the IPv6 stack on top of a IPv4 network stack. Read more about it here.

“Native” IPv6 is the one that’s nice to work with, but also the one not every ISP supports. I made sure to pick an ISP that has a native IPv6 stack and doesn’t use IPv6-over-IPv4

Getting started - figuring out what my ISP router is doing

The first step to getting IPv6 working was to actually figure out what my ISP router is doing when I plug it in. Specifically: How does it actually get the IPv6 prefix assigned, how does it authenticate, and what does the handhshake look like.

Now how do we actually do this? Wireshark and a couple of USB-C network dongles to the rescue!

Mac has a lot of nice networking features, one of them is the ability to bridge 2 network interfaces, effectively creating a tunnel between them. This can be done through the CLI, but also through the network preferences pane after plugging multiple network dongles into the Mac.

Specifically, we want to make it so the MacBook is in between the ISP Router and the network out and acting as a transparent bridge. So Internet <-> MacBook <-> ISP Router

Once the virtual bridge is created, we can use Wireshark to monitor the new device for any traffic that happens between the ISP router and the mothership. Now start Wireshark, plug the ISP router in, and….

… it works!

Lot’s of new information here for a non-network engineer like me, but also some familiar words: DHCPv6, ICMPv6, UDP, SSL. The *v6 protocols are IPv6 variants of the same things we have in IPv4, but beefed up and more important in IPv6 land.

Specifically, ICMPv6 is much more important than the IPv4 version that we use for pinging stuff and is used extensively for the Neighbor Discovery Protocol, that’s those “Router Solicitation” and “Router Advertisement” messages you see in the screenshot above.

So from this startup dump we can see:

• My ISP router inquires with ISP uplink through Router Solicitation where the uplink router is
• Uplink responds with Router Advertisement
• ISP router advertises itself through DHCPv6, uses a DUID (“DHCP unique identifier”) to identify itself, this DUID is generated off the mac address
• ISP responds to DHCPv6 request and assigns IPv6 prefix
• Neighbor Discovery Protocol continues with Neighbor Solicitation and Neighbor Advertisement

(the XID stuff you see in there is a transaction id that’s being calculated on each request)

So to summarize: This ISP is using fully native IPv6, the IPv6 prefix space is assigned through DHCPv6, and authentication happens through a DUID identifier that’s being used in all DHCPv6 requests.

If we are able to replicate this handshake on the UDM, we should have IPv6 working. Now how do we actually do that…

Teaching the UDM how to DHCPv6

From the wireshark dump above, we know that the ISP router is using DHCPv6 to authenticate and get its prefix assigned. We also know which DUID the ISP router uses for identifying itself, and we know the prefix size that we get assigned from the ISP (/48).

Let’s enable IPv6 through DHCPv6 through the UniFi admin on WAN:

Then grep for dhcp to see what’s happening on the UDM:

# ps aux | grep dhcp
 3537 root     /sbin/udhcpc --foreground --interface eth4 --script /usr/share/ubios-udapi-server/ubios-udhcpc-script -r 124.214.62.19
 3895 root     /usr/sbin/odhcp6c -R -f -e -v -s /usr/share/ubios-udapi-server/ubios-odhcp6c-script -P 48 eth4
 7715 root     grep dhcp

The UDM is spinning up udhcpc and odhcp6c with the closed-source binaries ubios-odhcp6c-script, both on the eth4 interface (my WAN port).

We can also print our own duid by running cat /config/dnsmasq.lease, and since the DUID being used is not the one that our ISP router uses, we have to do something about it.

odhcp6c that is being used here is is a DHCPv6 client by openwrt and it being open-source and widely used means we can dig deeper into what it’s doing with all those parameters:

Usage: odhcp6c [options] <interface>

Feature options:
        -S <time>       Wait at least <time> sec for a DHCP-server (0)
        -N <mode>       Mode for requesting addresses [try|force|none]
        -P <length>     Request IPv6-Prefix (0 = auto)
        -F              Force IPv6-Prefix
        -V <class>      Set vendor-class option (base-16 encoded)
        -u <user-class> Set user-class option string
        -c <clientid>   Override client-ID (base-16 encoded 16-bit type + value)
        -i <iface-id>   Use a custom interface identifier for RA handling
        -r <options>    Options to be requested (comma-separated)
        -R              Do not request any options except those specified with -r
        -s <script>     Status update script (/usr/sbin/odhcp6c-update)
        -a              Don't send Accept Reconfigure option
        -f              Don't send Client FQDN option
        -k              Don't send a RELEASE when stopping
        -t <seconds>    Maximum timeout for DHCPv6-SOLICIT (120)
        -m <seconds>    Minimum time between accepting updates (30)

The flag we’re after is -c <clientid> Override client-ID (base-16 encoded 16-bit type + value) . So theoretically, if we are able to pass -c into the startup of the odhcp6c command, we should be able to receive a valid IPv6 address from the ISP. Let’s give it a try!

Disable DHCPv6 again through the admin UI, wait until the process fully stopped, then manually run the command above, but with -c flag:

/usr/sbin/odhcp6c -R -f -c <DUID from our wireshark dump>  -e -v -s /usr/share/ubios-udapi-server/ubios-odhcp6c-script -P 48 eth4

Wait a bit, then ip -6

# ip -6 addr show scope global
10: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fd74:d39f:a55f::1/64 scope global
       valid_lft forever preferred_lft forever
    inet6 240f:78:<snip>/64 scope global dynamic
       valid_lft 108367sec preferred_lft 108367sec

Hello there, IPv6 :)

Back on the UDM admin, let’s enable IPv6 on the LAN network. We want to do prefix delegation and also advertise ourselves as a router (remember Neighbor Discovery Protocol above?)

Wait a bit, then let’s enable IPv6 on my mac and renew the DHCPv6 lease:

en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        inet6 240f:78:9464::<snip> prefixlen 64 dynamic
        inet6 240f:78:9464:0:<snip> prefixlen 64 autoconf secured
        inet6 240f:78:9464:0:<snip> prefixlen 64 autoconf temporary

Nice!!

And what about https://test-ipv6.com?

Persisting changes to the UDM

Now that we got an IPv6 address, let’s persist those changes so we don’t have to manually run odhcpv6 every time.

Pre-req: Extending the UDM with boot-scripts

UniFi gear is hackable which allows us to do more than just what’s possible through the admin UI. The USG had a very advanced customization flow baked in, but the UDM is a tad more annoying to work with.

Luckily there is this awesome project called udm-utilities by boostchicken, a “A collection of things to enhance the capabilities of your Unifi Dream Machine or Dream Machine Pro.”

Basically what this project does is, it installs a custom package that extends the UDM with the ability to run scripts on boot that persist in between reboots. We put our shell scripts into /mnt/data/on_boot.d, and udm-utilities makes sure those are run when the router boots.

There are a lot of really cool things you can do with it, from configuring the UDM to even running podman containers like homebridge or adguard, but we’ll be focusing on the network aspects only for this writeup.

Changing DUID on boot

Now the next problem is, how do we actually start odhcp6c with our own options? The UDM launches it on boot through ubios-udapi-server, which is closed-source. We could decompile the binary with ghidra and see if we can hack our own arguments in, how hard can it be?

But there is an easier (more hacky) way of achieving just that. What if the binary at /usr/sbin/odhcp6c isn’t actually a binary, but an executable script that runs odhcp6c for us? The UDM will continue to execute odhcp6c like usual and we can inject our custom flags. Here’s what I mean:

# cat /usr/sbin/odhcp6c
#!/bin/sh
# workaround script to inject odhcp6 options
/usr/sbin/odhcp6c-real -R -f -c 0003000<snip> "$@"

The custom script that we put in /usr/sbin is calling odhcp6c transparently and adds the extra -c flag.

Using udm-utilities, we can automate this with the following boot script:

#!/bin/sh
mv /usr/sbin/odhcp6c /usr/sbin/odhcp6c-real
cp /mnt/data/odhcp6c/odhcp6c /usr/sbin/odhcp6c
chmod +x /usr/sbin/odhcp6c
killall odhcp6c

1. We move odhcp6c to odhcp6c-real
2. Copy the custom odhcp6c to the old location
3. Make it executable
4. Kill odhcp6c

Once killed, ubios-udapi-server will automatically attempt to restart it by executing /usr/sbin/odhcp6c, which is no longer the real binary but our custom script.

And does it work?

# ps aux | grep dhcp
 3891 root     {odhcp6c} /bin/sh /usr/sbin/odhcp6c -e -v -s /usr/share/ubios-udapi-server/ubios-odhcp6c-script -P 64 eth4
 3895 root     /usr/sbin/odhcp6c-real -R -f -c 0003000<snip> -e -v -s /usr/share/ubios-udapi-server/ubios-odhcp6c-script -P 64 eth4

It absolutely does!

Was it worth it?

Worth it in terms of performance or network benefit? Probably not. The world doesn’t feel different when interacting with IPv6 and speed for me is pretty much on par as it was with IPv4.

But it has been a very nice learning experience. Without this venture into IPv6-land I would have never learned about the differences between IPv6 and IPv4.

My network is now fully IPv6 compatible which it wasn’t before, and that’s an achievement in itself.

A little backstory - Like most people I didn’t really take hearing protection serious until around my mid-twenties, when I realized that the faint high-pitched sound I hear when it’s very silent isn’t just my blood or body noises, but actual tinnitus.

Luckily it’s so silent that I never actually notice it unless I actively try to focus on the sound. I also started doing regular hearing checks and hey - it turns out my hearing is still far better than people way younger my age. (I’m proud to still be able to hear that high-pitched beeping in front of department stores and get annoyed by it)

But It was still an eye-opener.

A couple years back when I attended a music festival, I started learning about attenuating earplugs - or earplugs that don’t just seal the ear, but contain a small filter that attenuates (fancy word for lessens) the amount of sound that goes through them. I got a pair of blox ears-plugs from a booth that was available and have been using them religiously whenever I go to a club or festival ever since. The nice thing of attenuation over blocking is that you can still hear and hold conversations normally, for example.

These days I DJ a lot in clubs and am surrounded by loud sound more than I’d wanted to be. I usually keep my earplugs in at all time up until the moment I actually play, mainly because they still slightly distort the sound, but hey what do you expect from $30-50 plugs. I’m now taking music and DJing more serious, so after years of use I’ve decided I finally wanted something nicer.

The world of custom-molded audio things

My world expanded after I went down the audiophile rabbit-hole. Turns out you can custom mold a bunch of things - from custom in-ear monitors, hearing aids, but also earplugs!

There are a bunch of places that do custom orders for earplugs, from companies to music stores, to even just getting them at the local otolaryngologist made.

I wanted something that was specifically made for music, so something that protects my ears while keeping distortion to a minimum. Optimally something that artists use on stage (though that’s mostly IEMs). I singled out 2 companies that are known in this field, ACS and sensaphonics, but ACS requires you to send in a scan of our ear-canal which I obviously didn’t have yet.

Sensaphonics has a stronger presence in Japan full with Japanese website so that’s what I ended up picking. Even better, their musicians earplugs support swappable filters in -9db, -15db, and -25db steps which make them very versatile. There are also graphs available that show how these earplugs specifically affect the sound in different frequencies so they were arguably the best option scientifically as well.

Getting a ear-canal impression done

Contact and order was very painless through their product page. They sent through a questionnaire by email, confirmed the order details and asked me to pick a location where I’d like to do my ear-canal impression. Sensaphonics doesn’t do this themselves, but delegate it to local medical stores that have the necessary equipment for making hearing-aids.

The impression is done by filling the ear with a playdoh-like substance that hardens after a few minutes.

Besides the odd feeling of having your ear completely 100% sealed, this is painless.

The impression is then sent to a lab where it’s scanned and digitized, then sent to the manufacturer to be used for the mold. I asked if I could have access to the scan data, but Sensaphonics made it clear that they wouldn’t do that

And that was it. All that was left was a 3 weeks wait.

My custom molded attenuating earplugs

3 weeks later and we finally arrived at the title of this blog post - my earplugs arrived!

The earplugs feel well made. For some reason didn’t expect them to be so squishy, but it makes sense because otherwise how would they seal the ear without damaging it? They go all the way in and fit perfectly. Not uncomfortable, not hard to put in, it just fits.

I went with the -15db filter for my club and DJ adventures but am considering getting the -9db filter as well for more casual events. I am a little worried about the swappable filter thing, and eventual wear around the insertion point. Swapping them too often will probably cause it to get loose and allow more sound go through the cracks.

But how do they actually sound?? And how do these $200+ custom molded ones compare to the <$50 blox I used until now??

Well they sound… normal, duh. Everything sounds the same as it always does, just a bit quieter. I trialed them at a very loud club and sometimes even forgot I had them in. Post-club, no ear ringing so I call that a success!

Compared to the blox, a couple things - first of all they are different db levels. My custom ones are -15db where the blox were -19db. On the blox, things sound more muffled. Not bad, just more muffled and quiet. They have a stronger ‘earplug feeling’, and putting them in is harder.

So, should you go out and get custom ones made as well?

The answer is probably ’no’, but you absolutely should get attenuating earplugs if you are exposed to louder sounds semi-regularly.

Hearing loss is permanent (for now). There is currently no cure for tinnitus and bringing back what has been destroyed, so taking care of your ears is very important. Don’t act reactive when you started noticing loss, but proactive and try to prevent it.

If you have the cash and want the best of the best, then yes, get custom molded ones. They are definitely nice and I like mine a lot.

Ran into this little app called Anybox the other day while I was hunting for bookmark managers to try and ended up liking this one a lot

How I was doing things until now

Up until now, my way to manage bookmarks was to, well…, just bookmark them in my browser (duh), then use Alfred Workflows like this one (or more recently, just Raycast) to give me a dialog that can fuzzy complete bookmarks by name whenever I hit CMD-CTRL-B.

This started to feel a little insufficient when I switched from Software Engineer to Engineering Manager. All of a sudden I had so many links to things that I had to somehow keep track of - meeting documents, 1on1s, project overviews, Wiki pages with information on how to handle situations like employee resignation, and and and. The browser autocomplete for bookmarks worked, but I still often lost pages and had to re-find them in an ocean of slack messages, especially when the browser history got cleared.

I knew of raindrop.io and while it looks super sleek, it never clicked and I felt like I was trying to force myself to use it, without having a problem for it to solve. Instead of putting bookmarks into my browser, I was putting them into raindrop which just felt slower in every way.

What I wanted was what I already had: Some way to rapidly bookmark things, then get a Quick Open dialogue like in Sublime Text or Obsidian to fuzzy-find stuff. I also wanted it to be browser independent, so I could stop re-importing all my bookmarks every time I switch main browsers. Basically, I wanted a bookmark manager that is built for busy people that are used to features of IDEs.

Turns out that’s what Anybox is!

Anybox

Anybox is a mix between bookmark manager and ‘store everything’ app like DEVONthink (though far less advanced). It’s an app to, well, clip and manage bookmarks (duh).

Anybox is an invisible app to me. I barely ever interact with it through it’s main window, but what I do all the time is hit CMD-CTRL-B to conjure a textbox in the middle of my screen that lets me fuzzy-filter over all my bookmarks

It syncs through iCloud, it can quickly open bookmarks, it allows me to clip from any browser by right-clicking on something, and as a bonus: It has an iOS app that allows me to do the same! Neat!

Of course Anybox can do a lot more. It has support for other file-types like images (well the name is ‘Anybox’ and not ‘BookmarkBox’) and can even automatically archive bookmarks as PDF, Screen Shot and even Web Archive if you need that.

Everything in Anybox is built with an engineers mindset: CMD-P opens a quick switcher to jump around the app, CMD-K opens a Sublime-like command palette to do all the actions through a keyboard-interface

There are things I’m not happy with, mainly that using authenticated sites that require login/password are hard to deal with. In the beginning there were other issues with clipping from Firefox/Chrome and retaining of things like the site title (all my clips ended up becoming “Please login to JIRA”), but after a quick chat with the developer on twitter, he pushed an update to fix those things literally the next day! (I bought a lifetime license right afterwards even though I don’t use any of the pro features)

The dev has also added AppleScript automation support after users requested it and is in general very open to feature requests and suggestions.

So, do I really need a bookmark manager? Until recently I was convinced that the answer is a clear ’no’. Important stuff goes into DEVONthink for archiving, and for the rest, the browser autocomplete and bookmark management features are sufficient.

But Anybox is one of those things that doesn’t feel like it’s changing my workflow a bit. It’s intuitive and fits right into how I want-, and were doing things already anyway, so this one gets a big thumbs up.

Get it for free from anybox.cc. Subscriptions are $1.49 a month, $11.99 a year or $27.99 lifetime

Today, let’s talk a little about my favorite time-sink: Productivity!

I can’t even count how often I re-invented my entire information management stack on the hunt for the perfect solution. Tweaks here, tweaks there - putting some new app into the center focus and building everything around it (again).

Did I find the perfect solution? Nah, I don’t think it actually exists. On the contrary, tools that give me too many customization options turned out to be a pretty bad match for a brain like mine.

So I took it slower. I stopped overthinking the hell out of the perfect system and just let the software handle things for me. Over the course of the past months/years this resulted in a system that works incredibly well for me that I’ll be using to start 2022.

Today, let’s talk about these apps and what role they play in my day to day life. First on the list we have…

1) DEVONthink

DEVONthink has been my gem for the past I think 3 years. It’s maybe the most used, most important Mac app that I own. It’s also the most expensive app I’ve purchased so far.

When friends hear me rave about DEVONthink and they go and check it out, they usually end up even more confused about what it actually is than before.

DEVONthink is a tool for organizing information, but it’s one of these things that is something else for each person that’s using it: For researchers it could be a tool to clip content from the web and aggregate information. For academics it could be a database to classify and link content together. For me it’s my digital office.

Each document I receive like PDFs, images and scans go into DEVONthink. And I really mean each document: When I get a physical letter I scan (not OCR) it with Genius Scan and share it into DEVONthink through the iOS share sheet. When I plan a trip, I clip the PDFs for flight reservations and hotels into DEVONthink. When my cat gets a health-check, the document goes into DEVONthink, and so on. In DEVONthink I then use the built-in Abbyy OCR engine (Abbyy is arguably still king in OCR land) to process the documents, then classify and organize them.

DEVONthink is powerful enough that just by my habits of where certain documents go, and by looking at the content of new documents, it can already tell me with good accuracy where a document likely belongs. All I have to do is hit Ctrl-C and DEVONthink moves it for me. DEVONthink also warns me when I have duplicates, tells me about similar documents and puts logic into my huge pile of documents. No matter what I’m searching for, even if it’s some tax receipt from 3 years ago, I know exactly how to find it.

But another even more important reason why DEVONthink is in the middle of my digital life is its sync capabilities: DEVONthink allows me to setup multiple parallel sync targets like iCloud, Dropbox, Bonjour, WebDav, each with their own encryption key! So my Database could be stored in different locations, fully e2ee encrypted with different keys. No other software I have used is able to do this, which puts it miles ahead of directly using Dropbox/iCloud Drive for me. It gives me peace of mind that my data is secure and backed-up properly with a fail-safe.

In an age where every cloud provider is processing your data for their ML models, having true e2ee systems is rare.

2) Obsidian

I tried hard to use DEVONthink for notes and quick scribbles but the UI just doesn’t do it for me. It always felt awkward, like I’m working in a pile of .txt documents with a barebones notepad editor. I wanted something nicer, and after hopping through dozens of different apps (notable mention to Outlinely and Agenda) I somehow ended up at Obsidian. Ironic because Obsidian is another one of those apps that are hard to actually ‘get’, and surprising because I usually stay far away from electron-based apps. Yet Obsidian became a staple in my life.

I’d describe Obsidian as a knowledge management app. Didn’t I just say the same thing about DEVONthink? Kinda, both apps have some overlap in what they do. The website says “Obsidian is a powerful knowledge base on top of a local folder of plain text Markdown files.”

It’s a markdown editor that stores the files in a database (a “vault”), and gives you a Quick Open dialog. You can also customize literally everything about the app with plugins (I even wrote my own plugin).

Obsidian can become very complex if you spend too much time tweaking it, or trying to follow the productivity system of some internet guru that swear by it. It’s very easy to fall into this habit, I myself am a bit guilty of that. The best way to experience Obsidian is to just use it without thinking what you want to adjust and tweak. Don’t find a problem to solve with it, instead let your brain embrace it. This sounds preachy but I mean it! Just install it and see if it clicks or not.

I mentioned above that my brain is prone to overthink, over-tweak and over-optimize everything and that also happened when I started to use Obsidian. Honestly, it even stressed me.

What changed this was when I went through a very busy period in my life and had to just write things down quickly. I had no time to think about folders, tags, links and so on. I just wanted to write things down and do the rest later. Turns out that this was exactly what my brain needs - a dump of information that I can access at my fingertips the second I need to recall that information.

My brain doesn’t care a bit if my “House Music Sub-genres Differences” note is in a folder called music/, dj/ or even music/dj/ (see where I’m going with this?), because I will never access the note that way. What I will do is hit CMD-O and hammer in “house differences”, and boom - Obsidian gives me exactly what I wanted to find.

It takes away all the mental overhead of having to find a proper directory structure, tagging system or what not and allows me to focus on exactly what I want to focus on: The information.

I label all my notes by the question that note answers, for example “How to do hardlinks in APFS” or “How to construct Minor Chords”. If there is structure between notes, I also insert a link and maybe add them to a Map of Content, like a “Music MOC” that pulls together certain pieces of information with relation.

I never have to worry that I’ll forget certain bits of information because Obsidian got me covered. Obsidian is free, but I also pay for Obsidian Sync. The main reason for that is, you might have guessed it, e2ee encryption! Syncing with Obsidian Sync means my data is stored in encrypted form with a key set by me. If I were to store the Obsidian vault in iCloud Drive for example, it would again technically be accessible by Apple.

3) Things & To-Do management

Believe it or not, I hate todo-list apps the most so I’ll keep this short. I tried every todo management solution under the sun, but it comes down to the same issue: I am not actually doing what I put in my todo list. No clever OmniFocus-like system was able to change that.

I treat my todo-list app as a reminder to not forget something. When I have a thought, I immediately log it on my iPhone or with Siri, then later remember about the item once I open the app again. It takes the fear of forgetting off my brain.

The app I am using this year, the same as last year and the year before is Things. It’s the most simple and (arguably) best-looking option out there and fits my workflow pretty well. It’s also non-subscription which I value a lot.

Nothing too fancy here, I have areas of responsibility for private, work, productivity and shopping. When I get an email that I need to act on, I immediately use the quick-entry shortcut to create a todo out of it, so I don’t forget.

Maybe one productivity hack I am using is to use Zapier to automatically create new todos whenever someone assigns me a pull request on GitHub or issue on JIRA. Sometimes emails about PRs or mentions on GitHub go under, so automating it was the next step to put a bandaid on that issue.

I then prioritize the items, move them into my Today list when appropriate, and usually work out of whatever is in Today. Every couple days I go through all items in Things and look at them again, to make sure there isn’t something I am forgetting.

Now there is still the problem that I’m often not doing all the things that I put in my Today list. Sometimes it’s because lack of time, sometimes there are too many items, sometimes I’m just too busy. Sometimes there are other excuses. When you see the same items day after day, your brain (or at least my brain) starts to ignore them. They are still in the todo list, but my brain starts considering them part of the todo list, if that makes any sense.

I BulletJournal-ed in the past with an actual paper journal which was nice for keeping focus on certain things. This year I wanted to experiment with bringing some of that feeling back, without having to carry paper and pen with me all the time, so I picked up a ReMarkable2 tablet.

Not much to say about it yet besides that it’s an experiment. The logic of using a handwritten day-planner over a todo list is that I have to make the conscious decision of actually writing an item into the planner, with my hand. And I have to do that for each and every day that I decide that I want to do that item. If I don’t end up doing it on that day, yup, write it down again. That puts emphasis on the things I actually really want to do, and gives me an out if I decide something is no longer important.

(It also just feels nice to use a pen)

4) Other little things

I think those are the main big tools that I’ll be using for the foreseeable future. I might expand this post with other things if something else pops up, but a quick scan over my iPhone and currently running Mac apps confirms me that there isn’t much else to go into big detail.

A few bits and pieces though:

4.1) Arq

I think without Arq, I wouldn’t be able to properly sleep at night. My Mac has a 2TB disk, and all of that huge amount of data could potentially vanish in an instant if I were to drop it. Arq allows me to backup e2ee to different providers: I always have a local backup on my SSD, and then one in the cloud. I also backup Obsidian, DEVONthink and everything else I consider important up. Arq saved me so many times that it’s just fair to include here.

4.2) Syncthing

Syncthing is a newer tool I’m currently trialing. After buying that new 2TB Macbook Pro, I now have 2 main devices: A mac mini and a Macbook. Syncthing allows me to quickly sync folders across those 2 machines without having to touch the internet, fully P2P. Most of my stuff syncs e2ee, and DEVONthink already does P2P with Bonjour sync, so it’s more one of these things that come in handy now and then when I have to shovel big amounts of data around.

And as usual…

…these tools work incredibly well for me, but that obviously doesn’t mean they will for you as well. I’m just glad that I reached a point where I don’t have to waste further mental energy to adjust or tweak the system.

I know where to put things, how to access them and how to easily recall information again. I have peace of mind that my stuff is safe and secure for years to come, and that’s all I want.