🐥 YellowDuck.be

🐥 Faster, leaner MySQL backups and restores with mysqldump

2026-08-22T17:00:00Z

mysqldump is a perfectly fine tool for many production workloads — until your database grows large enough that backups take too long and restores take even longer. Before you reach for a more exotic solution, there are several flags and session-level variables that can make a meaningful difference with zero infrastructure changes.

This article focuses on plain MySQL, but shows where to wire up these settings when you use spatie/laravel-backup and stefanzweifel/laravel-backup-restore in a Laravel project.

Where to configure this in Laravel

Both packages read dump and restore options from config/database.php, inside the dump key of your connection:


// config/database.php
'mysql' => [
    // ... standard connection settings ...
    'dump' => [
        'excludeTables' => [...],
        'useSingleTransaction' => true,
        'add_extra_option' => '...', // passed to mysqldump (backup)
        'options' => '...',          // passed to mysql client (restore)
    ],
],

add_extra_option is forwarded to mysqldump by spatie/laravel-backup.
options is forwarded to the mysql client by laravel-backup-restore.

Everything below maps directly to one of those two keys.

Dump-time optimisations

Exclude noisy, non-essential tables

Some tables accumulate millions of rows that are useful for debugging but worthless in a restore. Excluding them shrinks the dump file and cuts restore time dramatically.


'excludeTables' => [
    'telescope_entries',
    'telescope_entries_tags',
    'telescope_monitoring',
],

Laravel Telescope is the classic example: it can easily outweigh the rest of your schema combined.

Use a single transaction

--single-transaction

In spatie/laravel-backup this is:


'useSingleTransaction' => true,

This wraps the dump in a START TRANSACTION so InnoDB tables are read from a consistent snapshot without locking them. Essential for any live database.

Increase `--net-buffer-length`

--net-buffer-length=16777216   # 16 MB (default is 1 MB)

mysqldump groups rows into multi-row INSERT statements. The default maximum per statement is 1 MB. Raising it to 16 MB reduces the total number of statements in the file, which directly cuts parse and execution time during a restore.

Pair it with --max_allowed_packet on the server side:

--max_allowed_packet=512M

Without this, large packets are silently rejected.

Skip unnecessary LOCK/UNLOCK statements

--skip-add-locks

mysqldump normally wraps each table dump in LOCK TABLES … WRITE / UNLOCK TABLES. This is redundant when you are already using --single-transaction and when you disable table locking at restore time (see below). Removing these statements makes the dump file smaller and the restore faster.

Suppress MySQL 8 column statistics

--column-statistics=0

MySQL 8's mysqldump emits ANALYZE TABLE statements to update column statistics after each table is imported. On large tables these can block progress noticeably. If you run your own statistics collection after a restore, suppress them.

Compress the dump on the wire

--compression-algorithms=zlib

Compresses data in transit between mysqldump and the MySQL server. Useful when the client and server are on separate hosts, reducing network I/O. For socket connections the benefit is minimal, but there is no downside.

Skip binary log tracking (GTID)

--set-gtid-purged=OFF

If you use GTID-based replication, mysqldump normally writes SET @@GLOBAL.gtid_purged into the dump. During a restore this can conflict with an existing gtid_executed set. Passing OFF omits that statement and keeps restores clean on secondary or development databases.

Avoid table-level locks

--lock-tables=false

When using --single-transaction, table-level locking is already unnecessary. Explicitly disabling it avoids a redundant flush.

Use `--quick` for large tables

--quick

By default mysqldump buffers an entire table in memory before writing. --quick streams one row at a time, keeping memory usage flat regardless of table size. Always enable it for any production database.

Restore-time optimisations

These are passed to the mysql client via an --init-command, which MySQL executes before the dump is processed:

SET sql_log_bin=0;
SET foreign_key_checks=0;
SET unique_checks=0;
SET autocommit=0;

In config/database.php:


'options' => '--init-command="SET sql_log_bin=0; SET foreign_key_checks=0; SET unique_checks=0; SET autocommit=0;"',

`SET sql_log_bin=0`

Disables binary logging for the session. There is no point logging every INSERT from a dump file into the binlog — it inflates the binlog and slows the restore. Safe when you control the import and are not relying on the binlog to propagate changes to replicas. This also reduces the amount of disk space needed for the restore.

`SET foreign_key_checks=0`

MySQL enforces referential integrity row-by-row during inserts. In a full dump, the parent and child rows are all present — you just may not have inserted the parent yet when a child arrives. Disabling this check lets MySQL trust the dump and skip the per-row lookups, which is one of the most impactful restore optimisations available.

Re-enable it (or restart the session) after the restore to verify the restored data is consistent.

`SET unique_checks=0`

Similar rationale: MySQL normally verifies unique constraints on every insert. A correctly generated dump has no duplicates, so this check is redundant overhead. Disabling it reduces index update cost during the restore.

`SET autocommit=0`

Turns off the implicit COMMIT after every statement. Combined with the large INSERT batches produced by --net-buffer-length, this means the storage engine can batch many rows into a single transaction, dramatically reducing fsync pressure.

Putting it together


// config/database.php
'dump' => [
    'excludeTables' => [
        'telescope_entries',
        'telescope_entries_tags',
        'telescope_monitoring',
    ],
    'useSingleTransaction' => true,
    'add_extra_option' => '--set-gtid-purged=OFF --lock-tables=false --quick'
        . ' --max_allowed_packet=512M --net-buffer-length=16777216'
        . ' --compression-algorithms=zlib'
        . ' --skip-add-locks --column-statistics=0',
    'options' => '--init-command="SET sql_log_bin=0; SET foreign_key_checks=0; SET unique_checks=0; SET autocommit=0;"',
],

These settings are safe for the vast majority of production MySQL 8 setups. The restore-side flags assume you trust the dump and will verify data integrity via application-level health checks afterwards — which is standard practice regardless of how you import.

The backup side produces a smaller file, faster. The restore side skips redundant constraint checking and batches commits. Together they can reduce both phases by a significant margin without changing anything about your infrastructure or your backup tooling.

More info

#laravel #best-practice #mysql #devops

🐥 Speeding up our PHP test suite with a GitHub Actions matrix strategy

2026-08-20T17:00:00Z

Our test suite had become a bottleneck. Every push to develop or main triggered three separate CI jobs — unit tests, feature tests, and integration tests — each running sequentially after the previous one finished and each downloading the same apt packages from scratch. The result: slow feedback loops and a lot of redundant work.

This post walks through the changes we made to fix that.

The Problem: Three Jobs, Lots of Duplication

Before this change, the CI pipeline defined three independent jobs:

phpunit-unit:
  name: "PHP Unit Tests"
  steps:
    - name: Install packages
      run: |
        sudo apt update
        sudo apt install -y package-a package-b package-c
    - name: Execute tests
      run: php artisan test --parallel tests/Unit

phpunit-feature:
  name: "PHP Feature Tests"
  steps:
    # same apt install block...
    - name: Execute tests
      run: php artisan test --parallel tests/Feature

phpunit-integration:
  name: "PHP Integration Tests"
  steps:
    # same apt install block again...
    - name: Execute tests
      run: php artisan test --parallel tests/Integration

Each job was:

Checking out the repo independently
Setting up PHP independently
Installing Composer dependencies independently (even with caching, cache restores take time)
Running apt-get install with the same packages — every single time, with no caching

Step 1: Collapsing Three Jobs into a Matrix

The first change replaced the three separate job definitions with a single phpunit job that uses GitHub Actions' matrix strategy:

phpunit:
  name: "PHP Tests (${{ matrix.suite }})"
  runs-on: ubuntu-latest
  strategy:
    fail-fast: false
    matrix:
      include:
        - suite: unit-a
        - suite: unit-b
        - suite: feature-a
        - suite: feature-b
        - suite: Integration

All suites run in parallel, with fail-fast: false so a failure in one suite doesn't cancel the others mid-run.

The Execute tests step became a single parameterised command:

- name: Execute tests
  run: php artisan test --parallel --testsuite=${{ matrix.suite }} --coverage-clover test-coverage/coverage-${{ matrix.suite }}.xml

Why More Suites Instead of Fewer?

The original split was coarse: all unit tests ran together, and all feature tests ran together. One large bucket would hold everything up even when most of the tests in it finished quickly.

By naming suites explicitly in phpunit.xml, we got finer-grained parallelism:

<testsuite name="unit-a">
  <directory suffix=".php">./tests/Unit/GroupAdirectory>
testsuite>
<testsuite name="unit-b">
  <directory suffix=".php">./tests/Unit/GroupBdirectory>
  <directory suffix=".php">./tests/Unit/GroupCdirectory>
testsuite>
<testsuite name="feature-a">
  <directory suffix=".php">./tests/Feature/GroupAdirectory>
testsuite>
<testsuite name="feature-b">
  <directory suffix=".php">./tests/Feature/GroupBdirectory>
  <directory suffix=".php">./tests/Feature/GroupCdirectory>
testsuite>

This keeps each matrix leg roughly balanced in terms of test count and makes the overall wall-clock time shorter than when one slow bucket holds everything up.

Step 2: Caching apt Packages

The most wasteful part of the old setup was reinstalling the same system packages on every runner, for every job, from scratch. apt-get install isn't slow in absolute terms, but multiplied across five parallel jobs and every CI run, it adds up.

We replaced the raw apt install block with awalsh128/cache-apt-pkgs-action:

- name: Install packages
  uses: awalsh128/cache-apt-pkgs-action@latest
  with:
    packages: package-a package-b package-c
    version: ${{ hashFiles('.apt-packages') }}

The version key is tied to a .apt-packages lockfile committed to the repo. The cache is invalidated only when that file changes — not on every push — so the common case is a fast cache hit rather than a full install.

Any post-install configuration that previously lived inside the install block becomes its own named step. The action calls it only on a cache miss, so it doesn't run unnecessarily on cache hits:

- name: Post-install configuration
  run: # your configuration command here

The Downstream Effect: Coverage Merging

Any job that previously depended on the three separate job names now depends on the single phpunit job:

some-downstream-job:
  needs: [
    "phpunit",   # was: phpunit-unit, phpunit-feature, phpunit-integration
    ...
  ]

And the coverage merge step includes one file per suite:

./clover-merge -o coverage.xml \
  unit-a/coverage-unit-a.xml \
  unit-b/coverage-unit-b.xml \
  feature-a/coverage-feature-a.xml \
  feature-b/coverage-feature-b.xml \
  integration/coverage-integration.xml

What This Changes

Before	After
3 jobs, one per test type	5 jobs, all parallel
`apt install` on every run, every job	Cached apt packages, invalidated only on lockfile change
Coarse path-based test selection	Named testsuites in `phpunit.xml`
~3× duplicated job boilerplate	Single parameterised job definition

The wall-clock time for the PHP test stage drops because the legs run concurrently and the apt install step is usually a cache hit. Maintenance is easier too — adding a new suite is a two-line change: one entry in phpunit.xml and one entry in the matrix include list.

#php #devops #github #testing

🐥 Advanced debugging in Elixir with IO.inspect

2026-08-17T17:00:00Z

When writing Elixir, most developers quickly get familiar with IO.inspect/2 as a quick way to see what's happening inside their code. But what many overlook is that IO.inspect is far more powerful than just "print this variable to the console."

In fact, with the right options and placement, IO.inspect can become a precise, highly targeted debugging tool, one that doesn't interrupt your program flow and works seamlessly with Elixir's functional pipelines.

This post will walk through both the fundamentals and advanced patterns for using IO.inspect effectively. By the end, you'll know how to control output formatting, label your prints for clarity, debug concurrent processes, and even integrate conditional or file-based inspection.

The Basics of `IO.inspect`

IO.inspect is defined like this:

IO.inspect(item, opts \\ [])

item: any Elixir value you want to inspect.
opts: keyword list of options for controlling how the term is printed as defined in Inspect.Opts.

By default, it writes the inspected value to the standard output (:stdio) and then returns the term unchanged.

That last part is key. Because it returns the term, you can drop it anywhere in a function or pipeline without breaking the flow.

Example:

defmodule Demo do
  def run do
    IO.inspect(%{name: "Alice", age: 30})
  end
end

Prints:

%{age: 30, name: "Alice"}

Caveat: When `IO.inspect` Is the Last Call in a Function

Because IO.inspect returns its argument, it also means that if it's the last expression in your function, that inspected value becomes the function's return value.

That's fine if you intend to return it, but it can lead to subtle bugs if you were only printing it for debugging and expected a different return.

Take this example:

def fetch_user(id) do
  Repo.get(User, id)
  |> IO.inspect(label: "Fetched user")
end

Here, the function returns the user struct as normal, which is fine.

Now, consider:

def log_and_return do
  IO.inspect("Done!")
end

This returns "Done!" instead of, say, :ok.

If you want to print something but return a different value, make the return explicit:

def log_and_return do
  IO.inspect("Done!")
  :ok
end

Or if you're in a pipeline:

value
|> IO.inspect(label: "Debug")
|> do_something_else()

Rule of thumb: if IO.inspect is the last expression in a function, be explicit about what you want to return.

Strategic Placement With the Pipe Operator

Because IO.inspect returns its argument, it fits perfectly in the middle of a pipeline:

users
|> IO.inspect(label: "Before filtering")
|> Enum.filter(& &1.active)
|> IO.inspect(label: "After filtering")
|> Enum.map(& &1.name)

This lets you peek into the data flow at exactly the point you want, without rewriting your code into intermediate variables.

A neat trick is to place multiple IO.inspect calls at different stages, each with a distinct label, so you can see how the data changes step by step.

Using Labels for Clarity

Without labels, multiple inspection outputs can be hard to tell apart. The label option solves this:

IO.inspect(data, label: "After filtering")

Prints:

After filtering: [%{name: "Alice"}, %{name: "Bob"}]

When debugging a pipeline with several inspect points, labels make the output self-describing. This is especially useful when you're debugging multiple similar data structures in the same run.

Pretty Printing and Formatting Output

Sometimes, especially with large maps or deeply nested lists, the default single-line output is hard to read. That's where formatting options defined in Inspect.Opts come in.

Option	Description	Default
`:pretty`	if set to `true` enables pretty printing.	`false`
`:limit`	limits the number of items that are inspected for tuples, bitstrings, maps, lists and any other collection of items, with the exception of printable strings and printable charlists which use the `:printable_limit` option. If you don't want to limit the number of items to a particular number, use `:infinity`. It accepts a positive integer or `:infinity`.	`50`
`:printable_limit`	limits the number of characters that are inspected on printable strings and printable charlists. You can use `String.printable?/1` and `List.ascii_printable?/1` to check if a given string or charlist is printable. If you don't want to limit the number of characters to a particular number, use `:infinity`. It accepts a positive integer or `:infinity`.	`4096`
`:width`	number of characters per line used when pretty is `true` or when printing to IO devices. Set to `0` to force each item to be printed on its own line. If you don't want to limit the number of items to a particular number, use `:infinity`.	`80`
`:charlists`	when `:as_charlists` all lists will be printed as charlists, non-printable elements will be escaped. When `:as_lists` all lists will be printed as lists. When the default `:infer`, the list will be printed as a charlist if it is printable, otherwise as list. See `List.ascii_printable?/1` to learn when a charlist is printable.	`:infer`

Example:

data = for i <- 1..100 do
  %{id: i, value: String.duplicate("x", i)}
end

IO.inspect(data, pretty: true, limit: 5)

Prints:

[
  %{id: 1, value: "x"},
  %{id: 2, value: "xx"},
  %{id: 3, value: "xxx"},
  %{id: 4, ...},
  %{...},
  ...
]

Here, we can see the first five entries, and the rest are summarized with ....

Coloring and Styling Output

Elixir's inspect options support syntax coloring, very handy when your terminal is full of logs.

Example:

IO.inspect(data,
  syntax_colors: [
    atom: :blue,
    string: :green,
    number: :red
  ],
  pretty: true
)

This makes atoms blue, strings green, and numbers red in your console output.

Colors can be any IO.ANSI.ansidata/0 as accepted by IO.ANSI.format/1.

If you want to use the default colors (like what is used in IEx), you can use IO.ANSI.syntax_colors/0:

IO.inspect(data,
  syntax_colors: IO.ANSI.syntax_colors(),
  pretty: true
)

Note: The colors are only visible if your terminal supports ANSI colors.

Conditional Inspection

Sometimes you want to inspect only if a certain condition is true, for example, when a debug flag is enabled or when a value crosses a threshold.

Here's an example with a debug flag:

def debug(term) do
  if Application.get_env(:my_app, :debug) do
    IO.inspect(term, label: "Debug")
  end
  term
end

You can use this function like this:

users
|> Enum.filter(& &1.active)
|> debug()
|> Enum.map(& &1.name)

You can toggle the output by setting config :my_app, :debug, true or false.

Capturing Inspect Output Instead of Printing

If you want to get the inspected form of a value without printing it, use inspect/2 (without the IO. prefix):

string_representation = inspect(data, pretty: true)

This is useful if you want to:

Write the debug output to a file
Send it to a logging service
Include it in an exception message

Example:

File.write!("debug.log", inspect(data, pretty: true))

You can also redirect IO.inspect output to another device:

IO.inspect(data, label: "Debug", device: :stderr)

Debugging Concurrency and Async Code

When you use IO.inspect in asynchronous code, the output may arrive out of order, making it hard to follow. Adding identifiers or timestamps can help.

Example:

tasks =
  for id <- 1..3 do
    Task.async(fn ->
      IO.inspect(self(), label: "PID #{id}")
      :timer.sleep(100)
      id * id
    end)
  end

Enum.map(tasks, &Task.await/1)

Prints:

PID 3: #PID<0.112.0>
PID 2: #PID<0.111.0>
PID 1: #PID<0.110.0>

Including the PID or a unique request ID in your label helps you trace which output belongs to which process.

Advanced Trick: Inspecting and Pattern Matching in One Go

You can combine pattern matching with inspection to see exactly what's being matched:

%{id: id} = IO.inspect(user, label: "User before insert")

This inspects the user variable before pattern matching extracts the id.

You can also place IO.inspect inside a guard to conditionally print:

case user do
  %{role: :admin} = u -> IO.inspect(u, label: "Admin user")
  _ -> :ok
end

In this last example, "Admin user" will only be printed if the user has the role :admin.

Using `dbg/2` for Richer Inspection

Since Elixir 1.14, we have dbg/2, a built-in debugging helper that works like IO.inspect but also shows the code expression that produced the value.

You can use it like this:

users |> Enum.filter(& &1.active) |> dbg()

Which prints:

users #=> [%{active: false, name: "Alice"}, %{active: true, name: "Bob"}]
|> Enum.filter(& &1.active) #=> [%{active: true, name: "Bob"}]

This makes it much easier to understand where in your code the inspected value is coming from, especially when you're inspecting multiple similar-looking values.

You can also pass options similar to IO.inspect:

dbg(users, label: "Active users", pretty: true)

Here are the key differences with IO.inspect:

Shows the code expression automatically.
Output format is slightly more verbose.
Same return behavior, returns the inspected value, so you can keep it in a pipeline.
Best for interactive debugging during development.

When `IO.inspect` Is Not Enough

While IO.inspect is a fantastic quick-and-dirty tool, there are times when you need more powerful debugging:

IEx.pry: drops you into an interactive REPL inside the running process.
:observer.start/0: Erlang's GUI for monitoring processes, memory, and more.

The trick is to know when to reach for IO.inspect and when to switch to one of these other tools.

Setting Default Options for `IO.inspect` in IEx

When using IEx, you can configure the default options for IO.inspect. You can do this using the IEx.configure/1 function:

IEx.configure(
  colors: [
    syntax_colors: [
      number: :light_yellow,
      atom: :light_cyan,
      string: :light_black,
      boolean: :red,
      nil: [:magenta, :bright]
    ],
    ls_directory: :cyan,
    ls_device: :yellow,
    doc_code: :green,
    doc_inline_code: :magenta,
    doc_headings: [:cyan, :underline],
    doc_title: [:cyan, :bright, :underline]
  ]
)

You can easily put this in your .iex.exs file so that it's applied automatically every time you open the shell.

Creating a Reusable Inspect Helper

Here's a neat way to wrap IO.inspect/2 into your own helper module with preferred defaults. This way, you can keep consistent inspection output without repeating options everywhere:

defmodule MyApp.Debug do
  @moduledoc """
  Convenience wrapper around `IO.inspect/2` with sensible defaults.
  """

  @default_opts [
    label: "DEBUG",
    pretty: true,
    limit: :infinity,
    width: 120,
    syntax_colors: IO.ANSI.syntax_colors()
  ]

  @doc """
  Inspects a value with default debug options.

  This is pipe-friendly: it returns the given value unchanged
  after inspecting it.

  ## Examples

      iex> [1, 2, 3]
      ...> |> Enum.map(&(&1 * 2))
      ...> |> MyApp.Debug.inspect()
      [2, 4, 6]
  """
  def inspect(term, opts \\ []) do
    term
    |> IO.inspect(Keyword.merge(@default_opts, opts))
  end
end

You can use it like this:

result =
  users
  |> Enum.filter(&(&1.active?))
  |> MyApp.Debug.inspect(label: "Active users")
  |> Enum.map(& &1.name)
  |> MyApp.Debug.inspect(label: "User names")

This way:

You get pretty printing (pretty: true) by default.
Lists and maps are not truncated (limit: :infinity).
A label is always shown (DEBUG unless overridden).
You can still override any option per call.
It works fine when it's in a pipeline.

Handy Visual Studio Code snippets

To make using IO.inspect faster and more consistent, you can configure editor snippets in Visual Studio Code so you don't have to type repetitive boilerplate each time.

In Visual Studio Code, open:

Code -> Settings… -> Configure Snippets -> Elixir

and add the following snippet definitions:

{
  "inpsect": {
    "prefix": "lin",
    "body": "IO.inspect($1, label: \"$1\", pretty: true)",
    "description": "IO.inspect with label."
  },
  "inpsectSelectedText": {
    "prefix": "sin",
    "body": "IO.inspect($TM_SELECTED_TEXT, label: \"${TM_SELECTED_TEXT/(.*)/${1:/upcase}/}$0\", pretty: true)",
    "description": "IO.inspect with selected text as label."
  },
  "pipeInspect": {
    "prefix": "pin",
    "body": "|> IO.inspect(label: \"$1\", pretty: true)",
    "description": "IO.inspect with pipe."
  },
  "pipeInspectWithFileReference": {
    "prefix": "pinf",
    "body": "|> IO.inspect(label: \"$TM_FILEPATH:$TM_LINE_NUMBER$0\", pretty: true)",
    "description": "IO.inspect with pipe and file reference."
  },
  "inspectFromClipboard": {
    "prefix": "cin",
    "body": "IO.inspect($CLIPBOARD, label: \"${CLIPBOARD/(.*)/${1:/upcase}/}$0\", pretty: true)",
    "description": "IO.inspect clipboard content."
  }
}

With these in place, you'll have short prefixes to insert commonly used IO.inspect/2 patterns:

Prefix	Description
`lin`	`IO.inspect` with label
`sin`	`IO.inspect` using selected text as label
`pin`	`IO.inspect` in a pipeline
`pinf`	`IO.inspect` in a pipeline with file and line reference
`cin`	`IO.inspect` clipboard content

This way, you can quickly drop in debug output with consistent formatting, colors, and labels, without breaking your flow.

Suppose you are transforming a list of user maps and want to check intermediate results inside a pipeline:

users
|> Enum.filter(&(&1.active))
|> IO.inspect(label: "After filter", pretty: true)
|> Enum.map(& &1.email)

With the snippet defined above, you don't have to type all that. Just type pin, hit Tab, and you get:

|> IO.inspect(label: "", pretty: true)

You can immediately type your label (e.g. "After filter") and continue coding. This keeps your debugging consistent, colorful, and fast, without breaking your flow.

Best practices

Use labels liberally, unlabeled output is harder to parse.
Limit output for large collections, use limit and pretty.
Avoid leaving it in production unless intentional.
Wrap it in helper functions when you want conditional control.
Tag concurrent output with PIDs, timestamps, or request IDs.
Credo can be used to detect unintentional calls to IO.inspect in a CI/CD pipeline.

Conclusion

IO.inspect may look like a humble debugging tool, but in Elixir it's a powerful way to see what's going on without breaking your code's flow. By combining it with labels, formatting, conditional output, and process context, you can get precise insights into your program's behavior, all without leaving your editor.

#pattern #best-practice #elixir

🔗 My thoughts on the future of Go in the AI era

2026-08-15T13:00:00Z

What if the rise of AI makes programming languages like Go more valuable? Alex Pliutau argues that while languages like TypeScript and Rust dominate the discussion, Go offers unique advantages in the AI landscape. With its massive standard library, fast compilation, and commitment to stability, Go could thrive even as coding agents proliferate. This article dissects Go's critical features—like explicit error handling and efficient concurrency—that align well with AI-generated code needs. Pliutau suggests that Go's simplicity and practicality may become invaluable in an era where comprehensible and maintainable code is essential.

Continue reading on packagemain.tech

🐥 Hosting your own APT repository for a Go CLI tool

2026-08-14T17:00:00Z

Distributing a command-line tool to Linux users is straightforward until you want to make it feel native. Tarballs work, but APT packages are what seasoned Linux users expect: a single apt install, automatic updates via apt upgrade, and clean removal with apt remove. This post walks through how we set up a self-hosted APT repository for a Go CLI tool using only a Makefile, a small shell script, and a clever Go utility called aptblob.

The goal

We want users to be able to run:

curl -fsSL https://packages.example.com/pubkey.gpg | sudo apt-key add -
echo "deb https://packages.example.com stable main" | sudo tee /etc/apt/sources.list.d/mycli.list
sudo apt update
sudo apt install mycli

To make that work, we need to produce properly structured .deb packages for each architecture, sign them with a GPG key, and publish an APT repository index that apt can parse.

Step 1: cross-compiling and packaging

Go makes cross-compilation trivial. The build-deb make target compiles the binary twice — once for arm64 and once for amd64 — and wraps each in a proper Debian package.

## build-deb: build the debian package
.PHONY: build-deb
build-deb:
    @rm -f ./$(APPNAME)-*
    $(call build-binary,linux,arm64,arm64)
    $(call build-deb,arm64)
    $(call build-binary,linux,amd64,x86_64)
    $(call build-deb,amd64)

The build-binary helper looks like this:

define build-binary
    @GOOS=$(1) GOARCH=$(2) go build -o $(APPNAME)
endef

The build-deb helper function does all the Debian packaging work. It creates the directory structure that dpkg-deb expects, writes a minimal DEBIAN/control file, places the binary into usr/bin/, and calls dpkg-deb --build to produce the .deb archive:

define build-deb
    @mkdir -p $(APPNAME)_$(VERSION)-1_$(1)/DEBIAN
    @mkdir -p $(APPNAME)_$(VERSION)-1_$(1)/usr/bin
    @mv $(APPNAME) $(APPNAME)_$(VERSION)-1_$(1)/usr/bin/$(APPNAME)
    @echo "Package: mycli"              > $(APPNAME)_$(VERSION)-1_$(1)/DEBIAN/control
    @echo "Version: $(VERSION)"        >> $(APPNAME)_$(VERSION)-1_$(1)/DEBIAN/control
    @echo "Maintainer: Team " >> $(APPNAME)_$(VERSION)-1_$(1)/DEBIAN/control
    @echo "Architecture: $(1)"         >> $(APPNAME)_$(VERSION)-1_$(1)/DEBIAN/control
    @echo "Description: My CLI tool"   >> $(APPNAME)_$(VERSION)-1_$(1)/DEBIAN/control
    @dpkg-deb --build -Zgzip $(APPNAME)_$(VERSION)-1_$(1)
    @rm -rf $(APPNAME)_$(VERSION)-1_$(1)
endef

The -Zgzip flag tells dpkg-deb to use gzip compression, which produces slightly larger packages than xz but is more universally compatible.

The version is injected automatically from the latest git tag:

VERSION := $(shell git describe --tags --abbrev=0)

So tagging a release in git is all it takes to produce the correct package version.

Step 2: the `aptblob` tool

The real hero of this setup is zombiezen.com/go/aptblob. It is a small Go utility that creates and maintains APT repositories without requiring you to install reprepro, aptly, or any other heavyweight tooling. Because it is fetched and run via go run, there is no separate installation step — Go's module system handles it.

aptblob supports multiple storage backends via URL: file:// for local disk, and cloud storage providers for production hosting.

Step 3: initialising the repository

Before uploading any packages, the repository needs to be bootstrapped with an InRelease file that describes the repository metadata. This is handled by init-repo.sh:

#!/usr/bin/env bash

go run zombiezen.com/go/aptblob@latest init -k $KEY_ID "file://`pwd`/apt-repo" stable <<EOF
Origin: stable
Label: My CLI Repository
Suite: stable
Codename: stable
Version: 1.0
Architectures: arm64 amd64
Components: main
Description: The My CLI software repository
EOF

The -k $KEY_ID flag tells aptblob to sign the repository metadata with a specific GPG key. The key ID is passed via the APT_SIGNING_KEY_ID environment variable so it never appears in source control. The stable argument at the end is the distribution name — the string users put in their sources.list entry.

Step 4: uploading the packages

With the repository initialised, the build-apt-repo target uploads both .deb files:

## build-apt-repo: build the apt repository
build-apt-repo: build-deb
    @echo "Building apt repository for version $(VERSION)"
    @rm -rf apt-repo
    @mkdir apt-repo
    @KEY_ID=$(KEY_ID) bash init-repo.sh
    @go run zombiezen.com/go/aptblob@latest upload -k $(KEY_ID) \
        "file://$(shell pwd)/apt-repo" stable $(APPNAME)_$(VERSION)-1_arm64.deb
    @go run zombiezen.com/go/aptblob@latest upload -k $(KEY_ID) \
        "file://$(shell pwd)/apt-repo" stable $(APPNAME)_$(VERSION)-1_amd64.deb
    @rm *.deb

Each upload call:

Copies the .deb into the repository's pool directory
Updates the Packages index files for the relevant architecture
Regenerates and re-signs the Release / InRelease metadata

The -k $(KEY_ID) flag is required on every upload because aptblob must re-sign the repository index each time a package is added.

After the upload, the temporary .deb files are cleaned up — the canonical copies now live inside apt-repo/.

Step 5: publishing

The apt-repo/ directory is a fully self-contained, static APT repository. Serving it is as simple as putting it behind any HTTP server or syncing it to a cloud storage bucket with public read access:

# Sync to S3 (or any S3-compatible service)
aws s3 sync ./apt-repo s3://my-packages-bucket/ --delete

# Or rsync to a VPS
rsync -avz ./apt-repo/ user@packages.example.com:/var/www/packages/

Why this approach works well

No daemon or database. Traditional APT repository tools like reprepro maintain a local database and run as a long-lived process. aptblob is entirely stateless — the repository metadata is the state.

Pure Go toolchain. The only external dependencies are gpg (for signing) and dpkg-deb (for building the .deb files). Everything else is managed by go run, which means CI machines don't need any special APT repository software pre-installed.

Reproducible builds. Every release starts from a clean rm -rf apt-repo, so there is no risk of stale packages accumulating silently.

Works with any storage. Today we use file:// for a local build; tomorrow we can switch to an S3 or GCS URL without changing anything else.

The full release workflow

# 1. Tag the release
git tag v1.2.3 && git push --tags

# 2. Build and publish
APT_SIGNING_KEY_ID=ABCDEF1234567890 make build-apt-repo
aws s3 sync ./apt-repo s3://my-packages-bucket/ --delete

That's two commands from tag to published APT package. For a small CLI tool, it is hard to imagine a lighter-weight setup that still delivers a genuine apt install experience.

Summary

Step	Tool	What it produces
Cross-compile	`go build` with `GOOS`/`GOARCH`	Linux binaries (arm64, amd64)
Package	`dpkg-deb --build`	`.deb` archives
Init repo	`aptblob init`	Signed `InRelease` / `Release` metadata
Upload packages	`aptblob upload`	Updated `Packages` index + pool
Serve	Any static HTTP server	Installable APT repository

If you are shipping a Go CLI and want proper APT distribution without the overhead of a full packaging pipeline, aptblob plus a handful of Makefile targets gets you there in under 50 lines.

#golang #tools #best-practice #devops #linux

🐥 Distributing a Go CLI via a Homebrew tap

2026-08-08T17:00:00Z

Homebrew is the package manager that macOS developers reach for first. If you want your CLI tool to feel like a first-class citizen on macOS — installable with a single command, updatable with brew upgrade, and auto-completed by the shell — a Homebrew tap is the right solution. This post covers how to set one up for a Go CLI, including the formula, CI validation, and the update workflow.

What is a tap?

A Homebrew tap is just a GitHub repository named homebrew-. Once a user runs brew tap org/something, Homebrew knows to look for formulas in that repo. From that point on, brew install org/something/mycli works exactly like installing any official Homebrew package.

The naming convention is the only magic: no registration, no approval process. Anyone can host a tap.

The formula

Formulas live in a Formula/ directory and are written in Ruby using Homebrew's DSL. For a pre-built Go binary, the formula is remarkably simple:

class MyCli < Formula
  desc "CLI interface for My App"
  homepage "https://example.com"
  url "https://cdn.example.com/mycli/5c7b6ef47a682ffe.../mycli-macos.tar.gz"
  version "1.47.0"
  sha256 "48e146382b0f527549328da4bed9d90f2d584e1b05a8f94846cf47c3c66a9343"
  license "MIT"

  def install
    bin.install "mycli"
  end

  test do
    system "#{bin}/mycli", "--version"
  end
end

There are only four things that change on each release:

url — points to the new tarball on the CDN
version — the semver string shown by brew info
sha256 — the checksum Homebrew verifies before unpacking

The install block copies the binary into the Homebrew prefix (/opt/homebrew/bin/ on Apple Silicon, /usr/local/bin/ on Intel). The test block is a smoke test Homebrew runs after install — here, just confirming the binary exits cleanly when asked for its version.

Building a universal macOS binary

The URL points to a .tar.gz containing a single binary. Rather than shipping separate arm64 and x86_64 tarballs and maintaining two formula entries, we use lipo to merge them into one universal binary:

define build-binary-mac
    @echo "Building for macos"
    GOOS=darwin GOARCH=arm64 go build -o mycli-arm .
    GOOS=darwin GOARCH=amd64 go build -o mycli-x86 .
    lipo -create -output mycli mycli-x86 mycli-arm
    tar czf ./mycli-macos.tar.gz mycli
    rm -f mycli-x86 mycli-arm mycli
endef

GOOS=darwin GOARCH=arm64 go build and GOOS=darwin GOARCH=amd64 go build produce two binaries. lipo -create stitches them into a Fat Binary that macOS automatically runs under the native architecture. The result: one tarball, one formula URL, works on both Apple Silicon and Intel Macs without any user-facing complexity.

The CDN URL strategy

The binary is uploaded to object storage (DigitalOcean Spaces in our case, but S3 or any public CDN works equally well). The URL path includes the git commit SHA:

https://cdn.example.com/mycli/{git-sha}/mycli-macos.tar.gz

Embedding the commit SHA rather than the version tag means:

Uploads are naturally immutable — the same SHA always points to the same build
Multiple release candidates for the same version cannot collide
The create-index.py script (which generates a index.json listing all build artifacts) uses the same SHA, keeping the release artefacts coherent

Updating the formula

When a new version ships:

Build and upload the tarball: make build-all produces mycli-macos.tar.gz; CI uploads it to the CDN under the new commit SHA.
Compute the SHA256 of the tarball: sha256sum mycli-macos.tar.gz
Update three lines in the formula: url, version, sha256
Open a PR against the tap repository

The tap CI then validates the change before it merges (more on that below).

CI: validating formulas with `brew test-bot`

The tap repo includes a GitHub Actions workflow that runs on every push and pull request:

name: brew test-bot
on:
  push:
    branches: [main]
  pull_request:

jobs:
  test-bot:
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest]
    runs-on: ${{ matrix.os }}
    steps:
      - uses: Homebrew/actions/setup-homebrew@main

      - run: brew test-bot --only-cleanup-before
      - run: brew test-bot --only-setup
      - run: brew test-bot --only-tap-syntax
      - run: brew test-bot --only-formulae
        if: github.event_name == 'pull_request'

      - name: Upload bottles as artifact
        if: always() && github.event_name == 'pull_request'
        uses: actions/upload-artifact@main
        with:
          name: bottles
          path: '*.bottle.*'

brew test-bot is Homebrew's own CI harness. The steps do progressively more:

Step	What it checks
`--only-tap-syntax`	Ruby syntax, formula naming conventions, `brew audit` rules
`--only-formulae`	Actually installs the formula and runs the `test do` block

Running on both ubuntu-latest and macos-latest catches issues specific to either platform — important because Homebrew has first-class Linux support (Linuxbrew) and many teams use it in Docker-based CI.

CI: Merging PRs with `brew pr-pull`

The second workflow handles the merge step:

name: brew pr-pull
on:
  pull_request_target:
    types: [labeled]

jobs:
  pr-pull:
    if: contains(github.event.pull_request.labels.*.name, 'pr-pull')
    runs-on: ubuntu-22.04
    steps:
      - uses: Homebrew/actions/setup-homebrew@main
      - uses: Homebrew/actions/git-user-config@main

      - name: Pull bottles
        env:
          HOMEBREW_GITHUB_API_TOKEN: ${{ github.token }}
          PULL_REQUEST: ${{ github.event.pull_request.number }}
        run: |
          brew pr-pull --debug --tap="$GITHUB_REPOSITORY" "$PULL_REQUEST"

      - uses: Homebrew/actions/git-try-push@main
        with:
          token: ${{ github.token }}
          branch: main

      - name: Delete branch
        if: github.event.pull_request.head.repo.fork == false
        run: git push --delete origin "$BRANCH"

Instead of merging PRs normally, a maintainer applies the pr-pull label. brew pr-pull then:

Downloads any pre-built bottles attached to the PR as GitHub Actions artifacts
Commits the bottle checksums into the formula
Pushes directly to main and deletes the PR branch

This mirrors the workflow used by the official Homebrew/homebrew-core tap and means the merge is always handled by Homebrew's own tooling rather than GitHub's merge button.

For a simple pre-built binary formula (no compilation step, no bottle needed), this is mostly ceremony — but it is good practice because it keeps the tap workflow consistent with the broader Homebrew ecosystem and makes it easy to add bottles later if the build-from-source path is ever needed.

The user experience

After all of this, the user-facing story is clean:

# One-time setup
brew tap myorg/mycli

# Install
brew install mycli

# Upgrade when a new version ships
brew upgrade mycli

And because Homebrew auto-discovers shell completions in share/zsh/site-functions/, share/bash-completion/, and similar paths, completions are picked up automatically if the binary generates them into those locations at install time.

Summary

Component	What it does
`homebrew-mycli` repo	The tap — a public GitHub repo Homebrew reads formulas from
`Formula/mycli.rb`	Declares the URL, version, checksum, install path, and smoke test
Universal binary (`lipo`)	One tarball covers both Apple Silicon and Intel in a single formula entry
CDN + commit SHA URL	Immutable, content-addressed artifact storage
`brew test-bot` CI	Validates formula syntax and install on every PR
`brew pr-pull` CI	Homebrew-native merge flow that handles bottle attachment

The tap repository itself is under 20 lines of real content. Most of the work is a one-time setup; after that, each release is a three-line diff to the formula and a label click to merge.

#golang #tools #mac

🐥 Adding 2FA to Phoenix LiveView with phx.gen.auth

2026-08-03T17:00:00Z

Two-factor authentication (2FA) is a great way to improve account security. This post walks through how to add TOTP-based 2FA to a Phoenix LiveView app using the built-in phx.gen.auth authentication system.

We’ll cover:

Adding required fields and dependencies
2FA setup flow using LiveView
TOTP challenge during login
LiveView-specific login handling

Add dependencies

Add the following libraries to mix.exs:

defp deps do
  [
    {:nimble_totp, "~> 1.0"},
    {:qr_code, "~> 2.2"}
  ]
end

Then run:

mix deps.get

Extend the user schema

Add two fields to your users table:

# priv/repo/migrations/*_add_totp_to_users.exs
alter table(:users) do
  add :totp_secret, :string
  add :totp_confirmed_at, :utc_datetime
end

Migrate:

mix ecto.migrate

In your user schema:

schema "users" do
  field :totp_secret, :string
  field :totp_confirmed_at, :utc_datetime
  # ...
end

Ensure your changeset/2 casts these fields:

def changeset(user, attrs) do
  user
  |> cast(attrs, [:email, :totp_secret, :totp_confirmed_at])
  |> validate_required([:email])
end

Add 2FA setup LiveView

Create a new LiveView at /settings/two_factor that generates a secret, renders a QR code, and lets the user enter their TOTP code.

defmodule MyAppWeb.TwoFactorSetupLive do
  use MyAppWeb, :live_view
  alias NimbleTOTP
  alias QRCode
  alias MyApp.Accounts

  def mount(_params, _session, socket) do
    user = socket.assigns.current_user

    if user.totp_secret do
      {:ok, assign(socket, setup?: false)}
    else
      secret = Base.encode32(NimbleTOTP.secret())
      uri = NimbleTOTP.otpauth_uri("MyApp:#{user.email}", Base.decode32!(secret), issuer: "MyApp")
      {:ok, png} = QRCode.create(uri, :png)
      b64 = Base.encode64(png)

      {:ok,
       assign(socket,
         setup?: true,
         secret: secret,
         qr_code: "data:image/png;base64,#{b64}",
         code: "",
         error: nil
       )}
    end
  end

  def handle_event("verify", %{"code" => code}, socket) do
    secret = Base.decode32!(socket.assigns.secret)

    if NimbleTOTP.valid?(secret, code) do
      {:ok, _} =
        Accounts.update_user(socket.assigns.current_user, %{
          totp_secret: socket.assigns.secret,
          totp_confirmed_at: DateTime.utc_now()
        })

      {:noreply,
       socket
       |> put_flash(:info, "2FA enabled.")
       |> push_redirect(to: "/settings")}
    else
      {:noreply, assign(socket, error: "Invalid code")}
    end
  end
end


<h1>Two-Factor Authenticationh1>

<%= if @setup? do %>
  <p>Scan this QR code in your authenticator app:p>
  <img src={@qr_code} />

  <form phx-submit="verify">
    <label>Code:label>
    <input type="text" name="code" />
    <button>Verifybutton>
  form>

  <%= if @error, do: content_tag(:p, @error, class: "text-red-500") %>
<% else %>
  <p>2FA is already enabled.p>
<% end %>

Add a TOTP challenge LiveView

When a user logs in with 2FA enabled, redirect them to a LiveView at /two_factor to enter their code.

defmodule MyAppWeb.TwoFactorChallengeLive do
  use MyAppWeb, :live_view
  alias MyApp.Accounts
  alias NimbleTOTP
  alias MyAppWeb.UserAuth

  def mount(_params, %{"pending_user_id" => id}, socket) do
    {:ok, assign(socket, user: Accounts.get_user!(id), code: "", error: nil)}
  end

  def handle_event("verify", %{"code" => code}, socket) do
    user = socket.assigns.user

    if NimbleTOTP.valid?(Base.decode32!(user.totp_secret), code) do
      {:noreply,
       socket
       |> put_flash(:info, "Logged in with 2FA.")
       |> UserAuth.log_in_user(user, %{})}
    else
      {:noreply, assign(socket, error: "Invalid code")}
    end
  end
end


<h1>Enter your 2FA codeh1>

<form phx-submit="verify">
  <input type="text" name="code" />
  <button>Verifybutton>
form>

<%= if @error, do: content_tag(:p, @error, class: "text-red-500") %>

Update login logic

After validating the user’s password (in your LiveView or controller), check if TOTP is enabled:

if user.totp_confirmed_at do
  socket
  |> Phoenix.LiveView.put_session(:pending_user_id, user.id)
  |> Phoenix.LiveView.redirect(to: "/two_factor")
else
  UserAuth.log_in_user(socket, user, %{})
end

In controllers, you’d use put_session(conn, ...) and redirect(conn, ...) instead.

Update `UserAuth` helper

Make sure your UserAuth module has a LiveView-compatible log_in_user/3:

def log_in_user(socket, user, _params \\ %{}) do
  token = MyApp.Accounts.generate_user_session_token(user)

  socket
  |> Phoenix.LiveView.put_session(:user_token, token)
  |> Phoenix.LiveView.redirect(to: "/")
end

Done

You now have a working TOTP 2FA flow in Phoenix LiveView:

Users opt in to 2FA in their settings
A QR code is shown and confirmed
2FA is required on next login
All handled with clean LiveView flows

From here, you can extend the system with:

Disabling 2FA
Backup codes
Remembering trusted devices

Let me know if you'd like a follow-up on any of those features.

#elixir #phoenix #auth

🐥 Finding unexpected HTTP responses in JSON logs with jq

2026-07-28T17:00:00Z

When debugging an application in production, I often want a quick overview of requests that didn't result in a "normal" HTTP response. If your web server writes JSON logs, jq makes this incredibly easy.

Here's a command I regularly use:

cat access_*.log \
  | jq -r '
      select(
        .status != 200 and
        .status != 206 and
        .status != 302 and
        .status != 304 and
        .status != 308 and
        .status != 101
      )
      | [.status, .request.uri]
      | @csv
    ' \
  | uniq

The output looks something like this:

404,"/api/v1/users/..."
403,"/admin/..."
429,"/api/v1/search"
500,"/api/v1/orders/..."

What this does

The command performs three simple steps:

Reads all matching access log files.
Filters out the expected HTTP status codes (200, 206, 302, 304, 308, and 101).
Outputs the remaining status code together with the requested URI as CSV.

Finally, uniq removes duplicate entries so you get a concise overview instead of thousands of repeated requests.

Why this is useful

This is an easy way to spot things like:

Missing routes (404)
Authorization issues (401/403)
Rate limiting (429)
Unexpected server errors (500)
Any other response that deserves investigation

Instead of searching through millions of log lines, you immediately get a list of unique problem endpoints.

A small improvement

If your logs aren't already grouped, consider sorting before calling uniq:

cat access_*.log \
  | jq -r '
      select(
        .status != 200 and
        .status != 206 and
        .status != 302 and
        .status != 304 and
        .status != 308 and
        .status != 101
      )
      | [.status, .request.uri]
      | @csv
    ' \
  | sort \
  | uniq

Or, even shorter:

...
| sort -u

It's a simple one-liner, but it's become one of my favourite ways to quickly identify unexpected responses in production logs.

#tools #logging #http

🐥 Katagelophobia and cyber security: when fear of ridicule becomes a vulnerability

2026-07-25T17:00:00Z

Katagelophobia — the fear of being laughed at — is rarely discussed in technical circles, yet it quietly shapes how people behave in security-critical situations. In cyber security, where human judgment is often the last line of defense, this fear can become an unexpected attack surface.

The hidden driver behind silence

Security incidents are often preceded by small signals:

A suspicious email that “looks off”
An unexpected MFA prompt
A system behaving slightly differently than usual

In an ideal environment, users report these signals immediately. In reality, many hesitate. Katagelophobia plays a role here: people avoid speaking up because they fear being wrong, overreacting, or being perceived as inexperienced.

This creates a dangerous dynamic: attackers rely on hesitation.

Social engineering thrives on psychological pressure

Modern phishing and social engineering attacks are designed to exploit emotion, not logic. Urgency, authority, and fear are well-known tactics, but fear of embarrassment is equally powerful.

Examples:

“This is urgent, don’t escalate”
“Only you can fix this quickly”
“Please don’t involve others yet”

These cues discourage validation and collaboration. A user already prone to avoiding ridicule is more likely to comply silently rather than question the request.

Organizational culture as a security control

Technical defenses can’t compensate for a culture where people are afraid to ask questions.

Teams that unintentionally reward “knowing everything” or penalize mistakes create an environment where katagelophobia flourishes. The result:

Underreporting of incidents
Delayed response times
Increased dwell time for attackers

In contrast, strong security cultures normalize uncertainty:

“If in doubt, report it” is actively reinforced
False positives are treated as learning opportunities
Junior and non-technical staff feel safe raising concerns

Designing systems that reduce fear

You can’t eliminate psychological traits, but you can design around them.

Practical approaches:

1. Lower the cost of being wrong

Make reporting trivial and low-friction:

One-click “Report phishing” buttons
Dedicated Slack/Teams channels
No requirement to justify suspicion

The easier it is, the less overthinking occurs.

2. Remove judgment from feedback loops

Avoid responses like:

“This was obviously safe”
“You should have known this”

Instead:

Thank the report
Explain briefly
Reinforce that reporting was correct behavior

3. Simulate safely

Phishing simulations shouldn’t shame users. If people feel tested rather than trained, katagelophobia increases.

Focus on:

Education over scoring
Trends over individual performance
Private feedback instead of public metrics

4. Lead by example

When senior engineers or leadership openly admit uncertainty or mistakes, it sets a powerful precedent.

Security improves when saying “I’m not sure” becomes acceptable.

The human layer is not optional

Cyber security discussions often focus on zero-days, encryption, and infrastructure hardening. Yet many breaches still start with a simple human interaction.

Katagelophobia highlights a key reality: people don’t just fail because they lack knowledge — they fail because of social pressure.

Addressing that pressure is not “soft” work. It’s a core part of building resilient systems.

Closing thought

Attackers exploit whatever works. If fear of ridicule prevents someone from reporting a suspicious email, that fear becomes part of the attack chain.

Reducing that fear may be one of the simplest — and most overlooked — security improvements you can make.

#best-practice #auth

🐥 Request::input() vs Request::string() in Laravel

2026-07-20T17:00:00Z

Laravel provides several ways to retrieve input from an HTTP request. Two of the most commonly used methods are input() and string(). While they look similar, they have an important difference that can make your code safer and more expressive.

`input()` returns mixed values

The input() method returns the raw value from the request.


$name = $request->input('name');

The return type depends entirely on the submitted data:

string
array
int
bool
null

This makes input() the right choice when you don't know or don't care about the exact type, or when you're expecting something other than a string.

For example:


$ids = $request->input('ids', []);
$published = $request->input('published', false);

`string()` always returns a Stringable object

If you expect textual input, string() is often the better choice.


$name = $request->string('name');

Instead of returning a plain string, it returns an Illuminate\Support\Stringable instance, allowing you to immediately chain string operations.


$username = $request
    ->string('username')
    ->trim()
    ->lower()
    ->replace(' ', '-');

No need for nested Str::of() calls or temporary variables.

Better type safety

One subtle advantage is that string() guarantees you're working with a string-like value.

With input(), it's easy to accidentally call a string function on an array:


// Potentially problematic
$name = trim($request->input('name'));

If name unexpectedly contains an array, PHP will throw a type error.

Using string() makes your intent explicit:


$name = $request
    ->string('name')
    ->trim()
    ->toString();

When to use which

As a general guideline:

Use input() when:

You expect arrays, booleans, integers, or mixed data.
You simply need the raw request value.

Use string() when:

You expect text input.
You plan to manipulate the value.
You want more expressive, fluent code.
You want to make your intent clear to future readers.

My recommendation

For textual fields like names, email addresses, search queries, slugs, or titles, prefer string(). It communicates that the value is expected to be text and gives you Laravel's fluent string API for free.

Reserve input() for cases where the value may legitimately be another type, such as arrays from multi-select fields, boolean flags, or numeric values.

It's a small change, but one that makes your codebase a little more readable, a little safer, and a little more idiomatic.

#laravel #php #best-practice

🐥 How to start and enable ClamAV service on Linux

2026-07-07T17:00:00Z

ClamAV is a powerful, open-source antivirus engine designed to detect malware, viruses, and trojans on Linux systems. If you have just installed it, you need to start its background services so that it can protect your system and keep its virus definitions up to date.

This guide will show you how to start, enable, and verify the ClamAV services using systemctl.

Prerequisites

Before running the commands, ensure you have:

A Linux distribution installed.
Administrative privileges (sudo or root access).

Step 1: Start the Antivirus Daemon

The primary service is clamav-daemon. This background process loads virus signatures into memory and handles on-access scanning. It also allows you to run high-speed scans using the clamdscan utility.

Run the following commands in your terminal:

Start the service:
```
sudo systemctl start clamav-daemon
```
Enable it to start automatically on boot:
```
sudo systemctl enable clamav-daemon
```
Verify that it is running correctly:
```
sudo systemctl status clamav-daemon
```

💡 Note: The daemon might take a few moments to change to an "active" status. This delay happens because it is loading a large database of malware definitions directly into your system's RAM.

Step 2: Start the Automatic Database Updater

An antivirus engine is only as good as its virus signatures. ClamAV uses a separate service called clamav-freshclam to look for and download the latest malware definitions automatically.

To get the updater running, execute these commands:

Start the updater service:
```
sudo systemctl start clamav-freshclam
```
Enable the updater to start on boot:
```
sudo systemctl enable clamav-freshclam
```
Verify the update status:
```
sudo systemctl status clamav-freshclam
```

Conclusion

Your Linux machine is now running ClamAV in the background with continuous, automated database updates. You can now safely run manual system scans or configure real-time protection.

#terminal #linux #sysadmin

🐥 Fixing Korean PDF rendering issues in Poppler on Ubuntu 22.04

2026-07-05T17:00:00Z

Poppler’s pdftoppm tool is commonly used to render PDFs into images for processing, previews, or conversion pipelines. On Ubuntu 22.04, the default Poppler version (22.02) can run into issues when rendering PDFs that contain Korean text, especially when fonts are missing or incorrectly substituted.

This post explains a practical fix that avoids upgrading Poppler entirely.

The problem

When rendering certain Korean PDFs with pdftoppm, the output may show:

missing characters (blank boxes or “tofu” glyphs)
incorrect font substitution
broken or unreadable text rendering

This is often not caused by Poppler itself, but by missing font configuration and CMap data required for CJK (Chinese, Japanese, Korean) text handling.

The root cause

Poppler relies on the system font stack and external mapping data:

Fontconfig: resolves font substitutions
CJK fonts: provide actual glyphs for Korean characters
Poppler CMap data: maps PDF encodings to Unicode correctly

On a minimal Ubuntu server installation, two key components are often missing:

poppler-data
CJK font packages such as Noto Sans CJK

The fix

Instead of upgrading Poppler, installing the correct font dependencies resolves the issue.

1. Install CJK fonts

sudo apt install fonts-noto-cjk fonts-noto-core fonts-unifont

2. Install Poppler CMap data

sudo apt install poppler-data

3. Rebuild the font cache

fc-cache -fv

Verify the fix

Re-run your rendering command:

pdftoppm input.pdf output

Korean text should now render correctly in the generated images.

Why this works

Poppler 22.x is capable of rendering CJK PDFs correctly, but only when:

the system provides appropriate glyph fonts
CMap mappings are available via poppler-data

Without these, Poppler falls back to incomplete or incorrect font substitution, leading to broken output.

When you actually need a newer Poppler

Upgrading Poppler is only necessary if:

rendering logic itself is broken (rare for CJK issues)
you need specific bug fixes or features in newer releases

For font-related issues, upgrading is usually unnecessary and introduces avoidable dependency complexity.

Conclusion

If you encounter broken Korean text rendering in pdftoppm on Ubuntu 22.04, the fix is typically not a Poppler upgrade but a missing font stack.

Installing fonts-noto-cjk and poppler-data resolves most cases immediately and keeps your system stable without rebuilding core libraries.

#pdf #best-practice #linux #sysadmin

🐥 The MySQL null-safe equality operator: <=>

2026-06-18T17:00:00Z

If you've worked with MySQL long enough, you've probably been bitten by NULL comparisons at least once. A query that should return results returns nothing. A WHERE clause that should exclude a row doesn't. The culprit is almost always the three-valued logic of SQL — and the null-safe equality operator <=> is one of the cleanest tools for dealing with it.

The problem with `=` and `NULL`

In SQL, NULL represents the absence of a value — the unknown. Because of this, any comparison involving NULL using the standard = operator yields NULL (not TRUE or FALSE), and NULL is falsy in a WHERE clause.

SELECT NULL = NULL;   -- NULL
SELECT NULL = 1;      -- NULL
SELECT 1 = 1;         -- 1 (TRUE)

This means:

SELECT * FROM users WHERE deleted_at = NULL;  -- returns nothing

The idiomatic fix is IS NULL / IS NOT NULL, but that only works for literal null checks. The moment you're comparing two columns — one or both of which might be NULL — things get awkward fast.

-- This silently drops rows where either column is NULL
SELECT * FROM orders WHERE shipping_address = billing_address;

Enter `<=>`

MySQL's null-safe equality operator <=> behaves exactly like =, except it treats NULL as a comparable value. Two NULLs are considered equal, and a NULL compared to any non-null value is FALSE.

SELECT NULL <=> NULL;   -- 1 (TRUE)
SELECT NULL <=> 1;      -- 0 (FALSE)
SELECT 1 <=> 1;         -- 1 (TRUE)
SELECT 1 <=> 2;         -- 0 (FALSE)

This makes it safe to compare nullable columns directly:

-- Correctly includes rows where both columns are NULL
SELECT * FROM orders WHERE shipping_address <=> billing_address;

A real-world example

Consider a polymorphic subscriptions join table that maps users to subscribable entities (posts, documents, threads, etc.). The goal: return all subscribers excluding the item's author, even when author_id might be NULL.

The naive approach breaks silently:

WHERE subscriptions.user_id != posts.author_id

When author_id is NULL, this evaluates to NULL, so the row is dropped — meaning a user who subscribes to a post with no author would incorrectly disappear from the result.

The null-safe fix:

NOT (subscriptions.user_id <=> (
    SELECT author_id FROM posts WHERE id = subscriptions.item_id
))

This reads as: "keep this row unless the subscriber's user_id exactly matches the author_id, treating NULL as a concrete equal value." When author_id is NULL and user_id is not, the <=> returns FALSE, so NOT FALSE is TRUE — the row is kept. Correct behaviour in all cases.

`<=>` vs. the alternatives

Approach	Handles NULL?	Readable?	Standard SQL?
`col = val`	No	Yes	Yes
`col IS NULL`	Yes (literal only)	Yes	Yes
`COALESCE(col, '') = COALESCE(val, '')`	Yes (with sentinel)	Passable	Yes
`(col = val OR (col IS NULL AND val IS NULL))`	Yes	Verbose	Yes
`col <=> val`	Yes	Yes	No (MySQL/MariaDB)

The COALESCE sentinel approach is fragile — you need to pick a value that can never appear in real data. The verbose OR (IS NULL AND IS NULL) pattern works but is noisy. <=> wins on brevity and correctness, at the cost of portability.

When to use it

<=> is a good fit when:

Comparing two nullable columns directly in a WHERE or JOIN condition.
Negating equality on nullable data (NOT (a <=> b) is cleaner than the alternative).
Upsert / deduplication queries where you need exact matching including null identity.
Generated columns or audit logic where you want to detect whether a value actually changed, including transitions to/from NULL.

Caveats

MySQL and MariaDB only. The <=> operator is not part of the SQL standard and is not available in PostgreSQL, SQLite, or SQL Server. If your codebase runs tests against SQLite (a common Laravel setup), any <=> in a raw query will fail there.

PostgreSQL's equivalent is IS NOT DISTINCT FROM / IS DISTINCT FROM:

-- PostgreSQL equivalent of <=>
col IS NOT DISTINCT FROM val

-- PostgreSQL equivalent of NOT (col <=> val)
col IS DISTINCT FROM val

If cross-database portability matters, abstract the comparison behind a query scope or use the verbose but portable OR (IS NULL AND IS NULL) form.

Summary

The null-safe equality operator <=> is one of those small MySQL features that, once you know it exists, saves you from a whole class of subtle bugs. It's most valuable when you need to compare nullable columns directly — particularly in negated conditions where the standard != would silently swallow NULL rows. Just keep portability in mind: it's a MySQL/MariaDB extension, so make sure your test database matches your production database before reaching for it.

#best-practice #mysql #sql

🐥 Testing that Laravel events fire after a transaction commits

2026-06-16T17:00:00Z

A common source of bugs in Laravel applications is dispatching events inside a database transaction. Listeners often kick off their own queries or even their own transactions — and if the outer transaction hasn't committed yet, you can end up with deadlocks, stale reads, or listeners that act on data that gets rolled back.

The fix is straightforward: dispatch events after the transaction commits. But how do you write a test that actually enforces this? Here's a reusable pattern.

The problem

Consider an OrderAction that saves an order inside a transaction and then fires an OrderWasPlaced event:


final class PlaceOrderAction
{
    public function __invoke(Cart $cart): Order
    {
        DB::transaction(function () use ($cart) {
            $order = Order::create([...]);
            $order->lines()->createMany($cart->lines());
            
            // ❌ Event fired inside the transaction — listeners run
            //    while the order row is still locked.
            Event::dispatch(new OrderWasPlaced($order));
        });
    }
}

Any listener that tries to read the same rows will block (or deadlock) because the transaction still holds row locks.

The correct version moves the dispatch outside:


final class PlaceOrderAction
{
    public function __invoke(Cart $cart): Order
    {
        $order = null;

        DB::transaction(function () use ($cart, &$order) {
            $order = Order::create([...]);
            $order->lines()->createMany($cart->lines());
        });

        // ✅ Transaction has committed — listeners can safely read/write.
        Event::dispatch(new OrderWasPlaced($order));

        return $order;
    }
}

The test pattern

The key idea: register a real event listener before the action runs, and capture DB::transactionLevel() at the moment the event fires. If the event fires inside the transaction the level will be elevated; if it fires after the commit it will match the baseline.


use App\Actions\PlaceOrderAction;
use App\Events\OrderWasPlaced;
use Illuminate\Support\Facades\Bus;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Event;
use PHPUnit\Framework\Attributes\Test;
use Tests\TestCase;

final class PlaceOrderActionTest extends TestCase
{
    #[Test]
    public function it_dispatches_order_was_placed_after_the_transaction_commits(): void
    {
        // Fake jobs/queues so side-effect listeners don't cascade.
        Bus::fake();

        // Capture the DB nesting depth before the action runs.
        // LazilyRefreshDatabase / DatabaseTransactions wraps every test
        // in its own transaction, so the baseline is typically 1, not 0.
        $baselineLevel = DB::transactionLevel();
        $levelAtDispatch = null;

        // Register a real listener — do NOT call Event::fake(), otherwise
        // the dispatcher is replaced with a mock and no listeners run.
        Event::listen(OrderWasPlaced::class, function () use (&$levelAtDispatch) {
            $levelAtDispatch = DB::transactionLevel();
        });

        $cart = Cart::factory()->withLines(3)->create();

        app(PlaceOrderAction::class)($cart);

        $this->assertEquals(
            $baselineLevel,
            $levelAtDispatch,
            'OrderWasPlaced must be dispatched after the transaction commits, not inside it.',
        );
    }
}

Why compare against `$baselineLevel` instead of `0`?

Most Laravel test suites use LazilyRefreshDatabase or DatabaseTransactions, which wrap every test in an outer transaction for easy rollback. That means DB::transactionLevel() starts at 1 when the test body begins — not 0. Comparing against the snapshot taken before the action runs is always correct, regardless of your test database strategy.

Handling listener cascades

Sometimes the event you want to observe triggers further actions that write to the database, causing failures when those writes reference data that doesn't exist in the current test context. Two strategies:

1. Fake only the cascading action

If a listener dispatches a secondary action (e.g. StartFulfillmentAction), fake just that class so the chain stops there while leaving the event dispatcher intact:


Action::fake(StartFulfillmentAction::class);

The event still fires and your transaction-level listener still runs.

2. Fake only the jobs

If listeners queue jobs (Horizon, etc.), Bus::fake() is usually enough to prevent the cascade without touching the event system at all.

Checking multiple events

To assert that all the events fired by an action respect the post-commit invariant, collect them all in a single map:


$levels = [];

foreach ([OrderWasPlaced::class, InventoryReserved::class, InvoiceQueued::class] as $eventClass) {
    Event::listen($eventClass, function () use ($eventClass, &$levels) {
        $levels[$eventClass] = DB::transactionLevel();
    });
}

app(PlaceOrderAction::class)($cart);

foreach (array_keys($levels) as $eventClass) {
    $this->assertEquals(
        $baselineLevel,
        $levels[$eventClass],
        "{$eventClass} must be dispatched outside the transaction.",
    );
}

// Also assert every expected event actually fired.
$this->assertSame(
    [OrderWasPlaced::class, InventoryReserved::class, InvoiceQueued::class],
    array_keys($levels),
);

Quick reference

Scenario	Approach
Single event	`Event::listen()` + `DB::transactionLevel()` snapshot
Multiple events	Loop over event classes, collect levels into a map
Listener cascade breaks the test	`Action::fake(CascadingAction::class)` or `Bus::fake()`
Using `Event::fake()`	❌ Replaces the dispatcher — listeners never run, pattern breaks
Transaction depth varies by test setup	Always snapshot `$baselineLevel` before the action, never hardcode `0`

The pattern is lightweight — no mocking frameworks, no custom test doubles, just a listener closure and a single assertion. Once you've added it to one action test, it's easy to copy across the codebase anywhere you need to enforce the same invariant.

#laravel #database #best-practice #testing

🐥 Speeding up S3 uploads in GitHub Actions with Bash parallelism

2026-06-14T17:00:00Z

When you're uploading multiple directories to S3 (or an S3-compatible CDN like DigitalOcean Spaces) in a CI pipeline, the naive approach runs each upload sequentially. If you have three directories and each takes 30 seconds, you're waiting 90 seconds. They're completely independent — there's no reason not to run them at the same time.

Here's how to parallelize them with nothing but bash.

The problem

A typical multi-directory upload step looks like this:

- name: Upload to CDN
  run: |
    s3cmd put public/build s3://my-bucket/assets/ --recursive --acl-public
    s3cmd put public/img   s3://my-bucket/assets/ --recursive --acl-public
    s3cmd put public/js    s3://my-bucket/assets/ --recursive --acl-public

Each s3cmd put blocks until it's done before the next one starts. Wall-clock time = sum of all three.

The fix

- name: Upload to CDN
  run: |
    pids=()
    s3cmd put public/build s3://my-bucket/assets/ --recursive --acl-public & pids+=($!)
    s3cmd put public/img   s3://my-bucket/assets/ --recursive --acl-public & pids+=($!)
    s3cmd put public/js    s3://my-bucket/assets/ --recursive --acl-public & pids+=($!)
    for pid in "${pids[@]}"; do wait "$pid" || exit 1; done

Wall-clock time = duration of the slowest upload.

How it works

& pids+=($!) — The & runs the command in the background. $! is bash's special variable for the PID of the last backgrounded process, and we immediately append it to the pids array before starting the next job.

for pid in "${pids[@]}"; do wait "$pid" || exit 1; done — We wait for each background job by PID and fail the step immediately if any one of them exits with a non-zero code. This is important: a plain wait without arguments returns the exit code of the last process it waited for, which means a failure in the first or second upload could go undetected.

Why not just `wait`?

# Dangerous — only checks the exit code of the last job
s3cmd put public/build ... &
s3cmd put public/img   ... &
s3cmd put public/js    ... &
wait

If public/build fails but public/js succeeds, this exits 0 and your CI run goes green with a broken CDN.

Waiting by PID and checking each one individually gives you the same safety guarantee as running sequentially, at the speed of the fastest possible parallel execution.

The general pattern

This technique works for any set of independent shell commands you want to parallelize:

pids=()
some-command arg1 & pids+=($!)
some-command arg2 & pids+=($!)
some-command arg3 & pids+=($!)
for pid in "${pids[@]}"; do wait "$pid" || exit 1; done

No extra tooling, no GNU Parallel, no xargs — just bash.

#tools #devops #github

🐥 Debugging PHPUnit notices in Laravel parallel tests

2026-06-12T17:00:00Z

Running Laravel's test suite in parallel speeds things up considerably, but it also makes it easy to miss PHPUnit notices. The parallel worker output gets interleaved and buffered, and notices about deprecated API usage or risky tests tend to scroll past unnoticed — or disappear entirely. This post shows the command I use to surface them reliably.

The command

LARAVEL_PARALLEL_TESTING=1 \
LARAVEL_PARALLEL_TESTING_RECREATE_DATABASES=1 \
vendor/brianium/paratest/bin/paratest \
  --colors=always \
  --configuration=/path/to/phpunit.xml \
  --runner=\\Illuminate\\Testing\\ParallelRunner \
  --display-phpunit-notices \
  --fail-on-all-issues \
  tests/Unit

What each part does

Environment variables

LARAVEL_PARALLEL_TESTING=1 activates Laravel's parallel testing support. It causes the framework to spin up separate database connections per worker (suffixed _1, _2, etc.) and seed each one independently.

LARAVEL_PARALLEL_TESTING_RECREATE_DATABASES=1 forces those databases to be dropped and recreated from scratch on every run. Without it, a previous run's leftover state can cause tests to pass or fail for the wrong reasons — particularly relevant when you change a migration between runs.

Invoking paratest directly

Laravel's php artisan test --parallel is a thin wrapper around brianium/paratest. Calling the binary directly gives you access to flags that the Artisan wrapper doesn't expose, particularly the notice-related ones below.

--runner=\\Illuminate\\Testing\\ParallelRunner tells paratest to use Laravel's own runner class, which handles the database token injection and other framework-specific setup that the default runner skips.

--configuration takes an absolute path to your phpunit.xml. When you invoke paratest from outside the project root — for example, from a CI script or a Makefile — relative paths silently resolve to the wrong location. Using an absolute path avoids that class of silent misconfiguration.

Surfacing notices

--display-phpunit-notices is the key flag. PHPUnit emits notices for things like:

calls to deprecated assertion methods (assertContains on a string instead of assertStringContainsString)
tests marked @covers that cover no code
tests with no assertions when beStrictAboutTestsThatDoNotTestAnything is enabled

In a parallel run these notices are buffered per worker and often never reach the terminal. This flag ensures they are printed to output regardless.

--fail-on-all-issues treats any notice, warning, or deprecation as a test suite failure. This is what makes the command useful for CI: the exit code becomes non-zero the moment any worker emits a notice, so the pipeline fails and forces you to deal with it rather than letting it accumulate.

Scoping to `tests/Unit`

Passing tests/Unit as the path restricts the run to unit tests, which tends to surface notices faster than a full suite run because unit tests don't need a running server or real database queries. Once you've cleared the unit test notices, you can repeat with tests/Feature or omit the path entirely.

Reading the output

When a notice fires, paratest prints it alongside the failing worker output. The format looks like:

NOTICE  tests/Unit/Services/OrderServiceTest.php:42
Method Illuminate\Testing\Assert::assertContains() is deprecated. Use assertStringContainsString() instead.

The file path and line number point directly to the test method that triggered it. If you see the same notice repeating across many tests, the issue is usually in a shared base class or a trait — check the stack trace for the actual call site rather than fixing every test individually.

Making it a habit

The goal is to run with --fail-on-all-issues in CI from the start, before notices accumulate. If you're adding this to an existing codebase, it's usually easier to tackle notices test-file by test-file: run against a single file first, fix what you find, then broaden the path incrementally.

# Start with one file
LARAVEL_PARALLEL_TESTING=1 \
vendor/brianium/paratest/bin/paratest \
  --runner=\\Illuminate\\Testing\\ParallelRunner \
  --display-phpunit-notices \
  --fail-on-all-issues \
  tests/Unit/Services/OrderServiceTest.php

Once the file is clean, commit and move on to the next. The --fail-on-all-issues flag acts as a ratchet: once a file is notice-free, CI will catch any regression immediately.

#laravel #php #best-practice #testing

🐥 From N+1 to 1: Extracting paged PDF text with a single pdftotext call

2026-06-10T17:00:00Z

When building full-text search for uploaded documents, we needed to extract text page-by-page from PDFs so we could index each page as a separate chunk. The naive approach worked but was painfully slow. Here's how a single Unix insight cut it down to one process spawn.

The problem: N+1 process spawns

pdftotext is the standard Unix utility for extracting text from PDFs. It supports a -f (first page) and -l (last page) flag, so extracting a single page looks like this:

pdftotext -f 3 -l 3 document.pdf -

The natural implementation for paged extraction is to call this in a loop:


public function getPagedTextFromFile(string $path): Collection
{
    $pagedText = collect();

    $pageCount = $this->pdfInfoService->getPageCountFromFile($path);

    for ($i = 1; $i <= $pageCount; $i++) {
        $pagedText->put($i, $this->getRawTextFromFile($path, $i));
    }

    return $pagedText;
}

This is clean and obvious. It is also, for any document with real content, expensive:

1 pdfinfo call to get the page count
N pdftotext calls, one per page

Each call is a separate process spawn: the OS forks, loads the binary, opens the PDF, seeks to the requested page, extracts text, and exits. For a 50-page contract, that's 51 process spawns. On a server handling concurrent uploads, those 51 spawns happen in sequence, blocking the queue worker the entire time.

The insight: pdftotext already separates pages

Run pdftotext without page flags and pipe the output to a hex viewer:

pdftotext document.pdf - | cat -A | grep -P '\f'

You'll see form feed characters (\f, \x0C, ASCII 12) separating each page. This is standard — pdftotext has always done this. It's even documented in the man page, buried under the output format description.

That means the full multi-page text is already structured. We don't need N calls. We need one call and a string split.

The solution


public function getPagedTextFromFile(string $path): Collection
{
    $pagedText = collect();

    try {
        if ($this->mimeTypeForPath($path) !== 'application/pdf') {
            return $pagedText;
        }

        $result = $this->runExternalProcess(
            [config('pdftools.pdftotext'), $path, '-'],
            180
        );

        if ($result->getStdErr() !== '') {
            throw new Exception($result->getStdErr());
        }

        // pdftotext separates pages with \f; rtrim strips any optional trailing \f
        $pages = explode("\f", rtrim($result->getStdOut(), "\f"));

        foreach ($pages as $i => $pageText) {
            $pageText = trim($pageText, " \t\n\r\0\x0B");

            if ($pageText !== '') {
                $lines = $this->splitInLines($pageText);
                if (!empty($lines) && preg_match(self::DOCUSIGN_HEADER_PATTERN, $lines[0])) {
                    array_shift($lines);
                    $pageText = trim(implode("\n", $lines), " \t\n\r\0\x0B");
                }
            }

            $pagedText->put($i + 1, $pageText !== '' ? $pageText : null);
        }
    } catch (Exception $e) {
        Log::error("pdftotext | {$e->getMessage()}");
    }

    return $pagedText;
}

Before: 51 process spawns for a 50-page PDF.
After: 1 process spawn, regardless of page count.

For a 100-page document the old approach invoked pdftotext 100 times, each time reloading and re-parsing the entire PDF file to seek to one page. The new approach loads it once and returns everything.

Edge cases worth knowing

Trailing form feed. Some versions of pdftotext append a \f after the last page. Splitting "page1\fpage2\f" naively gives ["page1", "page2", ""] — an extra empty element. The rtrim($output, "\f") before splitting removes it cleanly.

Blank pages. A blank page produces an empty string after trimming. Storing null for it preserves correct page numbering for subsequent pages (page 5 stays page 5, even if pages 3 and 4 are blank). The downstream indexing code skips nulls, so blank pages don't pollute the search index.

Page-level headers. We strip DocuSign envelope headers (DocuSign Envelope ID: XXXXXXXX-...) from the beginning of any page that has one. Since the single-call output is split by page before this check, the per-page stripping logic is identical to the per-call approach — just applied after the split instead of inside each getRawTextFromFile call.

Non-PDF files. The MIME type check at the top of the method returns early for anything that isn't application/pdf. This replaces the earlier dependency on pdfinfo to get the page count — for non-PDFs that check would have returned 0 and short-circuited the loop, but the MIME check is simpler and removes the pdfinfo dependency from this path entirely.

The broader pattern

This is an instance of a general optimisation: if a tool is designed to process a whole file, don't call it once per chunk. The tool already knows how to walk the file efficiently; let it.

The same principle applies to:

ffprobe for video metadata — one call for all streams, not one call per stream
exiftool — batch mode processes a directory in one pass rather than per-file invocations
Database queries — SELECT with IN (...) instead of N individual selects
Meilisearch document uploads — one addDocuments call with a batch, not one call per document

In each case the per-item call pattern feels natural and is easy to reason about. But the overhead of repeatedly invoking a tool that was designed for whole-file processing accumulates fast once documents are large or queues are busy.

The fix, when it exists, is usually as simple as this one: read the man page, find the output format, split a string.

#pdf #development #php #tools #best-practice

🐥 The silent saboteur: how --quiet can break your Laravel command sequences

2026-06-08T17:00:00Z

When you chain multiple Artisan commands together in Laravel, there's a subtle trap waiting for you — one that's easy to miss because the commands still run, they just stop talking.

The setup

A common pattern in Laravel applications is a "meta-command" that runs a sequence of other commands in order. Think of a nightly job runner:


public function handle(): int
{
    $commandsToRun = [
        SomeCommand::class        => [],
        AnotherCommand::class     => [],
        DatabasePruneCommand::class => ['--quiet' => true],
        MoreCommands::class       => [],
        FinalCommand::class       => [],
    ];

    foreach ($commandsToRun as $commandToRun => $parameters) {
        $this->output->title($commandToRun);
        Artisan::call($commandToRun, $parameters, $this->output);
    }

    return self::SUCCESS;
}

You pass $this->output into each Artisan::call so the sub-command's output flows through to the console. Clean and straightforward.

The problem

One of your commands — say, a prune command — is noisy by default. You don't want its output cluttering the logs, so you pass '--quiet' => true. The title still appears, the command runs, everything looks fine.

Then you notice something odd: the commands after that quiet one stop printing their titles. The log files confirm they're running, but the console goes dark after that one --quiet call.

Why it happens

When Laravel processes the --quiet flag on a command, it calls:


$output->setVerbosity(OutputInterface::VERBOSITY_QUIET);

The key word is on the output object you passed in. Because you shared $this->output across all Artisan::call invocations, that single setVerbosity call mutates the object in place. Every subsequent command — and every title(), info(), or line() call — now runs against a quietly-configured output. They're suppressed silently, with no error.

The commands still execute; they just can't speak.

The fix

Capture the verbosity before each call and restore it immediately after:


foreach ($commandsToRun as $commandToRun => $parameters) {
    $this->output->title($commandToRun);

    $verbosity = $this->output->getVerbosity();
    Artisan::call($commandToRun, $parameters, $this->output);
    $this->output->setVerbosity($verbosity);
}

Two lines. The sub-command can do whatever it likes to the output during its run; your sequence always gets a clean slate for the next iteration.

The lesson

Shared mutable objects are the classic source of action-at-a-distance bugs. The output object here is a perfect example: you pass it in expecting read-like behaviour (writing to the terminal), but the callee has full write access to its configuration too.

A few takeaways:

Passing an object "for output" also grants mutation rights. Laravel's OutputStyle is stateful — verbosity, decorations, and more can be changed by anyone holding a reference.
Commands that still run aren't necessarily commands that are working correctly. Silent output suppression looks identical to "nothing went wrong" if you're only checking exit codes or log files.
Test the observable side effects, not just execution. A test that asserts title() is called N times would have caught this immediately; a test that only checks Artisan::call was invoked N times would not.

Bonus: write the test first

If you'd written this test before the bug appeared, you'd have been protected from day one:


public function it_restores_verbosity_so_quiet_commands_do_not_suppress_subsequent_output(): void
{
    // Simulate a sub-command that sets quiet mode
    Artisan::shouldReceive('call')
        ->twice()
        ->andReturnUsing(function () use ($output): int {
            $output->setVerbosity(OutputInterface::VERBOSITY_QUIET);
            return 0;
        });

    // Both titles must still appear
    $output->expects($this->exactly(2))->method('title');

    $this->subject->runSequenceOfCommands($command, [
        'first:command'  => [],
        'second:command' => [],
    ]);
}

Shared mutable state is everywhere in framework code. A little defensive save-and-restore goes a long way.

#development #laravel #best-practice

🐥 How to check how much memory a systemd unit is actually using

2026-06-06T17:00:00Z

When investigating memory usage on a Linux server, it's common to look at processes using tools such as top, htop, or ps. However, when applications are managed by systemd, it is often more useful to inspect memory usage at the service level rather than at the individual process level.

This is especially important for services that spawn multiple worker processes.

Using systemctl

The easiest way to see the memory usage of a systemd unit is:

systemctl status my-service.service

Example output:

Memory: 423.1M
CPU: 12min 34.567s

This value represents the total memory usage of the service's cgroup, including all child processes managed by the unit.

Using systemctl show

For scripting and automation, use:

systemctl show my-service.service --property=MemoryCurrent

Example:

MemoryCurrent=443662336

The value is returned in bytes.

To convert it to megabytes:

systemctl show my-service.service --property=MemoryCurrent --value \
    | awk '{ printf "%.2f MB\n", $1 / 1024 / 1024 }'

Inspecting the cgroup directly

Systemd tracks resource usage through Linux cgroups.

You can inspect the memory consumption directly:

cat /sys/fs/cgroup/system.slice/my-service.service/memory.current

Or locate the cgroup first:

systemctl show my-service.service --property=ControlGroup

Example:

ControlGroup=/system.slice/my-service.service

Then inspect the corresponding cgroup files.

Finding the processes behind a service

To see which processes belong to a unit:

systemctl status my-service.service

systemctl show my-service.service --property=MainPID

You can then inspect individual processes:

ps -o pid,rss,vsz,cmd -p <PID>

Keep in mind that summing RSS values from multiple processes can overestimate actual memory usage because shared memory pages may be counted multiple times.

Monitoring memory usage over time

For live monitoring:

watch -n 1 \
    'systemctl show my-service.service --property=MemoryCurrent --value'

Or use:

systemd-cgtop

This tool displays CPU, memory, and I/O usage per cgroup and is often the most convenient way to identify memory-hungry services on a system.

Configuring memory limits

Systemd can also enforce memory limits.

Example:

[Service]
MemoryMax=1G

After updating the unit file:

sudo systemctl daemon-reload
sudo systemctl restart my-service.service

To verify the configured limit:

systemctl show my-service.service --property=MemoryMax

Conclusion

For services managed by systemd, MemoryCurrent is usually the most accurate representation of how much memory the service is actually consuming. Unlike process-level tools, it accounts for the entire cgroup, making it ideal for monitoring applications that use worker pools, background jobs, or multiple child processes.

Useful commands to remember:

systemctl status my-service.service
systemctl show my-service.service --property=MemoryCurrent
systemd-cgtop

These provide a much clearer picture of service-level memory usage than inspecting individual processes alone.

#best-practice #linux #sysadmin

🐥 Fixing PHPUnit 13 with*() without expects() deprecations in Laravel tests

2026-06-04T17:00:00Z

After upgrading to PHPUnit 13, you may run into this deprecation warning in Laravel test suites:

Using with*() without expects() is deprecated and will no longer be possible in PHPUnit 14.

At first sight, the code often already looks correct:


$mock->expects($this->once())
    ->method('runSequenceOfCommands')
    ->with([...]);

So what is actually causing the warning?

The real issue

In many Laravel applications, the problem comes from using Laravel’s createPartialMock() helper together with newer PHPUnit versions.

Example:


$class = $this->createPartialMock(
    NightlyCommand::class,
    ['runSequenceOfCommands']
);

While this worked fine in older PHPUnit versions, PHPUnit 13 tightened internal mock expectation handling and now emits deprecation warnings in some cases when using Laravel’s wrapper helpers.

The fix

Instead of createPartialMock(), use PHPUnit’s native mock builder API.

Replace this:


$class = $this->createPartialMock(
    NightlyCommand::class,
    ['runSequenceOfCommands']
);

With this:


$class = $this->getMockBuilder(NightlyCommand::class)
    ->onlyMethods(['runSequenceOfCommands'])
    ->getMock();

And for multiple mocked methods:


$class = $this->getMockBuilder(NightlyCommand::class)
    ->onlyMethods([
        'runSequenceOfCommands',
        'confirm',
    ])
    ->getMock();

Why this works

getMockBuilder() uses PHPUnit’s modern native mock API directly, avoiding the legacy compatibility layer behind Laravel’s partial mock helpers.

This makes your tests:

compatible with PHPUnit 13
future-proof for PHPUnit 14
less dependent on framework-specific mock wrappers

Final result

Your expectations can stay exactly the same:


$class->expects($this->once())
    ->method('runSequenceOfCommands')
    ->with(
        $this->isInstanceOf(NightlyCommand::class),
        [
            // expected commands
        ]
    );

But by switching to getMockBuilder(), the deprecation warning disappears cleanly.

#laravel #php #best-practice #testing

🐥 Why crypto.getRandomValues() matters in JavaScript

2026-06-02T17:00:00Z

Generating random values sounds simple, until you need randomness that is actually secure.

A lot of JavaScript developers reach for Math.random() out of habit. While that works fine for visual effects, games, or non-critical IDs, it should never be used for anything security-sensitive.

That’s where the Web Crypto API comes in.

The problem with `Math.random()`

Math.random() is not cryptographically secure.

Its output is deterministic and predictable enough that an attacker may be able to reproduce or guess generated values under certain conditions.

That makes it unsuitable for:

Session tokens
Password reset links
API keys
CSRF tokens
Encryption keys
Secure identifiers

Example:

const id = Math.random().toString(36).slice(2)

This may look random, but it is not secure.

Using `crypto.getRandomValues()`

Modern browsers provide a secure random number generator through the Web Crypto API.

Example:

const bytes = new Uint8Array(16)

crypto.getRandomValues(bytes)

console.log(bytes)

This fills the typed array with cryptographically secure random bytes provided by the operating system.

Generating a secure random token

A common use case is generating secure tokens or identifiers.

function generateToken(length = 32) {
  const bytes = new Uint8Array(length)

  crypto.getRandomValues(bytes)

  return Array.from(bytes)
    .map(byte => byte.toString(16).padStart(2, "0"))
    .join("")
}

console.log(generateToken())

Example output:

4f8b7d3a1f9e0c8d7a2b6c5d4e3f1a9c

This is significantly safer than using Math.random().

Typed arrays are required

crypto.getRandomValues() only works with integer-based typed arrays.

Supported examples include:

new Uint8Array(32)
new Uint16Array(16)
new Int32Array(8)

This will fail:

crypto.getRandomValues([])

Because regular JavaScript arrays are not supported.

Browser and runtime support

crypto.getRandomValues() is widely supported in modern browsers.

It is also available in runtimes like:

Node.js
Deno
Bun

Example in Node.js:

const bytes = new Uint8Array(16)

crypto.getRandomValues(bytes)

console.log(bytes)

In older Node.js versions, developers typically used:

require("crypto").randomBytes(16)

UUID generation

If your goal is generating UUIDs, modern runtimes also support:

crypto.randomUUID()

Example:

550e8400-e29b-41d4-a716-446655440000

Internally, this also uses cryptographically secure randomness.

Things to remember

Use Math.random() for non-security-related randomness only
Use crypto.getRandomValues() for anything security-sensitive
Prefer crypto.randomUUID() when generating UUIDs
Always generate randomness using the operating system’s secure RNG

For modern JavaScript applications, crypto.getRandomValues() should be the default choice whenever security matters.

#javascript #best-practice #auth

🐥 How to find all Jira issues ever assigned to someone (even historical ones)

2026-05-31T17:00:00Z

If you've ever tried to pull up all the tickets that Sarah Mitchell worked on last year, you've probably started with the obvious query:

assignee = "sarah.mitchell@example.com"

And then realised it only shows her current tickets — not the ones she handed off to Tom Bergkamp or closed six months ago.

Here's the fix.

The `was` operator

Jira's JQL has a was operator that checks the change history of a field, not just its current value. So to find every issue Sarah was ever assigned to:

assignee was "sarah.mitchell@example.com"

Simple, but powerful.

Checking multiple people at once

Need to audit the work of an entire sub-team? Use was in with a list:

assignee was in ("sarah.mitchell@example.com", "tom.bergkamp@example.com", "priya.nair@example.com")

This returns any issue that was assigned to any of those three people at any point in time.

Scoping to a specific year

Combine it with the during clause to limit results to a time window — useful for annual reviews, retrospectives, or handover audits:

assignee was in ("sarah.mitchell@example.com", "tom.bergkamp@example.com") during ("2025-01-01", "2025-12-31")

This only matches issues where one of them was the assignee at some point during 2025 — even if the ticket has since been reassigned or closed.

Putting it all together

A full, practical query might look like this:

project = "PLATFORM"
AND assignee was in ("sarah.mitchell@example.com", "tom.bergkamp@example.com")
AND during ("2025-01-01", "2025-12-31")
ORDER BY updated DESC

The was operator works on other fields too — status was "In Progress", priority was "High", etc. — so it's worth keeping in your JQL toolkit whenever you need to query history rather than current state.

#tools #best-practice

🐥 Downloading external images as squares from a Phoenix app

2026-05-29T17:00:00Z

A common need in admin tools: click a button, download a remote image as a square JPEG. Simple enough — until CORS gets in the way.

The CORS problem

When fetching a cross-origin image and drawing it onto a , the browser marks the canvas as "tainted". The moment you call canvas.toBlob() to read the pixel data back out, it throws a security error. Unless the image server sends explicit CORS headers — which most don't — you can't do canvas operations on cross-origin images.

The fix: a server-side proxy

The solution is to proxy the image through the Phoenix app. From the browser's point of view the image comes from the same origin, so the canvas stays clean.

def image_proxy(conn, %{"url" => url}) do
  with true <- allowed_url?(url),
       {:ok, response} <- Req.get(url) do
    content_type =
      response.headers
      |> Map.get("content-type", ["image/jpeg"])
      |> List.first()
      |> String.split(";")
      |> hd()

    conn
    |> put_resp_content_type(content_type)
    |> send_resp(200, response.body)
  else
    false -> send_resp(conn, 400, "Invalid URL")
    _ -> send_resp(conn, 502, "Failed to fetch image")
  end
end

The allowed_url?/1 function validates the host against a known allowlist — an important SSRF guard so the proxy can't be abused to fetch arbitrary internal URLs.

Center-cropping to a square in the browser

Once the image loads from the proxy, a small Canvas API snippet handles the crop. The logic is straightforward: use the shorter dimension as the square size, then offset into the longer dimension by half the difference to take the center slice.

window.addEventListener("phx:download-square-image", (event) => {
  const { url, filename } = event.detail;
  const proxyUrl = `/image-proxy?url=${encodeURIComponent(url)}`;

  const img = new Image();
  img.onload = () => {
    const size = Math.min(img.naturalWidth, img.naturalHeight);
    const canvas = document.createElement("canvas");
    canvas.width = size;
    canvas.height = size;

    const ctx = canvas.getContext("2d");
    const offsetX = (img.naturalWidth - size) / 2;
    const offsetY = (img.naturalHeight - size) / 2;
    ctx.drawImage(img, offsetX, offsetY, size, size, 0, 0, size, size);

    canvas.toBlob((blob) => {
      const a = document.createElement("a");
      a.href = URL.createObjectURL(blob);
      a.download = filename;
      a.click();
      URL.revokeObjectURL(a.href);
    }, "image/jpeg", 0.95);
  };
  img.src = proxyUrl;
});

For a landscape image the left and right edges are trimmed, keeping the center. For a portrait image the top and bottom are trimmed instead. The full shorter dimension is always preserved — no upscaling, no padding.

Wiring it up in LiveView

The download is triggered from the template using Phoenix's JS.dispatch/2, which fires a custom DOM event with the image URL and filename as detail:

<button
  class="cursor-pointer"
  phx-click={
    JS.dispatch("phx:download-square-image",
      detail: %{url: @image_url, filename: "image.jpg"}
    )
  }
>
  Download
button>

No LiveView round-trip needed — JS.dispatch fires the event directly in the browser, the window listener catches it, and the download happens entirely client-side after the one proxy fetch.

Why `naturalWidth` instead of `width`?

img.width returns the CSS-rendered size, which is meaningless for an image that isn't attached to the DOM. img.naturalWidth always returns the actual pixel dimensions of the image data — the right value to use when doing pixel-level canvas operations.

#javascript #frontend #elixir #phoenix

🐥 The Tailwind `enabled:` selector trick for disabled buttons

2026-05-27T17:00:00Z

When you add a disabled attribute to a

🐥 Hide a layout section from a specific child view in Laravel Blade

2026-05-25T17:00:00Z

Sometimes you have a shared layout that renders a block — a promotional banner, a sidebar widget, an image — that makes sense on most pages but not all. Rather than duplicating the layout or reaching for JavaScript, you can solve this cleanly with two built-in Blade directives: @section and View::hasSection().

The problem

Say your login layout renders a decorative image at the bottom of every page:

{{-- resources/views/layouts/login.blade.php --}}
@php $loginImage = Contractify::loginImage(); @endphp
@if ($loginImage->src)
    
        src }}" alt="{{ $loginImage->alt }}">
    
@endif

This is fine for the login and registration pages, but on the OAuth authorization page it's visual noise you'd rather skip.

The solution

Step 1 — Wrap the block in the layout with @unless(View::hasSection(...)):

{{-- resources/views/layouts/login.blade.php --}}
@unless(View::hasSection('hide-login-image'))
    @php $loginImage = Contractify::loginImage(); @endphp
    @if ($loginImage->src)
        
            src }}" alt="{{ $loginImage->alt }}">
        
    @endif
@endunless

Step 2 — Declare the empty section in the child view that should opt out:

{{-- resources/views/vendor/passport/authorize.blade.php --}}
@extends('layouts.login')

@section('hide-login-image')@endsection

@section('content')
    {{-- ... --}}
@endsection

How it works

View::hasSection('hide-login-image') returns true if the currently-rendering child view has declared a section with that name — even an empty one. The @unless then skips the block entirely.

Every other view that extends the layout leaves that section undeclared, so hasSection returns false and the image renders as normal.

Why this is better than the alternatives

No layout duplication. You keep a single source of truth.
No conditionals based on route names or controller names. Those couple your layout to your routing.
The opt-out lives in the view that needs it. Easy to find, easy to remove.

It's a tiny pattern, but it keeps your layouts clean and your child views in control of their own presentation.

#frontend #laravel #best-practice

🐥 YellowDuck.be

🐥 Faster, leaner MySQL backups and restores with mysqldump

Where to configure this in Laravel

Dump-time optimisations

Exclude noisy, non-essential tables

Use a single transaction

Increase --net-buffer-length

Skip unnecessary LOCK/UNLOCK statements

Suppress MySQL 8 column statistics

Compress the dump on the wire

Skip binary log tracking (GTID)

Avoid table-level locks

Use --quick for large tables

Restore-time optimisations

SET sql_log_bin=0

SET foreign_key_checks=0

SET unique_checks=0

SET autocommit=0

Putting it together

More info

🐥 Speeding up our PHP test suite with a GitHub Actions matrix strategy

The Problem: Three Jobs, Lots of Duplication

Step 1: Collapsing Three Jobs into a Matrix

Why More Suites Instead of Fewer?

Step 2: Caching apt Packages

The Downstream Effect: Coverage Merging

What This Changes

🐥 Advanced debugging in Elixir with IO.inspect

The Basics of IO.inspect

Caveat: When IO.inspect Is the Last Call in a Function

Strategic Placement With the Pipe Operator

Using Labels for Clarity

Pretty Printing and Formatting Output

Coloring and Styling Output

Conditional Inspection

Capturing Inspect Output Instead of Printing

Debugging Concurrency and Async Code

Advanced Trick: Inspecting and Pattern Matching in One Go

Using dbg/2 for Richer Inspection

When IO.inspect Is Not Enough

Setting Default Options for IO.inspect in IEx

Creating a Reusable Inspect Helper

Handy Visual Studio Code snippets

Best practices

Conclusion

🔗 My thoughts on the future of Go in the AI era

🐥 Hosting your own APT repository for a Go CLI tool

The goal

Step 1: cross-compiling and packaging

Step 2: the aptblob tool

Step 3: initialising the repository

Step 4: uploading the packages

Step 5: publishing

Why this approach works well

The full release workflow

Summary

🐥 Distributing a Go CLI via a Homebrew tap

What is a tap?

The formula

Building a universal macOS binary

The CDN URL strategy

Updating the formula

CI: validating formulas with brew test-bot

CI: Merging PRs with brew pr-pull

The user experience

Summary

🐥 Adding 2FA to Phoenix LiveView with phx.gen.auth

Add dependencies

Extend the user schema

Add 2FA setup LiveView

Add a TOTP challenge LiveView

Update login logic

Update UserAuth helper

Done

🐥 Finding unexpected HTTP responses in JSON logs with jq

What this does

Why this is useful

A small improvement

🐥 Katagelophobia and cyber security: when fear of ridicule becomes a vulnerability

The hidden driver behind silence

Increase `--net-buffer-length`

Use `--quick` for large tables

`SET sql_log_bin=0`

`SET foreign_key_checks=0`

`SET unique_checks=0`

`SET autocommit=0`

The Basics of `IO.inspect`

Caveat: When `IO.inspect` Is the Last Call in a Function

Using `dbg/2` for Richer Inspection

When `IO.inspect` Is Not Enough

Setting Default Options for `IO.inspect` in IEx

Step 2: the `aptblob` tool

CI: validating formulas with `brew test-bot`

CI: Merging PRs with `brew pr-pull`

Update `UserAuth` helper

`input()` returns mixed values

`string()` always returns a Stringable object

The problem with `=` and `NULL`

Enter `<=>`

`<=>` vs. the alternatives

Why compare against `$baselineLevel` instead of `0`?

Why not just `wait`?

Scoping to `tests/Unit`