Runbook: apt update fails because it asks for i386 files
grep -rn '^deb' /etc/apt/sources.list /etc/apt/sources.list.d/; dpkg --print-foreign-architectures; sudo sed -i -E '/^deb +[^[ ]/ s/^deb +/deb [arch=amd64] /' /etc/apt/sources.list
The report: a server would not update. apt update failed, and the files it was trying to fetch were i386 files, on a 64-bit (amd64) machine. The package repository was an internal, air-gapped one that has no i386 packages at all. The fix that worked was to put [arch=amd64] after "deb" at the start of every entry in sources.list, and the update worked.
This runbook explains why that fix works, how to recognise the problem, the other ways to fix it (some are better than others), and what to do on the repository side. The lab output is real: apt 2.8.3 on Ubuntu 24.04 was run as an ordinary user against a tiny local repository built for the purpose, so nothing on this machine was changed. Commands that need root, or that change dpkg, are shown without output.
The five-minute path
dpkg --print-architecture the machine's own architecture (amd64)
dpkg --print-foreign-architectures extra architectures enabled: "i386" here is the cause
grep -rn '^deb' /etc/apt/sources.list /etc/apt/sources.list.d/ every repository the machine uses, old format
grep -rn 'URIs\|Architectures' /etc/apt/sources.list.d/*.sources the same, newer deb822 format
sudo apt update read the first E: or W: line, it names the file and the repository
then either: add [arch=amd64] to each entry, or remove the i386 architecture (Step 3 and Step 6)
The pattern to remember: apt fetches an index of packages for EVERY architecture the machine has enabled, from EVERY repository that does not say otherwise. A repository that has no i386 files produces a warning or an error, depending on how it describes itself.
Why apt asks for i386 on a 64-bit machine
Debian and Ubuntu support "multiarch": a 64-bit system can have a second architecture enabled so that it can install 32-bit libraries or programs next to the 64-bit ones. dpkg keeps the list of extra ("foreign") architectures. Once i386 is in that list, apt update asks every repository for the i386 package index (for example main/binary-i386/Packages) as well as the amd64 one.
How a server ends up with i386 enabled: someone ran dpkg --add-architecture i386 (to install Wine, a 32-bit vendor tool or a driver), or a package or an installer script did it for them. It can happen long before anyone notices, and it stays quiet until a repository that lacks i386 is added or an internal mirror is built without it. The list is in /var/lib/dpkg/arch, and the first thing to check is the output of dpkg --print-foreign-architectures.
A repository that serves only amd64 is then missing something that apt expects. What happens next depends on the repository's Release file (the signed list that describes the repository):
- if it says "Architectures: amd64", apt understands that i386 is not offered and skips it silently
- if it has no Architectures line, apt warns that the repository "does not seem to provide" the i386 index
- if it lists the i386 index but the file is not there (a mirror that dropped i386, a broken sync), apt reports an error, the update is marked failed (exit status 100) and package lists may be stale
That third case is the one that stops patching.
Step 1: is a foreign architecture enabled?
$ dpkg --print-architecture
amd64
$ dpkg --print-foreign-architectures
$
On the lab machine the second command prints nothing: no foreign architecture, so apt asks for amd64 only and the problem cannot happen. On a server that has the problem it prints "i386". (The lab below pretends i386 is enabled by telling apt about it with an option, so that nothing in the real dpkg database was touched.)
Step 2: what the failure looks like (a real lab)
A tiny repository was built with one package for amd64 and no i386 files, and apt update was run against it with i386 enabled and with no [arch=] in the entry. Three versions of the repository's Release file give three different results. The repository was a local one (file:), so the messages say "File not found" where a web server would say "404 Not Found". (The line "exit status: N" under each result is printed by the small script that ran apt, not by apt itself.)
First version: the Release file says "Architectures: amd64".
$ apt-get update (entry: deb [trusted=yes] file:/srv/mirror-lab/repo stable main)
Reading package lists...
exit status: 0
Quiet success. apt read the Architectures line and did not ask for i386.
Second version: the Release file has no Architectures line.
$ apt-get update
Reading package lists...
W: Skipping acquire of configured file 'main/binary-i386/Packages' as repository 'file:/srv/mirror-lab/repo stable InRelease' does not seem to provide it (sources.list entry misspelt?)
exit status: 0
A warning, but the update completed (exit status 0). The text "sources.list entry misspelt?" is misleading: the entry is fine, the repository just does not provide i386.
Third version: the Release file lists the i386 indexes, but the files are not on the server.
$ apt-get update
Get:4 file:/srv/mirror-lab/repo stable/main i386 Packages [138 B]
Err:4 file:/srv/mirror-lab/repo stable/main i386 Packages
File not found - /srv/mirror-lab/repo/dists/stable/main/binary-i386/Packages.gz (2: No such file or directory)
Get:4 file:/srv/mirror-lab/repo stable/main i386 Packages [127 B]
Err:4 file:/srv/mirror-lab/repo stable/main i386 Packages
File not found - /srv/mirror-lab/repo/dists/stable/main/binary-i386/Packages (2: No such file or directory)
Reading package lists...
E: Failed to fetch file:/srv/mirror-lab/repo/dists/stable/main/binary-i386/Packages File not found - /srv/mirror-lab/repo/dists/stable/main/binary-i386/Packages (2: No such file or directory)
E: Some index files failed to download. They have been ignored, or old ones used instead.
exit status: 100
This is the failure from the report: the E: lines, "Some index files failed to download" and exit status 100. A script, a cron job or a patching tool that checks the exit status sees a failed update. Over HTTP the same line reads "404 Not Found" with the server's address, and the file name ends in binary-i386/Packages (or Packages.gz, or Packages.xz).
The messages always name the file: when you see "binary-i386" on an amd64 machine, this is the cause.
Step 3: the fix: [arch=amd64] on each line
Put an option block right after the word deb: deb [arch=amd64] URL suite component. apt then asks that repository for amd64 only, and never mentions i386.
$ apt-get update (entry: deb [arch=amd64 trusted=yes] file:/srv/mirror-lab/repo stable main)
Reading package lists...
exit status: 0
The same repository that failed a moment ago now updates cleanly. Notes:
- Several options go in one block, separated by spaces: deb [arch=amd64 signed-by=/usr/share/keyrings/vendor.gpg] https://repo.example.com/apt stable main
- Do it for EVERY deb line that points at a repository without i386: the main one and every third-party file in /etc/apt/sources.list.d/. One forgotten line keeps the update failing, and the error names which repository is the culprit.
- deb-src lines (source packages) do not need it.
- [arch=amd64] says "this repository, amd64 only". It does not remove i386 from the machine (see "Remove i386").
The same fix in the newer .sources format
Ubuntu 24.04 and recent Debian keep their repositories in /etc/apt/sources.list.d/*.sources (the "deb822" format), and the old one-line file may be empty. There the same setting is a field called Architectures:
Types: deb
URIs: file:/srv/mirror-lab/repo
Suites: stable
Components: main
Architectures: amd64
Trusted: yes
This was run for real against the lab repository and updated cleanly. For the stock Ubuntu and Debian files, the field goes inside the stanza that describes the repository (add the line Architectures: amd64 next to Suites and Components). Keep a copy of the original first.
A safe way to edit every line
Back up, change, and look at the difference before you run apt. This was run on a sample file with a plain Debian line, a line that already has options, a deb-src line and an indented line:
$ cat sample.list
# main mirror
deb http://mirror.example.com/debian bookworm main
deb http://mirror.example.com/debian bookworm-updates main
deb [signed-by=/usr/share/keyrings/vendor.gpg] https://repo.vendor.example/apt stable main
deb-src http://mirror.example.com/debian bookworm main
deb http://security.example.com/debian-security bookworm-security main
$ cp sample.list sample.list.bak
$ sed -i -E '/^deb +[^[ ]/ s/^deb +/deb [arch=amd64] /' sample.list
$ diff sample.list.bak sample.list
2,3c2,3
< deb http://mirror.example.com/debian bookworm main
< deb http://mirror.example.com/debian bookworm-updates main
---
> deb [arch=amd64] http://mirror.example.com/debian bookworm main
> deb [arch=amd64] http://mirror.example.com/debian bookworm-updates main
The pattern changes only lines that start with "deb" followed by something that is not an option block, so it does not double up. Read what it did NOT touch: the line that already had [signed-by=...] needs [arch=amd64] added by hand inside its brackets, the deb-src line is correctly left alone, and the indented line (leading spaces) was skipped, which is why you should always check with diff and never trust a one-liner on an important file. On a real server run it on a copy first, or edit by hand: there are usually only a handful of lines.
The better fix when you do not need 32-bit: remove i386
If nothing on the server needs 32-bit software, the cleanest fix is to remove the architecture, so that apt never asks for it from any repository, now or after the next repository is added. Not run here, because it changes the dpkg database; read each step.
dpkg --print-foreign-architectures confirm it is i386
apt list --installed 2>/dev/null | grep ':i386' see what i386 packages are installed
sudo apt remove --purge PACKAGE:i386 ... remove them (dpkg will not drop the architecture while they exist)
sudo dpkg --remove-architecture i386 remove the architecture
sudo apt update now clean for every repository
If the list of installed i386 packages is long, find out why before removing anything: an application may depend on them. Removing them on a production server is a change: schedule it like one.
When you do need i386
Some servers genuinely need 32-bit libraries (an old vendor agent, Wine, some drivers). Then keep i386 enabled and be exact per repository: [arch=amd64,i386] for a repository that has both, [arch=amd64] for one that only has 64-bit, [arch=i386] for a 32-bit-only one. The goal is that no entry asks for something its repository does not have.
Fix it at the source: the repository
In an air-gapped site you cannot fall back to a public mirror, so the internal repository is the single source of truth, and it is worth making it describe itself correctly. Look back at the lab: with "Architectures: amd64" in the Release file, apt simply skips i386 and no client needs to be touched. When you build the mirror, select the architectures on purpose and let the tool write the Release file from that selection (the mirroring and repository tools each have an architecture setting: apt-mirror, debmirror, aptly, reprepro and others: check your tool's documentation, not run here). Then:
- re-check the published Release file: grep -n Architectures dists/SUITE/Release
- make sure the index files the Release file lists are really in the tree (a half-synced mirror produces the third case above)
- if i386 is deliberately not mirrored but some clients have it enabled, either remove it from them (the previous section) or give those clients [arch=amd64]
Both ends matter: [arch=amd64] on the clients protects against any repository, and a correct Release file on the mirror protects every client without editing it.
Decision table: symptom to cause
E: Failed to fetch .../binary-i386/Packages (or 404) on amd64 i386 is enabled; the repository lists but lacks i386 -> [arch=amd64] on that entry, or remove i386
W: Skipping acquire of configured file 'main/binary-i386/Packages' i386 is enabled; the Release file does not say what it has -> [arch=amd64], or fix the Release file
update works, no mention of i386 Release says "Architectures: amd64", or no foreign architecture -> nothing to do
works for the main repository, fails for one file in sources.list.d a third-party repository without i386 -> [arch=amd64] on its lines
fails for amd64 files, not only i386 a different problem (network, key, wrong suite, expired Release) -> read the first E: line, see "When it breaks" in the package-management guide
[arch=amd64] added, error persists one repository was missed (sources.list.d), or a typo in the brackets -> grep every deb line again
Tips
- The first E: line names the repository and the file. Fix that repository, run apt update again, and read the next one.
- Check the exit status in automation (echo $?): an update that "mostly worked" can leave stale package lists and a patch run that installs nothing.
- apt update -o Debug::pkgAcquire::Worker=1 shows what apt requests; apt-get -s upgrade is a dry run that changes nothing.
- After a fix, run apt update twice: the second run should print no E: or W: lines at all.
- Write down the reason i386 was enabled, in the server's notes. The next person will wonder.
- The same idea exists elsewhere: dnf and zypper have no multiarch problem of this kind, but a repository that serves the wrong architecture breaks them too (check the baseurl and the $basearch variable).
- See also the package-management guide in this tab, and the runbooks for a full disk and for high load.
