$ ~/archive/ play xz-backdoor
transcript_decrypted.log
0.0 Five hundred milliseconds.
1.84 Half a second.
3.379 The margin by which the internet was not
6.559 hijacked.
6.919 March, 2024.
10.039 San Francisco.
11.56 A thirty-eight-year-old Microsoft engineer named Andres Freund is
15.298 benchmarking PostgreSQL on a Debian development system.
18.306 It is a routine task.
20.454 Freund is a committer on the PostgreSQL project.
23.893 He tests database performance against pre-release Linux distributions
27.332 as a regular part of his work.
30.339 Something is wrong with the test environment.
33.2 Failing SSH login attempts — the kind that
36.863 hit every public-facing server constantly, automated bots trying
40.228 random username and password combinations — are using
43.591 far more CPU than they should.
46.115 A failed login should terminate quickly.
48.637 These are not terminating quickly.
50.74 Freund notices.
52.32 He investigates further.
53.926 Successful SSH logins, on his own local machine,
57.406 are taking five hundred milliseconds longer than the
60.889 baseline of approximately one hundred milliseconds.
63.5 Half a second.
64.92 Freund runs the connection under Valgrind, a memory-debugging
69.763 tool.
70.17 Valgrind throws errors pointing at liblzma — a
73.414 compression library that the SSH daemon has no
76.658 legitimate reason to invoke.
78.28 This is when the investigation becomes urgent.
81.299 Over the next several days, Freund traces the
84.772 errors.
85.168 He finds that the sshd process on his
88.34 test machine is executing code from liblzma during
91.512 authentication.
91.908 He dissects the xz-utils source in the git
95.079 repository.
95.475 The source is clean.
97.061 But the release tarball — the compressed source
100.233 archive that Debian actually downloads and builds from
103.406 — contains a file called build-to-host.m4 that is
106.578 not in the git source.
108.56 That file contains an obfuscated script.
111.801 The script decodes a bash script hidden inside
115.722 a test file called bad-3-corrupt_lzma2.xz — a file
119.644 that is disguised as a broken test input
123.566 for the compression library's error handling.
126.507 The bash script decodes a second file, good-large_compressed.lzma,
130.429 using custom obfuscation combined with RC4 decryption.
133.86 What comes out at the end of the
136.834 chain is a compiled shared object.
138.84 The shared object is a backdoor.
141.979 On the evening of March twenty-ninth, 2024 —
146.679 while, by his own account, listening to a
149.778 security podcast during a cooking break — Andres
152.877 Freund posts his findings to the oss-security mailing
155.978 list at Openwall.
157.139 Within 24 hours, Red Hat has assigned it
160.9 a CVE number.
162.198 The severity score is ten point zero.
165.224 The highest possible.
166.522 CISA issues an emergency advisory.
168.685 Debian, SUSE, Fedora, Arch, and Kali all revert
172.145 the affected packages.
173.443 GitHub suspends the account of the maintainer who
176.902 committed the backdoor.
178.199 The backdoor, it turns out, was scheduled to
181.644 ship with stable Linux distributions within approximately two
184.787 weeks.
185.18 Freund caught it by accident.
187.08 This is the case file on what happened.
191.578 On what almost happened.
193.026 And on what the operation was.
195.199 The target of the attack was xz-utils.
199.06 xz-utils is a compression toolkit.
201.688 Its main tool, the xz command, produces .xz
205.414 files — the Unix equivalent of .zip files,
209.14 with higher compression ratios.
211.003 Beneath the command-line tool is a library called
214.73 liblzma, which provides the compression algorithm to other
218.456 programs that need it.
220.319 liblzma is loaded, directly or indirectly, by an
223.984 enormous amount of Linux system software.
226.507 Package managers link against it.
228.609 System utilities link against it.
230.711 And — through a chain that the attackers
234.075 specifically engineered — the OpenSSH daemon links against
237.439 it.
237.86 The chain runs like this.
240.689 On most major Linux distributions, sshd is patched
244.736 to support systemd's service notification mechanism, called sd_notify.
248.784 That patch causes sshd to load a library
252.831 called libsystemd.
253.843 And libsystemd, in turn, loads liblzma.
256.879 The result: on most production Linux servers, the
260.66 SSH daemon — the process that accepts remote
264.139 logins — loads a compression library at startup,
267.62 even though SSH does not compress authentication traffic.
271.1 Compromise the compression library, and you compromise the
274.696 daemon that controls remote access to every Linux
277.992 server on the internet.
279.639 At the moment of the backdoor's near-deployment, Linux
283.206 was running roughly ninety-six percent of the top
286.471 one million web servers in the world.
289.329 All five hundred of the world's most powerful
292.596 supercomputers.
293.004 Approximately ninety-two percent of virtual machines across Amazon
296.269 Web Services, Google Cloud, and Microsoft Azure.
299.127 Android, which powers about eighty-five percent of the
302.394 world's smartphones, is built on the Linux kernel.
305.66 The xz-utils backdoor, had it shipped to stable
308.971 distributions, would have been present in a substantial
311.982 fraction of all of the above.
314.24 Alex Stamos, the former chief security officer at
317.766 Facebook, described the intended outcome in one phrase:
320.99 a master key to any SSH server on
324.216 Earth.
324.62 This is why the CVE was scored ten
329.675 point zero.
330.54 The technical design of the backdoor is worth
334.487 understanding precisely.
335.399 The attacker exploited the fact that release tarballs
338.867 contain auto-generated files that do not exist in
342.033 the version-controlled git source.
343.617 Specifically, the build-to-host.m4 file in the tarball contained
346.783 a single line that had been modified to
349.949 inject an obfuscated script into the configure step
353.117 of the build process.
354.699 This meant that anyone reading the git source
357.783 code could not see the backdoor.
359.87 Anyone building from git could not trigger it.
362.653 Only Linux distributions, which build from the released
365.437 tarballs — which is effectively all of them
368.22 — would execute the injection.
369.959 The injection chain was three stages.
372.636 Stage one extracted a bash script from a
375.803 file disguised as a corrupt compression test input.
378.971 Stage two used that bash script to decode
382.139 a second test file into a compiled shared
385.307 object.
385.704 Stage three linked the shared object into the
388.872 compiled liblzma binary.
390.06 The shared object used a legitimate glibc feature
394.492 called IFUNC — indirect function resolvers — to
398.624 hijack a specific OpenSSH function called RSA_public_decrypt.
402.24 RSA_public_decrypt is the OpenSSH function that validates RSA
406.473 signatures during certificate authentication.
408.439 Every time a client attempts to connect using
412.372 an RSA certificate, sshd calls this function to
416.305 verify the signature.
417.779 With the backdoor active, sshd was calling the
421.92 attacker's code instead.
423.36 The attacker's code inspected the RSA public modulus
427.163 — the large integer value passed in the
430.668 client's certificate.
431.543 Normally, this value is used in standard RSA
435.048 verification.
435.485 In the backdoor, it was actually a payload
438.99 container.
439.427 The code decrypted the payload using a hardcoded
442.932 ChaCha20 symmetric key.
444.245 Then it verified the decrypted payload's signature using
447.75 a hardcoded Ed448 public key.
449.939 If the signature verified — meaning the payload
453.107 was signed by the attacker's private key —
455.973 the code executed the embedded shell commands as
458.841 root.
459.199 This is what security researchers call gated remote
464.192 code execution.
465.04 The backdoor activates only when the attacker presents
468.432 a valid cryptographic signature.
470.127 An ordinary attacker who stumbled across the backdoor
473.519 could not exploit it.
475.216 Only the holder of the Ed448 private key
478.608 could trigger it.
479.879 This detail matters.
481.367 A criminal actor building a vulnerability for sale
484.536 makes it usable by whoever buys it.
487.307 A state actor building a persistent access capability
490.475 makes it exclusive.
491.663 Only they, and anyone they explicitly authorize, can
494.831 use the key.
496.019 The xz-utils backdoor was designed for exclusive use.
499.875 It was not a vulnerability.
502.098 It was a strategic asset.
504.319 The operation that produced the backdoor began on
508.721 January twenty-sixth, 2021.
510.259 On that date, a GitHub account was created
514.092 under the username JiaT75.
515.859 The display name was Jia Tan.
518.508 The account had no prior digital footprint.
521.599 No social media presence under that name.
524.69 No conference talks.
526.014 No previous open-source contributions.
527.78 No appearances in data breaches.
529.989 The name appears to be a pseudonym.
533.08 JiaT75's first public contribution to xz-utils occurred on
537.379 October twenty-ninth, 2021 — nine months after the
541.379 account's creation.
542.379 It was a minor patch for an editor
546.379 configuration file.
547.379 Innocuous.
547.879 Unimportant.
548.379 The kind of contribution that establishes presence without
552.379 drawing scrutiny.
553.379 Over the next two years and five months,
557.476 JiaT75 authored more than five hundred commits to
561.274 open-source projects.
562.222 The vast majority were legitimate improvements — code
566.019 reviews, translations, continuous integration maintenance, bug fixes.
569.341 Useful work.
570.291 Genuinely helpful.
571.24 Approximately eight of those commits were malicious.
574.34 The ratio matters.
575.889 Sixty to one.
577.138 For every malicious commit, the operator produced sixty
580.47 pieces of real, useful work.
582.551 This is what made the pattern impossible to
585.883 detect through statistical anomaly analysis.
587.965 The attacker spent two and a half years
591.296 of dedicated effort producing genuinely valuable contributions, purely
594.626 to accumulate the trust required to commit the
597.957 eight changes that ultimately mattered.
600.039 The operation did not work alone.
603.919 Beginning in April 2022, a user calling himself
607.337 Jigar Kumar appeared on the xz-devel mailing list.
610.456 Kumar had no history on the list before
613.575 April 2022.
614.355 His entire presence consisted of pressure emails to
617.472 the project's lead maintainer, complaining about slow response
620.591 times and demanding that a new maintainer be
623.71 added.
624.1 In May 2022, a second account — Dennis
627.798 Ens — appeared, inquiring about the maintenance status
631.197 of the Java version of xz-utils, and following
634.596 up with additional pressure messages.
636.72 Both accounts had the same profile.
639.62 No digital presence before 2022.
641.787 No activity outside the xz-devel mailing list.
644.82 No participation in any other project before or
648.287 after the pressure campaign.
650.019 In February 2024 — weeks before the backdoor
653.788 was committed — a third account appeared.
656.823 Hans Jansen.
657.691 Jansen's role was to pressure Debian maintainers into
661.159 adopting the compromised version of xz-utils as soon
664.629 as possible.
665.495 On March twenty-fifth, 2024, Hans Jansen filed a
668.965 Debian bug report explicitly requesting the upgrade.
672.0 Four days later, Andres Freund posted his findings
676.043 to oss-security.
676.98 Two additional supporting accounts — krygorin4545 and misoeater91
682.991 — provided background pressure in various threads, following
687.403 the same pattern.
689.058 No pre-operation presence.
690.712 Participation only during specific pressure windows.
694.022 Disappearance after objectives were achieved.
696.779 Kaspersky's post-incident analysis noted that the geographic styling
701.02 of the sock-puppet names was inconsistent — Singaporean,
704.962 European, Indian — suggesting that the operator had
708.903 deliberately dispersed the cover identities to avoid the
712.845 appearance of coordination.
714.322 But the behavioral pattern — timing of appearances,
718.264 synchronized pressure points, collective disappearance after each objective
722.206 — suggested a single operator or tightly coordinated
726.147 team.
726.639 The pressure worked.
728.299 On June twenty-ninth, 2022, the lead maintainer of
731.697 xz-utils — a Finnish software developer named Lasse
734.796 Collin — replied to the mailing list and
737.894 stated that Jia Tan would have a bigger
740.99 role in the project going forward, and was,
744.089 in practice, already functioning as a co-maintainer.
746.799 This was the handover.
748.82 To understand what had just happened, it is
752.299 necessary to understand what Lasse Collin's position was.
755.48 Collin had maintained xz-utils, alone, since 2009.
758.51 The project was unpaid.
760.072 He worked on it in his free time.
763.192 Earlier that same month, in a June 2022
766.315 mailing list reply, Collin had publicly acknowledged that
769.436 he was dealing with long-term mental health issues.
772.557 He referred to the project, in his own
775.678 words, as an unpaid hobby project.
778.019 Lasse Collin is not responsible for what happened.
782.432 The security community has been unanimous on this
785.245 point.
785.596 Collin was doing unpaid hobby work.
787.706 He was dealing with documented health challenges.
790.168 He was under sustained, coordinated pressure from what
792.98 turned out to be a state-level intelligence operation.
795.793 The system that placed him in that position
798.606 failed him.
799.309 He did not fail the system.
801.419 What the operation exploited was not Collin's competence.
805.279 It was his structural position — alone, unpaid,
808.84 dealing with real-world stressors, responsible for maintaining infrastructure
812.399 used by billions of people, without institutional support.
815.96 The professional attacker identified that structure and targeted
819.993 it.
820.46 Once the handover was complete, the infrastructure preparation
824.635 began.
825.12 In October 2022, Jia Tan was added to
828.985 the Tukaani organization on GitHub.
831.214 In December 2022, he created the repository's organizational
834.779 structure and made his first release.
837.453 In March 2023, he changed the primary security
841.019 contact email at Google's OSS-Fuzz — the automated
844.586 vulnerability-scanning platform — from Lasse Collin's address to
848.15 his own.
849.043 From that point forward, any security-relevant bug report
852.609 would reach Jia Tan before it reached anyone
856.173 else.
856.62 In June 2023, Jia Tan introduced IFUNC resolver
860.393 infrastructure into liblzma.
861.695 The commits appeared to be legitimate performance optimization
865.168 work.
865.601 In retrospect, they were the hooks through which
869.075 the backdoor would eventually be injected.
871.679 In July 2023, Jia Tan opened a pull
875.525 request at OSS-Fuzz to disable IFUNC fuzzing for
879.072 xz.
879.514 The change was intended to prevent the fuzz-testing
883.061 pipeline from detecting anomalies in the IFUNC-based backdoor
886.606 when it eventually shipped.
888.379 In February 2024, a separate commit broke the
891.836 build-system detection of Landlock — a Linux kernel
894.99 security feature that restricts process privileges.
897.357 The commit disabled xz's own sandbox on systems
900.513 that would otherwise have enforced it.
902.879 On February twenty-third and March ninth, 2024, the
906.546 backdoor itself was committed in two stages, packaged
909.913 as part of the xz-utils 5.6.0 and 5.6.1
913.278 releases.
913.7 Nineteen days later, Andres Freund noticed a 500-millisecond
918.703 SSH delay.
919.879 Researchers analyzing the commit metadata left behind by
924.019 the operation noticed a pattern.
926.419 Jia Tan's commit timestamps clustered primarily in the
930.125 Central European or Eastern European time zones.
933.106 Working hours corresponded to approximately nine in the
936.513 morning to five in the afternoon Central European
939.918 Time.
940.345 Activity continued during major Chinese public holidays and
943.75 paused during several European ones.
945.879 The name, and the claimed operational geography, was
949.679 East Asian.
950.554 The actual work pattern was European.
953.179 This is what intelligence tradecraft analysts call signature
956.735 leakage.
957.14 An operator spent over thirty months maintaining a
960.395 cover identity.
961.21 But the automatic timestamps embedded in every git
964.464 commit periodically revealed the actual location of the
967.719 machine that was making the commit.
970.159 Three candidate attributions have appeared in public analysis
975.53 from security researchers with relevant expertise.
978.36 American security researcher Dave Aitel, a former NSA
982.128 computer scientist, publicly assessed that the operation fits
985.596 the pattern attributable to APT29 — the advanced
989.065 persistent threat group attributed by the United States
992.533 and United Kingdom governments to Russia's Foreign Intelligence
996.0 Service.
996.434 APT29 is known for long-duration espionage campaigns, including
999.903 the SolarWinds supply chain compromise disclosed in 2020.
1003.37 Working hours aligned with Moscow time zone match
1006.839 Jia Tan's pattern.
1008.139 Costin Raiu — the former director of Kaspersky's
1012.192 Global Research and Analysis Team, with three decades
1015.945 of experience attributing sophisticated operations — identified three
1019.697 plausible candidates in a February 2026 podcast interview.
1023.448 Russia's APT29 was one.
1025.325 China's APT41, associated with the Ministry of State
1029.076 Security, was a second.
1030.953 North Korea's Lazarus Group, already featured in earlier
1034.705 Fragment Zero coverage, was a third.
1037.519 As of April 2026, no intelligence service has
1041.03 publicly attributed the operation.
1042.635 No indictments have been filed.
1044.643 No arrests have been made.
1046.648 The real identity of Jia Tan remains unknown.
1049.859 What is not in dispute, among researchers with
1053.861 relevant tradecraft expertise, is that the operation's patience,
1057.561 operational security, cryptographic sophistication, and resource commitment are
1061.262 consistent with a nation-state intelligence service — or
1064.962 a close functional equivalent — and are inconsistent
1068.663 with individual criminal or hacktivist activity.
1071.44 This was not the work of a lone
1074.336 hacker.
1074.66 The xz-utils operation was possible because of a
1079.203 structural feature of how modern technology civilization builds
1083.448 its critical infrastructure.
1085.039 The software that runs the internet was built,
1088.229 to a substantial degree, by volunteers working on
1091.117 their own time.
1092.201 The companies that profit from this software have
1095.089 contributed back a small fraction of its economic
1097.979 value.
1098.339 xz-utils was bundled with every major Linux distribution
1102.17 and ran on an enormous fraction of global
1105.7 servers.
1106.141 Its maintainer was unpaid.
1107.905 OpenSSL, the library that provides cryptography to most
1111.435 of the internet, was notoriously understaffed before the
1114.965 Heartbleed vulnerability in 2014.
1116.73 Log4j, the Java logging library behind the Log4Shell
1120.26 vulnerability in 2021, was maintained by a handful
1123.79 of volunteers — behind enterprise infrastructure globally.
1126.88 In every case, a library treated as critical
1131.227 infrastructure by multi-billion-dollar companies was maintained with hobby-project
1135.273 resources.
1135.779 The xz-utils operation did not invent this structural
1140.532 vulnerability.
1141.089 It exploited it.
1142.759 On April eleventh, 2024 — two weeks after
1148.166 Freund's disclosure — the U.S.
1150.547 Cybersecurity and Infrastructure Security Agency published a formal
1154.355 statement acknowledging the structural problem.
1156.733 CISA's position: the burden of securing open-source infrastructure
1160.541 cannot fall on individual unpaid maintainers, and companies
1164.348 consuming open-source software must contribute back, either financially
1168.156 or through developer time, to produce a sustainable
1171.963 ecosystem.
1172.44 The recommendations were non-binding.
1174.653 They were best practices.
1176.567 They depended on voluntary adoption by companies whose
1180.393 incentive structure had not historically rewarded the investment.
1184.22 Within a month, the Linux Foundation and the
1187.789 Open Source Security Foundation published a joint alert
1191.058 warning that similar social engineering takeover attempts were
1194.328 already in progress against multiple other open-source projects.
1197.598 The OpenJS Foundation — which maintains Node.js, jQuery,
1200.868 and related JavaScript infrastructure — publicly disclosed that
1204.137 it had received a coordinated pressure campaign following
1207.407 the same pattern as xz-utils, and had rebuffed
1210.678 it only because the xz-utils disclosure had taught
1213.948 the community what to look for.
1216.4 A 2026 Linux Foundation report documented the broader
1220.178 pattern across the ecosystem.
1221.916 The report's core finding: what happened with xz-utils
1225.394 was not a singular incident.
1227.567 It was a method.
1229.307 The method is being attempted at scale.
1232.349 Most of the successful detections are happening because
1235.827 the xz-utils case provided a signature to match
1239.305 against.
1239.74 How many unsuccessful detections there are — operations
1242.836 already in progress that have not yet been
1245.634 caught — is by construction impossible to count
1248.431 from public sources.
1249.48 Two episodes ago on Fragment Zero, the case
1253.299 file on the Dark Forest hypothesis closed with
1256.819 an observation.
1257.7 The doctrine Liu Cixin formalized in 2008 —
1261.22 silence as survival, concealment as strategic necessity, revelation
1264.74 as existential hazard — is the oldest operational
1268.259 security principle in the history of human conflict.
1271.779 Every force that has ever operated under conditions
1275.584 of uncertain threat and asymmetric capability has converged
1279.087 on the same conclusion.
1280.839 Be quiet.
1282.553 Move carefully.
1283.967 Assume observation.
1285.38 The xz-utils operation is the Dark Forest executed
1291.164 inside a human trust relationship.
1293.779 The attacker did not breach a firewall.
1296.971 The attacker did not exploit a zero-day.
1299.862 The attacker did not bypass any cryptographic protection.
1303.164 The attacker did something much simpler.
1305.643 The attacker hid in plain sight for three
1308.946 years, producing useful work, building genuine credibility, behaving
1312.25 exactly like any other helpful contributor, while preparing
1315.555 — silently, patiently, at strategic patience unimaginable to
1318.857 most technical organizations — the moment at which
1322.162 the prepared infrastructure would be used.
1324.64 The attack succeeded almost entirely because it was
1328.385 silent.
1328.817 It was detected not by any security tool,
1332.262 not by any audit, not by any institutional
1335.709 defense, but by one engineer's accidental observation of
1339.154 five hundred milliseconds of unexplained latency.
1341.74 Andres Freund's statement on his own discovery, posted
1345.057 to Mastodon in the weeks after the disclosure,
1348.074 should close the case file.
1349.96 Relying on luck in the future is a
1353.172 bad strategy.
1353.9 The xz-utils backdoor was caught.
1358.22 The Dark Forest version of the operation —
1361.625 the one that targets trust relationships rather than
1364.731 computer systems — is being attempted right now,
1367.838 at this moment, against some indeterminate number of
1370.944 other critical open-source projects.
1372.498 The pattern works.
1373.663 The economic incentives that make it work have
1376.769 not meaningfully changed.
1377.933 The institutional responses have been real but insufficient.
1381.039 The engineer who finds the next one will
1383.832 also need luck.
1384.766 They will need to be looking at the
1387.258 right benchmark at the right time on the
1389.75 right system.
1390.373 They will need to care enough to trace
1392.865 an anomaly to its source.
1394.422 They will need to publish their findings before
1396.913 the operation's principal has already shipped the payload
1399.405 to stable releases.
1400.339 They will need, specifically, the five hundred milliseconds.
1404.4 That interval is what stood between the internet
1409.019 in March 2024 and a single cryptographic key
1412.039 held by an unknown actor that would have
1415.059 unlocked every Linux server running SSH on Earth.
1418.079 Fragment Zero will track the case file.
1422.24 The case file does not close.
1424.579 It waits.

The Internet's First Hijack: One Volunteer Almost Took Down Linux

RELATED INVESTIGATIONS
RELATED INVESTIGATIONS